The 40,000-SKU Search & Tokenizer Collapse
The Customer-Facing Symptom
Searches for "STM32F407VGT6" or "6ES7214-1AG40-0XB0" return 0 results or take 4.5 seconds to load.
Why This Matters to Buyers
Search interest for product catalog software (12.4) and distributor management systems spikes when catalogs cross 5,000 items. Standard ecommerce search fails immediately on alphanumeric codes with dashes and slashes.
Why Generic Web Agencies Walk Directly Into This Trap
Web agencies use standard Shopify, WooCommerce, or raw SQL queries with "LIKE %query%". In relational databases, leading wildcards bypass B-tree indexes, forcing a full table scan across 40,000 rows on every keystroke. Furthermore, standard Elasticsearch/Lucene analyzers strip hyphens and slashes, breaking industrial part numbers into meaningless single digits.
// Typical Agency Approach (Postgres / MySQL)
// Scans 40,000 rows sequentially on every keystroke
SELECT * FROM products
WHERE title ILIKE '%6ES7214-1AG40%'
ORDER BY id DESC LIMIT 20;
// Result: 3,200ms TTFB, CPU 100%, 0 results if hyphen stripped!// Our Production Meilisearch Configuration
// Tokenizer preserves hyphens, sub-20ms search-as-you-type
{
"searchableAttributes": ["mpn", "brand", "series", "category", "description"],
"separatorTokens": [], // Do NOT strip '-' or '/' in industrial MPNs!
"rankingRules": [
"words",
"typo",
"proximity",
"attribute",
"exactness"
]
}Our Engineered Solution
We deploy a dedicated lightweight Meilisearch instance configured with industrial tokenizer rules. Hyphens, underscores, slashes, and periods are explicitly preserved as alphanumeric connectors. Typo tolerance is dialed specifically for part numbers (allowing 1-character typo only on prefixes, zero tolerance on pin count numbers). Response latency drops below 20ms across 40,000+ SKUs.
Have questions about this architecture?
We build these custom search and static rendering pipelines specifically for electronics and industrial parts distributors. Talk directly with our core engineers.