Site search: 800 recipes, finally findable

Executive summary

A recipe site with around 800 published recipes had a search box that could not find a recipe by its ingredients. Every ingredient, step and summary lived in custom fields, and the search engine only read the title and the post body, which was empty. We rebuilt the search stack and made the results provably correct.

The challenge

Searching for ricotta returned nothing, on a site that has twenty recipes containing ricotta. The content was all there. The engine simply never looked at it, because the site stores recipes in custom fields rather than in the page body, and the default search reads the body.

The obvious fix, an index that covers the custom fields, gets you halfway. It also surfaces every recipe that merely mentions the word in step nine, ranked alongside the recipes that are actually about it. A search that returns fifty results for chocolate is not better than one that returns none.

French makes it harder. Plurals, accents and typos all break an exact-match index, and the visitors who most need the search are the ones typing courgettes with an s at eleven at night.

What we built

The stack

The ranking rules are PHP running on the site itself, reading the database directly. No external service sits in the search path, and there is nothing new to keep paying for. The site's existing index and filter plugins stayed in place; the work was making them agree with each other.

Results

Verified on six ingredients: confiture 5 of 5, pancake 10 of 10, ricotta 20 of 20, chocolat 56 of 56, courgette 33 of 33, fraise 15 of 15. Exactly the recipes holding that word in their title or ingredients, with nothing off-topic.

Results pages went from 7.4 seconds to about 4.5, from 893 SQL queries to 549, from 997 KB to 596. The search engine itself accounts for 0.08 seconds of that.

The remaining 2.2 seconds were traced by profiling to the page builder's theme engine, on every uncached page. That is documented rather than guessed at, so the next person knows where to look instead of blaming the search again.

What we learned

• When a site stores its content in custom fields, the default search is not slow or badly tuned. It is looking at an empty column.
• Relevance is subtraction. Indexing more fields makes the results worse until you add the rule that excludes passing mentions.
• Always keep a fallback path. Excluding weak matches is right until a typo means there are no strong ones, and then an empty page is the worst possible answer.
• Check what the plugins do to each other. The ranking work would have been invisible in production because another plugin was disconnecting the engine on exactly the queries that mattered.
• Profile before optimising. Most of the time on those pages was never in the search.