News ranking: a filter in front of the model
A chamber of commerce was sending every article from its RSS feeds straight to a language model for scoring, duplicates and thin posts included. Sensirel put a filter in front of the model and a cleaner behind it: a quarter of the articles now never reach the model, and the ranking got cheaper and sharper at the same time.
24.7%
Articles dropped
Before any model call is made
89 → 67
Articles per run
Duplicates and thin posts removed
20-30%
Fewer model calls
Processing time down by the same share
~$12.50
Saved per week
On a pipeline that runs every day
Executive summary
A chamber of commerce curates regional business news for its members. Every article from its RSS feeds went straight to a language model for scoring, duplicates and thin posts included. We put a filter in front of the model and a cleaner behind it, and the ranking got cheaper and sharper at once.
The challenge
RSS feeds repeat themselves. The same story arrives under three URLs and two headlines, and a third of what comes in is a title with no body. Each of those cost a model call and then sat in the ranking next to real articles.
The scoring is not generic. Articles are rated on four criteria weighted by the chamber, with bonus points for local success stories and workforce topics, and sorted into four tiers from must-share to discard. The weights belong to the client and cannot drift.
The model did not follow instructions to return raw JSON; it wrapped its answer in a markdown code block every time. Downstream nodes broke on that until it was handled.
What we built
A pre-filter before the model
A code node that runs once over the whole batch: duplicate URLs out, near-duplicate titles out on a 40-character normalised prefix, titles under 20 characters and descriptions under 30 out. Every skip is logged with its reason.
The ranker, with the client's weights
Strategic alignment 40%, regional relevance 30% with the home city scoring highest, member impact 20%, timeliness 10%. Plus 10 for a local success story, plus 5 for a workforce topic. 80 and above is must-share, 60 to 79 is the daily digest, 40 to 59 the weekly roundup, below 40 is dropped.
A cleaner after the model
The JSON is pulled out of the markdown block by pattern, normalised, and passed on. Every node keeps its try-catch so one bad answer does not drop the batch.
The stack
Self-hosted n8n runs the pipeline: RSS in, code node pre-filter, batch loop, LLM chain, code node cleaner, post-filter. The scoring weights sit in the prompt and are documented as untouchable without the client's sign-off.
Results
On a typical run, 89 articles in and 67 out of the pre-filter: 22 dropped, 24.7%, before any model call.
20 to 30% fewer LLM calls, and processing time down by the same proportion.
About $12.50 a week saved at current volume, which is small money and a large share of the bill for a pipeline that runs every day.
What we learned
- The cheapest model call is the one you do not make. Deduplication is not glamorous and it paid for the whole build.
- Never assume the model returns what you asked for. Extract, then validate.
- Test with pinned data. Replaying the same 89 articles through each node is the only way to know what a change did.
- The weights are the client's editorial line. Changing them is a meeting, not a commit.
How it works
Drop what is not an article
Duplicates, near-duplicates, empty shells.
Score with the client's criteria
Four weighted criteria, two bonuses, four tiers.
Clean the answer
Pull the JSON out of whatever the model wrapped it in.
Log every decision
Skip reasons and scores, readable in the run.
See what your team
can hand to AI
A free 30-minute call. We map one workflow together and you leave with a plan you can use, with or without us.
Book a Free Strategy Call