Picture this. You have a file-sharing tool with a nice Netflix-style poster grid. You share your Looney Tunes collection with a friend. They open the page and see... the Kill Bill poster on Volume 1. Guardians of the Galaxy on Volume 2. Some obscure Japanese anime on Volume 5. Classy.
That's exactly what happened with ShareBox, my side project. The idea was simple: take the folder name, search TMDB, display the poster. Except folder names look like this:
Naruto.INTEGRALE.MULTI.VFF.1080p.BluRay.x264-AMB3R
Despicable.Me.COLLECTION.(2010-2024).MULTI.VFF.2160.4KLight.HDR10
Vol 1
S01My regex cleaner handled 80% of cases. The other 20% was pure chaos. "Vol 1" matched Kill Bill. "S01" matched a random Chinese series. "Movie" matched Scary Movie (I should have seen that one coming). And "Naruto" consistently returned Naruto Shippuden because Shippuden is more popular on TMDB.
The hall of shame: a bestiary of false positives
Before talking about solutions, some gems. Here's what TMDB returned when given raw folder names:
| Folder name | TMDB match | Reaction |
|---|---|---|
| Vol 1 | Kill Bill: Volume 1 | ... |
| Vol 2 | Guardians of the Galaxy Vol. 2 | Ah. |
| Covers | Alien: Covenant | Excuse me? |
| PLAYLIST | Zoey's Extraordinary Playlist | No. |
| Movie | Scary Movie | Should have seen that coming. |
| STREAM | Stream (the film) | There's a film called Stream??? |
| Wii | Wii_ja! | I don't even want to know. |
First lesson: TMDB always finds something. Even for "CLIPINF" (a Blu-ray technical folder), it returned a result with a poster. You need to know when not to search.
The approach: stack AI layers like safety nets
Rather than writing a superhuman regex, I treated the problem in layers. The principle: the free stuff does the heavy lifting, AI only catches what slips through.
- Regex + TMDB: basic name cleanup, TMDB search, first result. Free, instant.
- Pass 1 — "What's this title?": when regex fails, send the raw name to Claude Haiku to guess the real title.
- Pass 2 — "Is this the right match?": send all {folder name, TMDB match} pairs to AI to spot obvious errors.
- Pass 3 — "Which one do you pick?": when AI says "wrong match," fetch 15 TMDB candidates and let AI choose the right one.
A cron runs every 30 minutes for passes 1 and 2. Pass 3 only triggers when pass 2 finds a false positive. Out of 290 folders, pass 3 ran only 9 times.
Pass 1: the prompt that skips too much
The initial prompt was reasonable: "extract the proper title from this filename." Tested on 290 real names: 72 false skips. The AI saw "Naruto.INTEGRALE" and thought: "INTEGRALE isn't a title, it's a descriptor. Skip." Same for "Pokemon La Series" and all the Pikachu short films.
The fix came down to three rules: "when in doubt, skip=false" (a false positive will be caught later, a false skip is lost forever), "KEEP collections and integrales," and "translate known English titles to French." Result: 72 → 41 skips. Zero regressions.
Pass 2: when AI doesn't know your architecture
Pass 2 sent pairs like {"S01", "Season 1"} and asked: "is this correct?"
It flagged 55 errors. Problem: 46 were false alarms.
The AI didn't know my system had a dedicated season mechanism. When it saw "S01"
matched to "Season 1," it thought: "S01 alone means nothing, this must be wrong."
But in reality, it's the Season 1 poster fetched via TMDB's
/tv/{id}/season/1 API from the parent series. Perfectly correct.
34 Simpsons seasons. 11 Walking Dead seasons. 4 Batman seasons. All flagged as false positives when they were perfectly fine.
The fix: explain the context. A "special cases" section telling the AI that season folders matched to their number are correct. Translations are correct. Collections are correct. Result: 55 → 9 incorrects. All 9 are real problems.
Pass 3: TMDB and the popularity trap
Pass 2 detected that "Naruto" was badly matched and suggested searching "Naruto" on TMDB. Great. Except TMDB returns results by popularity. Naruto Shippuden is more popular than original Naruto. Taking the first result repeats the exact same mistake.
The solution: fetch 15 candidates and let AI choose with the filename as context. "INTEGRALE" in the name → it's the complete series, not a spin-off. The AI correctly picks Naruto (2002) over Shippuden.
Technical gotcha: the AI sometimes adds explanations after the JSON response.
{"idx": 1} because INTEGRALE indicates the complete series...
Classic json_decode() chokes on this. Fix: extract
{"idx": N} via regex instead of parsing the whole text.
The numbers
| Pass | Before | After | Improvement |
|---|---|---|---|
| Title extraction | 72 false skips | 41 | -43% |
| Match verification | 55 false negatives | 9 real issues | -84% |
| Candidate selection | 4 parse failures | 0 | -100% |
What I learned
The 80/20 rule applies to prompts. 46 out of 55 false negatives came from one pattern: season folders. One line in the prompt eliminated 84% of errors. No need to rewrite the whole prompt — find the dominant pattern and address it.
"When in doubt, do nothing" is often the best choice. Pass 1: "when in doubt, skip=false." Pass 2: "when in doubt, correct=true." Later passes will catch mistakes. Better a false positive you'll fix than a false negative you'll lose.
AI doesn't know your architecture. When the verification prompt saw "S01 → Season 1," it didn't know that was a legitimate match from a dedicated season mechanism. You need to explain your domain context, not just ask AI to verify.
Parsing matters as much as the prompt. The prompt can be perfect — if AI adds an explanation after the JSON and your parser crashes, same result. Parse defensively.
As in my article on prompt iteration loops: the best prompt isn't the one with the most instructions. It's the one that precisely describes the edge cases. "Seasons are CORRECT" is worth more than 20 lines of generic rules.