Novi · Group Product Manager · Shipped February 2026

Where to put the LLM

Keeping product badges accurate and auditable when retailer ingredient data is messy.

• Shipped Feb 2026
• ~12 brands corrected
• <100 products corrected
• 0 reported errors since

Retailers use Novi to award product badges, like the ones that tell a shopper a product meets a retailer's ingredient standards. Every badge depends on mapping a product's ingredient list to our ingredient database. That mapping has to be deterministic: retailers need every decision to be consistent, and they need an audit trail when someone asks why a product earned its badge.

The problem

During the onboarding and testing for our first multi-category retailer, they flagged formulated products that would be assigned inaccurate badges. I led the investigation.

The root cause

The mapper was working. The problem was upstream, in the ingredient data itself. Formatting quirks in the new categories made our parser read some problematic ingredients as sub-ingredients and drop them, so a product could pass a check it should have failed.

Looking across our badging errors, formatting was the single biggest class of problem:

• missing or non-standard separators
• marketing claims and other noise mixed into the ingredient list
• inconsistent parent-child formatting of ingredients

It had been a minor issue while our retailers sold mostly beauty products. Once food, supplements, and household goods arrived together, it became impossible to miss. No set of rules could cover every format.

The fix

Brand-submitted data on retailer sites will always be a little messy. Even careful retailers can't enforce strict formatting on every product page, so unexpected patterns are a given. We had already taken deterministic rules and edge-case handling as far as they could go, and there would never be enough rules to catch every case.

Engineering suggested an LLM. We didn't want one parsing ingredients, because that would cost us the consistency and audit trail that make a badge trustworthy. So I set the constraint: the mapping decision stays deterministic, and the LLM gets one tightly defined job.

Engineering proposed that job: clean up the formatting, then hand the normalized string to the same deterministic mapper we already trusted. I tested it and refined the prompt, then made the call to go with it, and engineering built it.

Testing the prompt

I tested the prompt against three sets of ingredient lists:

1. the problem lists that started the investigation
2. randomly selected lists from other retailers
3. examples from every product category, from data I had already gathered from prospective retailers

Testing changed the design. Adding the product category to the prompt improved the cleanup a lot, because supplements follow almost the opposite formatting conventions from food, and many household-goods strings aren't ingredient lists at all.

Rolling it out

We shipped right away for the new retailer, since there were no live badges to break. For an existing retailer, many brands already had live badges, so we measured every badge the change would affect before releasing. It touched about a dozen brands and fewer than 100 products, and every change was a correction: removing a badge that had been issued in error. We let those brands know first, then released.

The result

Since launch in February 2026, there have been no reported errors from this class of problem.

What I'd do next time

We used the problem lists for the initial testing only. Next time, I'd keep them as a regression set that every future prompt change has to pass, so the fix keeps holding as the prompt evolves.

What I took from it

Separate the layers before you choose a fix: the rules weren't wrong, the input was. And put the LLM where its mistakes are cheap and easy to check, never where the decision has to be auditable.