Notes from the Saytu team

What we learn building a Live2D character that answers preorder, sizing and authenticity questions on anime storefronts.

Latest

Understood, Not Searched

Build logSeptember 25, 20265 min read

The model understood the shopper. The catalogue did not.

A shopper asked naturally for a product, the model understood, but the catalogue never opened because an intent gate missed the phrasing. We changed the routing rule rather than the model, because natural-language support has to include everything around the LLM too.

More from the team

We kept the subject for 30 days instead of keeping the conversation

A shopper returned the next morning with the same session but without yesterday's transcript. Instead of retaining more conversation, we kept only the subject of a successful product search for the session's 30-day lifetime.

Build logSeptember 24, 20265 min read

The answer was right. The black shirt underneath it made it wrong.

We matched a shopper's request for white to a product that really had a white variant, then showed its black main photograph. The reply was technically right and the experience was wrong. We changed the rule so product cards have to support the answer a shopper actually sees.

Build logSeptember 24, 20265 min read

Five Near-Matches

Five near-matches kept the exact product away from the model

The exact product was already in Shopify, but five broad search results kept it out of the model's context. We moved the hard limit after ranking instead of before it.

Build logSeptember 23, 20266 min read

The AI started talking again. The human request stayed open.

A handoff nobody picked up used to silence the assistant forever. We now let the AI resume after staff silence without clearing the human request, because helping again is not the same as resolving it.

Build logSeptember 23, 20265 min read

Twenty chunks did not mean we had the whole document

A source with twenty stored chunks could mean either complete or truncated. We started recording the pre-cap count so cut, complete and unknown became three measurable states instead of one guess.

Build logSeptember 23, 20266 min read

Semantic retrieval was off in production, and the fallback hid it

Semantic retrieval had been failing with 401s while keyword fallback kept the assistant looking healthy. The first embeddings backfill exposed that production was authenticating with a literal environment-variable name.

Build logSeptember 23, 20265 min read