Fractal Tech (NYC): Context-Enabled Semantic Caching

Location: New York City, NY (Fractal Tech)
Role: Speaker (technical talk, architecture walkthrough)
Audience Size: Startup/Builder community meetup (40 attendees)

I presented a concept and implementation approach for context-enabled semantic caching: retrieve semantically similar cached responses, then adapt them using user context (and optionally retrieval context) to produce outputs that are fast and personalized. The point of the talk was to show a pragmatic alternative to “always call the model”: treat caching as a first-class component of LLM application architecture, not an afterthought.

The session emphasized why this matters in real systems: latency budgets, cost control, and consistent user experience. I walked through the mental model, the architecture, and how Redis enables this pattern with vector search and flexible data structures for storing cached outputs and user context.

(Topics covered in the talk):

• Avoiding “generic cached answers”: how user context changes the output and why that matters for relevance.

• Deciding what counts as “context”: preferences, role, history, permissions, and how to represent it safely.

• Preventing leakage and cross-user contamination: scoping cached artifacts and controlling what can be reused.

• Choosing between retrieval augmentation vs. cache reuse vs. both: when each tool is the right hammer.

• Evaluation: how to test that personalization improves outcomes without silently harming correctness.

• Presented a clear, reusable architecture pattern for personalized LLM caching with strong cost/latency economics.

• Strengthened the narrative used in subsequent demos and published materials on context-enabled semantic caching.