The problem

One of Lindy’s main product offerings was drafting emails on behalf of users. However, an agent without up-to-date context about the user can cause more harm than good. Work priorities change, coworkers leave, and projects end. This was the problem I was tasked with solving.

The system in action

You get an email from a prospective client asking to tweak one detail in an MSA. You're not a legal expert so you need to loop in the Head of Legal, who just joined the company.

You open your inbox and check the drafted reply. Your agent has inferred the staffing change from your recent inbox activity and composed a draft with the new head of legal cc'd. You send the email!

How it was built

There are a few important requirements that we identified early on for the system.

1. The system needs to be able to initialize a document representing the users work context, important contacts, and stylistic preferences.
2. The system needs to keep this user context up-to-date.
3. The system needs to avoid poisoning the user context.

I'll quickly go over how I fulfilled the first requirement; the second and third are far more interesting in my opinion.

Initialization

The initial piece was fairly straight forward to stand up.

A part of the onboarding journey for users is delegating email authorization to the platform, granting us scopes to read and write. This meant we already had everything we needed to start pulling in emails for the design.

Considering users wouldn't need drafts immediately meant that firing some sort of asynchronous workflow was needed. Deciding on what technology to run these workflows on was made easier because we already had Temporal as a dependency and Temporal has properties that work perfectly for this problem, more on that later.

With those pieces out of the way, I designed a workflow with two separate activities.

1. Fetch, Format, Save: Fetch the past few months of their inbox, sanitize and format emails into threads, then save it for short term storage.
2. Compose User Context: Pull in the email thread document from the last activity, prompt an LLM to infer important facts about the user, then save the model output for long term storage.

Quick(ish) aside on the design here.

Why temporal? Temporal makes sense here because inbox ingestion is long-running and failure-prone; think rate limits, auth errors, Anthropic is down, etc. Temporal ensures durable, retryable activities out of the box.

Why the two separate activities? There are two parts of the workflow that are potentially error prone and therefore should be independently retryable; fetching user emails across a large timeframe and the LLM calls.

Why claim check pattern? In Temporal, data passed between activities is serialized into the workflow's event history, which has hard size limits. So activity one needed to save its large output to a table so that it could just hand the primary key to activity two.

Now lets dive into how I fulfill the other two requirements; keeping user context up-to-date and preventing poisonous context.

Multi-agent self improvement

Having shipped the initialization step, qualitative and quantitative data started coming in — and as we suspected, initialization alone wasn't going to cut it.

Some customers reported that drafting occasionally missed the mark, both stylistically and in substance. The metrics dashboard we spun up to track drafting performance told the same story from another angle: nearly 40% of the drafts our agent produced were heavily edited before the user sent them.

Digging into it, we traced the frustration back to a single root cause — staleness of user context. The document we'd composed at onboarding was a snapshot, and snapshots go stale the moment work priorities shift or a coworker leaves.

That reframed the problem. Keeping context fresh pushes you to update aggressively, but llm's are inherently unpredictable and may write something wrong into the document. So the real question wasn't how to update the context — it was how to update it aggressively without blindly trusting any given update. That immediately ruled out the naive approach of just re-running initialization on a schedule: it would inherit all of initialization's blind spots, only now on a loop.

From there the pieces were mostly forced. If I couldn't trust an edit on assertion, I needed something objective to check it against — and the user's sent emails were exactly that, a stream of facts and preferences the user actually signed off on. That gave me a ground truth to measure edits against: replay the draft call with a proposed change and see whether it moved the output closer to what the user really sent.

This reasoning lead me to design a nightly Temporal Workflow with four activities containing three distinct agents - an agent to propose edits to user context, one that would replay the draft with only the updated user context changed, and one to judge whether the new draft is better than the original one.

1. Fetch Sent-Draft Pairs: Get user's draft-sent email pairs for the day, only if they differed significantly, then store the findings in a short-lived collection (claim check pattern).
2. Propose: With the days draft-sent pairs, prompt a LLM to update the user's context document to reflect new facts or preferences that contradict or that are genuinely new.
3. Replay: With the proposed edit we replayed the originating llm call with just the user context document changed. Here we reused LLM replay infrastructure we had built for evals.
4. Judge: Prompt an LLM to pick which from the old and new draft was closer to the email sent by the user. If the judge selected the draft from the replay step more often we accepted the proposed edit and saved it as the users new context document.

Results

The same dashboard that surfaced the problem got to tell the ending. In the weeks after shipping the self-improvement loop, the share of drafts that users heavily edited before sending fell from roughly 40% to 20% — half of the failure mode, gone. The qualitative side moved with it: the complaints about drafts missing the mark on tone or substance quieted down, and we started seeing moments like the one at the top of this post, where the agent picked up a staffing change nobody had told it about.

Just as telling was what didn't happen. The judge rejected a meaningful share of proposed edits every night, and that was the system working as intended — each rejection was a plausible-sounding inference that would have been silently written into the context document under a naive update scheme. The gate wasn't overhead; it was the reason we could run the loop aggressively at all.

Closing thoughts

With every system in software engineering the work is never really done and that certainly applies to this project. A few things that come to mind in terms of gaps in the system that I wish we had time for.

1. Initialization: There's an irony in the final design — every nightly edit had to survive a replay-and-judge gate, but the initial context document, the foundation everything else amended, was accepted on a single LLM call's word. The initialization workflow deserved the same treatment: replay drafts against the user's historical sent mail and validate the composed context before ever trusting it.
2. Cadence: In addition to the reasons above, the nightly was chosen because felt like the right starting point, not because a day is provably the right learning interval. A heavy email user generates enough sent-draft pairs to learn from by lunchtime; a light user might need a week to produce one meaningful signal. An adaptive cadence — triggered by accumulated pairs rather than the clock — probably serves both better than a fixed schedule serves either.
3. Prompts: The whole system rested on three prompts we largely hand-tuned and eyeballed. The judge is the sharpest edge: it's an LLM whose verdicts we treated as ground truth, but we never rigorously measured it against human preferences, or checked it for the known failure modes of LLM-as-judge setups like positional or length bias. Given that we'd already built GEPA and evals for prompt optimization, pointing it at these three prompts was an obvious move we never got to.