In May 2026, we decided to open source our conglomeration of Caseworker Empowerment Tools.

With this work, Nava Labs maintains rigorous, multi-layered generative AI guardrails to ensure everything we ship meets the needs of caseworkers serving real people. We use strategies like automated schema enforcement (i.e. Zod), deterministic processing (simple function calls), observability with Open Telemetry standards, and persistent logging to ensure we can trace requests and responses from start to finish. Most critically: every tool allows for a “human in the loop” at various review gates to ensure safety and accuracy.

Our commitment to safety and technical integrity of rolling out these solutions is non-negotiable.

Why guardrails matter

Generative AI can fail convincingly. An answer can sound useful while creating or citing the wrong policy, exposing sensitive data, or taking an action the user never intended. We’ve all been there when we slightly push back against Claude or ChatGPT and receive a you are absolutely right message (see: Stochastic Parrot).

In public-benefit work, those failures could cause irreparable harm to caseworkers and the people they intend to serve. That is unacceptable.

Guardrails belong throughout the entire software development lifecycle.

How we embed AI guardrails across the software development lifecycle

Guardrails aren’t an afterthought or a quick prompt adjustment — they belong woven directly into the software development lifecycle. Here is how Nava Labs embeds safety, control, and observability from initial design to active maintenance:

1. Plan by setting hard boundaries

Before writing a single line of code, we define the system’s exact perimeter. Nava Labs does this by explicitly mapping what data the model can see. What state changes it can trigger. And which high-stakes decisions an automated tool is strictly forbidden from making.

2. Build by coding for least privilege

We don’t rely on the model to “behave” itself. We attempt to enforce this behavior in code. We do this by validating every input and output with strict type enforcement (like Zod schemas), sandboxing untrusted web content inside virtual machines, scoping retrieval strictly to vetted policy manuals and documents, and granting tools the absolute minimum access they need to operate.

3. Test by breaking things on purpose

Models fail in weird ways. So, we actively try to trip them up before caseworkers ever touch them. We can do this by running golden-set benchmarks, monitoring for model drift over time, and running aggressive security stress-tests. Specifically, targeting direct and indirect prompt injection attacks (including sneaky aria-tag injections hidden in scraped websites or documents).

4. Operate by watching the wires in real time

Shipping to production is just the start. We maintain total visibility by tracing every end-to-end request. Typically by using OpenTelemetry standards, tracking real-time failure modes, and keeping a fully tested, one-click rollback path ready if a model update acts up.

5. Review by keeping humans in the loop

At the end of the day, the machine is just a tool helping an individual. We review what is proposed by surfacing code-verified citations, confidence scores, and missing-field alerts directly to caseworkers, ensuring human navigators always have the context, time, and explicit authority to approve, correct, or shut down a workflow.

We released our Caseworker Empowerment Toolkit as open source on GitHub so that other teams can inspect, challenge, and improve these patterns with us. Feel free to dive in!

Questions to keep in mind when building with generative AI

These questions below are really for teams building with and implementing generative AI to think about throughout the entire delivery lifecycle.

• What can the model see, and what can it change?

• What happens when a retrieved document or webpage contains malicious instructions?

• Can we trace every important claim or action to its source?

• Which failures stop the workflow instead of triggering another model call?

• What tests run when the model, prompt, data, or tools change?

• Who can disable or roll back the system when something goes wrong?

• Does the human reviewer have the evidence, time, and authority to disagree?

• Etc.

Have you been doing any work on implementing LLM guardrails? We’d love to hear about it in the comments!

Government agencies and processes already work this way. Think about how a benefits application moves: from an intake caseworker to an eligibility technician to a supervisor. Or background checks: criminal records, credit, employment verification, and education verification — running concurrently then combined at the end.

The handoffs, the specialization, the escalation to a human with signature authority: that architecture already exists. Often it is written into statutes, regulations, standard operating procedures, and so on. Similar to legacy software being globbed onto over the decades.

I’ve found myself wondering about:

• what pattern(s) government employees, processes, and technologies already use

• what breaks when you swap a human with decades of experience for an AI agent

• how we effectively measure and evaluate the AI agent and outcomes

• how we combat automation bias and rubber-stamping outputs

• and so on…

Let’s explore the eight core patterns from the Google post and how to apply them in a government context:

1. Sequential Pipeline

2. Coordinator / Dispatcher

3. Parallel Fan-Out

4. Hierarchical Decomposition

5. Generator and Critic

6. Iterative Refinement

7. Human in the Loop

8. Composite Patterns

The legal constraints that change the architecture

Before getting into the AI agent patterns, I want to highlight six requirements rooted in U.S. federal law. These six requirements shape which patterns are safe where and how they can be applied. Each requirement has some architectural consequences.

1. Every handoff is a federal record. Under the Federal Records Act, agency decision-making must be preserved and auditable. In a multi-agent system, the state passed between AI agents is part of the decision record. If an eligibility agent writes a determination recommendation to session state and a downstream agent overwrites it, you have destroyed a record. This makes ADK’s output_key discipline part of compliance control.

2. Adverse determination recommendations require an explanation the claimant can appeal. A person that is denied benefits gets told why, in plain-language terms, specific enough to contest. An AI agent chain that produces a correct recommendation with no traceable reasoning is a liability — even when the answer it provides is correct. Every pattern below needs to be able to answer: can you reconstruct which AI agent concluded what, and on what evidence?

3. Plain language is a statutory requirement. The Plain Writing Act of 2010 requires covered documents to be written clearly. Any AI agent generating citizen-facing text needs a critic checking reading level.

4. Section 508 applies to generated output. If an AI agent produces a PDF, a form, or a web response, accessibility is a requirement. Generated output needs an accessibility check in the loop.

5. The Privacy Act limits what crosses AI agent boundaries. Records in a system of records have use limitations. When a coordinator routes a case to three specialist AI agents, you have potentially disclosed personally identifiable information (PII) to three contexts. AI agent boundaries are disclosure boundaries. I look at this as the Principle of Least Privilege.

6. ATO boundaries are system boundaries. An AI agent touching a FedRAMP-authorized system and an AI agent touching an unauthorized one cannot share state. The authorization boundary constrains the implementation.

Keep these six constraints in the back of your mind while reading the patterns below. They help determine which ones you can deploy and which need a human in the loop.

1. Sequential pipeline

Government processes are often sequential with defined handoffs.

Where this could fit:

• FOIA request handling: Request Intake → Classification → Records Search → Redaction → Review → Response Generation

• Permit applications: Application Parser → Code Compliance Checker → Environmental Impact Reviewer → Fee Calculator → Approval Generator

• Benefits application processing: Intake → Eligibility Recommendation → Documentation Review → Approval → Notification

ADK’s SequentialAgent runs sub-agents in order. Each writes to shared session.state under an output_key, and the next AI agent reads it.

2. Coordinator / dispatcher

A coordinator AI agent reads intent and routes to a specialist, then steps out of the way.

Where this could fit:

• 311 citizen service centers routing requests to Trash and Recycling, Permits, Tax Questions, Public Safety, Parks & Recreation

• Health system navigation routing to Benefits, Healthcare Scheduling, Prescription Management, Claims Status

• Taxpayer assistance routing to Filing Questions, Payment Plans, Audit Support, Tax Law Interpretation, Refund Status

As you can see, routing runs on LLM-driven delegation. You define a parent with sub_agents, and ADK’s AutoFlow transfers control based on each child’s description.

My gut tells me that the Privacy Act constraint mentioned above applies at the routing decision. The coordinator needs only enough information to route and no more.

3. Parallel fan-out and gather

Independent checks run at the same time, and a synthesizer combines them.

Where this could fit:

• Background checks: Criminal Records, Credit, Employment Verification, Education Verification, running concurrently

• Building permit review: Structural Safety, Fire Code, Electrical Code, Plumbing Code, Zoning Compliance, reviewed simultaneously

• Grant application screening: Eligibility, Conflict of Interest, Budget Review, Technical Merit, concurrent

ParallelAgent runs sub-agents at once, then a downstream AI agent reads all their outputs. Two things of note:

1. Having many parallel AI agents running in separate threads with a shared state will create race conditions.

2. Do not let the output consolidator flatten the findings. For example, do not remove cited fire code specifics; include why something passed or failed with grounded information.

4. Hierarchical decomposition

A parent delegates part of a task, waits for the result, and continues its own reasoning. While this may seem the same as the sequential routing example above, this one is a bit different. The hierarchical parent gets an answer back and keeps working.

Where this could fit:

• Environmental impact assessment: a project coordinator AI agent delegates to Air Quality, Water Resources, Wildlife Impact, and Cultural Resources specialists, then writes an integrated assessment that reasons across all four

• Federal procurement review: a procurement officer AI agent calls Federal Acquisition Regulation Compliance, Small Business Set-Aside Analysis, and Price Reasonableness, then makes an award recommendation informed by all three

• Policy research: a lead AI agent calls a research assistant that manages Federal Register search, case law lookup, and Government Accountability Office report retrieval, then drafts against what came back

My understanding is that you can wrap a sub-agent in AgentTool to make this work. The parent calls the sub-agent’s entire workflow as a single function call and receives the result.

5. Generator and critic

A good rule of thumb that I like to follow when creating agentic systems is to split creation from validation. A generator drafts output, a critic checks against explicit criteria. Failures route back with specific feedback until the draft passes. This is also known as a Ralph Loop or setting a Goal in Codex.

Where this could fit:

• Guidance documents: the critic validates against the Code of Federal Regulations, plain-language standards, and Section 508 accessibility requirements

• Public notices: the critic checks accuracy, required disclosures, and reading level

• Grant scoring: the critic validates against the published rubric and flags scoring inconsistent with it

This is the pattern that statutory constraints demand. The Plain Writing Act and Section 508 are “pass/fail” criteria, which is exactly what a critic is for.

Since this is essentially a where loop, you should have explicit and enumerated criteria for the critic to hit, along with escalation paths or failsafes (like max_iterations) to ensure that the loop does not run indefinitely.

6. Iterative refinement

The same loop shape aimed at improvement rather than correctness. Generator drafts, critic writes optimization notes, refiner rewrites.

Where this could fit:

• Rulemaking: the critic identifies legal ambiguity and economic impact and the refiner incorporates comments

• Congressional reports: the critic checks factual accuracy, citation quality, and balance

You should be cautious with this approach since the refiner will overwrite text on each pass. Ensure that you are tracking draft history and how the language evolved with a version/iteration key.

7. Human in the loop

This is arguably the most important pattern in this list as it relates to government services.

AI agents do the groundwork and a human authorizes anything irreversible or high-consequence.

Where this could fit:

• Benefits recommendation: it could surface other programs for individuals with a human caseworker approving before sending

• Security clearance recommendations: AI could flag patterns and a human adjudicator decides

• Regulatory recommendations: an officer approves before action

For a large share of government decisions this is not optional. Where a statute or delegation of authority assigns a determination to a named official, an AI agent can prepare the decision, but it cannot make it.

Two really important items to capture when implementing this are:

1. Who reviewed/approved the AI output

2. Whether or not they changed the AI recommendation

Also, being able to think about and combat Automation Bias in AI is critical.

8. Composite systems

I’d be remiss not to mention that no government system or process is singular. Most combine the patterns above. Take a few moments to think about a government process that you interacted with recently. This could be as mundane as renewing your passport or applying for a grant.

These flows appear simple on the surface, but have decades of processes built up behind the scenes. Both of these examples end the same way: with a human making an informed decision.

Final thoughts

For work in government and civic tech, we should be thinking about where can we implement automation to free us humans up to do more impactful work. How can we add AI agents to reason and act where mistakes are cheap and reversible?

FOIA request classification, permit intake parsing, and 311 routing could be good first targets. They’re high volume, the failure mode is a rerouted request rather than a denied benefit, and every one of them has a human already positioned to catch errors.

Adjudication is the last thing you automate — not the first — and even then the AI agent prepares the decision for a human who signs it.

Have you been exploring multi-agent architectures? Do you see opportunities or concerns?

But before we get to good, let’s talk about the current state.

The current state

There’s a lot of conversation about “AI readiness,” including what leadership, governance, and infrastructure should be in place for public-sector agencies to deploy AI tools in benefit programs like SNAP. At this stage, both federal and state legislative guidance is thin (only three states have AI laws that explicitly impact AI use in public benefit programs1), and will likely stay that way. Instead, we’ve seen many states turn to governors’ executive orders, AI task force reports, and administrative policies to add adequate controls and guardrails around how AI can be deployed, seeking to prevent harm.

Once the policies and infrastructure are in place, public benefit agencies often decide they’re ready for their first AI pilots, often starting with internal-facing tools that aim to reduce caseworker burden. Many states have begun piloting numerous AI tools including policy chatbots, case note generators, document quality verifiers, and the like — tools Nava has had a hand in delivering.

And if the state agency has the right infrastructure and capacity, or if they’re working with a trusted vendor, hopefully they’re also doing some version of the third step: AI evaluation.

With all three steps in place, the process would look something like this:

This is a good start, and agencies who’ve developed some form of this process should applaud their progress — then push a little further.

And while most states establish governance before adoption, it doesn’t always happen in a linear fashion. States who fail to govern, or who establish strict guardrails that don’t allow for safe experimentation, may see shadow AI adoption by frustrated employees looking to reduce workload burden behind the scenes.

In states who’ve established some form of generic AI governance as the first step, we’re also starting to see a pattern where AI adoption is increasing faster than the two things that should keep it accountable: evaluation and governance. AI technology is evolving faster than the other two can keep up.

Current gaps, and how states might address them

A few common problems we’ve identified across many states with this current process:

1. Evaluating technical performance but not impact

Many states are missing a critical step in the above process: AI impact evaluation. Any AI tool deployed for any reason is typically evaluated by the deployer for a certain period of time to ensure quality, safety, reliability, and other performance metrics.

But human service agencies don’t care solely about the performance of the tool — they care whether it’s making a difference in the program performance metrics they’re expected to meet: Did it reduce the SNAP Payment Error Rate? Did it decrease churn among participants subject to work requirements? Did it reduce denials for failure to return verifications? These are the questions state agency leaders are responsible for answering to their legislatures when it comes time to fund their programs.

Despite this need, we’ve found that states are often lacking in AI impact evaluations, which are the types of evaluations that actually answer those questions. This may be due to lack of data infrastructure, lack of appropriate staff and resources, or lack of understanding around how to conduct impact evaluations. Whatever the reason, this is an area we’ve seen for further growth in many states.

We believe evaluations — including impact evaluations — are a key component of AI adoption in the public sector that should be prioritized. These evaluations help ensure the tool not only works but it actually meets the outcomes set by the agency’s original goals. We believe at a minimum, all AI tools deployed in public assistance programs should be able to answer these three questions:

1. Does it reduce caseworker burden?

2. Does it improve client access?

3. Does it cause any unintended harm?

For states who are struggling to get started with impact evaluations, Nava has put together some great resources you can start with.

2. Not sharing learnings across deployments

Another common problem we’ve seen is that the different AI tools are being piloted by different vendors, across different programs, and by different agencies. What one agency learns in their user research and evaluations rarely makes it to the next, causing duplicated work and inconsistent deployments that don’t build on each other.

We believe states that want to build AI tools effectively, quickly, and consistently in their benefit programs should have a system in place to share learnings across different deployments. This ensures that the same mistakes don’t get repeated from program to program.

There are now approximately 14 states that are publishing or are required to publish inventories for either “high risk” or all AI use cases within government agencies.2 We think an AI inventory — at a minimum for “high risk” use cases — is a good start, but we think this can go even further.

First, states need to ensure that the AI use case inventory process isn’t so burdensome and restrictive that their agencies are afraid to populate it. This is a pattern we’ve seen repeated a few times across states: inventories that require detailed submission, review, and approval. They ultimately scare agencies away from disclosing certain use cases they think will take too long to approve, or won’t be approved at all. We would caution that a burdensome inventory process can be equivalent to no inventory at all (and uses significantly more resources to set up).

Second, we believe that good AI inventories don’t just include a list of the use cases, but also share their evaluation results and any lessons learned. If other agencies are using the inventory as a way to understand what exists and how they can deploy it, these results will be critical for helping them make informed decisions and ask potential vendors the right questions.

A good AI inventory might also include space for the following information:

• Agency owner of this deployment, or who to contact for more information

• User research and evaluation findings or how to access them

• Best practices from the deployment. For example:

  • “We found a good way to ask users about consent to using AI. Here’s the pattern and language we used…”

  • “We were able to safely engage users of this particular demographic by doing…”

  • “We developed a way of ensuring meaningful human review of the final output. Here’s what we did…”

• Negative outcomes or harms discovered that should be shared with others developing a similar tool

3. Missing mechanisms for policy revision

We’ve found that in states that have written AI policies, almost none have a process in place to review and revise that policy. Once published, there’s often no clear owner reviewing the AI policy to make sure it did what it was intended to do. Did the guardrails protect who they were meant to? Are there any additional ones needed? AI technology is changing rapidly. What once made sense at the time of publishing may no longer be sufficient to protect against the capabilities of newer models or agentic AI.

We believe that AI governance should be revisited regularly, especially early in a state’s experimentation with AI. While legislation is a lot harder to change, there are more flexible policy vehicles like statewide executive orders and government-wide policies. This is the level at which states can maneuver more quickly and effectively. Good AI governance will only get better over time, and this can be enhanced and expedited if it operates as a loop — where governance feeds adoption, which feeds evaluation, which feeds back to governance and adoption.

We believe forming a strong loop from the start is what will separate states that simply adopt AI from states that get better at it.

Next steps

If your state is just starting out on your journey to AI adoption, or if you have a process and you’re looking to refine it, consider how you’ll build a loop. This could look like many things, including:

1. Build up your agency’s data infrastructure to allow for better impact evaluations

2. Write evaluation criteria into your RFPs

3. Create a post-deployment review of every AI use case that helps identify best practices and negative outcomes that might warrant adjustments to policy

4. Develop an AI inventory that also includes evaluation and learning data that’s shareable at least across your agencies, if not publicly

5. Codify and share any best practices that have proven effective in your program

6. Set a regular review cadence of your AI policies

7. Designate one clear owner of an AI policy whose job is to revise based on learnings

8. Consider joining feedback channels and working groups with other states that are interested in cross-state sharing of learnings

If you’re also thinking about this topic, we’d love to hear from you about your experience. Is there anything we’re missing? Let us know in the comments, or email us at labs@navapbc.com.

Government teams want to experiment with AI. But two factors can make that challenging:

• Cost: Cloud AI inference APIs cost money, with pricing often very opaque

• Security: Extremely sensitive data can’t leave the building

Local language models can help solve both problems. No per-token bills. Nothing leaves your machine.

Local language models follow a pattern similar to the open source movement: free to use, free to tinker with, free to run on your own terms.

Local language models’ strategic advantages for public-sector AI

Beyond just data security, having sovereignty on your tech stack — including language models — allows you to not be beholden to a single frontier lab or cloud provider, with their accompanying price increases, deprecations, etc.

Especially if you’re not an expert in FedRAMP and other regulatory compliance, hosting language models locally simplifies compliance. Your data stays in your control.

Research shows that fine-tuned smaller language models outperform larger models on a series of tasks — even within government domains. This affords you flexibility to load models into memory quicker, generate responses faster, and continue to close the gap with frontier models.

I’d be remiss not to mention that there is a maintenance and update tradeoff. Fine-tuning, model versioning, and other MLOps is operational overhead if you are looking to expand beyond yourself.

What’s out there

Some tools that make this easy to experiment with local language models include Ollama, n8n, and LM Studio.

To keep it simple, we’ll use the following:

• Ollama is a way to serve language models locally, right from your own machine. Think of it as a local API server for open models.

• n8n is a no-code workflow automation layer on top. You wire together nodes instead of writing scripts.

Put these two together, and you get 100% private, self-hosted AI agents and automation workflows. No paid cloud APIs. Everything processed locally.

Note: n8n has a self-hosted-starter-kit repository, but if you are on a Mac you cannot expose your GPU to Docker (similar to vLLM). It just takes a little more setup.

What I built

Back in June of 2025, I started prototyping with local language models on a new machine: a MacBook M4 Pro with 24 GB of RAM.

Both Ollama and n8n run in Docker, so I’m not too worried about a model going rogue and taking my machine offline (e.g. 370+ GB of data; deleting production databases; and more).

Some of the first automations I built are:

1. An email drafter

2. A Slack channel summarizer

3. A local document search, using RAG

4. An offline plain-language checker

See the GitHub repo here.

Anecdotally, for small, narrow tasks — like summarization, extraction, and classification — a well-prompted local model can perform quite well.

Where it is now and what’s next

This is extremely nascent. These are quick experiments, not production tools.

A few open questions I’m still working through:

• How far local models can really go on more complex reasoning tasks with vector stores, multistep reasoning and acting, etc.

• How well a small local model performs when fine-tuned on domain-specific documents versus a frontier model

• Whether the setup holds up outside a single laptop (e.g. running on a server on-premise)

If you’re experimenting with local AI for public-sector work, I’d love to hear how it’s going. Comment below or reach us at labs@navapbc.com.

Resources:

• Self-hosted AI starter kit

• n8n.io

• ollama.com

• Sample n8n Workflows

When someone needs support for food, health insurance, veterans benefits, and other public programs, the experience is driven by documents. Someone uploads a proof of income, or medical expenses, or proof of enrollment in an educational program. Staff work with those documents to make sure they’re accurate, valid, and meet requirements.

Much of the work to improve the experience of these programs is about making that flow better: getting files uploaded correctly, making sure they’re readable, and letting people know as early as possible if they need to resubmit something, so they don’t wait weeks to hear back.

As part of my role on the Nava Labs program, I work on our Caseworker Empowerment Toolkit, which supports staff who connect people to benefits. In this toolkit, one of the questions we’re exploring is: how might document AI tools help staff classify documents, extract information, and figure out whether what someone’s shared will actually get them the benefits they’re looking for?

And as we develop this tool, I keep hitting a challenge of finding realistic documents to test with that don’t include people’s sensitive data. This is a challenge common to any project trying to improve benefit program experiences.

Why testing document workflows is hard

The main obstacle is creating or gathering documents that account for different scenarios applicants may be in. Many benefits require proof of income, which could come from salaried employment, self-employment, or the gig economy, to name a few.

To address this, teams either hand-create or find sample documents and hope they cover enough of the situations people are in. Another path is using real documents, but this requires securely handling people’s most sensitive data, so it’s not a good fit for early development and testing.

There are so many permutations of scenarios, document types, and input methods, and there’s a need to test as many as possible before anything is put in front of the public or staff. Synthetic documents add another testing method to the toolkit, one that doesn’t depend on real sensitive information.

Why now

Previously, ChatGPT and other image-generation models were unreliable. Many of us have seen examples of images where a person has six fingers, or text with typos and garbled characters. The quality was so low that the output was essentially useless for testing.

Gpt-image-2 was released this spring and now an entire document can now be generated. Not perfectly, but fairly consistently: full-page layouts, different document structures, text that refers to the same person throughout, and dates and amounts that line up well enough to support testing.

A quick comparison of models

Here’s the same prompt for a synthetic pay stub, run through the old model and the new one.

Test 1: Paystubs

gpt-image-1: Synthetic pay stub generated by gpt-image-1, showing garbled words like “Duringar” and “Gotalc” and numbers that don’t add up.

gpt-image-2: Synthetic pay stub generated by gpt-image-2, with a clean layout, itemized earnings and deductions, and year-to-date totals that reconcile (at least at first glance).

Test 2: SNAP award letters

The same jump shows up in a SNAP award letter. The gpt-image-1 version misspells “Effective” and “Caseworker” and signs off with “Sercrndy.” The gpt-image-2 version produces a full approval notice with a benefit summary, EBT card instructions, and fair hearing rights, all consistent with one made-up household.

The synthetic SNAP award letter from gpt-image-2 makes a much more internally consistent approval notice with a benefit summary and fair hearing rights.

Note: If anything, Image2 makes this document look too polished, since many real SNAP award letters are simple B&W text in a letter with an organization logo in the header

What this makes possible

This lets us start to more systematically test document experiences, rather than relying purely on ad-hoc examples from the internet or handmade documents from a few people on an internal team. If someone has income from self-employment or the gig economy, we can generate documentation from the different apps and services they use and see how our systems would handle it at a first pass. And if we hear about a scenario where experiences are poor, we can generate documents for that scenario specifically. So we get speed, we get depth of experimentation, and we can do all of that without using people’s real information.

This process also has potential applications in detecting, or training staff to detect, fake documents created with AI.

The flipside

The images are better, but they can still contain errors, and as the quality goes up, the errors become harder to notice. Fewer sixth fingers, more income amounts off by a few dollars. No synthetic document is a replacement for human-generated documents and human testing. It’s all a supplement and a way to expand the tools we have available to make getting and receiving benefits less onerous for the public and staff.

Further work

• Deeper evaluation: The images are great for coarse document classification and generally seem to follow instructions for household composition, dates, income amounts, and other details. But further human validation is needed before scaling up synthetic document creation as a testing method.

• Improving tools to support batch creation + sharing of documents

• Tailoring prompts: Continuing to improve layouts that don’t match real documents, increasing formats (phone photos vs. scans vs. handwritten notes) and quality (blurriness, stains, crumpling, etc.)

When they meet with clients, caseworkers have to make a choice over and over: pay attention to the person in front of them, or look away to type notes into a computer. Do the first, and they’re left trying to remember the conversation at the end of a long day. What the client said about their income, their rent, who lives in the household, and which documents they need to find. The alternative is spending most of the interaction with their focus on a screen instead of a person.

One client interaction can turn into a note in a case management system, required forms and other outputs. Often times different providers and funders want the same information reported in slightly different ways. Additionally, if the caseworker spoke with a client in another language, they’ll have to write an English record in their case management system, then write back to the client in their preferred language. All of this documentation burden adds stress to staff who support people who are often already in a crisis, leading to high rates of burnout and turnover.

We’ve been prototyping a tool for those interactions. With consent, the tool transcribes the conversation, pulls out the case facts, and drafts a case note for the worker to review and correct before anything gets saved. The tool does the first pass and the caseworker can correct and edit any outputs. This prototype is a way for us to explore the implications of voice AI on case note accuracy, administrative burden, privacy and consent.

What’s out there

“Ambient AI” has been maturing, with model providers like Assembly AI offering a Realtime Speech-to-Text API that can transcribe interactions and even assign speaker labels on the fly. There are many applications of models like these, especially in the healthcare space, where providers like Abridge offering documentation support for clinical providers.

Case workers we’ve talked to have been curious about the idea, with reservations about client privacy and the accuracy of the outputs. But so far, the only real application we’ve heard from case workers is them adding notes (with sensitive info manually removed) to ChatGPT to format them into the appropriate template.

Prototype walkthrough

To explore this concept, the prototype walks through five steps: set up the session, capture the conversation, review the facts, review the drafts, and finalize.

It opens by asking who’s in the meeting. The worker adds themselves and the client by name before any recording starts, and consent is confirmed here too.

Session setup. Add participants and confirm consent before the conversation begins.

During the conversation, the tool transcribes in real time. In the demo scenario, a SNAP renewal where the client speaks Spanish, the bilingual transcript shows each turn in both languages, color-coded by speaker. When the conversation ends, the transcript is ready to review before moving to fact extraction.

Live record. The bilingual transcript shows both languages side by side, speaker-labeled and timestamped.

After the conversation, the worker reviews extracted facts grouped into “Needs review” and “Accepted.” High-confidence details (address, household members, income, expenses) are pre-accepted but editable. Lower-confidence items surface at the top for explicit review.

Fact review. Lower-confidence items surface first. Every fact has Accept, Reject, and Source controls.

Every fact links back to the exact moment in the conversation it came from. The evidence dialog shows the bilingual transcript scroll alongside a usage panel: the source, confidence band, timestamp, transcript quote, and which note sections the fact was pulled into.

Evidence. The transcript quote, confidence, timestamp, and the note sections it feeds into, all in one place.

Only after the facts are reviewed does the tool draft the case note. It opens a rich text editor alongside the full transcript, so a worker can read both at once, edit freely, and then send a client-facing summary separately.

Case note. Generated from the reviewed facts, editable in place, with the full transcript alongside for reference.

Where it is now

While the value of this tool seems clear, there are many challenges to overcome on the road to responsible implementation. Accuracy is key, especially considering the nuanced situations people actually show up with:

• several family members talking in the same room,

• someone switching languages mid-conversation,

• Accents,

• Correctly recording a specific names or $ amounts, or

• a client who says one thing early on and corrects it later.

All of this has to land in the record correctly, and getting it wrong is the risk the tool is supposed to reduce. Live speaker labeling, for instance, can drift in the first few seconds of a conversation while the model figures out who’s talking.

Evaluating a voice tool is harder than evaluating a chatbot. When most of the interaction is audio rather than text, there’s no clean transcript to check the output against by default, so it’s tougher to measure whether the tool heard things right.

Look forward to our team sharing more about this focus area, including things like:

• How we approach evaluating a tool like this

• Feedback and ideas from caseworkers

• How to implement tools like this responsibility and securely

• And more!

If you do benefits casework, or build tools for people who do, we’d like to hear how this lands. Reach us at labs@navapbc.com. See more about the caseworker tools we’re prototyping, piloting and scaling at caseworker.navapbc.com.

Nava Labs is a philanthropically-funded R&D team within Nava, focused on prototyping innovations to the social safety net that government agencies would benefit from but can’t fund directly. We build and test new approaches to delivering public services, evaluate what works, and advocate for scaling proven solutions.

While we already regularly share news, case studies, insights, and toolkits on our main website, this Substack will offer something different: a look behind the scenes. Content will include:

• Posts from our new Technologist-in-Residence team on their experimental work as they prototype and test emerging approaches to public service delivery

• Updates on our program evaluation work, including data deep dives, while analysis is still in progress

• Our latest thought processes and frameworks, even as we’re still mulling them over

• Other field observations from our work

You can subscribe and get all of our posts straight to your email inbox (or notifications on the Substack mobile app), or you can simply check out posts like a normal blog.

We’re excited to engage fellow practitioners in this space, building toward the goal of improving service delivery for the public. We’d love to exchange ideas with you in the comments on our posts.

Let’s learn together!