Skip to main content

Voice Engineering Review -- Acceptance Specs

Reviewer: Voice Agent Engineer Date: 2026-03-28 Specs reviewed: starter-voice.md, starter-both.md, pro-both.md, suite-both.md, cancelled.md, interview-action-items.md Code reviewed: voice-agent-livekit/ (main.py, session.py, verticals/church/agents.py, verticals/church/config.py, verticals/church/prompts.py, verticals/church/tools.py, verticals/church/integrations/supabase_church.py, core/tools.py, safety.py)

Summary​

  • Accurate claims: 16
  • Inaccurate claims: 7 (must fix)
  • Unverifiable claims: 6 (need testing or external verification)

Inaccuracies (specs claim something the code does not support)​

1. [ALL SPECS] Agent count: Pro/Suite = 4 agents (Discipleship + Stewardship)​

Spec claim: Starter = 2 agents (Care + Coordinator). Pro and Suite = 4 agents (Care, Coordinator, Discipleship, Stewardship).

What the code actually has:

  • verticals/church/agents.py defines exactly 2 agent classes: CoordinatorAgent and CareAgent. There is no DiscipleshipAgent or StewardshipAgent class.
  • verticals/church/config.py TIER_AGENTS shows ["care", "coordinator"] for ALL tiers (starter, pro, suite).
  • verticals/church/prompts.py has a build_stewardship_prompt() function, but no corresponding agent class uses it. It is dead code.
  • No Discipleship prompt function exists at all.
  • features.yaml (the canonical source) also lists only ["Care Agent", "Coordinator Agent"] for ALL tiers.
  • The Coordinator already handles giving/stewardship topics directly (see TOPIC AWARENESS in prompts.py lines 330-374 and GIVING & STEWARDSHIP section lines 377-411).

Verdict: The voice agent has exactly 2 agents at ALL tiers. The "4 agents at Pro/Suite" claim is aspirational but not implemented. The specs must be corrected to reflect reality: 2 agents at all tiers. The dashboard may display 4 agent cards for Pro/Suite as a UI feature, but the voice agent backend only routes between Coordinator and Care. Giving/stewardship is handled by the Coordinator directly, not a separate agent.

Affected specs: starter-both (Touchpoint 28: "4 AI Agent cards" at Pro), pro-both (Touchpoints 27, 28, 29, 35, 44: "4 agents"), suite-both (Touchpoints 27, 28, 35: "4+ agents").

Fix required: Either (a) update specs to say 2 agents at all tiers (matching code and features.yaml), or (b) build Discipleship and Stewardship agent classes with handoff routing. Option (a) is recommended because the Coordinator already handles these topics well. The dashboard can still show 4 agent "personalities" as a UI concept (chatbot side) without them being separate voice agents.

2. [AGENT DEFINITION / CODE] Voice IDs mismatch with agent definition​

Agent definition claim: Carson = Cz0K1kOv9tD8l0b5Qu53, Cindy = TPjXlGQDlUeF9rIDIfaE.

What the code actually has:

  • config.py: Carson (male) = 86e30c1d-714b-4074-a1f2-1cb6b552fb49, Cindy (female) = 1242fb95-7ddd-44ac-8a05-9e8a22a6137d
  • sales/agents.py: Carson = 86e30c1d-714b-4074-a1f2-1cb6b552fb49

Verdict: The agent definition doc (at top of this review session) has stale voice IDs from an earlier Cartesia API version. The code uses the correct IDs. The specs themselves do not mention specific voice IDs, so the specs are fine -- but the agent definition's Voice IDs table needs updating.

Fix required: Update agent definition's Voice IDs table to match code.

3. [ALL VOICE SPECS] Greeting script stored as welcome_greeting​

Spec claim (Touchpoint 35): "Greeting script configuration" as a textarea in the Agents sub-tab.

What the code actually has: supabase_church.py loads welcome_greeting from church_voice_agents table. agents.py CoordinatorAgent.on_enter() uses self.church.get("welcome_greeting", "") to build the greeting. If set, it inserts "I'm an AI assistant" after the first sentence.

Verdict: The backend fully supports custom greetings via the welcome_greeting field. However, it is stored per-church (on church_voice_agents), not per-agent. The specs imply a per-agent greeting configuration. In reality, the greeting is only for the Coordinator (who answers the call). The Care Agent has a hardcoded introduction: "Hi, I'm here with you now. Take your time, what's on your heart?"

Fix required: Minor spec clarification. Change "Greeting script configuration" to "Coordinator greeting script" and note that Care Agent has a fixed introduction that is not configurable.

4. [ALL VOICE SPECS] Voice picker: "Cartesia voices" dropdown​

Spec claim (Touchpoint 35): "Voice picker dropdown (Cartesia voices)" in Training > Agents.

What the code actually has: supabase_church.py loads voice_id from church_voice_agents. main.py uses it to build the TTS descriptor. Supports values: a specific Cartesia voice UUID, "random" (picks male or female randomly), or empty (uses default). Care Agent automatically uses the opposite gender voice.

Verdict: The backend supports per-church voice selection, but the code has NO list of available voices and NO voice preview/picker logic. The voice picker is a dashboard UI feature that would need to be built in the admin dashboard. The spec is describing desired dashboard UI, which is fine -- but the spec should clarify that this is a "not yet built" UI feature that writes to church_voice_agents.voice_id.

Fix required: Add "NOT YET BUILT" annotation to voice picker references in specs. The backend field exists and works; the dashboard UI does not.

5. [ALL SPECS] Tool count claims (12 / 35 / 39) vs actual tools​

Spec claim: Starter = 12 tools, Pro = 35 tools, Suite = 39 tools.

What the code actually has:

CoordinatorAgent has 14 @function_tool methods (transfer_to_care, capture_visitor_contact, submit_prayer_request, request_callback, send_directions_link, send_giving_link, send_sms_link, register_for_event, check_availability, book_appointment, get_service_times, get_upcoming_events, get_staff_directory, end_call). CareAgent has 3 (submit_prayer_request, request_callback, end_call).

There is NO tier-based tool gating in the voice agent code. All tools are available to all tiers. The TIER_AGENTS config dict in config.py is not used anywhere in the agent initialization path -- _build_church_path() in main.py always creates a CoordinatorAgent with all tools, regardless of plan.

Verdict: The "12/35/39 tools" claim comes from features.yaml and is a product-level marketing number (including chatbot tools). The voice agent does not gate tools by tier and has approximately 14+3=17 unique tools total (with overlap). The tool count in specs is a chatbot/product-level number, not a voice-agent-specific number. This is acceptable as a product claim if the chatbot has tier-gated tools, but the voice agent side has no tier gating.

Fix required: Either (a) implement tier-based tool gating in the voice agent, or (b) clarify in specs that tool counts are across both voice and chat, and the voice agent currently provides its full tool set at all tiers. Recommendation: (b) for now, with a backlog item for (a).

6. [PROVISIONING DOC] add-voice-agent.md is stale​

add-voice-agent.md claim: References Cartesia managed endpoint, Twilio forwarding, and column names that don't match current schema.

What actually happens: Per voice-provisioning.md, new customers use Telnyx numbers with direct SIP to LiveKit Cloud. The provisioning involves: (1) Buy Telnyx number, (2) Create LiveKit SIP inbound trunk, (3) Create dispatch rule, (4) Update church_voice_agents.twilio_phone_number, (5) Optional PHONE_REGISTRY addition.

Verdict: knowledge/runbooks/voice-ops/add-voice-agent.md is completely out of date. It references Cartesia forwarding and old Twilio provisioning. The canonical provisioning runbook is now knowledge/runbooks/voice-provisioning.md.

Fix required: Either update add-voice-agent.md to match voice-provisioning.md, or mark it as superseded with a redirect.

7. [TOOLS DOC] tools.md references old SDK patterns​

tools.md claim: References @loopback_tool, ToolEnv, is_background=True, yield-based background tools.

What the code actually has: LiveKit Agents SDK uses @function_tool() decorator. Tools receive RunContext context objects. There is no background tool concept -- tools are async functions that return results. No ToolEnv or @loopback_tool exists.

Verdict: knowledge/products/voice-agent/tools.md is written for the old Cartesia LINE SDK and has not been updated for the LiveKit migration. Tool names and DB operations are still accurate, but the SDK patterns, decorators, and context objects are all wrong.

Fix required: Update tools.md for LiveKit Agents SDK patterns (@function_tool, RunContext, Agent._call_context).

Accurate (confirmed by code)​

1. [STARTER-VOICE] Provisioning is NOT instant -- up to 1 business day​

Confirmed: The provisioning flow in voice-provisioning.md requires manual steps (Telnyx number purchase, LiveKit trunk/dispatch creation, DB update, test call). This cannot be instant. The two-email flow (Email 1 immediate, Email 2 after provisioning) correctly reflects this.

2. [ALL SPECS] Transcript-only, no audio recording​

Confirmed: main.py captures transcripts via session.on("conversation_item_added") which accumulates {"role", "content"} text entries. update_call_log_end() writes the transcript as JSONB to voice_call_logs.transcript. There is no audio recording or storage. recording_enabled is loaded from the DB but is never used anywhere in the LiveKit agent code -- it is a vestigial field.

3. [ALL SPECS] Multi-tenant routing: one agent serves all churches​

Confirmed: session.py:resolve_route() maps dialed numbers to agent types. _build_church_path() in main.py loads church-specific data from Supabase and injects it into the CoordinatorAgent. ONE deployed agent (churchwiseai-voice) handles all churches. Per-church config is loaded at call time from church_voice_agents + churches + premium_churches tables.

4. [ALL SPECS] Care Agent accessible via voice even without web Care hub​

Confirmed: The voice Coordinator has a transfer_to_care tool that creates a CareAgent with warm handoff. This works independently of the dashboard Care tab visibility. The Care Agent is a voice-side concept -- it handles pastoral conversations during phone calls regardless of whether the web Care hub is enabled.

Starter Voice hides the Care tab in the dashboard but still lists Care Agent as one of 2 active agents -- this is correct. The Care Agent works on the voice side (handling prayer, grief, pastoral needs during calls) even though the web-based Care hub (subscriber management, broadcasts) is hidden.

5. [ALL SPECS] Custom greeting support​

Confirmed: supabase_church.py loads welcome_greeting from church_voice_agents. CoordinatorAgent.on_enter() uses it: if custom greeting is set, it injects "I'm an AI assistant" after the first sentence; otherwise it defaults to "Thank you for calling {church_name}. I'm an AI assistant. How can I help you today?"

6. [ALL SPECS] Opposite-gender voice for Care Agent handoff​

Confirmed: config.py:get_opposite_voice() swaps male/female voices. CareAgent.__init__() uses the coordinator's resolved voice_id to determine the opposite voice. The CareAgent sets its own TTS descriptor via the tts= parameter in super().__init__(). LiveKit switches voice when the handoff occurs.

7. [ALL SPECS] Per-church voice selection support​

Confirmed: supabase_church.py loads voice_id from church_voice_agents. main.py resolves "random" to a specific voice, builds the TTS descriptor, and passes the resolved voice_id to the agent. Churches can override with a specific Cartesia voice UUID in the DB.

8. [ALL SPECS] Prayer request, callback, visitor contact tools work​

Confirmed: verticals/church/tools.py has _submit_prayer_request(), _request_callback(), _capture_visitor_contact(), _register_for_event() -- all write to the correct tables (voice_prayer_requests, voice_callback_requests, voice_visitor_contacts). Both Coordinator and Care agents expose these as @function_tool methods.

9. [ALL SPECS] Cal.com and Planning Center integration tools exist​

Confirmed: CoordinatorAgent has check_availability, book_appointment (Cal.com) and get_service_times, get_upcoming_events, get_staff_directory (Planning Center). These are gated on credentials presence (not tier), which is correct -- if the church has configured these integrations, they work.

Confirmed: CoordinatorAgent.transfer_to_care() creates a CareAgent and returns it as a handoff. The prompt explicitly requires the Coordinator to: (1) empathize first, (2) ask permission, (3) wait for affirmative response before calling the tool. The tool docstring reinforces this.

11. [ALL SPECS] E.164 phone number normalization​

Confirmed: main.py normalizes both dialed_number and caller_phone by prepending + if missing. Telnyx may send numbers without the + prefix, and this is handled.

12. [ALL SPECS] Call classification and summary​

Confirmed: session.py:classify_call() uses Gemini 2.5 Flash to generate a post-call classification with summary, sentiment, topics, category, urgency, follow-up-needed, and suggested-assignee. This is written to voice_call_logs columns.

13. [ALL SPECS] Church data cached 5 minutes​

Confirmed: supabase_church.py uses _CHURCH_CACHE_TTL = 300 (5 minutes) with stale-while-revalidate fallback on error.

14. [ALL SPECS] Repeat caller history (privacy-gated)​

Confirmed: session.py:load_repeat_caller_history() queries last 5 calls within 90 days for the same phone+church pair. Injects summary into RAG context with explicit instruction: "Do NOT mention these unless the caller brings them up first."

15. [ALL SPECS] Safety/moderation layer​

Confirmed: safety.py defines SafeAgent base class that all agents inherit. It runs check_threat(), check_crisis(), check_abuse(), and should_filter() before the LLM sees user text. Crisis response directs to 988. Threat response ends the call. Abuse response sets a boundary.

16. [ALL SPECS] End call via room deletion​

Confirmed: Both CoordinatorAgent.end_call() and CareAgent.end_call() use job_ctx.api.room.delete_room() to properly disconnect SIP calls. They wait 4 seconds for TTS farewell to finish before deleting the room.

Unverifiable (cannot determine from code alone -- needs live testing)​

1. [CANCELLED] 30-day phone number retention on Telnyx​

Spec claim: Phone number held for 30 days after cancellation, then released.

Why unverifiable: Telnyx number management is a manual process (provisioning runbook). There is no automated system for number retention/release. Whether a number can be "reserved but inactive" on Telnyx for 30 days at ~$1/mo requires Telnyx account verification.

Recommendation: This is a business process question, not a code question. The spec claim is reasonable (Telnyx numbers cost ~$1/mo to hold), but the automation for this does not exist. It will be a manual founder/agent task.

2. [CANCELLED] What callers hear after cancellation​

Spec claim: "Callers hear standard carrier 'number not in service' message" or "forwarding reverts to church's original phone system."

Why unverifiable: What callers hear depends on: (a) whether the church set up call forwarding at the carrier level, (b) how Telnyx handles calls to a deactivated number, (c) whether the LiveKit dispatch rule is removed. Need to test with an actual disconnected Telnyx number.

Recommendation: Test by deactivating a Telnyx number and calling it. Document the actual behavior.

3. [ALL SPECS] Voice agent stops answering when subscription lapses​

Spec claim: Voice agent stops answering calls when subscription ends.

Why unverifiable: The code has call limit enforcement (calls_limit in supabase_church.py line 90-98 -- returns None if limit reached), but there is no subscription status check. The load_church_data() function does not check premium_churches.status. A lapsed subscription would need either: (a) the church_voice_agents row to be deactivated, (b) calls_limit set to 0, or (c) a new status check added to load_church_data().

Recommendation: This is a gap. Add a subscription status check to load_church_data() that returns None (blocking the call) if premium_churches.status is not active. Currently, a cancelled church's voice agent would keep answering calls indefinitely until someone manually deactivates it.

4. [ALL SPECS] Notification emails with "(Voice Call)" subject prefix​

Spec claim: Notification emails include "(Voice Call)" in subject for voice-sourced requests.

Why unverifiable from reviewed code: The notification logic is in core/notifications.py which was not fully reviewed. The tool implementations in verticals/church/tools.py do not include notification dispatch -- they only write to the database. The notification may be triggered by a Supabase webhook or a separate notification path.

Recommendation: Review core/notifications.py to verify "(Voice Call)" subject line pattern.

5. [ALL SPECS] 1-business-day provisioning timeline​

Spec claim: Voice line provisioned within 1 business day of checkout.

Why unverifiable: This is a human SLA, not a code feature. The provisioning steps are manual (search number, buy, create trunk, create dispatch rule, update DB, test). A skilled operator can do this in 15-30 minutes. Whether it happens within 1 business day depends on operational responsiveness.

Recommendation: The timeline claim is reasonable. Consider adding a Supabase trigger or webhook that alerts the founder/agent when a voice plan checkout completes so provisioning starts immediately.

6. [ALL SPECS] Two-email provisioning flow​

Spec claim: Email 1 (immediate post-checkout) + Email 2 (after provisioning with phone number + carrier-specific forwarding instructions).

Why unverifiable: The email sending logic lives in the web app (churchwiseai-web/src/) which was not reviewed. Email 2 is described as "NEW -- needs to be built" in the starter-voice spec (Touchpoint 18B). The carrier-specific forwarding instructions library also needs to be built (per interview-action-items.md item 4).

Recommendation: Email 2 is NOT YET BUILT. This should be flagged as a pre-launch requirement, not just a spec claim.

Provisioning Flow Validation​

  • Telnyx number creation: Matches spec. voice-provisioning.md Step 2 covers buying a Telnyx number with the correct connection_id.
  • SIP trunk setup: Matches spec. Step 3 creates a LiveKit SIP inbound trunk with Telnyx auth credentials.
  • LiveKit dispatch rule: Matches spec. Step 4 creates a dispatch rule routing to churchwiseai-voice agent.
  • DB update: Matches spec. Step 5 updates church_voice_agents.twilio_phone_number (column name is legacy but functional).
  • 1-business-day timeline: Realistic but depends on operational SLA. No automation exists to trigger provisioning on checkout.
  • Email 2 (line is live): NOT YET BUILT. Manual process today.

Recommendations​

Must Fix Before Launch​

  1. Correct agent count in specs. All tiers have 2 voice agents (Coordinator, Care). The "4 agents at Pro/Suite" claim must be corrected to match both the code and features.yaml. The chatbot side may show 4 agent personality cards in the dashboard, but the voice agent only routes between Coordinator and Care.

  2. Add subscription status check to load_church_data(). Currently, a cancelled church's voice agent would keep answering calls indefinitely. Add a check: if premium_churches.status is not active, return None from load_church_data() so the call falls through to the sales agent or is rejected.

  3. Build Email 2 (Voice Line Live). The personalized "your line is live" email with phone number and carrier-specific forwarding instructions is NOT YET BUILT. This is a key part of the voice onboarding experience described in every voice spec.

  4. Update add-voice-agent.md runbook. It references Cartesia managed endpoints and old Twilio provisioning. Either update it to match voice-provisioning.md or mark it as superseded.

  5. Update tools.md for LiveKit SDK. References old @loopback_tool, ToolEnv, and background tool patterns. Must reflect @function_tool, RunContext, and current code patterns.

Should Fix​

  1. Annotate voice picker as "NOT YET BUILT" in specs. The backend supports per-church voice selection via voice_id column, but there is no admin dashboard UI for browsing/previewing Cartesia voices. The specs describe this UI feature without noting it needs to be built.

  2. Annotate greeting script as Coordinator-only. The specs imply per-agent greeting configuration, but only the Coordinator uses welcome_greeting. The Care Agent has a fixed introduction.

  3. Update agent definition Voice IDs. The Voice IDs table in the agent definition doc shows old Cartesia IDs. Update to match config.py values.

Backlog​

  1. Tier-based tool gating for voice agent. Currently all tools are available at all tiers. Consider whether Starter should have fewer tools (matching the 12/35/39 marketing claim), or whether the marketing claim should be clarified as "tools across both voice and chat."

  2. Automated provisioning trigger. Add a webhook or Supabase function that fires when a voice plan checkout completes, alerting the founder/agent to start provisioning immediately. This supports the 1-business-day SLA.

  3. Test Telnyx number deactivation behavior. Verify what callers actually hear when a Telnyx number is deactivated, to validate the cancelled spec claims.

Cross-Reference: features.yaml vs Specs​

Featurefeatures.yamlSpecsMatch?
Starter agentsCare, CoordinatorCare, CoordinatorYES
Pro agentsCare, CoordinatorCare, Coordinator, Discipleship, StewardshipNO -- specs wrong
Suite agentsCare, CoordinatorCare, Coordinator, Discipleship, StewardshipNO -- specs wrong
Starter tools1212YES (marketing number)
Pro tools3535YES (marketing number)
Suite tools3939YES (marketing number)
Voice agent tool gatingNot implementedImplied by tool countsGAP

Files Reviewed​

FilePathRole
main.pyvoice-agent-livekit/main.pyEntrypoint, routing, session setup
session.pyvoice-agent-livekit/session.pyPhone registry, cache, call lifecycle
agents.pyvoice-agent-livekit/verticals/church/agents.pyCoordinatorAgent, CareAgent
config.pyvoice-agent-livekit/verticals/church/config.pyTIER_AGENTS, voice IDs
prompts.pyvoice-agent-livekit/verticals/church/prompts.pyPrompt builders
tools.pyvoice-agent-livekit/verticals/church/tools.pyTool implementations
supabase_church.pyvoice-agent-livekit/verticals/church/integrations/supabase_church.pyChurch data loading
safety.pyvoice-agent-livekit/safety.pyPre-LLM moderation
sales/agents.pyvoice-agent-livekit/verticals/sales/agents.pySales, Demo, DemoRouter agents
features.yamlknowledge/data/features.yamlCanonical feature definitions
voice-provisioning.mdknowledge/runbooks/voice-provisioning.mdCurrent provisioning runbook
add-voice-agent.mdknowledge/runbooks/voice-ops/add-voice-agent.mdStale provisioning runbook
tools.mdknowledge/products/voice-agent/tools.mdStale tools documentation