Agent Testing Guide

Sample prompts and validation checklists for testing the VS Code custom agents.

Prerequisites

Agent Configuration Reference

Current YAML frontmatter for each agent (must match .github/agents/ files):

Master Orchestrator (codewiki.agent.md)

name: CodeWiki
description: Master agent that routes your request to the right CodeWiki specialist
model: GPT-5.3-Codex
tools:
  [read, agent, codewiki-mcp/*]
agents:
  [CodeWiki Researcher, CodeWiki Code Review, CodeWiki Architecture Explorer, CodeWiki Comparison, CodeWiki Synthesizer]

Subagents

# Researcher, Code Review, Architecture Explorer, Comparison:
model: GPT-5 mini
user-invokable: false
tools:
  [read, codewiki-mcp/*]

# Synthesizer (needs stronger reasoning for multi-repo integration):
model: GPT-5.3-Codex
user-invokable: false
tools:
  [read, codewiki-mcp/*]

Routing Quick Reference

1. CodeWiki Researcher (General Exploration)

Routing trigger: General "what is", "explain", "tell me about" questions.

Sample Prompts

@codewiki What is facebook/prophet and what are its main features?

@codewiki Explain the key concepts behind pallets/flask

@codewiki What topics does CodeWiki have for microsoft/vscode?

# Bare keyword — resolves automatically (v1.2.0+)
@codewiki What is prophet and what are its main features?

Expected Behaviour

Validation Checklist

• Master does not answer from its own knowledge
• Master uses the agent tool (not direct tool calls to read/search)
• Researcher cites CodeWiki sections in its answer
• Response contains real documentation content, not generic descriptions
• Master presents the full subagent response (not a brief summary)
• If a bare keyword was used, response is prefixed with a resolution note (e.g. > Resolved: keyword "prophet" → facebook/prophet)

2. CodeWiki Code Review (Module / Function Analysis)

Routing trigger: "review", "analyse", "what does module X do", code-level questions.

Sample Prompts

@codewiki Review the forecaster module in facebook/prophet — what does it do?

@codewiki What code patterns are used in the routing module of pallets/flask?

@codewiki Analyse the error handling approach in fastapi/fastapi

Expected Behaviour

Validation Checklist

• Master delegates to CodeWiki Code Review, not Researcher
• Reviewer focuses on code structure, patterns, and implementation details
• Response references specific modules, classes, or functions
• No hallucinated code — all content sourced from CodeWiki
• Master presents the full subagent response (not a brief summary)

3. CodeWiki Architecture Explorer (System Design)

Routing trigger: "architecture", "design", "how is X structured", "component hierarchy".

Sample Prompts

@codewiki Explain the overall architecture of facebook/react

@codewiki How is the plugin system architected in vitejs/vite?

@codewiki Describe the component hierarchy and data flow in vuejs/core

Expected Behaviour

Validation Checklist

• Master delegates to CodeWiki Architecture Explorer
• Response covers high-level design (layers, components, data flow)
• Includes or references structural breakdowns from CodeWiki
• Does not devolve into code-level details (that's the Reviewer's job)
• Master presents the full subagent response (not a brief summary)

4. CodeWiki Comparison (Multi-Repo)

Routing trigger: "compare", "vs", "difference between", "X or Y".

Sample Prompts

@codewiki Compare fastapi/fastapi vs pallets/flask — architecture, performance, and developer experience

@codewiki Compare facebook/react vs vuejs/core in terms of rendering strategy

@codewiki What are the differences between expressjs/express and koajs/koa?

Expected Behaviour

Validation Checklist

• Master delegates to CodeWiki Comparison, not Researcher
• Agent fetches documentation from both repos (not just one)
• Comparison is grounded in CodeWiki content, not generic knowledge
• Response includes a structured comparison (table, bullet list, or sections)
• Master presents the full subagent response (not a brief summary)

5. Request Indexing (Unindexed Repo — Subagent Handles It)

Routing trigger: Repo that returns NOT_INDEXED from any CodeWiki tool.

New in v1.3.0: The tool now uses MCP Elicitation to confirm with the user before submitting an indexing request.

Sample Prompts

@codewiki Check if Snowflake-Labs/agent-world-model is available on CodeWiki

@codewiki What does CodeWiki have for some-org/obscure-repo?

Expected Behaviour

Validation Checklist

• Master does not call any MCP tools directly (it delegates via agent)
• A subagent detects NOT_INDEXED and calls codewiki_request_indexing
• User is asked to confirm indexing via elicitation prompt (v1.3.0+)
• User is informed the repo has been submitted for indexing
• Master presents the full subagent response (not a brief summary)

6. CodeWiki Synthesizer (Multi-Repo Solution Building)

Routing trigger: User wants to BUILD something new by combining parts from multiple repos. Distinct from Comparison which evaluates/contrasts.

Sample Prompts

@codewiki I want to build an API server that uses the routing system from pallets/flask and the async handling from fastapi/fastapi. Help me design it.

@codewiki Take the plugin architecture from vitejs/vite and the component model from vuejs/core — design a new framework that combines both.

@codewiki Combine the authentication approach from supabase/supabase with the event pipeline from apache/kafka into a real-time auth notification system.

# Intentionally vague — Synthesizer should DISCOVER which parts to take
@codewiki Can you combine the best parts from fastapi/fastapi and pallets/flask into a new web framework solution?

Expected Behaviour

Validation Checklist

• Master delegates to CodeWiki Synthesizer, not Comparison
• Synthesizer fetches documentation from all mentioned repos
• Response includes a Parts Extracted table citing source repos
• Response includes Compatibility Analysis (conflicts + resolutions)
• Response includes Integration Architecture (Mermaid diagram or description)
• Response includes Directory Structure for the new project
• Response includes Implementation Guide with actionable steps
• For vague requests: includes Parts Selected table with reasoning
• All content is grounded in CodeWiki data, not generic knowledge
• Master presents the full subagent response (not a brief summary)

7. Keyword Resolution & Disambiguation (Bare Product Names)

Routing trigger: Any prompt using a bare keyword instead of owner/repo format.

Sample Prompts

@codewiki What is vue?

@codewiki Explain the architecture of react

@codewiki Compare vue vs react

@codewiki What topics does openclaw have?

Expected Behaviour

Validation Checklist

• Bare keyword “vue” triggers elicitation with multiple options (vuejs/vue, vuejs/core, etc.)
• User can select vuejs/core (Vue 3) instead of auto-picking vuejs/vue (Vue 2)
• Bare keyword “openclaw” auto-resolves to openclaw/openclaw (canonical match, NO elicitation)
• Bare keyword “react” triggers elicitation showing facebook/react and alternatives
• Resolution note appears at the top of the response with star count
• Alternative candidates are listed
• Declining/cancelling elicitation falls back to heuristic selection
• owner/repo format still works as before (no resolution note, no elicitation)
• Full URLs still work as before (no resolution note, no elicitation)

Full Workflow Scenario

This single prompt is designed to trigger all 5 workflow steps (Discover → Navigate → Read → Search → Synthesize) of the Researcher subagent.

@codewiki I want a deep technical explanation of the React Compiler's
compilation pipeline.

Specifically:
1. First, check what topics Google CodeWiki has for facebook/react.
2. Then look at the full table of contents to find sections about
   the compiler.
3. Read the section on "React Compiler Internals" to understand the
   multi-stage compilation pipeline, the IRs (HIR and ReactiveFunction),
   and the key optimization passes.
4. Search for "How does the React Compiler handle memoization and
   reactive scopes?" to get implementation-level details.
5. Combine everything into a single technical summary that covers:
   - The overall compilation pipeline (AST → HIR → ReactiveFunction → codegen)
   - The key intermediate representations and their purpose
   - How reactive scopes are inferred and merged
   - How the compiler replaces manual useMemo/useCallback
   - Cite which CodeWiki sections your answer comes from

Expected Tool Calls

Full Workflow Validation

• Step 1: Returns status: "ok" with topic list (expect ~26 sections)
• Step 2: Returns status: "ok" with hierarchical section structure
• Step 3: Returns status: "ok" with detailed content about HIR, ReactiveFunction, compilation passes
• Step 4: Returns status: "ok" OR RETRY_EXHAUSTED (upstream timeout is a known CodeWiki issue)
• Step 5: Agent produces a coherent summary citing specific CodeWiki sections
• All responses include content_hash and idempotency_key
• Subsequent identical calls return in <10ms (cache hit)
• Agent does NOT call the same tool >2 times for the same repo

Alternative Repos for Testing

If a repo is too large or slow during testing, try these. You can use bare keywords (v1.2.0+) — the server resolves them automatically:

See also: Agentic AI Guide — full agent definitions, architecture, and lessons learned.