Build Your Knowledge Agent with AG2

​

What You’ll Build

• An AG2 (AutoGen 2) agent that behaves like a documentation expert.
• Triggered only when invoked (e.g., @knowledge).
• Retrieves & cites snippets from an on-disk knowledge store.
• Integrated into CometChat conversations with streaming responses.

​

Prerequisites

• Python 3.10+ and pip.
• OPENAI_API_KEY in a .env file (used for embeddings + chat completions).
• Optional overrides:
  • KNOWLEDGE_AGENT_MODEL (default gpt-4o-mini)
  • KNOWLEDGE_AGENT_EMBEDDING_MODEL (default text-embedding-3-small)
  • KNOWLEDGE_AGENT_TOP_K, KNOWLEDGE_AGENT_TEMPERATURE, etc.
• CometChat app credentials (App ID, Region, Auth Key).

​

Quick links

• Repo (examples): ai-agent-ag2-examples/ag2-knowledge-agent
• Agent source: agent.py
• FastAPI server: server.py
• Sample web embed: web/index.html

​

How it works

• Ingests sources into namespaces using the KnowledgeStore. POST payloads to /tools/ingest with URLs, local files, or raw text. Files (.pdf, .md, .txt) are parsed and chunked before embeddings are stored under knowledge/<namespace>/index.json.
• Searches namespaces via /tools/search, ranking chunks with cosine similarity on OpenAI embeddings (defaults to top 4 results).
• Streams answers from /agent. The server wraps AG2’s ConversableAgent, emits SSE events for tool calls (tool_call_start, tool_call_result) and text chunks, and always terminates with [DONE].
• Cites sources. The agent compiles the retrieved snippets, instructs the LLM to cite them with bracketed numbers, and appends a Sources: line.
• Hardens operations: namespaces are sanitized, ingestion is thread-safe, and errors are surfaced as SSE error events so the client can render fallbacks.

• agent.py — KnowledgeAgent, store helpers, embedding + retrieval logic.
• server.py — FastAPI app hosting /agent, /tools/ingest, /tools/search, /health.
• knowledge/ — default storage root (persisted chunks + index).
• web/index.html — CometChat Chat Embed sample that points to your deployed agent.

​

Setup

​

Project structure

• Dependencies: requirements.txt
• Agent logic: agent.py
• Knowledge store: knowledge/
• API server: server.py
• Web sample: web/index.html

​

Core configuration

OPENAI_API_KEY=sk-your-key
KNOWLEDGE_AGENT_MODEL=gpt-4o
KNOWLEDGE_AGENT_TEMPERATURE=0.2
KNOWLEDGE_AGENT_TOP_K=4
KNOWLEDGE_AGENT_EMBEDDING_MODEL=text-embedding-3-small
KNOWLEDGE_AGENT_DEFAULT_NAMESPACE=docs
KNOWLEDGE_AGENT_STORAGE_PATH=knowledge

POST /tools/ingest
{
  "namespace": "docs",
  "sources": [
    {
      "type": "url",
      "value": "https://example.com/handbook",
      "title": "Handbook"
    },
    {
      "type": "file",
      "path": "/absolute/path/to/policies.pdf"
    },
    {
      "type": "text",
      "title": "Inline FAQ",
      "value": "Q: How do refunds work?\nA: ..."
    }
  ]
}

data: {"type":"tool_call_start","tool_call_id":"call_123","tool_name":"search_docs","args":{"namespace":"docs","query":"@agent refund policy"}}

data: {"type":"tool_call_result","tool_call_id":"call_123","result":{"matches":[{"chunk_id":"...","score":0.82,"source":"knowledge/docs/...","title":"Refund Policy","preview":"..."}]},"is_frontend_tool":false}

data: {"type":"text_message","delta":"Refunds are processed within 5 business days "}
data: {"type":"text_message","delta":"once the request is approved.[1] Sources: [1] Refund Policy"}
data: [DONE]

​

Step 1 - Ingest documentation

​

Step 2 - Ask the knowledge agent

curl -N -X POST http://localhost:8000/agent \
  -H "Content-Type: application/json" \
  -d '{
    "thread_id": "demo-thread",
    "run_id": "demo-run",
    "messages": [
      { "role": "user", "content": "@knowledge What does our refund policy cover?" }
    ],
    "tool_params": { "namespace": "docs" }
  }'

​

Step 3 - Deploy the API

• Host the FastAPI app (Render, Fly.io, ECS, etc.) or wrap it in a serverless function.
• Expose /agent, /tools/ingest, and /tools/search over HTTPS.
• Keep KNOWLEDGE_AGENT_STORAGE_PATH on a persistent volume so ingested docs survive restarts.
• Harden CORS, rate limiting, and authentication before opening the endpoints to the public internet.

​

Step 4 - Configure CometChat

For more on CometChat AI Agents, see: Overview · Instructions · Custom agents

​

Step 5 - Customize in UI Kit Builder

​

Step 6 - Integrate

Widget Builder

React UI Kit

Pre Built UI Components

The AG2 Knowledge agent you connected is bundled automatically. End-users will see it immediately after deployment.

​

Step 7 - Test end-to-end

​

Security & production checklist

• Protect ingestion routes with auth (API key/JWT) and restrict who can write to each namespace.
• Validate URLs and file paths before fetching to avoid SSRF or untrusted uploads.
• Rate limit the /agent and /tools/* endpoints to prevent abuse.
• Keep OpenAI keys server-side; never ship them in browser code or UI Kit Builder exports.
• Log tool calls (namespace, duration) for debugging and monitoring.

​

Troubleshooting

• Agent replies without citations: confirm the retrieval step returns matches; check embeddings or namespace.
• Empty or slow results: ensure documents were ingested successfully and the top-k limit isn’t set to 0.
• 503 Agent not initialized: verify server logs—KnowledgeAgent requires OPENAI_API_KEY during startup.
• SSE errors in widget: make sure your reverse proxy supports streaming and disables response buffering.

​

Next steps

• Add new tools (e.g., summarize, link_to_source) by extending the AG2 agent before responding.
• Swap in enterprise vector stores by replacing KnowledgeStore with Pinecone, Qdrant, or Postgres.
• Schedule re-ingestion jobs when docs change, or watch folders for automatic refreshes.
• Layer analytics: log tool_call_result.matches for insight into content coverage.