Building an AI-Native Knowledge Layer: From Markdown to Semantic Search, Knowledge Graphs, and MCP

Executive summary

Most personal websites, documentation sites, and long-form writing collections are designed for humans.

You write Markdown, a static-site generator renders HTML, search engines crawl the pages, and visitors navigate through links, tags, archives, and search.

That model still works.

But AI agents consume information differently.

An agent should not need to crawl ten pages, reverse-engineer navigation, scrape HTML, reconstruct relationships, and guess what information is authoritative every time it wants to understand a body of work.

It also should not burn context by grepping a repo, globbing Markdown trees, or stuffing large HTML pages into the prompt just to answer one focused question. Structured retrieval returns the right canonical items — IDs, titles, topics, URLs, and bounded neighbors — so the model spends tokens on reasoning instead of hunting.

I wanted something different for my own site.

I wanted me.itsecurity.network to remain a normal human-readable website while also becoming a structured knowledge system that AI clients could safely explore.

Most importantly, I wanted to preserve my authoring model:

write Markdown -> git push -> done

I did not want to create a second database that had to be maintained alongside the website.

I did not want to manually update a graph every time I published an article.

I did not want to manually generate embeddings.

I did not want an AI system modifying old articles simply to keep relationships current.

And I certainly did not want to give a public AI agent GitHub, shell, deployment, or filesystem permissions just so it could answer questions about material I had already published.

Instead, I built the system in layers.

Today the architecture provides:

• normal HTML pages for people
• traditional lexical search for exact terms
• structured JSON knowledge APIs
• canonical topic normalization
• a deterministic knowledge graph
• semantic natural-language search
• bounded related-content discovery
• an interactive graph UI
• llms.txt
• explicit agent guidance
• a production Model Context Protocol endpoint
• five bounded read-only MCP tools
• independent development and production retrieval stacks
• evaluation sets for search, relationships, and MCP tool selection

The architecture looks roughly like this:

  flowchart TB
    Author["Author"]
    Git["Private Git repository<br/>Markdown + Front Matter"]
    Hugo["Hugo Build"]
    Pages["Cloudflare Pages<br/>me.itsecurity.network"]

    HTML["Human-readable HTML"]
    Items["/api/items.json"]
    Topics["/api/topics.json"]
    Graph["/api/graph.json"]
    Catalog["/api/search-catalog.json"]
    LLMSTXT["/llms.txt"]
    Agents["/agents/"]

    SearchUI["Human Semantic Search"]
    GraphUI["Interactive Knowledge Graph"]

    Semantic["Cloudflare Semantic API Worker<br/>me-itsecurity-semantic-api-production"]
    AISearch["Cloudflare AI Search<br/>me-itsecurity-search-production"]

    MCP["Knowledge MCP Worker<br/>mcp.itsecurity.network/mcp"]

    Clients["MCP Clients / AI Agents"]

    Author --> Git
    Git --> Hugo
    Hugo --> Pages

    Pages --> HTML
    Pages --> Items
    Pages --> Topics
    Pages --> Graph
    Pages --> Catalog
    Pages --> LLMSTXT
    Pages --> Agents

    HTML --> SearchUI
    HTML --> GraphUI

    SearchUI -->|POST /api/search| Semantic
    GraphUI -->|GET /api/related| Semantic

    Semantic -->|Private AI Search binding| AISearch
    Semantic -->|search| Catalog
    Semantic -->|related| Items

    Clients -->|Streamable HTTP| MCP

    MCP -->|get_item| Catalog
    MCP -->|explore_topic| Topics
    MCP -->|explore_topic| Items
    MCP -->|get_connections| Graph

    MCP -->|Private Cloudflare service binding| Semantic

    LLMSTXT -. discovery .-> Clients
    Agents -. retrieval guidance .-> Clients

This is not simply “RAG for my blog.”

It is a deliberately separated knowledge architecture where:

Markdown       = authored source of truth
Hugo           = deterministic projection engine
JSON APIs      = machine-readable canonical knowledge
Knowledge Graph= explicit relationships
AI Search      = semantic candidate retrieval
Semantic Worker= retrieval policy and normalization
MCP            = agent capability interface
Client LLM     = reasoning and synthesis

The rest of this article explains why I separated those responsibilities, how requests actually move through the architecture, what I learned building it, and how someone else could reproduce the same pattern.

Current production architecture at a glance

Markdown stays the authored source of truth. Hugo projects deterministic knowledge APIs and a graph. Cloudflare Pages serves humans and machines. A Semantic Worker owns retrieval policy in front of AI Search. A read-only MCP Worker exposes five bounded tools. Details and request sequences are in Current production architecture.

Related-policy note: production related retrieval uses RELATED_MAX_CHUNKS=50 and accepts a neighbor only when it shares a canonical topic and scores at least 0.50.

The design principle: one source of truth, multiple interfaces

The most important architectural decision was not choosing MCP, Cloudflare, or even semantic search.

It was deciding what remains authoritative.

For me, the answer is simple:

Authored Markdown in Git remains the canonical knowledge source.

Everything else is derived.

  flowchart TD
    Markdown["Markdown + Front Matter<br/>Canonical Authored Knowledge"]

    Markdown --> Human["Human projections"]
    Markdown --> Machine["Machine projections"]

    Human --> HTML["HTML"]
    Human --> RSS["RSS"]
    Human --> SearchUI["Search UI"]
    Human --> GraphUI["Graph UI"]

    Machine --> Items["Items API"]
    Machine --> Topics["Topics API"]
    Machine --> Graph["Graph API"]
    Machine --> Catalog["Search Catalog"]

    Catalog --> Semantic["Semantic Retrieval"]
    Items --> MCP["MCP"]
    Topics --> MCP
    Graph --> MCP
    Semantic --> MCP

This means I do not maintain one knowledge base for humans and another one for AI.

I maintain content.

The platform projects that content into whatever forms different consumers need.

That distinction is important.

Once AI infrastructure becomes its own authoritative content store, synchronization becomes an architectural problem:

Did the vector store update?
Did the graph update?
Did someone edit the CMS but not Git?
Does the MCP return the old title?
Which database contains the real relationship?

By keeping authored knowledge in Git, those questions become much easier.

The zero-touch publishing contract

I explicitly designed around this workflow:

write Markdown
      |
      v
git push
      |
      v
build
      |
      v
done

Not:

write Markdown
      |
update knowledge graph
      |
assign vector metadata
      |
create relationships
      |
update MCP config
      |
update another database
      |
publish

The second approach works for a demo.

It does not scale as a publishing system.

My CI even contains a regression test that creates an ordinary temporary article without a custom knowledge: front-matter block and verifies that it is automatically represented in the generated items, canonical topics, and graph.

The desired contract is enforced as architecture, not just documented as an aspiration.

Current production architecture

Here is the current production architecture in more detail.

  flowchart TB

    subgraph Authoring["AUTHORING / SOURCE CONTROL"]
        User["Author"]
        Repo["Private Git repository<br/>Markdown + topic map"]
        MD["Markdown content<br/>Blog / Projects / About"]
        TopicMap["Canonical topic mapping<br/>data/knowledge_topics.yml"]

        User --> Repo
        MD --> Repo
        TopicMap --> Repo
    end

    subgraph Build["DETERMINISTIC BUILD PLANE"]
        Hugo["Hugo"]
        GitInfo["Git-backed Lastmod"]
        BuildValidation["Knowledge Layer CI<br/>IDs / Topics / Graph / Provenance<br/>Zero-touch publishing / Freshness"]
    end

    Repo --> Hugo
    Repo --> BuildValidation
    GitInfo --> Hugo

    subgraph Public["PUBLIC CONTENT PLANE — CLOUDFLARE PAGES"]
        Site["me.itsecurity.network"]
        Blog["/blog/"]
        Projects["/projects/"]
        About["/about/"]
        Explore["/search/"]
        KG["/knowledge/"]

        Items["/api/items.json"]
        Topics["/api/topics.json"]
        GraphAPI["/api/graph.json"]
        SearchCatalog["/api/search-catalog.json"]
        Fuse["/index.json"]

        LLMS["/llms.txt"]
        AgentGuide["/agents/"]
    end

    Hugo --> Site

    Site --> Blog
    Site --> Projects
    Site --> About
    Site --> Explore
    Site --> KG

    Site --> Items
    Site --> Topics
    Site --> GraphAPI
    Site --> SearchCatalog
    Site --> Fuse
    Site --> LLMS
    Site --> AgentGuide

    subgraph SemanticPlane["SEMANTIC RETRIEVAL PLANE"]
        Semantic["Semantic API Worker<br/>me-itsecurity-semantic-api-production"]
        SearchRoute["POST /api/search"]
        RelatedRoute["GET /api/related"]
        AIBinding["Cloudflare AI Search binding"]
        AIIndex["AI Search Instance<br/>me-itsecurity-search-production"]

        SearchRoute --> Semantic
        RelatedRoute --> Semantic
        Semantic --> AIBinding
        AIBinding --> AIIndex
        Semantic -->|search| SearchCatalog
        Semantic -->|related| Items
    end

    Explore --> SearchRoute
    KG --> RelatedRoute

    subgraph MCPPlane["AGENT INTERFACE PLANE"]
        MCPDomain["mcp.itsecurity.network/mcp"]
        MCPWorker["me-itsecurity-knowledge-mcp-production<br/>Stateless Streamable HTTP"]
        Tools["5 read-only tools<br/><br/>search_knowledge<br/>get_item<br/>find_related<br/>get_connections<br/>explore_topic"]

        MCPDomain --> MCPWorker
        MCPWorker --> Tools
    end

    subgraph AgentClients["MCP CLIENTS"]
        Claude["Claude / Claude Code"]
        Inspector["MCP Inspector"]
        Custom["Other MCP-capable agents"]
    end

    Claude --> MCPDomain
    Inspector --> MCPDomain
    Custom --> MCPDomain

    Tools -->|search_knowledge / find_related<br/>Private service binding| Semantic

    Tools -->|get_item| SearchCatalog
    Tools -->|get_connections| GraphAPI
    Tools -->|explore_topic| Topics
    Tools --> Items

    subgraph Controls["SECURITY / GOVERNANCE"]
        EdgeRL["Cloudflare edge rate limiting"]
        WorkerRL["Worker rate limiting"]
        HostOrigin["Host / Origin validation"]
        Schema["Zod schemas + bounded results"]
        Timeout["12s upstream timeout"]
        Size["512 KiB MCP upstream ceiling"]
        Headers["no-store / nosniff / no-referrer"]
        NoWrite["No GitHub / Shell / FS / Admin / Publishing"]
    end

    MCPDomain --> EdgeRL
    EdgeRL --> WorkerRL
    WorkerRL --> HostOrigin
    HostOrigin --> MCPWorker

    Tools --> Schema
    Tools --> Timeout
    Tools --> Size
    MCPWorker --> Headers
    MCPWorker --> NoWrite

There are several things in this diagram that are easy to miss but architecturally important.

The MCP Worker does not simply proxy every request to AI Search.

It has two distinct retrieval paths.

Deterministic retrieval path

These operations read canonical static data directly from the published site:

get_item
      |
      v
/api/search-catalog.json

get_connections
      |
      v
/api/graph.json

explore_topic
      |
      +--> /api/topics.json
      |
      +--> /api/items.json

No embedding model is needed.

No vector search is involved.

No LLM is involved.

Those answers come from deterministic build artifacts.

Semantic retrieval path

Only the tools that actually require meaning-based retrieval call the semantic layer:

search_knowledge
      |
      v
Semantic Worker
      |
      v
AI Search

find_related
      |
      v
Semantic Worker
      |
      v
AI Search

In production, MCP-to-semantic communication uses a private Cloudflare service binding.

The MCP does not depend on making a public HTTP round trip back through semantic.itsecurity.network.

That creates a cleaner service-to-service boundary:

Internet
   |
   v
MCP Worker
   |
   | private service binding
   v
Semantic Worker
   |
   | private AI Search binding
   v
AI Search

That is much closer to how I would design the same pattern inside an enterprise environment.

Public interfaces you can inspect yourself

Most of the architecture is publicly observable. Full curl examples and MCP samples are in Try the architecture.

Human site:      https://me.itsecurity.network/
Search UI:       https://me.itsecurity.network/search/
Knowledge graph: https://me.itsecurity.network/knowledge/
llms.txt:        https://me.itsecurity.network/llms.txt
Agent guidance:  https://me.itsecurity.network/agents/
Items API:       https://me.itsecurity.network/api/items.json
Search catalog:  https://me.itsecurity.network/api/search-catalog.json
Topics API:      https://me.itsecurity.network/api/topics.json
Graph API:       https://me.itsecurity.network/api/graph.json
Lexical index:   https://me.itsecurity.network/index.json
Semantic search: POST https://me.itsecurity.network/api/search
Related API:     GET  https://me.itsecurity.network/api/related
MCP endpoint:    https://mcp.itsecurity.network/mcp

The source repository stays private. What is public is the deployed knowledge surface above.

The architecture landed in four production milestones:

1. Knowledge Layer V1
   Deterministic Hugo projections: items, topics, graph, search catalog,
   interactive graph UI, and zero-touch publishing validation.

2. Semantic Search V1
   Controlled Cloudflare Worker in front of AI Search, with origin checks,
   schema validation, timeouts, and bounded response shapes.

3. Production Semantic Search
   Isolated production retrieval stack, separate from development,
   with accepted search and related-content benchmarks.

4. Controlled Knowledge MCP V2
   Five read-only tools over Streamable HTTP, private Worker-to-Worker
   binding into semantic search, and layered rate limiting.

Request flow 1: publishing a new article

The system starts with a normal publishing workflow.

There is no MCP-specific publishing process.

  sequenceDiagram
    autonumber

    actor Author
    participant Git as GitHub
    participant CI as Knowledge Layer CI
    participant Hugo as Hugo Build
    participant Pages as Cloudflare Pages
    participant APIs as Generated Knowledge APIs
    participant Crawl as AI Search Crawler
    participant Index as AI Search Index

    Author->>Git: Commit/push normal Markdown article

    Git->>CI: Trigger CI

    CI->>CI: Validate stable IDs
    CI->>CI: Validate topic mappings
    CI->>CI: Validate graph references
    CI->>CI: Validate edge provenance
    CI->>CI: Zero-touch publishing regression
    CI->>CI: Git-backed Lastmod regression

    Git->>Hugo: Build site

    Hugo->>Hugo: Parse Markdown/front matter
    Hugo->>Hugo: Normalize canonical topics
    Hugo->>APIs: Generate items.json
    Hugo->>APIs: Generate topics.json
    Hugo->>APIs: Generate graph.json
    Hugo->>APIs: Generate search-catalog.json

    Hugo->>Pages: Publish static build

    Pages-->>Author: Human-readable article live

    Crawl->>Pages: Crawl eligible canonical content
    Note over Crawl,Pages: Blog, projects and About<br/>main content only

    Pages-->>Crawl: Canonical page content + metadata
    Crawl->>Index: Update semantic index

    Note over Author,Index: Author only wrote Markdown.<br/>Knowledge projections and retrieval layers update downstream.

This is what I mean when I say the system is zero-touch from the authoring perspective.

The platform performs the knowledge transformation.

The author writes.

Why stable IDs matter

Once several systems refer to the same item, URLs and titles alone become fragile.

Titles change.

URLs sometimes change.

UI labels change.

A stable machine identity should not.

Conceptually:

{
  "id": "agent-skills-the-new-supply-chain-attack-vector",
  "title": "Agent Skills: The New Supply Chain Attack Vector",
  "type": "essay",
  "url": "/blog/agent-skills-the-new-supply-chain-attack-vector/",
  "topics": [
    "ai-security",
    "supply-chain-security"
  ]
}

The same identity can then be shared across:

Hugo
 |
 +--> JSON APIs
 |
 +--> Graph
 |
 +--> Search catalog
 |
 +--> Semantic metadata
 |
 +--> Related retrieval
 |
 +--> MCP
 |
 +--> Benchmarks

Without stable IDs, every integration ends up inventing another reconciliation mechanism.

Topic normalization without rewriting old content

I also did not want to force every historical article to adopt a brand-new taxonomy.

Hugo tags are useful for publishing, but they are not necessarily a good long-term knowledge ontology.

Over time you might accumulate things like:

AI
AI Security
AI-security
agent security
AI agents
Agentic AI

Treating every spelling variation as a graph node creates noisy topology.

Instead I maintain a deliberately smaller canonical topic vocabulary.

For example:

ai-security:
  label: "AI Security"

mcp:
  label: "Model Context Protocol (MCP)"

zero-trust:
  label: "Zero Trust"

Existing content metadata is normalized into those canonical topics.

The original articles do not need to be rewritten simply to satisfy the graph.

Separate authored facts from machine inference

This is one of the most important parts of the architecture.

Not every edge means the same thing.

I model three provenance planes:

deterministic
inferred
curated

Deterministic

No model judgment is required.

Example:

Docker Sandboxes
       |
       | has_topic
       v
   AI Security

This can be generated directly from known metadata.

A deterministic graph edge might conceptually be:

{
  "source": "item:docker-sandboxes...",
  "target": "topic:ai-security",
  "type": "has_topic",
  "origin": "deterministic"
}

Today’s live /api/graph.json is entirely this plane: every edge is has_topic with origin=deterministic. That is intentional for a corpus this size.

Inferred

These are machine-derived suggestions.

Example:

Agent Skills
      |
      | related_to
      |
Docker Sandboxes

That may be useful.

But the relationship was not authored by me.

The system therefore exposes it as:

origin = inferred
relationship = related_to
direction = undirected

Inferred neighbors are computed on demand by /api/related and MCP find_related. They are not written back into the static graph.

Curated

The schema and MCP get_connections tool can preserve author-created overrides when they exist — richer types such as:

supports
challenges
extends
references

That is a governance capability, not routine publishing labor. The current public graph does not yet contain curated edges; publishing a normal article still only requires Markdown plus optional topic metadata.

Similarity is not a knowledge claim

This sounds obvious, but many semantic knowledge systems blur the distinction.

A vector model saying:

cosine similarity = high

does not mean:

Article A supports Article B

It does not even necessarily mean:

Article A is meaningfully related to Article B

It means the vectors are close under a particular embedding representation.

That is candidate evidence.

Not truth.

This is why I do not generate a permanent all-to-all semantic graph.

Avoid the O(N²) semantic graph trap

For N documents, all-pairs comparison produces approximately:

N(N - 1)
---------
    2

pairs.

At 1,000 documents:

499,500

At 5,000:

12,497,500

Instead the architecture is bounded:

  flowchart LR
    Source["Selected document"]
    Vector["Vector retrieval"]
    Candidates["Bounded nearest-neighbor candidates"]
    Dedupe["Chunk/document deduplication"]
    Policy["Relationship acceptance policy"]
    Output["0..M inferred related items"]

    Source --> Vector
    Vector --> Candidates
    Candidates --> Dedupe
    Dedupe --> Policy
    Policy --> Output

The 0 is important.

A retrieval system should be allowed to conclude:

Nothing is sufficiently related.

Top K does not mean K facts.

Semantic search architecture

The semantic layer intentionally sits behind an application boundary.

  flowchart LR
    Browser["Browser / API Client"]
    Route["POST /api/search"]
    Worker["Semantic API Worker"]
    Validation["Validation / Bounds"]
    Binding["AI Search Binding"]
    Search["Cloudflare AI Search"]
    Normalize["Chunk Deduplication<br/>Canonical Mapping"]
    Catalog["search-catalog.json"]
    Response["Bounded Canonical Results"]

    Browser --> Route
    Route --> Worker
    Worker --> Validation
    Validation --> Binding
    Binding --> Search
    Search --> Normalize
    Catalog --> Normalize
    Normalize --> Response
    Response --> Browser

I deliberately did not expose Cloudflare AI Search’s built-in public endpoint directly.

A search platform is not the same thing as my application’s trust boundary.

The Worker owns policy.

Request flow 2: semantic natural-language search

Consider this request:

How should enterprises isolate AI coding agents?

The full sequence looks like this:

  sequenceDiagram
    autonumber

    actor User
    participant UI as me.itsecurity.network/search/
    participant Worker as Semantic API Worker
    participant RL as Semantic Rate Limiter
    participant AI as Cloudflare AI Search
    participant Catalog as search-catalog.json

    User->>UI: Ask natural-language question

    UI->>Worker: POST /api/search<br/>{"query":"How should enterprises isolate AI coding agents?","limit":3}

    Worker->>Worker: Validate method/content type
    Worker->>Worker: Enforce 2 KiB request ceiling
    Worker->>Worker: Validate query + result bounds

    Worker->>RL: Check request allowance

    alt Rate limit exceeded
        RL-->>Worker: Reject
        Worker-->>UI: 429 bounded response
    else Allowed
        RL-->>Worker: Continue

        Worker->>AI: Hybrid search<br/>up to 15 candidate chunks

        AI-->>Worker: Ranked chunks

        Worker->>Worker: Group chunks by knowledge ID
        Worker->>Worker: Deduplicate documents
        Worker->>Worker: Keep strongest document evidence

        Worker->>Catalog: Resolve IDs to canonical records
        Catalog-->>Worker: Title / URL / type / topics

        Worker->>Worker: Bound result count
        Worker-->>UI: Canonical document results
        UI-->>User: Display source documents
    end

    Note over Worker,AI: 12-second backend timeout
    Note over Worker,UI: no-store + nosniff

Notice what the response does not contain.

It does not return a generated answer.

It returns canonical sources.

That separation is intentional.

Why normalize vector chunks back into documents?

Embedding systems commonly index document chunks.

Users think in documents.

Suppose one article generates four highly relevant chunks:

Article A / chunk 4
Article A / chunk 2
Article B / chunk 1
Article A / chunk 7
Article C / chunk 3

Returning this directly would make search results look terrible.

So the application boundary converts:

chunks
   |
   v
knowledge ID grouping
   |
   v
best evidence retained
   |
   v
canonical catalog mapping
   |
   v
documents

The public API speaks my domain model.

Not my vector backend’s implementation model.

This abstraction later became critical for MCP too.

Current semantic production policy

The current production Worker configuration intentionally bounds retrieval.

Search currently uses:

Maximum canonical documents: 10
Maximum hybrid candidate chunks: 15
Backend timeout: 12 seconds
Worker coarse limiter: 30 requests / minute

Related retrieval is independently tuned:

RELATED_MAX_CHUNKS:            50
Maximum returned documents:    5

RELATED_MIN_VECTOR_SCORE:      0.58  (reported as `threshold`; config ceiling only)
RELATED_SHARED_TOPIC_MIN:      0.50  (actual acceptance floor)

Acceptance rule:
  shared canonical topic AND vectorScore >= 0.50

Production sets RELATED_MAX_CHUNKS=50 (the maximum the Worker accepts). An earlier misconfiguration used 60, which fell outside the validated 1..50 range and silently fell back to the default of 30. The config is now aligned with the intended candidate pool.

The acceptance rule is stricter than a generic similarity floor: a candidate is returned only when it shares a canonical topic and scores at least 0.50. RELATED_MIN_VECTOR_SCORE=0.58 is still configured and returned in the JSON as threshold, but it is not an alternate accept path for high-scoring, topic-mismatched neighbors.

The effective policy looks like this:

// Production vars
RELATED_MAX_CHUNKS = "50"
RELATED_MIN_VECTOR_SCORE = "0.58"               // reported threshold only
RELATED_SHARED_TOPIC_MIN_VECTOR_SCORE = "0.50"  // real acceptance floor

const maxChunks = configuredInteger(env.RELATED_MAX_CHUNKS, DEFAULT_MAX_CHUNKS, 1, 50);
// => 50

const mapped = mapDocumentsToCatalog(selected, catalog)
  .filter((item) => item.id !== source.id)
  .filter((item) =>
    sharesCanonicalTopic(source, item) &&
    (item.vectorScore ?? 0) >= sharedTopicThreshold  // 0.50
  )
  .slice(0, limit);

Those numbers are implementation policy for this corpus, not universal recommendations.

They came from testing this corpus.

That distinction matters.

Never copy a similarity threshold from somebody else’s RAG tutorial and assume it has semantic meaning for your data.

Request flow 3: related-content inference

Related-content retrieval follows a different path.

Suppose the selected document is:

agent-skills-the-new-supply-chain-attack-vector

The sequence is roughly:

  sequenceDiagram
    autonumber

    actor User
    participant Graph as Knowledge Graph UI
    participant Worker as Semantic API Worker
    participant Catalog as Canonical Catalog
    participant AI as AI Search Vector Retrieval

    User->>Graph: Select Agent Skills article

    Graph->>Worker: GET /api/related?id=<stable-id>&limit=5

    Worker->>Worker: Validate ID / limit
    Worker->>Catalog: Resolve source from items.json
    Catalog-->>Worker: Canonical source + topics

    Worker->>AI: Vector retrieval<br/>exclude source knowledge ID

    AI-->>Worker: Up to 50 candidate chunks

    Worker->>Worker: Deduplicate by document
    Worker->>Worker: Exclude self
    Worker->>Worker: Map candidates to canonical records

    loop Each candidate
        Worker->>Worker: Evaluate vector score

        alt shared canonical topic AND score >= 0.50
            Worker->>Worker: Accept candidate
        else
            Worker->>Worker: Reject candidate
        end
    end

    Worker-->>Graph: 0..5 inferred related documents

    Graph->>Graph: Render inferred relationship<br/>as dashed / undirected

    Note over Graph,Worker: "related_to" is an inferred suggestion,<br/>not an authored claim.

That final note is central to the architecture.

The graph should visually communicate uncertainty and provenance rather than laundering model output into an apparent fact.

The knowledge graph is not the vector index

It is useful to keep these concepts separate.

The deterministic graph represents explicit structure.

Article
  |
  +-- has_topic --> AI Security
  |
  +-- has_topic --> MCP

Semantic retrieval represents similarity.

Article
  |
  +-- related_to? --> another article

They answer different questions.

Graph

What explicit structure exists?

Semantic retrieval

What content appears similar in meaning?

A future Graph RAG system may combine them.

That does not mean they should be collapsed into the same datastore or semantics today.

Why MCP came last

MCP was not the foundation.

It was the final adapter.

By the time I implemented MCP, the site already had well-defined capabilities:

semantic search
canonical item lookup
related retrieval
graph traversal
topic exploration

The MCP tool design therefore became almost obvious.

This is a pattern I would recommend elsewhere:

Build and understand your domain capabilities first. Then expose them through MCP.

Do not use MCP as the place where your data model gets invented.

The production MCP architecture

The MCP server is a separate Cloudflare Worker:

me-itsecurity-knowledge-mcp-production

Public endpoint:

https://mcp.itsecurity.network/mcp

Transport:

Stateless Streamable HTTP

The service exposes exactly five tools:

search_knowledge
get_item
find_related
get_connections
explore_topic

And deliberately nothing else.

MCP tool routing

  flowchart TD
    Client["MCP Client"]
    MCP["Knowledge MCP"]

    Client --> MCP

    MCP --> Decision{"Requested capability"}

    Decision -->|Meaning / question| SK["search_knowledge"]
    Decision -->|Known stable ID| GI["get_item"]
    Decision -->|Semantic neighbors| FR["find_related"]
    Decision -->|Graph neighborhood| GC["get_connections"]
    Decision -->|Canonical topic| ET["explore_topic"]

    SK --> Semantic["Semantic Worker"]
    FR --> Semantic

    GI --> Catalog["search-catalog.json"]
    GC --> Graph["graph.json"]

    ET --> Topics["topics.json"]
    ET --> Items["items.json"]

    Semantic --> AI["Cloudflare AI Search"]

This design intentionally avoids overlapping tools.

Tool 1: search_knowledge

Purpose:

Search my public security writing, projects, and profile by meaning.

Example intent:

What has Sammy written about securing autonomous coding agents?

Input:

{
  "query": "securing autonomous coding agents",
  "limit": 3
}

The schema bounds:

query:
  minimum 2 characters
  maximum 300 characters

limit:
  minimum 1
  maximum 5

The result is canonical public source metadata.

It does not generate an answer.

Tool 2: get_item

Purpose:

Retrieve one known public record by stable knowledge ID.

Conceptually:

get_item(
  "agent-skills-the-new-supply-chain-attack-vector"
)

This is deterministic lookup.

No vector search is necessary.

Tool 3: find_related

Purpose:

Find a bounded semantic neighborhood around a known article or project.

Conceptually:

find_related(
  id="agent-skills-the-new-supply-chain-attack-vector",
  limit=3
)

The MCP response explicitly preserves:

relationship = related_to
direction = undirected
origin = inferred

This avoids representing semantic proximity as author intent.

Tool 4: get_connections

Purpose:

Traverse explicit graph connections.

Example:

get_connections("topic:ai-security")

The MCP reads:

https://me.itsecurity.network/api/graph.json

and returns the matching node, bounded edges, and connected nodes.

Current limit:

1–20 edges

Tool 5: explore_topic

Purpose:

Explore a canonical topic and the content associated with it.

Example:

explore_topic(
  topic="ai-security",
  limit=5
)

The MCP combines:

/api/topics.json
/api/items.json

Again, no vector query is required.

Request flow 4: MCP initialization and semantic search

Now consider an actual MCP client asking about my work.

  sequenceDiagram
    autonumber

    actor User
    participant Client as MCP Client / AI Agent
    participant Edge as Cloudflare Edge
    participant MCP as Knowledge MCP Worker
    participant Limiter as MCP Worker Limiter
    participant Semantic as Semantic API Worker
    participant AI as AI Search
    participant Catalog as Canonical Catalog

    User->>Client: "What has Sammy written about securing AI coding agents?"

    Client->>Edge: Connect to<br/>https://mcp.itsecurity.network/mcp

    Edge->>Edge: Edge abuse/rate policy

    Edge->>MCP: Streamable HTTP MCP request

    MCP->>Limiter: Check coarse per-client limit
    MCP->>MCP: Validate Host / Origin

    MCP-->>Client: MCP initialize response

    Client->>MCP: tools/list

    MCP-->>Client: search_knowledge<br/>get_item<br/>find_related<br/>get_connections<br/>explore_topic

    Client->>Client: Model selects search_knowledge

    Client->>MCP: tools/call search_knowledge<br/>query + limit

    MCP->>MCP: Zod schema validation

    MCP->>Semantic: Private Cloudflare service binding<br/>POST /api/search

    Semantic->>AI: Hybrid semantic retrieval

    AI-->>Semantic: Candidate chunks

    Semantic->>Semantic: Deduplicate / normalize

    Semantic->>Catalog: Resolve canonical records

    Catalog-->>Semantic: Published source metadata

    Semantic-->>MCP: Canonical bounded results

    MCP->>MCP: Remove unnecessary internals
    MCP->>MCP: Enforce response ceiling

    MCP-->>Client: MCP structuredContent + text

    Client->>Client: LLM synthesizes answer

    Client-->>User: Answer with canonical sources

    Note over MCP,Semantic: Private service binding<br/>No public same-zone HTTP loop

This diagram is probably the best single representation of why I built the architecture this way.

The LLM is at the edge of the system.

It is not embedded inside every infrastructure component.

Request flow 5: MCP deterministic graph exploration

If the client instead asks:

Show me the work connected to AI Security.

the model can select get_connections or explore_topic.

For graph traversal:

  sequenceDiagram
    autonumber

    actor User
    participant Agent as MCP Client / LLM
    participant MCP as Knowledge MCP Worker
    participant Site as me.itsecurity.network
    participant Graph as /api/graph.json

    User->>Agent: Show the authored graph connections for AI Security

    Agent->>MCP: tools/call get_connections<br/>id="topic:ai-security"

    MCP->>MCP: Validate graph ID
    MCP->>MCP: Enforce max edges <= 20

    MCP->>Site: GET /api/graph.json
    Site->>Graph: Serve static generated artifact
    Graph-->>MCP: Nodes + edges

    MCP->>MCP: Locate topic node
    MCP->>MCP: Select incident edges
    MCP->>MCP: Collect connected nodes

    MCP-->>Agent: Node + bounded edges + nodes

    Agent-->>User: Explain graph neighborhood

    Note over MCP,Graph: No vector search<br/>No inference<br/>No LLM required server-side

That is an important characteristic.

Not every “AI agent” operation needs AI infrastructure.

Why only five tools?

MCP servers can easily become giant remote procedure-call catalogs.

That is rarely a good idea.

Agents perform better when tools have clear semantic separation.

My tool-selection model is approximately:

Natural-language question?
        |
        v
search_knowledge

Already know the ID?
        |
        v
get_item

Looking for semantically similar work?
        |
        v
find_related

Need explicit graph structure?
        |
        v
get_connections

Want everything around a canonical topic?
        |
        v
explore_topic

I also created a versioned MCP tool-selection benchmark.

It includes positive prompts for all five tools and negative prompts for capabilities the server does not possess.

Examples of negative intent:

Publish this article.

Read my private GitHub repository.

Run a shell command.

Modify the website.

Expected result:

no applicable MCP tool

Good tool selection includes knowing when not to use a tool.

Request flow 6: tool selection as a security boundary

  sequenceDiagram
    autonumber

    actor User
    participant Model as Client LLM
    participant MCP as Knowledge MCP

    User->>Model: "Publish a new article to Sammy's site"

    Model->>Model: Inspect available tools

    Note over Model,MCP: Available:<br/>search_knowledge<br/>get_item<br/>find_related<br/>get_connections<br/>explore_topic

    Model->>Model: No write/publishing capability exists

    Model-->>User: Cannot perform publishing through this MCP

    Note over MCP: No GitHub binding<br/>No filesystem<br/>No shell<br/>No admin API<br/>No deployment capability

This is why capability minimization is more powerful than trying to prompt an agent into behaving safely.

The dangerous tool simply does not exist.

MCP as an IAM problem

I think of an MCP server more like an IAM role than a chatbot plugin.

The question is not:

What could this integration potentially do?

The question is:

What is the minimum capability required for its purpose?

The purpose of my MCP is:

Allow agents to explore intentionally public knowledge.

Therefore its capability envelope is:

READ:
  published items
  topics
  graph relationships
  semantic search results
  semantic related suggestions

WRITE:
  nothing

Specifically, the Worker has no capability for:

GitHub writes
private repository access
filesystem access
shell execution
browser automation
code execution
content publishing
site administration
deployment

If a future version needs those capabilities, I would treat that as a different security architecture.

Authentication decision: intentionally public

The production MCP currently does not require authentication.

That was a deliberate product/security decision rather than an accidental omission.

The reasoning is straightforward.

Every exposed record is already intentionally public.

The MCP is simply another bounded retrieval interface over that public corpus.

Authentication would add friction to legitimate agent discovery without protecting private information that does not exist behind the interface.

But:

public != unrestricted

The system still applies:

• Host validation
• browser Origin restrictions
• strict schemas
• bounded results
• Worker rate limiting
• Cloudflare edge rate limiting
• timeouts
• response-size ceilings
• generic upstream errors
• private service bindings
• no write-capable bindings

If the service ever exposes:

private knowledge
personalized results
cost-sensitive operations
write actions
GitHub access
administrative controls

then the authentication architecture changes immediately.

OAuth or Cloudflare Access becomes appropriate.

Security controls should match actual capabilities.

Trust-boundary view

Another useful way to understand the architecture is by trust boundary.

  flowchart LR

    subgraph Untrusted["UNTRUSTED / EXTERNAL"]
        Human["Browser"]
        Agent["MCP Client"]
    end

    subgraph Edge["CLOUDFLARE EDGE BOUNDARY"]
        WAF["Edge controls"]
        Rate["Rate limiting"]
        Host["Host / Origin checks"]
    end

    subgraph App["APPLICATION POLICY BOUNDARY"]
        MCP["MCP Worker"]
        Semantic["Semantic Worker"]
    end

    subgraph Canonical["CANONICAL PUBLIC DATA"]
        Static["Hugo static APIs"]
        Pages["Published content"]
    end

    subgraph Retrieval["DERIVED SEMANTIC DATA"]
        AI["AI Search Index"]
    end

    Human --> Edge
    Agent --> Edge

    Edge --> MCP
    Edge --> Semantic
    Edge --> Pages

    MCP --> Static
    MCP -->|private service binding| Semantic

    Semantic --> Static
    Semantic -->|AI Search binding| AI

    Pages --> Static

    AI -. "derived / rebuildable" .-> Pages

The static authored site remains the canonical public content layer.

The semantic index is derived and rebuildable.

That matters when deciding what should be trusted.

Detailed data classification by architecture plane

I mentally classify the system like this:

This prevents a generated answer from being confused with source material.

Production vs development isolation

I keep persistent development and production stacks.

  flowchart TB

    subgraph Dev["DEVELOPMENT"]
        DevMCP["mcp-dev.itsecurity.network"]
        DevMCPW["me-itsecurity-knowledge-mcp-dev"]
        DevSem["semantic-dev.itsecurity.network"]
        DevSemW["me-itsecurity-semantic-api"]
        DevAI["me-itsecurity-search-dev"]
        Preview["Cloudflare Pages branch preview"]

        DevMCP --> DevMCPW
        DevMCPW --> DevSemW
        DevSemW --> DevAI
        DevMCPW --> Preview
        DevAI --> Preview
    end

    subgraph Prod["PRODUCTION"]
        ProdMCP["mcp.itsecurity.network"]
        ProdMCPW["me-itsecurity-knowledge-mcp-production"]
        ProdSem["semantic.itsecurity.network"]
        ProdSemW["me-itsecurity-semantic-api-production"]
        ProdAI["me-itsecurity-search-production"]
        Site["me.itsecurity.network"]

        ProdMCP --> ProdMCPW
        ProdMCPW -->|Private service binding| ProdSemW
        ProdSem --> ProdSemW
        ProdSemW --> ProdAI
        ProdMCPW --> Site
        ProdAI --> Site
    end

The environments do not share:

AI Search index
Worker service
MCP hostname
content origin
rate-limit namespace

The public /llms.txt and /agents/ documentation advertise only production.

Development remains an engineering environment, not an accidental public API.

Current public MCP security controls

The production MCP Worker currently applies several layers of protection.

Route isolation

Only:

/mcp

is a valid application route.

Unknown paths return:

404

Unsupported protocol behavior is rejected rather than silently handled.

Strict IDs

Stable IDs use constrained patterns rather than arbitrary strings.

Conceptually:

[a-z0-9][a-z0-9_-]{0,127}

Graph identifiers additionally support:

item:<id>
topic:<id>

This prevents identifiers from becoming arbitrary URL fetches or injection primitives.

Result bounds

Current MCP maximums include:

search_knowledge    <= 5 results
find_related       <= 5 results
get_connections    <= 20 edges
explore_topic      <= 10 items

There is intentionally no:

limit = 100000

option.

Upstream timeout

Upstream requests are bounded to:

12 seconds

The agent does not get to hold the Worker open indefinitely because a backend becomes unhealthy.

Upstream response ceiling

The MCP rejects oversized upstream responses beyond:

512 KiB

This is different from merely limiting the number of requested results.

You need both logical and byte-level bounds.

Response hardening

Responses include protections such as:

Cache-Control: no-store
X-Content-Type-Options: nosniff
Referrer-Policy: no-referrer

Error normalization

Internal upstream errors are logged operationally.

Clients receive bounded public messages.

That avoids turning backend exception handling into an information disclosure channel.

Layered rate limiting

The MCP has a Worker-level coarse limiter.

Configured application rate:

20 requests / 60 seconds

Development and production use separate Worker limiter namespaces so traffic accounting does not cross environments.

Testing showed an important practical detail: Cloudflare’s Worker rate-limit binding is designed as a permissive, eventually consistent control rather than exact global accounting.

So I did not treat that as the only control.

The production architecture adds an edge rule as well.

  flowchart LR
    Client["Client IP"]
    Edge["Cloudflare zone rate rule"]
    WorkerLimit["Worker rate limiter"]
    MCP["MCP handler"]
    Tool["Tool execution"]

    Client --> Edge
    Edge --> WorkerLimit
    WorkerLimit --> MCP
    MCP --> Tool

During testing, the edge control was able to enforce the expected blocking behavior under burst traffic.

This is a good example of a general lesson:

Configuration is not evidence. Test the actual security control.

Retrieval evaluation is part of architecture

Semantic retrieval is probabilistic.

That does not mean its quality should be subjective.

I version retrieval benchmarks alongside the site and treat them as part of the architecture, not as optional QA notes.

The recorded V2 production semantic acceptance run on August 14, 2026 (isolated production semantic hostname) included:

Semantic search: 13 / 13
Related content: 6 / 6
Latency:         717.2 ms p50, 2941.4 ms p95

Notable acceptance behaviors from that run:

Identity / background / capability queries
  -> canonical About profile returned

SolarWinds supply-chain related seed
  -> Agent Skills suggestion accepted at ≈0.521
     (required shared-topic floor 0.50; 0.58 alone is not enough)

Known weak MCP related seed
  -> zero suggestions
     (prevents the prior false-positive "always invent a neighbor" failure)

The production index policy also mattered: an earlier all-pages crawl indexed about 150 URLs and was rejected because navigation and utility pages polluted retrieval. The accepted production index only admits blog posts, project pages, and the About profile.

I also created explicit negative controls.

That is especially important for related-content inference.

A retrieval system that always finds something is not necessarily a good system.

Sometimes the correct output is:

{
  "results": []
}

MCP tool-selection evaluation

The MCP layer has its own benchmark.

It contains:

2 positive prompts per tool
+
negative/no-tool prompts

covering:

search_knowledge
get_item
find_related
get_connections
explore_topic

publishing
private repository access
shell execution
unrelated requests

The server contract mapped the acceptance set unambiguously to the intended tool—or to no tool.

Individual client models still need their own evaluation because tool selection ultimately happens client-side.

That distinction matters:

Server:
defines capabilities.

Client model:
decides when to use them.

Independent MCP client validation

I did not consider raw handcrafted JSON-RPC calls sufficient evidence of compatibility.

The endpoint was also validated using independent clients.

Those included:

MCP SDK client
Official MCP Inspector CLI

Tests covered:

initialize
tools/list
tools/call

against production.

This is another principle I try to apply broadly:

Test protocols using real independent implementations, not just your own server tests.

llms.txt as discovery, MCP as capability

The site exposes:

https://me.itsecurity.network/llms.txt

That document tells a machine client what exists.

For example:

Items:
https://me.itsecurity.network/api/items.json

Topics:
https://me.itsecurity.network/api/topics.json

Knowledge graph:
https://me.itsecurity.network/api/graph.json

Semantic search:
https://me.itsecurity.network/api/search

Related content:
https://me.itsecurity.network/api/related

MCP:
https://mcp.itsecurity.network/mcp

And:

https://me.itsecurity.network/agents/

explains when to use each retrieval mechanism.

I think of the roles as:

llms.txt
    =
machine discovery

/agents/
    =
retrieval semantics and policy

MCP
    =
typed runtime capabilities

Those are complementary—not competing—interfaces.

Why I did not expose Cloudflare’s built-in MCP endpoint

Cloudflare AI Search can provide a convenient built-in MCP-style retrieval endpoint.

I intentionally did not make that the public architecture.

Convenience was not the only requirement.

I wanted control over:

tool names
tool descriptions
tool schemas
canonical source mapping
result limits
provenance
graph access
topic exploration
failure handling
rate limits
future compatibility

My MCP also combines multiple data planes.

AI Search alone cannot provide:

deterministic graph traversal
canonical topic exploration
static item lookup

The MCP therefore operates at the domain layer, not merely the vector-store layer.

MCP is an adapter, not the knowledge system

This distinction summarizes much of the architecture.

Wrong mental model:

MCP
 |
 +--> "the knowledge"

My model:

Authored knowledge
        |
        +--> deterministic projections
        |
        +--> semantic retrieval
        |
        +--> graph
        |
        v
       MCP
        |
        v
     agents

If I removed MCP tomorrow, the knowledge system would still exist.

If I replaced MCP with another agent protocol, the source architecture would still exist.

That is intentional loose coupling.

Failure domains and rollback

I wanted AI features to fail independently.

  flowchart TD
    Static["Static Hugo site"]
    Search["Semantic Search"]
    Graph["Knowledge Graph"]
    MCP["Knowledge MCP"]

    Static --> Search
    Static --> Graph

    Search -. optional .-> MCP
    Graph -. optional .-> MCP

    FailMCP["MCP outage"]
    FailSearch["AI Search outage"]

    FailMCP -. does not break .-> Static
    FailMCP -. does not break .-> Search
    FailMCP -. does not break .-> Graph

    FailSearch -. does not break .-> Static
    FailSearch -. deterministic graph still works .-> Graph

The MCP Worker can be disabled or rolled back without affecting:

Hugo
the website
semantic search UI
the deterministic Knowledge Graph
static APIs

Likewise, AI Search can fail without taking the authored website offline.

This is how I prefer to introduce AI infrastructure:

additive first, foundational only when there is strong reason.

Operational failure behavior

Why I do not let the MCP generate the final answer

None of the server tools is named:

ask_sammy
answer_question
generate_blog_answer

That is deliberate.

The server retrieves evidence.

The client model reasons over it.

  flowchart LR
    MCP["Knowledge MCP"]
    Evidence["Canonical evidence"]
    LLM["Client LLM"]
    Answer["Generated answer"]

    MCP --> Evidence
    Evidence --> LLM
    LLM --> Answer

This gives multiple clients freedom to use the same source material differently.

The server does not need to own the final prose.

It owns retrieval semantics.

Search vs graph vs MCP vs RAG

These terms often get collapsed together.

They solve different problems.

I find this separation more useful than calling the entire thing “RAG.”

How to reproduce this architecture

You do not need Hugo.

You do not need Cloudflare.

The pattern is portable.

Phase 1 — establish canonical knowledge

Start with one source of truth:

Markdown
CMS
documentation repository
database

Every public item should have:

stable ID
title
canonical URL
content type
summary
topics

Phase 2 — create deterministic machine projections

Generate endpoints equivalent to:

/items
/topics
/graph
/search-catalog

Do this without any LLM.

Make the deterministic layer useful on its own.

Phase 3 — build a controlled taxonomy

Do not automatically turn every arbitrary tag into a permanent ontology node.

Normalize tags into a smaller controlled topic set.

Phase 4 — add explicit graph relationships

Start with high-confidence deterministic relationships such as:

has_topic

That alone makes the graph useful. Richer curated types (references, extends, supports) can come later when a human actually needs to assert them. Every edge should include provenance from day one.

Phase 5 — introduce semantic search

Index only content that should actually be discoverable.

Do not blindly crawl:

navigation
archives
pagination
utility pages
tag indexes

Restrict indexing to canonical knowledge objects.

Phase 6 — place an application boundary in front of retrieval

Your API should own:

validation
limits
normalization
timeouts
errors
rate limits
canonical mapping
logging policy

Do not let the vector backend become your public contract.

Phase 7 — build retrieval benchmarks

Create questions with expected documents.

Include:

semantic cases
exact cases
negative cases

Version them in Git.

Phase 8 — add related retrieval

Use vector search to generate candidates.

Do not automatically make every candidate a graph edge.

Support:

0 results

Phase 9 — expose a small MCP surface

Only after the domain capabilities are stable.

For a knowledge system, five tools similar to mine are enough for a surprisingly large range of tasks:

search
lookup
related
connections
topic exploration

Phase 10 — test what the MCP cannot do

Negative tests are part of the security design.

Try:

write content
run code
fetch arbitrary URL
access private data
change configuration

The expected result should not depend solely on the LLM refusing.

The capability should be absent.

What I deliberately did not add

Architecture earns complexity.

I did not automatically add:

Neo4j
Postgres
Redis
Pinecone
D1
a dedicated graph database
a separate headless CMS
a permanent AI relationship database
an author approval queue

Could some of these become useful later?

Of course.

But the current system does not need them.

Static JSON is perfectly capable of representing a modest deterministic knowledge graph.

Cloudflare AI Search handles semantic candidate retrieval.

Workers enforce policy.

Adding a graph database simply because the architecture contains a graph would increase operational complexity without solving a demonstrated problem.

Where this can go next

The next corpus expansion I want is my book at bash.itsecurity.network.

Today the knowledge layer covers the site’s essays, projects, and profile. Bringing the book in as a first-class source would let semantic search, related retrieval, topic exploration, and MCP tools cite chapters and passages the same way they already cite blog posts — without asking an agent to scrape the whole book site or load large HTML into context.

That fits the existing model: authored chapters remain the source of truth; Hugo or an equivalent build projects canonical items and topics; AI Search indexes only the book surfaces that should be discoverable; MCP stays a bounded read-only interface over the combined corpus.

The current inferred relationship is deliberately conservative:

related_to

The interesting future work is richer directional reasoning.

Potential relationship types include:

supports
challenges
extends
contradicts
evolves_from
references

But these cannot safely be inferred from embedding similarity.

A future bounded inference pipeline could look like:

  flowchart LR
    Source["Changed / selected document"]
    NN["Vector nearest neighbors"]
    Candidates["Small candidate set"]
    Classifier["Relationship classifier"]
    Evidence["Passage-level evidence validation"]
    Schema["Strict relationship schema"]
    Output["0..M accepted edges"]

    Source --> NN
    NN --> Candidates
    Candidates --> Classifier
    Classifier --> Evidence
    Evidence --> Schema
    Schema --> Output

A relationship might eventually carry:

{
  "source": "item-a",
  "target": "item-b",
  "type": "extends",
  "origin": "inferred",
  "confidence": 0.91,
  "evidence": [
    {
      "source_excerpt": "...",
      "target_excerpt": "..."
    }
  ],
  "model": "model-id",
  "policy_version": "relationship-policy-v2"
}

The key word is evidence.

Vector similarity alone is not enough for directional claims.

Graph RAG without runaway traversal

A future Graph RAG layer could combine semantic seeds with explicit graph traversal.

  flowchart TD
    Q["Question"]
    Semantic["Semantic seed retrieval"]
    Seeds["Top canonical seed documents"]
    Graph["Trusted graph expansion"]
    Bound["Depth + node caps"]
    Rank["Evidence ranking"]
    Context["Bounded context"]
    LLM["Client model"]

    Q --> Semantic
    Semantic --> Seeds
    Seeds --> Graph
    Graph --> Bound
    Bound --> Rank
    Rank --> Context
    Context --> LLM

But graph traversal would need:

strict depth cap
strict node cap
origin-aware edge policies
confidence requirements
cycle handling
stale inference invalidation

Otherwise a weak inferred edge can recursively amplify into a large amount of weak context.

Lessons from building it

1. Start below MCP

The quality of an MCP server depends heavily on the quality of the domain underneath it.

MCP is not a substitute for information architecture.

2. Keep authorship boring

Publishing should remain:

write
commit
push

Infrastructure complexity should not leak into routine authoring.

3. Stable identity is foundational

Search, graph, semantic retrieval, MCP, and evaluation all become easier when they share one stable ID system.

4. Separate facts from inference

A deterministic has_topic edge and a vector-derived related_to suggestion are fundamentally different objects.

Model them differently.

5. Similarity is candidate generation

Embeddings are excellent for finding possibilities.

Do not mistake them for semantic truth.

6. Normalize at the application boundary

Never make your public API or MCP contract mirror every field returned by the underlying vendor.

Expose your domain.

7. A small tool surface is a feature

Five tools with clear meaning are better than thirty overlapping RPC methods.

8. Capability absence beats prompting

My public knowledge MCP cannot deploy the site because it has no deployment capability.

That is much stronger than telling the LLM:

Please don’t deploy anything.

9. Test negative behavior

Security testing should include:

What happens when the correct result is nothing?
What happens when the model selects the wrong capability?
What happens under burst traffic?
What happens when an upstream fails?
What happens when the backend returns too much data?

10. AI should remain replaceable

Cloudflare AI Search is an implementation dependency.

It should not become my domain model.

Because the semantic Worker normalizes results, I could replace the backend later without redesigning the MCP protocol.

The larger idea: the web is becoming dual-interface

Historically a website exposed something roughly like:

HTML
CSS
JavaScript
RSS

Increasingly, technically mature sites can expose another surface:

structured knowledge
semantic retrieval
agent guidance
typed capabilities

I do not think this means replacing the human web.

I think it means deriving both interfaces from the same source.

  flowchart TD
    Source["Canonical authored knowledge"]

    Source --> Human["Human interface"]
    Source --> Machine["Machine interface"]

    Human --> HTML
    Human --> SearchUI["Search UI"]
    Human --> Visualization["Knowledge Graph UI"]

    Machine --> APIs["Structured APIs"]
    Machine --> Semantic["Semantic retrieval"]
    Machine --> Discovery["llms.txt / agents"]
    Machine --> MCP["MCP tools"]

That is a much more interesting architecture than putting a chat bubble in the corner of a website.

Try the architecture

You can inspect the public layers directly.

Human site

https://me.itsecurity.network/

Explore/search

https://me.itsecurity.network/search/

Knowledge graph

https://me.itsecurity.network/knowledge/

Machine discovery

https://me.itsecurity.network/llms.txt

Agent guidance

https://me.itsecurity.network/agents/

Items

https://me.itsecurity.network/api/items.json

Topics

https://me.itsecurity.network/api/topics.json

Graph

https://me.itsecurity.network/api/graph.json

Search catalog

https://me.itsecurity.network/api/search-catalog.json

Example canonical item shape

{
  "id": "docker-sandboxes-enterprise-security-for-ai-coding-agents",
  "type": "essay",
  "title": "Docker Sandboxes: Why Your AI Coding Agents Should Never Run on Bare Metal",
  "topics": ["ai-security", "coding-agents", "zero-trust"],
  "url": "https://me.itsecurity.network/blog/docker-sandboxes-enterprise-security-for-ai-coding-agents/"
}

Semantic search example

curl -X POST \
  https://me.itsecurity.network/api/search \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "How should enterprises isolate AI coding agents?",
    "limit": 3
  }'

Related-content example

curl \
  'https://me.itsecurity.network/api/related?id=agent-skills-the-new-supply-chain-attack-vector&limit=5'

MCP endpoint

Streamable HTTP at https://mcp.itsecurity.network/mcp (browser GET returns 405 by design).

Initialize:

curl -X POST https://mcp.itsecurity.network/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-03-26",
      "capabilities": {},
      "clientInfo": { "name": "blog-reader", "version": "1.0" }
    }
  }'

List tools:

curl -X POST https://mcp.itsecurity.network/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}'

That returns exactly five tools: search_knowledge, get_item, find_related, get_connections, and explore_topic.

Call get_item:

curl -X POST https://mcp.itsecurity.network/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{
    "jsonrpc": "2.0",
    "id": 3,
    "method": "tools/call",
    "params": {
      "name": "get_item",
      "arguments": { "id": "sammy-farida-profile" }
    }
  }'

Truncated structured result:

{
  "item": {
    "id": "sammy-farida-profile",
    "type": "profile",
    "title": "About Me",
    "url": "https://me.itsecurity.network/about/"
  }
}

Call get_connections for a topic neighborhood:

curl -X POST https://mcp.itsecurity.network/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{
    "jsonrpc": "2.0",
    "id": 4,
    "method": "tools/call",
    "params": {
      "name": "get_connections",
      "arguments": { "id": "topic:ai-security", "limit": 5 }
    }
  }'

Edges in that response are currently has_topic with origin=deterministic, matching the live static graph.

Why the repository stays private

The interesting parts are already exposed through the public interfaces above: the JSON knowledge APIs, semantic endpoints, knowledge graph UI, agent guidance, and MCP tools. Keeping the private Git history, CI internals, Worker config secrets, and unfinished experiments offline avoids dead GitHub links for readers while still letting agents and people inspect the live system.

Final architecture summary

The entire system can be reduced to a few principles:

Markdown is authored truth.

Hugo turns authored truth into deterministic structure.

The graph exposes explicit relationships.

AI Search finds semantic candidates.

The Semantic Worker decides what retrieval means.

MCP exposes bounded domain capabilities.

The client LLM performs reasoning.

No AI component gets to silently redefine authored truth.

Or visually:

  flowchart LR
    Markdown["Markdown<br/>AUTHORED"]
    Hugo["Hugo<br/>DETERMINISTIC"]
    Knowledge["Knowledge APIs + Graph<br/>STRUCTURED"]
    Search["AI Search<br/>INFERRED RETRIEVAL"]
    Policy["Semantic Worker<br/>POLICY"]
    MCP["MCP<br/>CAPABILITY"]
    Agent["Agent / LLM<br/>REASONING"]

    Markdown --> Hugo
    Hugo --> Knowledge
    Hugo --> Search
    Search --> Policy
    Knowledge --> MCP
    Policy --> MCP
    MCP --> Agent

That separation is the part of this project I value most.

The site is still a static website.

But it is also a public knowledge API.

A deterministic graph.

A semantic retrieval system.

And an MCP-accessible source for AI agents.

All without replacing the way I write.

That is the architecture I would build again.

Not:

Add an LLM to the website.

But:

Make the knowledge explicit, make inference distinguishable from fact, put policy around retrieval, and give agents only the capabilities they actually need.