# Vector 5 documentation

Source: https://docs.vector5.ai

# Introduction

Vector 5 is cookieless analytics that measures humans, AI crawlers, and assistant referrals — with a public collection manifest anyone can fetch.

Every privacy-first analytics tool runs in the browser. GPTBot, ClaudeBot, and PerplexityBot do not. Humans who click a ChatGPT citation often arrive with no referrer and get filed as Direct. Vector 5 was built for that gap.

It is a clean-room project — not a Plausible fork. The server and dashboard are AGPL-3.0. The tracker, middleware, MCP server, and agent skills are MIT so you can paste them into any site.

- **~1 KB** Tracker size (no cookies, no localStorage)
- **5** Actors (classified in Go, server-side)
- **6** MCP tools (read-only aggregates)
- **1** Database (Postgres. No extra warehouse.)

## Two ingest paths

The browser script counts people. The server middleware counts everything that never runs JavaScript. Both post to the same Go API, which classifies every hit with an actor and discards the IP after hashing and a country lookup.

| Path | Who it sees | Endpoint | Package |
| --- | --- | --- | --- |
| v5.js | Browsers that execute JavaScript | POST /api/event | @vector5/tracker · MIT |
| Middleware | Crawlers, agents, and optional humans | POST /api/hit | @vector5/middleware · MIT |

## Five actors

- `human` — browser visitor
- `human_via_ai` — arrived from an assistant
- `ai_crawler` — GPTBot and peers
- `ai_agent` — headless agent
- `bot` — other automation

Dashboards default to humans. The AI Traffic panel reports crawlers and assistant referrals. Agent metrics are task-oriented — not bounce rate. Read the full model in Actors.

## What you get

- A 1 KB cookieless tracker with SPA, outbound, and download events out of the box.
- Server middleware for Next.js, Express, Hono, Cloudflare Workers, and Node.
- A public collection manifest at /api/sites/{domain}/manifest — cookies, IPs, and fingerprints are all false.
- AI insights that only ever see rollups. Ollama, OpenAI, Anthropic, or Gemini.
- An MCP server so Claude, Cursor, or Codex can query the same aggregates the dashboard shows.
- Self-host with Docker Compose: Postgres, the API, and the dashboard.

## What you do not get

- Who a visitor is.
- Whether they returned after midnight UTC.
- Cross-site or cross-device identity.
- Exact address (city is optional and off by default).
- Anything about visitors who block the request entirely.

> **Hosted product.** vector5.ai is invite-only. Request access or self-host today.

## Licence

Server and dashboard: AGPL-3.0. Tracker, middleware, MCP, and skills: MIT. See Licence and ADR 0001.

- [Quick start](/quick-start) — Compose up, add a site, drop the tag.
- [Concepts](/concepts) — Actors, hashes, rollups, and manifests.
- [Install the tracker](/tracker) — One script before </head>.
- [Add middleware](/middleware) — See the crawlers JavaScript never sees.


---

# Quick start

Self-host Vector 5 with Docker Compose, add a site, and ship the tracker plus middleware in under ten minutes.

Self-host with Docker Compose. Postgres, the Go API, and the Next.js dashboard come up together. On a self-hosted instance you sign in with the operator account from .env.

## 1. Start the stack

```bash
git clone https://github.com/vector5-ai/analytics
cd analytics
cp .env.example .env
docker compose up --build

# Dashboard  http://localhost:3000
# API        http://localhost:8080
```

> **Change the defaults.** V5_ADMIN_PASSWORD and V5_JWT_SECRET in .env are placeholders. Do not ship them to a public host.

## 2. Sign in

Open http://localhost:3000/app/login. Default operator account: admin@localhost / changeme. Add a site — the domain must match what you will put in data-domain.

## 3. Add the tracker

```html
<script defer data-domain="example.com"
  src="http://localhost:8080/js/v5.js"></script>
```

The script is about 1 KB, sets no cookies, and writes nothing to localStorage. Full options live in Tracker.

## 4. Add the middleware

Crawlers never load v5.js. Install @vector5/middleware so GPTBot and friends still count.

```ts
import { vector5 } from '@vector5/middleware/next'

export const middleware = vector5({
  endpoint: 'http://localhost:8080/api/hit',
  domain: 'example.com',
})
```

Same package ships Express, Hono, Workers, and Node adapters. See Middleware.

## 5. Verify

1. **Hit a page as yourself.** Open the site. The dashboard realtime strip should show a human within a few seconds.
2. **Simulate a crawler.**  curl -A "GPTBot" https://example.com/docs. The AI Traffic panel should increment GPTBot.
3. **Fetch the manifest.** GET /api/sites/example.com/manifest should report cookies: false, ip_stored: false, and the daily-hash formula.

## Hosted instead

The hosted product at vector5.ai is invite-only. Request access. For your own cloud deploy set V5_AUTH_MODE=supabase and point the app at your Supabase project — see Supabase auth.


---

# Concepts

The vocabulary Vector 5 uses everywhere: sites, actors, visitor hashes, rollups, manifests, and insights.

If you only read one page besides the quick start, read this. Every dashboard filter, API query, and MCP tool uses the same nouns.

## Site

A site is a hostname you register in the dashboard (example.com). The tracker's data-domain and the middleware's domain must match it. Events for unknown domains are rejected with 404 unknown site.

## Event vs hit

|  | `POST /api/event` | `POST /api/hit` |
| --- | --- | --- |
| Sender | v5.js in the browser | Server middleware |
| JavaScript executed? | Yes — jsExecuted = true | No |
| Typical actor | human, human_via_ai | ai_crawler, ai_agent, bot |
| Payload | URL, referrer, UTMs, screen, props | method, path, status, UA, referer, IP |

## Actor

A deterministic label on every row. Classification happens in Go (apps/api/internal/classify) from user-agent, referrer, UTM, and whether JS ran. See Actors.

## Visitor hash

```text
visitor_hash = SHA-256(daily_salt || site_id || ip || user_agent)
```

The salt rotates at 00:00 UTC per site. After hashing, the IP is dropped. A visitor who returns tomorrow is counted as new. That is intentional. Visitor hash.

## Rollups

Raw events are partitioned by month. The dashboard and MCP never scan them for charts — they read hourly and daily rollups computed inside the API process. No extra workers. Insights and anomaly flags are generated from rollups only. ADR 0002 and ADR 0005.

## Collection manifest

Every site publishes a machine-readable JSON document at /api/sites/{domain}/manifest. It states what is collected, what is not, how uniques work, and the retention window. Anyone — including an auditor or an agent — can fetch it without auth. Manifest.

## Insights

Optional. Configure a provider per team (PATCH /api/team/llm): Ollama (llama3.2 by default), an OpenAI-compatible endpoint, Anthropic, or Gemini. The model receives period totals, crawler counts, and crawl→referral pairs — never event rows. Every derived number is labelled observed or inferred.


---

# Architecture

A Go ingest API, a Next.js dashboard, Postgres, and four MIT client packages. One compose file. No extra workers.

Vector 5 is a monorepo: apps/api (Go), apps/web (Next.js dashboard + marketing), apps/docs (this site), and MIT packages under packages/.

## Repository layout

| Path | Role | Licence |
| --- | --- | --- |
| apps/api | Go binary v5d — ingest, classify, rollup, query, auth | AGPL-3.0 |
| apps/web | Dashboard, marketing site, waitlist | AGPL-3.0 |
| apps/docs | Documentation (docs.vector5.ai) | AGPL-3.0 |
| packages/tracker | v5.js — ~1 KB browser script | MIT |
| packages/middleware | Next / Express / Hono / Workers / Node | MIT |
| packages/mcp | stdio MCP server, six tools | MIT |
| packages/skills | Ready-made agent prompts | MIT |

## What happens on a pageview

1. The browser posts { d, n, u, r, h, w, p, utm_* } to /api/event via sendBeacon.
2. The API looks up the site, loads today's salt, parses the UA, classifies the actor, looks up country, hashes the visitor, discards the IP.
3. The row is written to events. Hourly/daily rollups update in-process.
4. If the visitor was GPTBot, this step never happened — the middleware posted /api/hit instead, with jsExecuted = false.

## Storage

Postgres 17. Events and agent hits are partitioned by month. Dashboard queries hit rollups. A storage interface is left so ClickHouse can be added later; it is not required. See ADR 0002.

## Auth modes

- V5_AUTH_MODE=admin — single operator from .env. Default for self-host and local.
- V5_AUTH_MODE=supabase — hosted / multi-user. JWT from your Supabase project.

API keys (v5_live_…) are created under Settings and sent as Authorization: Bearer. They are scoped to the team and work for every read route plus MCP.


---

# Hosted vs self-host

Same software. Invite-only cloud at vector5.ai, or Docker Compose on your metal.

## Side by side

|  | Hosted (vector5.ai) | Self-host |
| --- | --- | --- |
| Access | Invite-only waitlist | Clone and compose up |
| Auth | Supabase | Admin user, or your own Supabase |
| Tracker URL | https://<your-host>/js/v5.js | http://localhost:8080/js/v5.js or your domain |
| Data residency | Our region | Yours |
| LLM | Optional, your key or ours later | Ollama on-prem, or any provider |
| Updates | We ship them | You pull |
| Licence | Same AGPL + MIT split | Same — cloud resellers must publish modifications |

## When to self-host

- You need the data to stay in your VPC.
- You want Ollama and zero third-party model calls.
- You are evaluating before requesting hosted access.

## When to use hosted

- You do not want to operate Postgres.
- You want the dashboard on a Vector 5 domain with backups handled.
- You are fine with invite-only while the product is early.

- [Docker Compose](/self-host/docker) — The supported self-host path.
- [Supabase auth](/self-host/supabase) — Multi-user cloud on your project.
- [Request access](https://vector5.ai/access) — Hosted waitlist on vector5.ai.


---

# Tracker script

A ~1 KB cookieless script. One tag before </head>. No cookies, no localStorage identifiers, no fingerprints.

v5.js is the browser half of Vector 5. It counts people who execute JavaScript. Crawlers that never run JS are the middleware's job.

## Install

```html
<script defer data-domain="example.com"
  src="https://analytics.example.com/js/v5.js"></script>
```

The API serves the script at /js/v5.js (5-minute cache). You can also vendor packages/tracker/dist/v5.js and point data-api at your ingest host.

- [Framework recipes](/tracker/install) — Next.js, WordPress, Shopify, Astro, Remix, SvelteKit, Webflow.
- [Attributes](/tracker/attributes) — data-domain, data-api, data-hash.
- [Custom events](/tracker/events) — window.v5 and data-v5-event.
- [SPAs](/tracker/spa) — pushState is already hooked.

## What it sends

| Field | JSON key | Source |
| --- | --- | --- |
| Domain | d | data-domain or location.hostname |
| Event name | n | pageview unless you pass one |
| URL | u | location.href |
| Referrer | r | document.referrer |
| Hostname | h | location.hostname |
| Screen width | w | screen.width |
| Props | p | Optional object from window.v5 |
| UTMs | utm_* | From location.search if present |

The server derives referrer host, country, device, browser, and OS, then discards the IP. The script never sends a visitor id.

## Automatic events

- Pageviews on load, pushState, replaceState, and popstate.
- Hash changes when data-hash="true".
- Outbound Link on clicks to another host.
- File Download for pdf, zip, csv, xlsx, docx, png, mp4.
- data-v5-event on any clicked element.

## Ignore yourself

```ts
localStorage.v5_ignore = '1'
```

The script reads that key once at boot and exits. It is the only localStorage access, and it is opt-in on your machine. Ignore visits.


---

# Install on any stack

Drop v5.js into Next.js, WordPress, Shopify, Astro, Remix, SvelteKit, Webflow, or a raw HTML file.

The script must load on every public page. Put it in the document head with defer. Do not load it on admin or preview routes if you want to keep editors out of the numbers — or use v5_ignore.

## HTML

```html
<script defer data-domain="example.com"
  src="https://analytics.example.com/js/v5.js"></script>
```

## Next.js

```ts
// App Router · app/layout.tsx
import Script from 'next/script'

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        {children}
        <Script
          src="https://analytics.example.com/js/v5.js"
          data-domain="example.com"
          strategy="afterInteractive"
        />
      </body>
    </html>
  )
}
```

```ts
// Pages Router · pages/_document.tsx
import { Html, Head, Main, NextScript } from 'next/document'

export default function Document() {
  return (
    <Html>
      <Head>
        <script defer data-domain="example.com"
          src="https://analytics.example.com/js/v5.js" />
      </Head>
      <body>
        <Main />
        <NextScript />
      </body>
    </Html>
  )
}
```

Pair this with @vector5/middleware/next so crawlers that never hydrate still count.

## WordPress

Add the tag in your theme's header.php before wp_head(), or inject it with a small plugin / "Insert Headers and Footers" style tool. Use the production domain, not localhost.

```html
<script defer data-domain="example.com"
  src="https://analytics.example.com/js/v5.js"></script>
<?php wp_head(); ?>
```

## Shopify

Online Store → Themes → Edit code → theme.liquid, just before </head>.

## Astro, Remix, SvelteKit, Webflow

- Astro — put the script in src/layouts/Layout.astro <head>.
- Remix — app/root.tsx inside <head> via a <script> tag.
- SvelteKit — src/app.html or +layout.svelte with <svelte:head>.
- Webflow — Project Settings → Custom Code → Head.


---

# Script attributes

data-domain, data-api, and data-hash. Everything else is inferred server-side.

| Attribute | Default | Purpose |
| --- | --- | --- |
| data-domain | location.hostname | Must match a registered site |
| data-api | {script host}/api/event | Override the ingest URL |
| data-hash | off | Count #hash changes as pageviews |

## data-domain

Always set this in production. If you omit it, www.example.com and example.com become two sites unless you registered both. Vector 5 normalizes the domain on ingest, but the dashboard site you created is the source of truth.

## data-api

If you vendor the script from your own CDN, you must set data-api or events will post to the CDN host.

```html
<script defer
  data-domain="example.com"
  data-api="https://analytics.example.com/api/event"
  src="https://cdn.example.com/v5.js"></script>
```

## data-hash

Set data-hash="true" for docs sites and older SPAs that route on the fragment. Modern pushState apps do not need it — those navigations are already hooked.


---

# Goals and custom events

Call window.v5, mark an element with data-v5-event, then create a matching goal in the dashboard.

## window.v5

```ts
// Custom event with props
window.v5('signup', { plan: 'pro' })

// A pageview you triggered yourself
window.v5('pageview')
```

window.v5 is assigned after the script boots. If you fire events from a bundle that may load first, guard with window.v5?.('signup') or call after load.

## data-v5-event

```html
<button data-v5-event="Signup">Create account</button>
<a href="/pricing.pdf" data-v5-event="Pricing PDF">Download</a>
```

Clicks on [data-v5-event] win over the automatic outbound / download detectors on the same element.

## Goals

Create a goal in the dashboard (or POST /api/sites/{id}/goals) with the same name. Conversion rates are filterable by actor, so agents never inflate them. See Dashboard → Goals.

## Props

Props are a JSON object on the event (p). Use them for plan names, variants, or file URLs. Do not put emails, user ids, or anything that could identify a person — the privacy model assumes event props are non-identifying. The API accepts at most 64 KB per request.


---

# SPAs and hash routing

pushState, replaceState, and popstate are hooked automatically. Hash routing is opt-in.

## History API

The script wraps history.pushState and history.replaceState, and listens for popstate. Next.js, Remix, SvelteKit, Vue Router, and React Router all go through those primitives, so client-side navigations count without extra code.

> **Next.js App Router.** You do not need a usePathname effect. The tracker already sees the URL change.

## Hash routing

```html
<script defer data-domain="example.com" data-hash="true"
  src="https://analytics.example.com/js/v5.js"></script>
```

## Manual pageviews

If you have a custom router that does not touch history (rare), call window.v5('pageview') after you update the URL.


---

# Ignore your own visits

A single localStorage flag on your machine. The script never writes it — you do.

## The flag

```ts
localStorage.v5_ignore = '1'
```

On boot the script reads localStorage.v5_ignore inside a try/catch and returns if it is set. That is the only localStorage access. The collection manifest still reports local_storage: false because Vector 5 never writes an identifier there.

To resume counting, delete localStorage.v5_ignore and reload.

## Staff and preview

- Do not load the script on /admin, /studio, or CMS preview iframes.
- Staging hostnames should be a separate site, or omitted from the dashboard.
- The middleware never sees this flag — it only applies to v5.js.


---

# Next.js tracker

Script in the root layout, middleware for crawlers, and optional hash tracking for docs.

## Root layout

```ts
import Script from 'next/script'

export default function RootLayout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <body>
        {children}
        <Script
          src={process.env.NEXT_PUBLIC_V5_SCRIPT!}
          data-domain={process.env.NEXT_PUBLIC_V5_DOMAIN}
          strategy="afterInteractive"
        />
      </body>
    </html>
  )
}
```

## Pair with middleware

App Router middleware runs on the edge before the document is sent. That is where GPTBot is visible. Follow Next.js middleware.


---

# WordPress

Add v5.js to the theme head. Keep wp-admin out of the numbers.

## Theme head

Insert the script in header.php or via a header-injection plugin. Use your production analytics host, not the WordPress admin URL.

## Skip wp-admin

Do not enqueue the script on is_admin(). Editors clicking around /wp-admin will otherwise look like highly engaged humans.


---

# Shopify

One tag in theme.liquid. Checkout pages are Shopify-hosted and will not run your script.

## theme.liquid

Online Store → Themes → Edit code → layout/theme.liquid, before </head>.

## Checkout

Shopify checkout is a different origin. The tracker will not run there. Track "Add to cart" and "Begin checkout" as custom events on your storefront instead, and create goals for those names.


---

# Server middleware

Crawlers and agents never load your script. @vector5/middleware inspects the user agent and forwards non-human requests to /api/hit.

If you only install v5.js, your AI Traffic panel will be empty. GPTBot does not execute JavaScript. The middleware is the other half of Vector 5.

## Why it exists

Client-side analytics is blind to training crawlers, retrieval bots, and most headless agents. @vector5/middleware runs inside your server, checks the UA, and fire-and-forgets a hit. It never awaits the request and never blocks your response.

## Install

```bash
pnpm add @vector5/middleware
```

```ts
// Next.js · middleware.ts
import { vector5 } from '@vector5/middleware/next'

export const middleware = vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
})
```

```ts
// Express · server.ts
import express from 'express'
import { express as vector5 } from '@vector5/middleware'

const app = express()
app.use(vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
}))
```

```ts
// Hono · app.ts
import { Hono } from 'hono'
import { hono as vector5 } from '@vector5/middleware'

const app = new Hono()
app.use('*', vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
}))
```

```ts
// Workers · worker.ts
import { workers as vector5 } from '@vector5/middleware'

const track = vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
})

export default {
  async fetch(request, env, ctx) {
    ctx.waitUntil(track(request))
    return fetch(request)
  },
}
```

```ts
// Node · server.ts
import http from 'node:http'
import { node as vector5 } from '@vector5/middleware'

const track = vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
})

http.createServer((req, res) => {
  track(req, res)
  // ...your handler
}).listen(3000)
```

## Behaviour

- By default only non-human actors are recorded. The UA is matched against a crawler/bot/headless regex.
- Pass recordHumans: true for sites without JavaScript.
- Sends are keepalive fetch calls. Failures are swallowed — analytics must never 500 your app.
- On Next.js, x-forwarded-for is forwarded for geo, then discarded by the API.

## Options

| Option | Type | Required | Notes |
| --- | --- | --- | --- |
| endpoint | string | yes | Full URL to POST /api/hit |
| domain | string | yes | Registered site hostname |
| recordHumans | boolean | no | Default false |

## Hit payload

```json
{
  "domain": "example.com",
  "method": "GET",
  "path": "/docs/middleware",
  "status": 0,
  "bytes": 0,
  "ua": "GPTBot",
  "referer": "",
  "ip": "203.0.113.10"
}
```

The API classifies the UA as ai_crawler / ai_agent / bot, hashes the visitor, looks up country, and drops the IP. Status and bytes are optional — the Next adapter does not wait for the response, so status is often 0.


---

# Next.js middleware

vector5() from @vector5/middleware/next. Edge-friendly, fire-and-forget, matcher-aware.

## Setup

```ts
import { vector5 } from '@vector5/middleware/next'

export const middleware = vector5({
  endpoint: process.env.V5_HIT_URL!,
  domain: 'example.com',
})

export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
}
```

The helper reads user-agent, referer, and x-forwarded-for, then returns without producing a NextResponse. Next.js continues the request as usual.

## Matcher

Exclude static assets or you will record every chunk as a crawler hit. The matcher above is a good default. Also skip /api/* if those routes are not pages you care about.

## Compose with your own middleware

```ts
import { NextResponse } from 'next/server'
import { vector5 } from '@vector5/middleware/next'

const track = vector5({
  endpoint: process.env.V5_HIT_URL!,
  domain: 'example.com',
})

export function middleware(request: Request) {
  track(request)
  return NextResponse.next()
}
```


---

# Express

app.use(vector5({ endpoint, domain })). Calls next() immediately.

## Setup

```ts
import express from 'express'
import { express as vector5 } from '@vector5/middleware'

const app = express()
app.use(vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
}))
```

Uses req.originalUrl, req.ip, and the incoming UA. Place it before your routes. It always calls next().


---

# Hono

Works on Node, Bun, Deno, and Cloudflare via Hono middleware.

## Setup

```ts
import { Hono } from 'hono'
import { hono as vector5 } from '@vector5/middleware'

const app = new Hono()
app.use('*', vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
}))
```


---

# Cloudflare Workers

Use ctx.waitUntil so the hit outlives the response.

## Setup

```ts
import { workers as vector5 } from '@vector5/middleware'

const track = vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
})

export default {
  async fetch(request: Request, env: unknown, ctx: { waitUntil: (p: Promise<unknown>) => void }) {
    ctx.waitUntil(track(request))
    return fetch(request)
  },
}
```

> **waitUntil.** Without waitUntil, the isolate may freeze before the hit is sent. Always schedule it.


---

# Node HTTP

A function you call from http.createServer. It does not wrap the response.

## Setup

```ts
import http from 'node:http'
import { node as vector5 } from '@vector5/middleware'

const track = vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
})

http.createServer((req, res) => {
  track(req, res)
  res.writeHead(200, { 'content-type': 'text/plain' })
  res.end('ok')
}).listen(3000)
```


---

# recordHumans

For sites without JavaScript. Counts every request, including browsers.

## When to use it

Static sites, Gemini-generated HTML, or anything that never ships v5.js. Pass recordHumans: true and the middleware records every UA that hits the matcher.

```ts
export const middleware = vector5({
  endpoint: 'https://analytics.example.com/api/hit',
  domain: 'example.com',
  recordHumans: true,
})
```

## Do not double-count

> **Pick one path for humans.** If you also load v5.js, browsers will appear twice — once as a hit, once as an event. Leave recordHumans off whenever the tracker is installed.


---

# Actor classification

Every event gets one of five actors. Classification happens server-side in Go and is deterministic.

Client-side scripts never see AI crawlers. User-agent lists alone are spoofable. Vector 5 classifies every ingest in apps/api/internal/classify using a curated UA list, assistant referrers / UTMs, and headless heuristics. ADR 0004.

## The five actors

- `human` — browser visitor
- `human_via_ai` — arrived from an assistant
- `ai_crawler` — GPTBot and peers
- `ai_agent` — headless agent
- `bot` — other automation

| Actor | Rule |
| --- | --- |
| ai_crawler | UA matches a known AI crawler (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Bytespider, CCBot, Amazonbot, Meta-ExternalAgent, …). Mapped to a source (chatgpt, claude, perplexity, gemini, …). |
| human_via_ai | Referrer host is an assistant (chatgpt.com, perplexity.ai, claude.ai, gemini.google.com, copilot.microsoft.com, grok.com, chat.deepseek.com) or utm_medium is ai_search / ai-assistant, or utm_source names an assistant. |
| ai_agent | Headless signature (headless, playwright, puppeteer, crawl4ai, agent…) and no JavaScript executed. |
| bot | Generic automation: bot, crawler, spider, uptime monitors, link previewers, Lighthouse. |
| human | Everything else. |

## Evaluation order

1. Known AI crawler UA → ai_crawler.
2. Generic bot UA → bot, or ai_agent if it also looks headless and JS did not run.
3. Assistant referrer or AI UTM → human_via_ai.
4. Headless UA and no JS → ai_agent.
5. Otherwise → human.

Browser events from v5.js pass jsExecuted = true, so they almost never become ai_agent. Middleware hits pass false.

## Source and AI source

ai_source is the assistant or vendor (chatgpt, claude, perplexity, gemini, copilot, grok, deepseek). source is the report-friendly name — crawler name, UTM, or a mapped host (Google, Bing, Hacker News). The full host map lives next to the classifier.

## Verified crawlers

The result includes a verified flag reserved for vendor IP / reverse-DNS checks when publishers document them. Today the curated UA list is the primary signal. Pull requests adding crawlers are welcome — the public list is GET /api/crawler-index.

- [Crawler index](/actors/crawlers) — Every recognised AI crawler and its source.
- [human_via_ai](/actors/human-via-ai) — Assistant referrals and AI UTMs.
- [Agents](/actors/agents) — Headless, Playwright, crawl4ai.
- [Bots](/actors/bots) — Uptime, previews, Lighthouse.


---

# AI crawlers

GPTBot, ClaudeBot, PerplexityBot, and the rest of the curated index.

## Recognised crawlers

| Needle (UA) | Name | ai_source |
| --- | --- | --- |
| gptbot | GPTBot | chatgpt |
| oai-searchbot | OAI-SearchBot | chatgpt |
| chatgpt-user | ChatGPT-User | chatgpt |
| claudebot | ClaudeBot | claude |
| claude-user | Claude-User | claude |
| anthropic-ai | Anthropic | claude |
| perplexitybot | PerplexityBot | perplexity |
| perplexity-user | Perplexity-User | perplexity |
| google-extended | Google-Extended | gemini |
| googleother | GoogleOther | gemini |
| bytespider | Bytespider | other |
| applebot-extended | Applebot-Extended | other |
| ccbot | CCBot | other |
| amazonbot | Amazonbot | other |
| meta-externalagent | Meta-ExternalAgent | other |
| cohere-ai | Cohere | other |
| youbot | YouBot | other |

## Public index

GET /api/crawler-index returns this list without auth. Use it to generate robots.txt allow-lists or to audit which bots you have actually seen via ai_traffic.

## robots.txt

Vector 5 does not change your robots policy. It only measures who arrived. The Audit AI crawler access skill drafts an allow-list from observed traffic.


---

# human_via_ai

A person who arrived from ChatGPT, Perplexity, Claude, Gemini, Copilot, Grok, or DeepSeek.

This is the actor that makes Vector 5 different from "we added a GPTBot row to a Plausible clone." It is a human, attributed to an assistant.

## Assistant referrers

| Host | ai_source |
| --- | --- |
| chatgpt.com, chat.openai.com | chatgpt |
| perplexity.ai, www.perplexity.ai | perplexity |
| claude.ai | claude |
| gemini.google.com | gemini |
| copilot.microsoft.com | copilot |
| grok.com | grok |
| chat.deepseek.com | deepseek |

## UTM overrides

If an assistant strips the referrer, you can still mark the click: utm_medium=ai_search or utm_medium=ai-assistant, or utm_source of chatgpt, perplexity, claude, gemini (and prefixes of chatgpt).

## The Direct problem

Many ChatGPT citations open without a referrer. Those visits look like Direct in every other tool. Vector 5 cannot invent a referrer that was not sent — but crawl→referral on the same path tells you which pages assistants cite versus which ones people click. See AI traffic and the Cited but not clicked skill.


---

# AI agents

Headless browsers acting for a person. Task-oriented metrics, not bounce rate.

## Signals

A UA containing headless / playwright / puppeteer / crawl4ai / agent, and JavaScript did not execute (middleware hit). Browser events from a real Chrome window stay human even if someone spoofs a string — because v5.js ran.

## How to read them

Agents fetch a page to do a job. Bounce rate is the wrong lens. Use goals ("agent completed signup") and the pages they requested. The dashboard treats ai_agent as its own series so it never dilutes human conversion.


---

# Generic bots

Uptime monitors, link previewers, Lighthouse, and everything else that looks automated.

## Generic needles

bot, crawler, spider, slurp, bingpreview, facebookexternalhit, pingdom, uptimerobot, headlesschrome, phantomjs, lighthouse.

## Filtering

Dashboards default to humans. Pass actor=bot on any read API, or pick Bot in the UI, when you want to see monitors and previewers. They are stored so you can audit them — they are not mixed into human KPIs.


---

# Privacy model

Vector 5 measures audiences, not people. No cookies, no localStorage IDs, no fingerprints, no stored IPs.

> Vector 5 Analytics measures audiences, not people.
> — docs/privacy-model.md

## What we collect

| Field | Why | Stored? |
| --- | --- | --- |
| Page path | Traffic reports | Yes |
| Referrer host / source | Attribution | Yes (host only) |
| UTM parameters | Campaigns | Yes |
| Country / region | Geo breakdown | Yes (IP discarded after lookup) |
| Device, browser, OS, screen width | Device reports | Yes |
| Event name and optional props | Goals | Yes |
| Actor and AI source | AI traffic | Yes |
| Visitor hash | Unique visitors for one UTC day | Yes |
| IP address | Hash + geo only | Never stored |
| Cookies / localStorage IDs | — | Never |
| Fingerprints | — | Never |

## Unique visitors

```text
visitor_hash = SHA-256(daily_salt || site_id || ip || user_agent)
```

The salt rotates at 00:00 UTC per site. After hashing, the IP is dropped. A visitor who returns tomorrow is counted as new. Visitor hash.

## What we cannot know

- Who a visitor is.
- Whether they returned after midnight UTC.
- Cross-site or cross-device identity.
- Exact address (city is optional and off by default).
- Anything about visitors who block the request entirely.

## Source of truth

The machine-readable version is GET /api/sites/{domain}/manifest. The long-form document is docs/privacy-model.md in the repository. The marketing site's privacy page is the plain-language twin.


---

# Daily-salted visitor hash

SHA-256 of daily salt, site id, IP, and user agent. The IP never hits disk.

## Why a hash

Cookieless analytics still needs a way to count unique visitors inside a day without storing an IP. ADR 0003.

## Formula

```text
visitor_hash = SHA-256(daily_salt || site_id || ip || user_agent)
```

Implemented in apps/api/internal/hash. The salt is generated per site per UTC day and stored only for that calendar day. The IP is an argument to the hash function, then forgotten.

## Rotation

00:00 UTC. A reader in California who visits at 4pm and again at 6pm the same local evening may cross the UTC boundary and count twice. That is disclosed in "What we cannot know."

## Limits we accept

- No cross-day journeys.
- Shared NATs collapse to one hash per UA for that day.
- UA changes (browser update mid-day) create a new hash.
- We will not add cookies "just for returning visitors."


---

# Collection manifest

A public JSON document every site serves. Auditable by humans and agents.

## Shape

```json
{
  "name": "Vector 5 Analytics",
  "site": "example.com",
  "cookies": false,
  "local_storage": false,
  "fingerprinting": false,
  "ip_stored": false,
  "visitor_hash": "sha256(daily_salt || site_id || ip || user_agent); salt rotates 00:00 UTC",
  "retention_days": 730,
  "fields": ["path", "referrer_host", "utm", "country", "device",
             "browser", "os", "actor", "ai_source", "event_name"],
  "cannot_know": [
    "Who a visitor is",
    "Whether they returned after midnight UTC",
    "Cross-site or cross-device identity",
    "Exact address or city",
    "Anything about visitors who block the request"
  ]
}
```

## Fetch it

No auth. CORS is open. Anyone can curl it. retention_days comes from the site row (default 730).

```bash
curl https://analytics.example.com/api/sites/example.com/manifest
```

## Verify with an agent

The Collection manifest check skill asks an agent to fetch the manifest and confirm the deployed tracker still matches the claims.


---

# Retention

Site-level retention_days. Rollups outlive raw rows only as long as you keep them.

## Default

730 days, published on the manifest. Patch the site (PATCH /api/sites/{id}) to change it. Raw events are partitioned by month so dropping old partitions is cheap.

## Export before you shrink it

GET /api/sites/{id}/export returns aggregates as JSON. POST /api/sites/{id}/import accepts CSV if you are migrating history in.


---

# GDPR and CCPA

No consent banner required for the default configuration. You still own the legal review.

> **Not legal advice.** This page describes the product. Your counsel decides whether it is enough for your jurisdiction and your props.

## No cookie, no consent wall

The tracker sets no cookies and stores no identifier in localStorage. The IP is not retained. Unique visitors are a daily hash. That is the same class of processing cookieless analytics tools have used to avoid a cookie banner — plus an actor field.

## Roles

- Self-host — you are the controller. Vector 5 is software you run.
- Hosted — you are the controller of your site's analytics; Vector 5 processes it on your instructions. A DPA will ship with hosted accounts.

## Access requests

Because there is no persistent identity, there is no row to retrieve for "all data about Jane." You can export aggregates and confirm the manifest. You cannot reconstruct a person from a visitor hash after the salt rotates.

Do not put emails or user ids in event props. If you do, you have created personal data Vector 5 did not design for.


---

# Dashboard

Sites, filters, goals, AI traffic, insights, and settings. Defaults to humans.

The dashboard is the Next.js app at apps/web. After login you land on /app (site list) and then /app/{domain}.

## Layout

- KPIs — visitors, views, and the actor mix for the selected window.
- Timeseries — daily rollups, filterable by actor.
- Breakdowns — pages, sources, countries, devices.
- AI Traffic — crawlers, assistant referrals, crawl→referral.
- Goals — conversions that never include agents unless you ask.
- Realtime — humans in the last five minutes.
- Insights / anomalies — optional, aggregates only.

## Humans first

Opening the dashboard without a filter shows human. Switch to human_via_ai to see assistant-attributed people, or to ai_crawler when you are negotiating robots.txt. Filters.

- [Sites](/dashboard/sites) — Add a domain, get a snippet.
- [AI traffic](/dashboard/ai-traffic) — Crawled vs cited vs clicked.
- [Insights](/dashboard/insights) — Ollama or a hosted model.
- [Settings](/dashboard/settings) — Keys, LLM, retention.


---

# Sites

A site is a hostname. The tracker and middleware must use the same one.

## Add a site

From /app, create a site with the production hostname (example.com). Avoid www unless that is what users type — or register both.

## Install snippet

The site page shows the tracker tag and a middleware stub pointed at your V5_PUBLIC_URL. Copy them as-is. Unknown domains 404 on ingest.


---

# Filters

days and actor. The same two query params the API and MCP use.

## Window

1 / 7 / 30 days. Rollups are hourly and daily; longer ranges are a follow-up, not a missing ClickHouse cluster.

## Actor

Every chart respects the actor chip. all is available when you want the raw mix — useful for a weekly review, noisy for conversion rate.


---

# Goals

Named events with conversion rates that agents cannot inflate.

## Create

Name must match window.v5('signup') or data-v5-event="signup" (case-sensitive). You can also POST /api/sites/{id}/goals.

## Filter by actor

A Playwright suite that submits your form will land as ai_agent if it came through the middleware without JS, or human if it loaded v5.js. Either way you can exclude it from the human conversion rate.


---

# AI insights

Optional. The model only ever sees rollups. Ollama by default.

## Enable

Settings → LLM, or PATCH /api/team/llm. Self-hosters can keep inference on-prem with Ollama (OLLAMA_HOST, model llama3.2). ADR 0005.

## observed vs inferred

Every number the model repeats from rollups is observed. Recommendations are inferred. The UI keeps that distinction so a founder briefing cannot launder a guess as a count.


---

# Realtime

Humans in the last five minutes. Crawlers do not belong here.

## Five minutes

GET /api/sites/{id}/realtime powers the live strip. It is scoped to people (human, human_via_ai) so a GPTBot sweep cannot make the office look busy.


---

# AI traffic

Crawlers, assistant referrals, and the crawl-to-referral gap.

## Panels

- Which bots fetched you (GPTBot, ClaudeBot, PerplexityBot, …).
- Which pages they fetched.
- Which assistants sent humans (human_via_ai).
- Pages with high crawl and low referral — the "cited but not clicked" list.

## What to do with it

Give those pages a clear first paragraph, a stable title, and schema.org. The Cited but not clicked skill writes the brief from ai_traffic.


---

# Settings

API keys, LLM provider, operator account, retention.

## API keys

Create, name, revoke. Paste into .mcp.json as V5_API_KEY. API keys.

## LLM

Provider + model for insights. Leave none if you do not want a model in the loop. Insights still never receive event rows.

## Retention

Per-site retention_days, published on the manifest. Retention.


---

# HTTP API

Ingest is public. Reads take a Bearer API key. days and actor filter most query routes.

## Base URL

Self-host default: http://localhost:8080. Hosted: your project URL. The dashboard talks to NEXT_PUBLIC_API_URL.

## Authentication

Create a key under Settings → API keys (POST /api/keys). Send it on every authenticated route:

```bash
Authorization: Bearer v5_live_…
```

Login (POST /api/auth/login) returns a JWT for the admin operator. The dashboard uses that. Agents and scripts should use API keys. Authentication.

## Shared filters

| Query | Values | Applies to |
| --- | --- | --- |
| days | 1, 7, 30 | stats, breakdown, ai, goals |
| actor | human, human_via_ai, ai_crawler, ai_agent, bot, all | same |
| field | source, page, country, device | breakdown only |

## Catalog

- `POST /api/waitlist` — Public early-access request
- `POST /api/auth/login` — Operator login → JWT
- `GET /api/me` — Current principal
- `POST /api/event` — Browser events from v5.js
- `POST /api/hit` — Server hits from the middleware
- `GET /api/sites/{domain}/manifest` — Public collection manifest
- `GET /api/crawler-index` — Recognised AI crawlers
- `GET /api/sites` — List sites
- `POST /api/sites` — Create a site
- `PATCH /api/sites/{id}` — Update a site
- `GET /api/sites/{id}/stats` — Totals + daily series
- `GET /api/sites/{id}/breakdown` — source | page | country | device
- `GET /api/sites/{id}/realtime` — Humans in the last 5 minutes
- `GET /api/sites/{id}/ai` — Crawlers, referrals, crawl→referral
- `GET /api/sites/{id}/goals` — Goal conversions
- `POST /api/sites/{id}/goals` — Create a goal
- `DELETE /api/sites/{id}/goals/{goalID}` — Delete a goal
- `GET /api/sites/{id}/insights` — Generated insights
- `GET /api/sites/{id}/anomalies` — Anomaly flags
- `GET /api/sites/{id}/export` — Export aggregates as JSON
- `POST /api/sites/{id}/import` — Import history from CSV
- `GET /api/keys` — List API keys
- `POST /api/keys` — Create an API key
- `DELETE /api/keys/{id}` — Revoke an API key
- `PATCH /api/team/llm` — Configure the insight provider
- `GET /health` — Liveness
- `GET /js/v5.js` — Tracker script


---

# Authentication

Admin JWT for the dashboard. API keys for scripts, MCP, and CI.

## Operator login

```bash
curl -X POST http://localhost:8080/api/auth/login \
  -H 'content-type: application/json' \
  -d '{"email":"admin@localhost","password":"changeme"}'
```

Returns { "token": "…" }. Used when V5_AUTH_MODE=admin. Hosted / Supabase mode uses the Supabase session instead.

## API keys

Create under Settings or POST /api/keys. The secret is shown once. Prefix v5_live_. Send as Authorization: Bearer. API keys.

## GET /api/me

Returns the current principal when a valid JWT or key is present. Some GET routes currently degrade without auth for local demo — treat production as authenticated.


---

# POST /api/event

Browser ingest. 202 on success. 404 if the domain is not a registered site.

Called by v5.js via sendBeacon or fetch + keepalive. CORS is open. Body is capped at 64 KB.

## Body

```json
{
  "d": "example.com",
  "n": "pageview",
  "u": "https://example.com/docs/middleware",
  "r": "https://chatgpt.com/",
  "h": "example.com",
  "w": 1440,
  "p": { "plan": "pro" },
  "utm_source": "chatgpt",
  "utm_medium": "ai_search",
  "utm_campaign": "",
  "utm_term": "",
  "utm_content": ""
}
```

d can also be supplied as ?d=. The server uses X-Forwarded-For / X-Real-IP for hash + geo, then discards the IP. UA comes from the request header. jsExecuted is true.

## Response

| Status | Meaning |
| --- | --- |
| 202 | Accepted |
| 400 | Bad JSON |
| 404 | Unknown site |


---

# POST /api/hit

Server ingest from @vector5/middleware. Same 202 / 404 contract as /api/event.

## Body

```json
{
  "domain": "example.com",
  "method": "GET",
  "path": "/docs/middleware",
  "status": 200,
  "bytes": 18432,
  "ua": "Mozilla/5.0 (compatible; GPTBot/1.2)",
  "referer": "",
  "ip": "203.0.113.10"
}
```

If ua or ip is omitted, the API fills them from the incoming request. Classification runs with jsExecuted = false, which is how headless UAs become ai_agent instead of human.


---

# Stats and realtime

Totals, daily series, and humans in the last five minutes.

## GET /api/sites/{id}/stats

?days=1|7|30&actor=human. Returns period totals and a daily series from rollups. This is what the dashboard sparkline and the MCP query_stats tool read.

```bash
curl -H "Authorization: Bearer $V5_API_KEY" \
  "$V5_API_URL/api/sites/$SITE_ID/stats?days=7&actor=human"
```

## GET /api/sites/{id}/realtime

Humans (and human_via_ai) in the last five minutes. Used by the live strip on the dashboard.


---

# GET /api/sites/{id}/breakdown

Top sources, pages, countries, or devices for a period and actor.

## field=

source · page · country · device. Combine with days and actor. MCP maps this to top_pages and sources.

```bash
curl -H "Authorization: Bearer $V5_API_KEY" \
  "$V5_API_URL/api/sites/$SITE_ID/breakdown?field=page&days=30&actor=human_via_ai"
```


---

# GET /api/sites/{id}/ai

Crawlers, pages crawled, assistant referrals, and crawl-to-referral.

## What you get

- Crawler counts by bot name (GPTBot, ClaudeBot, …).
- Pages those crawlers fetched.
- human_via_ai referrals by assistant.
- crawl_to_referral — pages with crawl volume vs click-through from assistants.

This is the payload behind the AI Traffic panel and the MCP ai_traffic tool. It is also the input to the "cited but not clicked" skill.


---

# Goals API

List, create, and delete goals. Conversions are always actor-aware.

## List and conversions

GET /api/sites/{id}/goals?days=30&actor=human returns each goal and its conversion count for the window. Names must match what you send from window.v5 or data-v5-event.

## Create

```json
{ "name": "signup" }
```

DELETE /api/sites/{id}/goals/{goalID} removes the definition. Historical events stay; they just leave the goals report.


---

# Insights, anomalies, LLM

Aggregates only. Configure the provider with PATCH /api/team/llm.

## GET /insights

Generated copy from the last run. Empty if no provider is configured (V5_LLM_PROVIDER=none).

## GET /anomalies

Flags from the rollup series — spikes in a crawler, a sudden drop in humans, a page that started getting crawled but never referred. Numbers are labelled observed or inferred.

## PATCH /api/team/llm

```json
{
  "provider": "ollama",
  "model": "llama3.2"
}
```

Providers: ollama (default model llama3.2), OpenAI-compatible endpoints, anthropic, gemini. Env fallbacks: V5_LLM_PROVIDER, V5_LLM_MODEL, OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, OLLAMA_HOST.

## What the model sees

```ts
// Everything the model receives. Nothing else.
{
  "period": "2026-08-31 → 2026-09-06",
  "visitors": { "human": 12480, "human_via_ai": 1731, "delta_wow": 0.08 },
  "ai_referrals": { "chatgpt": 1190, "perplexity": 402, "claude": 139 },
  "crawlers": { "GPTBot": 2210, "ClaudeBot": 980, "PerplexityBot": 712 },
  "crawl_to_referral": [
    { "page": "/pricing", "crawled": 340, "referred": 2 },
    { "page": "/docs/middleware", "crawled": 128, "referred": 61 }
  ]
}
```


---

# Export and import

JSON export of aggregates. CSV import for history.

## Export

GET /api/sites/{id}/export — aggregates, not raw event rows. Safe to hand to a spreadsheet or another Vector 5 instance.

## Import

POST /api/sites/{id}/import with a CSV body. Use this when you are leaving another tool and want a baseline. It does not invent actors the source did not have — those rows land as human unless the CSV includes an actor column Vector 5 recognises.


---

# Sites

List, create, and patch sites. Domain is the join key for ingest.

## GET /api/sites

Every site on the team. { id, domain, retention_days, … }.

## POST /api/sites

```json
{ "domain": "example.com" }
```

## PATCH /api/sites/{id}

Update retention or display name. Changing domain after you have shipped the tracker will 404 ingest until you update data-domain.


---

# API keys

Team-scoped secrets for MCP, curl, and CI. Shown once.

## Create

POST /api/keys with an optional name. Response includes the secret once. Store it in your password manager or the MCP env. GET /api/keys lists prefixes and names, not secrets.

## Revoke

DELETE /api/keys/{id}. Existing dashboard JWTs are unaffected. Rotate keys if a laptop is lost.


---

# GET /api/sites/{domain}/manifest

Public, unauthenticated, CORS-open. The machine-readable privacy policy.

## Domain, not id

This route is keyed by hostname so anyone can fetch it without knowing the internal UUID. Full shape: Collection manifest.


---

# GET /api/crawler-index

Public list of recognised AI crawlers. PRs welcome.

## Use it

Generate robots.txt, document which bots you allow, or diff against what ai_traffic actually observed. See AI crawlers.


---

# POST /api/waitlist

Public early-access request for the hosted product.

## Body

```json
{ "email": "you@example.com", "site": "example.com" }
```

Validates a plausible email. Used by the homepage form on vector5.ai. Self-hosters can ignore it.


---

# MCP server

Give Claude, Cursor, or Codex read-only access to the same aggregates the dashboard shows.

@vector5/mcp is a stdio MCP server. Six tools, all reads, all aggregates. It never sees event rows and it cannot ingest.

## Install

```json
{
  "mcpServers": {
    "vector5": {
      "command": "npx",
      "args": ["@vector5/mcp"],
      "env": {
        "V5_API_URL": "https://analytics.example.com",
        "V5_API_KEY": "v5_live_…",
        "V5_SITE_ID": "example.com"
      }
    }
  }
}
```

Cursor reads .mcp.json from the project root. Claude Desktop uses claude_desktop_config.json. Codex uses its MCP settings panel. Cursor setup.

## Environment

| Var | Required | Notes |
| --- | --- | --- |
| V5_API_URL | no | Default http://localhost:8080 |
| V5_API_KEY | yes in prod | Bearer key from Settings |
| V5_SITE_ID | yes | Site domain or id the API expects |

## Tools

Each tool accepts days and actor. Details: Tools reference.

| Tool | Upstream |
| --- | --- |
| query_stats | GET /api/sites/{id}/stats |
| top_pages | GET /api/sites/{id}/breakdown?field=page |
| sources | GET /api/sites/{id}/breakdown?field=source |
| ai_traffic | GET /api/sites/{id}/ai |
| goals | GET /api/sites/{id}/goals |
| insights | GET /api/sites/{id}/insights |


---

# MCP tools

Six read-only tools. Input schema is days + actor.

## Shared input

```json
{
  "type": "object",
  "properties": {
    "days": { "type": "number" },
    "actor": { "type": "string" }
  }
}
```

## Each tool

- query_stats — period totals and the daily series. Start here for a briefing.
- top_pages — paths by views for the actor.
- sources — Google, ChatGPT, Direct, …
- ai_traffic — crawlers, referrals, crawl→referral. Required for the crawler audit and "cited but not clicked" skills.
- goals — conversion counts. Always pass actor=human unless you mean otherwise.
- insights — last generated insight text. Empty if LLM is none.


---

# Cursor, Claude, Codex

Drop the same server into any MCP host.

## Cursor

Project-level .mcp.json (shown on MCP) or the global MCP settings. Reload the window after changing env. Ask: "Using Vector 5 query_stats, what changed this week?"

## Claude Desktop

Add the same command / args / env block under mcpServers in the Claude Desktop config, then restart Claude.

## Codex

Register @vector5/mcp as a stdio server. The protocol version is 2024-11-05 (initialize, tools/list, tools/call).


---

# Agent skills

Ready-made prompts in packages/skills. MIT. Copy them into Claude, Cursor, or Codex.

These prompts assume MCP is connected. They only use observed numbers.

## Audit AI crawler access

> Using Vector 5 ai_traffic, list which bots crawled this site in the last 30 days, which pages they fetched, and whether robots.txt or llms.txt would block them. Propose a robots.txt allow-list for GPTBot, ClaudeBot, PerplexityBot, and OAI-SearchBot.

## Cited but not clicked

> Using ai_traffic.crawl_to_referral, find pages that AI crawlers fetch but that send almost no human_via_ai referrals. Suggest title, schema, and first-paragraph changes so assistants are more likely to cite a clickable link.

## Weekly review

> Read query_stats, sources, top_pages, and insights. Write a founder briefing: what changed, why, and one thing to do this week. Only use observed numbers.

## Collection manifest check

> Fetch /api/sites/{domain}/manifest and confirm the public claims (no cookies, no IP storage, daily hash) match the deployed tracker.


---

# llms.txt

This docs site publishes /llms.txt and /llms-full.txt so assistants can ingest the corpus.

## The two files

- /llms.txt — index of every docs page with a one-line description.
- /llms-full.txt — the full markdown corpus. Large; for offline / RAG ingest.

Both are generated from the same catalog as the sidebar and the ⌘K search. If a page exists here, an agent can find it there.

## On your own site

Vector 5 does not write llms.txt for you. Publish one next to robots.txt. Then use the crawler-audit skill to see whether GPTBot and ClaudeBot actually fetch it.


---

# Self-hosting

A Go binary, a Next.js app, and Postgres. docker compose up --build runs all three.

Vector 5 is designed to be operated by one compose file. Rollups run inside the API process; there are no extra workers.

## Ports

| Service | Port | Image |
| --- | --- | --- |
| Postgres 17 | 5432 | postgres:17-alpine |
| API (v5d) | 8080 | apps/api/Dockerfile |
| Web (dashboard + marketing) | 3000 | apps/web/Dockerfile |
| Docs (this site) | 3001 | apps/docs — deploy separately to docs.vector5.ai |

## Run pieces locally

```bash
docker compose up postgres -d
cd apps/api && go run ./cmd/v5d
pnpm --filter @vector5/web dev
pnpm --filter @vector5/docs dev
```

## This docs app

apps/docs is a standalone Next.js app. Host it on docs.vector5.ai. The marketing site already points there. Set NEXT_PUBLIC_SITE_URL=https://docs.vector5.ai so Open Graph and sitemap URLs are correct.

- [Docker Compose](/self-host/docker) — Supported path.
- [Environment](/self-host/environment) — Every variable explained.
- [Supabase auth](/self-host/supabase) — Multi-user cloud.
- [Production](/self-host/production) — TLS, backups, hardening.


---

# Docker Compose

postgres, api, web. Copy .env.example and compose up.

## Bring it up

```bash
git clone https://github.com/vector5-ai/analytics
cd analytics
cp .env.example .env
# set V5_ADMIN_PASSWORD and V5_JWT_SECRET
docker compose up --build
```

The compose file wires DATABASE_URL to the postgres service, sets V5_AUTH_MODE=admin, and publishes 3000 / 8080 / 5432.

## Health

```bash
curl -s http://localhost:8080/health
# {"ok":"true"}
```


---

# Local development

Go API + pnpm workspaces. Homebrew Postgres is fine.

## Database

.env.example uses your OS username on macOS (Homebrew / Postgres.app), not postgres. Create a vector5 database. Compose uses postgres/postgres instead.

## API

```bash
cd apps/api && go run ./cmd/v5d
```

## Web and docs

```bash
pnpm install
pnpm --filter @vector5/web dev      # :3000
pnpm --filter @vector5/docs dev     # :3001
```

Root scripts: pnpm dev (web), pnpm dev:docs, pnpm build:docs.


---

# Environment variables

API, auth, web, and optional LLM. Nothing here is sent to the browser except NEXT_PUBLIC_*.

## API

| Variable | Default | Purpose |
| --- | --- | --- |
| V5_LISTEN | :8080 | Bind address |
| V5_PUBLIC_URL | http://localhost:8080 | Public origin for the tracker URL |
| DATABASE_URL | local postgres | App connection |
| DATABASE_DIRECT_URL | falls back to DATABASE_URL | Migrations / admin |

## Auth

| Variable | Default | Purpose |
| --- | --- | --- |
| V5_AUTH_MODE | admin | admin or supabase |
| V5_ADMIN_EMAIL | admin@localhost | Operator |
| V5_ADMIN_PASSWORD | changeme | Change this |
| V5_JWT_SECRET | dev-secret | Sign dashboard tokens |
| SUPABASE_URL |  | When mode is supabase |
| SUPABASE_ANON_KEY |  | Server |
| SUPABASE_SERVICE_ROLE_KEY |  | Server only — never NEXT_PUBLIC |

## Web

| Variable | Purpose |
| --- | --- |
| NEXT_PUBLIC_API_URL | Browser → API |
| NEXT_PUBLIC_SITE_URL | Canonical, OG, sitemap (https://vector5.ai) |
| NEXT_PUBLIC_SUPABASE_URL | Hosted auth |
| NEXT_PUBLIC_SUPABASE_ANON_KEY | Hosted auth (publishable) |

## LLM

| Variable | Purpose |
| --- | --- |
| V5_LLM_PROVIDER | none | ollama | openai-compatible | anthropic | gemini |
| V5_LLM_MODEL | Override the default model |
| OLLAMA_HOST | Default http://localhost:11434 |
| OPENAI_API_KEY / ANTHROPIC_API_KEY / GEMINI_API_KEY | Provider secrets |

## Docs

| Variable | Purpose |
| --- | --- |
| NEXT_PUBLIC_SITE_URL | https://docs.vector5.ai |
| NEXT_PUBLIC_MARKETING_URL | https://vector5.ai |


---

# Supabase auth

V5_AUTH_MODE=supabase for multi-user cloud. Never put the service role in the browser.

## Flip the mode

Set V5_AUTH_MODE=supabase and point both the API and the Next app at the same project. The dashboard uses @supabase/ssr. The API validates JWTs from that project.

## Keys

- NEXT_PUBLIC_SUPABASE_ANON_KEY / SUPABASE_ANON_KEY — publishable. Fine in the browser.
- SUPABASE_SERVICE_ROLE_KEY — server only. Never prefix with NEXT_PUBLIC_.

> **service_role.** A leaked service role bypasses RLS. Rotate immediately if it has ever been in a client bundle.


---

# Production hardening

TLS, secrets, backups, and the few things compose does not do for you.

## Secrets

- Rotate V5_ADMIN_PASSWORD and V5_JWT_SECRET before the first public packet.
- Do not commit .env.
- API keys are shown once — treat them like passwords.

## TLS and URLs

Terminate TLS in Caddy / nginx / your PaaS. Set V5_PUBLIC_URL and NEXT_PUBLIC_SITE_URL to the public https origins. Forward X-Forwarded-For so geo and the visitor hash see the client, not the proxy.

## Backups

Postgres is the only datastore. Snapshot the volume or pg_dump. Rollups can be rebuilt from events if you still have the partitions.

## Upgrading

```bash
git pull
docker compose up --build
```

Read the changelog before you pull. Migrations live with the API.


---

# Changelog

What shipped. Dates are UTC.

## 0.1.0 — 2026-09-06

- Cookieless v5.js tracker (~1 KB) with SPA, outbound, and download events.
- @vector5/middleware for Next.js, Express, Hono, Workers, and Node.
- Five-actor classification in Go (human, human_via_ai, ai_crawler, ai_agent, bot).
- Daily-salted visitor hash. IP discarded after hash + geo.
- Public collection manifest and crawler index.
- Dashboard: stats, breakdowns, realtime, AI traffic, goals, insights, anomalies.
- HTTP API + API keys.
- @vector5/mcp with six read-only tools and packages/skills prompts.
- Aggregates-only LLM (Ollama / OpenAI / Anthropic / Gemini).
- Docker Compose self-host. Admin or Supabase auth.
- Standalone docs app at docs.vector5.ai (apps/docs) with ⌘K search and llms.txt.


---

# Glossary

The words the UI, API, and MCP all share.

## Terms

| Term | Meaning |
| --- | --- |
| Actor | One of five labels on every event / hit. |
| AI source | Vendor behind a crawler or assistant referral (chatgpt, claude, …). |
| Event | Browser ingest via /api/event. |
| Hit | Server ingest via /api/hit. |
| Manifest | Public JSON privacy document for a site. |
| Rollup | Hourly / daily aggregate. What charts and LLMs read. |
| Site | A registered hostname. |
| Visitor hash | SHA-256 unique for one site, one UTC day. |
| human_via_ai | A person who arrived from an assistant. |
| jsExecuted | Whether v5.js ran. True for events, false for hits. |


---

# FAQ

Short answers. The long ones live on the other pages.

## General

- Is this a Plausible fork? No. Clean-room. ADR 0001.
- Do I need ClickHouse? No. Postgres + rollups. ADR 0002.
- Can I use it without JavaScript? Yes — middleware with recordHumans: true.
- Is hosted free? Self-host is free (AGPL). Hosted is invite-only.

## Privacy

- Do I need a cookie banner? Not for the default tracker. Confirm with counsel. GDPR.
- Do you store IPs? No. Hash + geo, then drop.
- Can I identify a returning user tomorrow? No. The salt rotates at 00:00 UTC.

## AI traffic

- Why is AI Traffic empty? You did not install the middleware.
- Why is ChatGPT traffic Direct? Many citations omit a referrer. Use crawl→referral and AI UTMs. human_via_ai.
- Does the LLM see my events? No. Rollups only. ADR 0005.


---

# Troubleshooting

Events missing, crawlers missing, 404 unknown site, empty insights.

## No browser events

1. Is v5.js loading? Check the network tab for /js/v5.js and POST /api/event (202).
2. Did you set localStorage.v5_ignore while testing?
3. Does data-domain match the site you created?
4. If you vendored the script, did you set data-api?
5. Ad blockers: the path /js/v5.js is short on purpose. A filter list that blocks plausible will not match it, but a generic "analytics" list might.

## No crawlers

1. Middleware installed and deployed? v5.js cannot see GPTBot.
2. Matcher excluding the pages crawlers hit?
3. Simulate: curl -A "GPTBot" https://your.site/.
4. Confirm /api/hit returns 202, not 404.

## 404 unknown site

The domain on the event / hit is not in sites. Create it in the dashboard. www vs apex is the usual mismatch.

## Empty insights

V5_LLM_PROVIDER is none, or Ollama is down (OLLAMA_HOST). Insights also need rollups — a brand-new site has nothing to say.

## Double-counted humans

You set recordHumans: true and loaded v5.js. Pick one path for browsers. recordHumans.


---

# Licence

AGPL-3.0 for server and dashboard. MIT for tracker, middleware, MCP, and skills.

## The split

| Work | Licence |
| --- | --- |
| apps/api, apps/web, apps/docs | AGPL-3.0 |
| packages/tracker, middleware, mcp, skills | MIT |

See LICENSE, NOTICE, and DCO in the repository. Vector 5 is not affiliated with Plausible Analytics.

## Why

Client packages should be frictionless to paste into any site. The server should stay auditable, and anyone who offers Vector 5 as a network service must publish their modifications. ADR 0001.


---

# Compare

How Vector 5 differs from cookieless analytics that only run in the browser.

The marketing compare page is the public version. This is the operator version.

## Matrix

|  | Typical cookieless GA alt | Vector 5 |
| --- | --- | --- |
| Browser pageviews | Yes | Yes (v5.js) |
| AI crawlers | Invisible | Server middleware |
| Assistant referrals | Often Direct | human_via_ai + AI UTMs |
| Actor on every row | No | Five actors, deterministic |
| Public manifest | Rare | GET /api/sites/{domain}/manifest |
| LLM insights | Sometimes raw rows | Rollups only, Ollama included |
| MCP | Rare | Six read-only tools |
| Database | ClickHouse or Postgres | Postgres first |
| Licence | Varies | AGPL + MIT split |
| Fork of Plausible | Sometimes | No. Clean-room. |


---

# Architecture decisions

The ADRs in docs/decisions. Short, accepted, dated.

Source files live in docs/decisions/.

## 0001 — Clean-room and licence

Write every line ourselves. Reuse only non-copyrightable principles (daily-salted hash, no cookies, session windows). AGPL on the server; MIT on client packages so they can be pasted into any site. Cloud resellers must publish modifications.

## 0002 — Postgres, not ClickHouse

The cloud database is Supabase Postgres 17. Self-hosters expect one compose file. Partition events by month, serve the dashboard from rollups, leave a storage interface for ClickHouse later. Fast until tens of millions of events.

## 0003 — Daily-salted visitor hash

SHA-256(daily_salt || site_id || ip || user_agent). Salt rotates 00:00 UTC. IP discarded. A returning visitor tomorrow is a new hash. The limit is displayed in "What we cannot know."

## 0004 — Layered actor classification

Five actors. Curated UA list, vendor IP / rDNS when published, headless heuristics. Assistant referrers mark human_via_ai. Dashboards default to humans. Agent metrics are task-oriented, not bounce rate.

## 0005 — Aggregates-only LLM

Insights receive rollup rows only. Providers are pluggable, including local Ollama. Every derived number is observed or inferred. The privacy guarantee holds when AI is enabled.


---

# Contributing

DCO, crawler PRs, and how to run the docs app.

## DCO

Sign off commits (git commit -s). See DCO in the repository.

## New crawlers

Add a needle + name + ai_source in apps/api/internal/classify and a test. The public index is generated from that list. PRs that come with a link to the vendor's published UA are merged faster.

## Docs

This site is apps/docs. Pages live as typed catalogs under content/. After you add a page, it appears in the sidebar, ⌘K, sitemap, and llms.txt automatically.

```bash
pnpm --filter @vector5/docs dev
```


---