98083cd8d6
Operators cannot add a document today: ingestion is CLI-only, needs a host
login and a TTY for confirm_header(), /datadisk/ai-docs is mounted read-only,
ai-api has no identity, and nothing in the stack has a role that can write
doc_chunks. This designs the way in, the way out, and control over what is in
the retrieval pool. Design and schema only - no router, worker or UI code yet.
Documents in (16.1-16.9, db/004):
upload -> pre-scan -> review -> approve -> published, with the header
confirmation moved from a terminal prompt to a review screen and recorded
rather than discarded. A CHECK constraint refuses an approved row without a
confirmed number, revision and effective date, so an API bug cannot skip it.
Three roles: agent_ro unchanged, uploads_rw writes the queue only, ingest_rw
writes doc_chunks and has no HTTP surface.
Documents out (16.10-16.11, db/005):
--supersede needs a revision to keep, so a cancelled procedure cannot be
withdrawn at all. Adds withdraw (immediate, reversible, audited), restore
(refused while another revision is live) and purge (off by default). A
column grant plus a trigger let the web-facing role make a document less
citable and never more.
The pool (16.13-16.15, db/006):
pool_enabled, orthogonal to superseded: one is a claim about the document,
the other about the corpus. Retrieval requires both, so re-enabling a
withdrawn document does not make it citable. Named profiles and a
per-request override let a demo trim the corpus without mutating state on a
shared live host, and every reduced-pool answer carries a banner with the
document count, following the used_fixture_data precedent.
Two existing defects found and documented while designing this:
- ai-ingest takes PGUSER=agent_ro from api.env, a SELECT-only role, so the
Phase 3 command in the README cannot write doc_chunks (16.1).
- ingest_file() always inserts superseded = FALSE, so `--all` re-ingests a
superseded revision as live. --supersede survives only until the next bulk
run (16.10).
One commit rather than three: the upload, withdrawal and pool designs
interleave in the same spec, README and compose files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
87 lines
3.7 KiB
Text
87 lines
3.7 KiB
Text
# =============================================================================
|
|
# ai-routes.caddy - the blocks to paste into ~/Caddyfile on lin001.
|
|
#
|
|
# This file is NOT deployed as-is. ~/Caddyfile is a single hand-maintained file
|
|
# with many .bak-* snapshots beside it. Append these blocks, then:
|
|
#
|
|
# cp ~/Caddyfile ~/Caddyfile.bak-ai-$(date +%Y%m%d)
|
|
# docker exec caddy caddy reload --config /etc/caddy/Caddyfile
|
|
#
|
|
# `import authelia` is the shared AD + Duo gate. OMITTING IT SILENTLY MAKES THE
|
|
# SERVICE PUBLIC. Every block below keeps it - there is no deliberate exception
|
|
# anywhere in this stack. (Forgejo omits it only because forward-auth breaks
|
|
# git clients; that reason does not apply to anything here.)
|
|
#
|
|
# DNS is not managed on this host. Each hostname needs an A record ->
|
|
# 20.211.144.151 before Caddy can issue a certificate. Ask Dan.
|
|
#
|
|
# Azure hairpin: LAN hosts cannot reach the VM public IP from inside the VNet.
|
|
# For an operator on cicore1 to reach ai.yokogawa.tech by hostname, the DC needs
|
|
# a pinpoint record -> 10.0.0.17, the same treatment influx.yokogawa.tech has.
|
|
# Raise this early - it is a dependency on someone else and will not surface
|
|
# until Phase 7.
|
|
#
|
|
# Add each block at the phase that needs it, not all at once. A hostname with a
|
|
# Caddyfile block and no Authelia rule is a hole.
|
|
# =============================================================================
|
|
|
|
# --- Phase 2 -----------------------------------------------------------------
|
|
lf.yokogawa.tech {
|
|
import authelia
|
|
reverse_proxy langfuse:3000
|
|
}
|
|
|
|
# --- Phase 5 -----------------------------------------------------------------
|
|
cube.yokogawa.tech {
|
|
import authelia
|
|
reverse_proxy cube:4000
|
|
}
|
|
|
|
# --- Phase 6 -----------------------------------------------------------------
|
|
api.yokogawa.tech {
|
|
import authelia
|
|
reverse_proxy ai-api:8000
|
|
}
|
|
|
|
# --- Phase 9 -----------------------------------------------------------------
|
|
# Document upload needs two changes to the block above. No new hostname: the
|
|
# upload endpoints live under /docs on the existing API.
|
|
#
|
|
# 1. ai-api must learn WHO is calling. Authelia returns Remote-User,
|
|
# Remote-Name, Remote-Email and Remote-Groups from the forward-auth
|
|
# subrequest; Caddy only passes them upstream if it is told to.
|
|
#
|
|
# CHECK ~/Caddyfile's shared `authelia` snippet FIRST. If it already sets
|
|
# copy_headers, this is done and duplicating it is a no-op at best. Do NOT
|
|
# edit the shared snippet to add it - every other service on the host
|
|
# imports it, and this is not the change to make on their behalf.
|
|
#
|
|
# If the snippet does not copy them, replace the Phase 6 block with:
|
|
#
|
|
# api.yokogawa.tech {
|
|
# forward_auth authelia:9091 {
|
|
# uri /api/verify?rd=https://auth.yokogawa.tech
|
|
# copy_headers Remote-User Remote-Name Remote-Email Remote-Groups
|
|
# }
|
|
# request_body {
|
|
# max_size 50MB # match MAX_UPLOAD_MB; Caddy refuses larger
|
|
# }
|
|
# reverse_proxy ai-api:8000
|
|
# }
|
|
#
|
|
# - matching the shared snippet's own forward_auth arguments, which must be
|
|
# read off the host rather than assumed from this comment.
|
|
#
|
|
# 2. ai-api trusts those headers only because nothing outside the proxy network
|
|
# can reach it. If anything here ever gains a published port, that trust is
|
|
# gone and the /docs endpoints are open to whoever can reach the port.
|
|
#
|
|
# The group restriction is NOT expressed here. It goes in the Authelia rule for
|
|
# ^/docs/.* (authelia/access-rules.md), and is re-checked in ai-api. A Caddy
|
|
# matcher would be a third place to keep in step, and the first to be forgotten.
|
|
|
|
# --- Phase 7 -----------------------------------------------------------------
|
|
ai.yokogawa.tech {
|
|
import authelia
|
|
reverse_proxy ai-web:80
|
|
}
|