yau-plant-assistant/api/config.py
Claude fd85e62ebf Add the document library screens: upload, review, withdraw, restore
Phase 9's operator path, built ahead of Phase 8 at the customer's direction and
live at api.yokogawa.tech/documents. Upload, convert, review, approve, withdraw
and restore. The pool screen is explicitly out of scope.

Served by ai-api rather than ai-web, and mounted at /documents rather than
/docs. ai.yokogawa.tech is SCADA-only since 2026-08-28 and passes through no
Authelia, so it has no identity to record; publishers arrive on
api.yokogawa.tech where the forward-auth headers still do. /docs stays with
Swagger, which the customer is keeping - two things under one prefix with two
different access policies is what gets misread during a later edit.

Conversion is text extraction, not document parsing: pypdf, python-docx and
openpyxl. Docling would be better at this and pulls torch, which lin001 has
neither the memory to install nor the business running next to the demo plant's
PLC. The cost is real - no layout, no table structure, and a scan cannot be read
at all, so it is refused rather than stored empty. It is acceptable only because
the converted text is shown to a person before the document can be cited, which
is the same safety net the design already required for the header. convert.py is
the one file to change if that stops being true.

Chunking is mirrored from ingest.py rather than shared, because the two live in
different images. They must stay identical: if they drift, the same document
chunks differently depending on who loaded it, and the assistant answers or
fails to answer depending on that. The step-sequence rule is locked by a test.

Identity is self-asserted for the demo - the actor is typed on the form, which
section 16 forbids, and the publisher list is one name with no password. Rows are
written as `demo:<name>` with actor_groups = 'DEMO-UNVERIFIED' so that when real
auth goes on, a name somebody typed stays tellable from a name Authelia proved.
doc_actions cannot be deleted from, so an ambiguity there would be permanent.

Two rules the code enforces rather than documents: uploading is open to anyone
who reaches the page, because uploading changes nothing an operator can see -
approving does, and that is what is gated; and an empty publisher list means
nobody, not everybody.

Verified on the host end to end: withdraw as a non-publisher 403s, with a short
reason 400s, and as admin flips 5 chunks and writes a complete audit row;
restore puts them back and keeps both rows. The corpus is unchanged afterwards.

Requirements are split so the document dependencies install in their own layer -
a change there costs four small wheels instead of re-resolving fastapi,
langgraph and langfuse on a 2 vCPU shared host.

The five divergences from section 16 are recorded in section 14. The one with
teeth: files published through the UI stay in the inbox, so `ai-ingest --all`
cannot see them and the two paths must not be used on the same document.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 14:13:17 +10:00

191 lines
8.8 KiB
Python

"""Configuration, read once from the environment.
Values come from ~/ai/api.env on lin001 (0600, not in Git). .env.example in the
repo root lists every key with no values. Nothing here has a secret default,
and nothing here is ever logged.
"""
from __future__ import annotations
import os
from functools import lru_cache
from pydantic import BaseModel
class Settings(BaseModel):
# --- imh (pending) -----------------------------------------------------
use_fixtures: bool = True
# --- local -------------------------------------------------------------
pghost: str = "pg-ai"
pgport: int = 5432
pgdatabase: str = "plant"
pguser: str = "agent_ro"
pgpassword: str = ""
# --- Azure OpenAI ------------------------------------------------------
azure_openai_endpoint: str = ""
azure_openai_api_key: str = ""
azure_openai_api_version: str = ""
chat_deployment: str = "" # flagship - final prose only
cheap_deployment: str = "" # classifier, entities, tool selection
embed_deployment: str = "" # text-embedding-3-small
# --- no-LLM stub mode --------------------------------------------------
# Off by default and must stay that way. See api/stub.py for what it does
# and, more importantly, what it does not prove.
no_llm_stub: bool = False
# --- behaviour ---------------------------------------------------------
classifier_confidence_threshold: float = 0.7
site_timezone: str = "Australia/Sydney"
# Where the CONTROLLED copy of a procedure actually lives. A site fact, not
# a document fact - it is the same for every document, so it is not in
# doc_chunks. It is here rather than left to the model because sending an
# operator to a controlled copy that does not exist is worse than telling
# them nothing. The default says who to ask, which is always true.
controlled_copy_location: str = (
"Ask the WRPS document controller - this assistant does not hold "
"controlled copies."
)
max_rows_returned: int = 5000
query_timeout_seconds: int = 30
max_output_tokens: int = 1200
# --- Documents (Phase 9) -----------------------------------------------
# Two extra roles, and the split is the safety boundary, not bookkeeping.
#
# uploads_rw the queue, and withdraw. db/005 carries a trigger that
# refuses to let this role set superseded = FALSE, so it can
# make a document LESS visible and never more.
# ingest_rw writes doc_chunks: approve and restore. Anything that makes
# a document citable uses this one.
#
# In the design these are two components - ai-api and ai-docs-worker, the
# second with no HTTP surface. There is no worker here, so both connections
# live in this process. That is a real deviation and it is recorded in
# BUILD-AI-CONTAINERS.md S14: the trigger still blocks the web role, but the
# process holding uploads_rw also holds ingest_rw, so the separation is now
# a code boundary rather than a deployment one. Splitting the worker out
# later is a config change and a compose file, not a redesign.
uploads_db_user: str = "uploads_rw"
uploads_db_password: str = ""
ingest_db_user: str = "ingest_rw"
ingest_db_password: str = ""
# Where uploaded files land before review. Never inside /datadisk/ai-docs:
# that folder means "the documents this plant runs on", and an unreviewed
# upload is not one of those.
docs_inbox: str = "/inbox"
max_upload_mb: int = 25
# --- Document identity -------------------------------------------------
# "authelia" - the actor is Remote-User from the forward-auth headers, and
# a missing header is a 401. This is the design.
# "demo" - the actor is TYPED BY THE PERSON on the form. Self-asserted,
# unverified, and exactly what the design forbids ("identity
# comes from the headers, never from the request body").
#
# Demo mode is off unless asked for, the screens carry a banner saying the
# name is unverified, and every row it writes is stored as `demo:<name>`
# with actor_groups = 'DEMO-UNVERIFIED'. That marking is the point: when
# real auth goes on, a self-asserted audit row must still be tellable from
# an authenticated one. Without it they are indistinguishable forever.
doc_identity_mode: str = "authelia"
# Who may change anything. In the design this is the AD group
# AI_DocPublishers; with no AD group available it is a name list, swapped
# for the group later with one config change. Empty means nobody, and the
# API fails closed - approve, withdraw and restore all 403.
doc_publishers: tuple[str, ...] = ()
# --- Cube --------------------------------------------------------------
cubejs_api_url: str = "http://cube:4000/cubejs-api/v1"
cubejs_api_secret: str = ""
# --- Langfuse ----------------------------------------------------------
langfuse_host: str = "http://langfuse:3000"
langfuse_public_key: str = ""
langfuse_secret_key: str = ""
def dsn(self) -> str:
"""Postgres DSN. Never log the result - it carries the password."""
return (
f"postgresql://{self.pguser}:{self.pgpassword}"
f"@{self.pghost}:{self.pgport}/{self.pgdatabase}"
)
def docs_dsn(self, role: str) -> str:
"""DSN for one of the document roles. Never log the result.
`role` is "uploads" or "ingest" - spelled out at every call site rather
than defaulted, because picking the wrong one is the difference between
a web request that can withdraw a document and one that can publish it.
"""
user, password = {
"uploads": (self.uploads_db_user, self.uploads_db_password),
"ingest": (self.ingest_db_user, self.ingest_db_password),
}[role]
return (
f"postgresql://{user}:{password}"
f"@{self.pghost}:{self.pgport}/{self.pgdatabase}"
)
def redacted(self) -> dict[str, object]:
"""Safe to log and safe to return from /healthz."""
secret = {"pgpassword", "azure_openai_api_key", "cubejs_api_secret",
"langfuse_secret_key", "uploads_db_password",
"ingest_db_password"}
return {
k: ("set" if v else "unset") if k in secret else v
for k, v in self.model_dump().items()
}
@lru_cache
def settings() -> Settings:
env = os.environ
return Settings(
use_fixtures=env.get("USE_FIXTURES", "true").lower() == "true",
pghost=env.get("PGHOST", "pg-ai"),
pgport=int(env.get("PGPORT", "5432")),
pgdatabase=env.get("PGDATABASE", "plant"),
pguser=env.get("PGUSER", "agent_ro"),
pgpassword=env.get("PGPASSWORD", ""),
no_llm_stub=env.get("NO_LLM_STUB", "false").lower() == "true",
azure_openai_endpoint=env.get("AZURE_OPENAI_ENDPOINT", ""),
azure_openai_api_key=env.get("AZURE_OPENAI_API_KEY", ""),
azure_openai_api_version=env.get("AZURE_OPENAI_API_VERSION", ""),
chat_deployment=env.get("CHAT_DEPLOYMENT", ""),
cheap_deployment=env.get("CHEAP_DEPLOYMENT", ""),
embed_deployment=env.get("EMBED_DEPLOYMENT", ""),
classifier_confidence_threshold=float(
env.get("CLASSIFIER_CONFIDENCE_THRESHOLD", "0.7")
),
site_timezone=env.get("SITE_TIMEZONE", "Australia/Sydney"),
controlled_copy_location=env.get(
"CONTROLLED_COPY_LOCATION",
"Ask the WRPS document controller - this assistant does not hold "
"controlled copies.",
),
max_rows_returned=int(env.get("MAX_ROWS_RETURNED", "5000")),
query_timeout_seconds=int(env.get("QUERY_TIMEOUT_SECONDS", "30")),
max_output_tokens=int(env.get("MAX_OUTPUT_TOKENS", "1200")),
uploads_db_user=env.get("UPLOADS_DB_USER", "uploads_rw"),
uploads_db_password=env.get("UPLOADS_DB_PASSWORD", ""),
ingest_db_user=env.get("INGEST_DB_USER", "ingest_rw"),
ingest_db_password=env.get("INGEST_DB_PASSWORD", ""),
docs_inbox=env.get("DOCS_INBOX", "/inbox"),
max_upload_mb=int(env.get("MAX_UPLOAD_MB", "25")),
doc_identity_mode=env.get("DOC_IDENTITY_MODE", "authelia").lower(),
# Comma-separated. Blank entries dropped so a trailing comma in an env
# file cannot silently authorise "".
doc_publishers=tuple(
n.strip() for n in env.get("DOC_PUBLISHERS", "").split(",") if n.strip()
),
cubejs_api_url=env.get("CUBEJS_API_URL", "http://cube:4000/cubejs-api/v1"),
cubejs_api_secret=env.get("CUBEJS_API_SECRET", ""),
langfuse_host=env.get("LANGFUSE_HOST", "http://langfuse:3000"),
langfuse_public_key=env.get("LANGFUSE_PUBLIC_KEY", ""),
langfuse_secret_key=env.get("LANGFUSE_SECRET_KEY", ""),
)