What to log
Request timestamp, user ID, prompt, response, model version, latency, cost, safety classifier scores, tool calls, IPs.
Advertisement
PII considerations
Log full prompt? PII risk. Options: hash PII, sample-log, encrypt at rest. Balance investigability + privacy.
Advertisement
Retention
Regulator + business need dictate. Financial: 7 years. Healthcare: variable. Delete when not needed.
Query access
SIEM (Splunk, Datadog). Purpose-built LLM observability (LangSmith, Helicone, Arize).