Audit Diff System

Restricted Access: This documentation is only accessible to @tenzo.ai and @salv.ai email addresses.

Overview

The Audit Diff System records what changed — not just that something changed. When an admin updates a setting, template, or integration rule, the system captures a before/after snapshot and produces a human-readable diff string. This diff is stored in the audit_events table and rendered in the Audit Events page with red (removed) and green (added) highlighting, including word-level change detection.

Storage

The diff string is stored as plain text in the details column (Text, nullable) on the audit_events table. Structured metadata like template_id, template_type, and action goes in the event_metadata column (JSONB). The key columns:

Column	Type	What it stores
`event_type`	`String(64)`	Category — e.g., `SETTINGS_UPDATED`
`entity_type`	`String(32)`	What was changed — e.g., `ORGANIZATION`
`entity_id`	`UUID`	ID of the changed entity
`details`	`Text`	The diff string (plain text with `-` /`+` line prefixes)
`event_metadata`	`JSONB`	Structured context — template type, action, IDs
`user_id`	`UUID`	Who made the change

The diff string format uses a simple convention:

Field Label
- old value
+ new value

Another Field
- removed line
+ added line

Lines prefixed with - are removals, + are additions, and non-prefixed lines are section headers (field names). This format is generated by difflib.ndiff on the backend and parsed line-by-line on the frontend.

Frontend Rendering

DiffRenderer.tsx receives the raw details string from GET /org/{org_id}/audit-events and parses it line-by-line:

Line pattern	Rendered as
Starts with `+`	Green background, dark green text (addition)
Starts with `-`	Red background, dark red text (removal)
Non-empty, no prefix	Bold section header (field name)
Matches `id: ...`	Dimmed metadata text

Word-level highlighting: When a removed line (- ) and an added line (+ ) appear near each other (within 3 lines), the component pairs them and runs diffChars (from the diff npm library) to highlight exactly which characters changed within the line. Changed words get a darker background shade — darker green for additions, darker red for removals.

End-to-End Flow

Architecture

The system has three layers. The core engine knows nothing about data sources — adapters bridge the gap between your data format and the engine, and feature modules handle domain-specific logic.

Layer Details

Layer 1: Core Engine — `diff_engine.py`

A single pure function that does all the diffing:

def build_diff(old, new, fields: list[DiffField]) -> str | None

It takes a list of DiffField descriptors and two objects, reads values from both, and produces a diff string. It has no knowledge of Pydantic, SQLAlchemy, or Cosmos — it works with any object that supports attribute or key access.

Layer 2: Adapters

Each adapter knows how to extract DiffField descriptors from its data source:

Adapter	Data Source	How fields are defined
`model_diff.py`	Pydantic models	Reads `json_schema_extra={"diff_policy": {...}}` from model fields. All fields are diffed by default.
`sqlalchemy_diff.py`	SQLAlchemy models	Reads `info={"diff_label": "..."}` from column definitions. Only columns with `diff_label` are diffed (opt-in).
`cosmos_config_diff.py`	Cosmos config dicts	Reads from `UIConfigModel` classes. Converts values to display format via `to_ui_schema()` (e.g., `0.8` → `80` for percentages).

Layer 3: Feature Modules

These call the adapters and handle domain-specific logic:

Module	What it diffs
`template_diff.py`	All 16 template types (uses Pydantic adapter)
`script_diff.py`	Campaign script fields + screening questions (uses Pydantic adapter)
`campaign_audit_types.py`	Job and Job Template settings (uses Pydantic adapter)
`org_settings_audit.py`	Organization settings and Question Bank changes
`admin_settings_diff.py`	ATS integration config (uses SQLAlchemy adapter)
`integration_rule_diff.py`	Rules, actions, priorities (uses diff engine directly)
`template_audit.py`	Not a diff module — shared helper that emits audit events for templates

How to Add Audit Logging

Step 1: Choose Your Adapter

Pydantic model? → Use model_diff.py (or template_diff.py if it’s a template)
SQLAlchemy model? → Use sqlalchemy_diff.py — add info={"diff_label": "..."} to columns you want diffed
Cosmos config dict? → Use cosmos_config_diff.py
Something custom? → Use diff_engine.py directly with a hand-built DiffField list (see integration_rule_diff.py)

Step 2: Annotate Your Model

Pydantic — all fields are diffed automatically. Override behavior with diff_policy:

class MyTemplate(CosmosBaseModel):
    title: str = ""                    # Diffed, auto-labeled as "Title"
    instructions: str = ""             # Diffed, auto-labeled as "Instructions"
    internal_id: str = Field(          # Skipped
        default="",
        json_schema_extra={"diff_policy": {"skip": True}}
    )
    items: list[Item] = Field(         # List with key matching
        default_factory=list,
        json_schema_extra={"diff_policy": {"list_key": "id", "ordered": True}}
    )

SQLAlchemy — only columns with diff_label are diffed:

start_stage: Mapped[str | None] = mapped_column(
    String(255), nullable=True, info={"diff_label": "Start Stage"}
)
# Columns without diff_label are ignored by the diff system

Step 3: Wire Into the Endpoint

Audit logging must never crash or delay the primary operation. On request paths, capture the old state, complete the primary write, then schedule the audit emitter with create_task_with_error_logging. Keep diff construction inside the async emitter so failures are reported by the task wrapper instead of escaping from the endpoint. If a call site must await audit persistence, wrap that awaited call in try/except instead.

# UPDATE
old = await dao.fetch_item(item_id=id, ...)       # 1. Capture old state
new = build_updated_model(old, validated_payload) # 2. Build the validated new state
await dao.patch_item(item_id=id, ...)             # 3. Complete the primary write
create_task_with_error_logging(                   # 4. Diff and audit asynchronously
    emit_my_audit(old=old, new=new, org_id=..., user_id=...),
    error_message="Audit event creation failed for ...",
)

# CREATE
await dao.create_item(item=new_item, ...)        # 1. Complete the primary write
create_task_with_error_logging(                  # 2. Diff (old=None) and audit
    emit_my_audit(old=None, new=new_item, org_id=..., user_id=...),
    error_message="Audit event creation failed for ...",
)

# DELETE
old = await dao.fetch_item(item_id=id, ...)     # 1. Capture before delete
await dao.delete_item(item_id=id, ...)          # 2. Complete the primary write
create_task_with_error_logging(                 # 3. Diff (new=None) and audit
    emit_my_audit(old=old, new=None, org_id=..., user_id=...),
    error_message="Audit event creation failed for ...",
)

Step 4: Add Auth Context

Most audit endpoints need auth to get user_id and org_id. If the endpoint doesn’t already have it, add it as a parameter:

auth: Annotated[AuthContext, Depends(get_auth_context)]

DiffField Reference

DiffField controls how each field is compared and displayed:

Property	Type	Default	Description
`name`	`str`	required	Field name to read from the object
`label`	`str \| None`	`None`	Human-readable label. If `None`, auto-generated from `name` (`my_field` → “My Field”, `callIntro` → “Call Intro”)
`skip`	`bool`	`False`	Skip this field entirely (for internal IDs, etc.)
`ordered`	`bool \| None`	`None`	For lists: does order matter? `True` = compare by position, `False` = sort before comparing, `None` = treat as ordered
`indexed`	`bool`	`False`	For lists: show `[1]`, `[2]` prefixes
`list_key`	`str \| None`	`None`	For list-of-models: match items by this field instead of position (e.g., `"id"`)
`annotation`	`type \| None`	`None`	Python type hint, used to pick formatting (str → text diff, bool → true/false, int/float → number)
`nested_builder`	`callable \| None`	`None`	For nested Pydantic models: function to recursively diff the nested object

Coverage Matrix

Area	Create	Update	Delete
ATS Integration Config	—	✅	—
Dynamic ATS Config (Cosmos)	—	✅	✅ (reset)
Integration Rules	✅	✅	✅
Rule Priorities	—	✅	—
User Invitations	✅	—	—
User Role Assignments	—	✅	—
Role Permission Sets	—	✅	—
Campaign Scripts	—	✅	—
Job/Campaign Settings (Cosmos, including Job Templates)	—	✅	—
Organization Settings	—	✅	—
Organization Question Bank	—	✅	—
Job SQL Fields (rename, owner, folder, results email, location)	—	✅	—
Background Info Templates	✅	✅	✅
Script Generation Templates	✅	✅	✅
Summary Templates	✅	✅	✅
Thank You Email Templates	✅	✅	✅
Rejection Email Templates	✅	✅	✅
Custom Booking Outro Templates	✅	✅	✅
Redaction Templates	✅	✅	✅
Suggested Follow Up Templates	✅	✅	✅
Note Presets / Templates	✅	✅	✅
Flagged Event Templates	✅	✅	✅
Candidate Question Extraction Templates	✅	✅	✅
Question Constraints	✅	✅	✅
Meeting Note ATS Summary Sync	✅	—	—
Intake ATS Job Creation	✅	—	—
Intake ATS Job Fields Sync	—	✅	—
Candidate Custom Fields ATS Sync	—	✅	—
ATS Stage / Disposition Move	—	✅	—
ATS Post-Call Results Writeback	—	✅	—
ATS Call Stop Note Post	✅	—	—
ATS Candidate Application Creation	✅	—	—
ATS Custom Fields / Consent Writeback	—	✅	—

File Index

File	Purpose
`audit/diff_engine.py`	Core engine — `build_diff()`, `DiffField`, formatting
`audit/model_diff.py`	Pydantic adapter — extracts `DiffField` from model fields
`audit/sqlalchemy_diff.py`	SQLAlchemy adapter — extracts `DiffField` from column `info`
`audit/cosmos_config_diff.py`	Cosmos adapter — extracts from `UIConfigModel` classes, converts display values
`audit/script_diff.py`	Campaign script diff (screening questions + model fields)
`campaigns/campaign_audit_types.py`	Job and Job Template settings diff and audit emission
`audit/org_settings_audit.py`	Organization settings and Question Bank audit emission
`audit/integration_rule_diff.py`	Integration rule CRUD + actions + priority reordering
`audit/user_permission_audit.py`	User invitation, role assignment, and role-permission audit helpers
`audit/template_diff.py`	Diff builders for all 16 template types
`audit/template_audit.py`	`emit_template_audit()` — shared helper to create audit events
`audit/meeting_note_audit.py`	Audit event emitters for meeting note ATS writes (summary sync, job create, field sync)
`audit/ats_audit.py`	Audit event emitters for ATS writes (stage moves, call results, notes, candidate apps, field updates)
`audit/enums.py`	`AuditTemplateType`, `AuditAction`, `AuditEventType`, `AuditEntityType`
`audit/diff_helpers/`	Low-level formatters: text diff, line diff, bool/number/list formatting
`ui/.../DiffRenderer.tsx`	Frontend — renders diff strings with red/green + word highlighting

User and Permission Events

User-management auditing records the administrator who initiated each change in user_id and the affected user or role in the audit entity and metadata.

Event	Emitted when	Details
`USER_INVITED`	A Stytch `member.created` webhook creates the invited local user	Invited email, assigned role, and whether the invite came from the single or bulk flow
`ROLE_CHANGED`	An administrator assigns, replaces, or clears a user’s role	Target user and previous/new role names
`ROLE_PERMISSIONS_CHANGED`	An administrator changes the permissions assigned to an editable role	Role name and previous/new permission keys

The invite endpoint stores the inviting administrator’s user ID and invite source in server-controlled Stytch trusted metadata. The member.created webhook emits the event after the local user exists, allowing entity_type=USER and entity_id to reference a real user. Duplicate webhook deliveries do not emit duplicate invitation events because the existing-user guards return before emission. Audit failures remain non-blocking: a successful invitation or permission update is not rolled back if its audit event cannot be written.

ATS Write Actions

ATS auditing captures outbound write actions to third-party ATS systems (dispositions, results, notes, field extractions, application creation). High-frequency background polling loops and read operations are intentionally excluded to keep the audit log focused on user- and decision-impacting mutation events.

Event	Emitted when	Details
`ATS_STAGE_CHANGED`	Candidate stage or disposition changes in ATS via system action or recruiter action	Target stage name, reason, actor type (system vs recruiter), candidate/job IDs
`ATS_CALL_RESULTS_WRITTEN`	Post-call scorecard/results writeback succeeds	Candidate, campaign, interview, call status, scores
`ATS_NOTE_POSTED`	Notes or call-stop summary posted to candidate record in ATS	Note preview, candidate, campaign, interview IDs
`ATS_APPLICATION_CREATED`	Application created in ATS for an existing or ingested candidate	Application ID, candidate name/ID, ATS provider
`ATS_FIELD_UPDATES_WRITTEN`	Custom fields or consent flags written back to candidate in ATS	Extracted field names, candidate/campaign IDs

Every ATS audit emitter uses safe_emit_audit_event and resolves entity references (CAMPAIGN, CANDIDATE, INTERVIEW) to provide clickable links in the audit UI.

Internal Documentation

Modules

Fraud Risk

Outreach

ATS Implementations

Overview

Storage

Frontend Rendering

End-to-End Flow

Architecture

Layer Details

Layer 1: Core Engine — `diff_engine.py`

Layer 2: Adapters

Layer 3: Feature Modules

How to Add Audit Logging

Step 1: Choose Your Adapter

Step 2: Annotate Your Model

Step 3: Wire Into the Endpoint

Step 4: Add Auth Context

DiffField Reference

Coverage Matrix

File Index

User and Permission Events

ATS Write Actions

​Overview

​Storage

​Frontend Rendering

​End-to-End Flow

​Architecture

​Layer Details

​Layer 1: Core Engine — diff_engine.py

​Layer 2: Adapters

​Layer 3: Feature Modules

​How to Add Audit Logging

​Step 1: Choose Your Adapter

​Step 2: Annotate Your Model

​Step 3: Wire Into the Endpoint

​Step 4: Add Auth Context

​DiffField Reference

​Coverage Matrix

​File Index

​User and Permission Events

​ATS Write Actions

Overview

Storage

Frontend Rendering

End-to-End Flow

Architecture

Layer Details

Layer 1: Core Engine — `diff_engine.py`

Layer 2: Adapters

Layer 3: Feature Modules

How to Add Audit Logging

Step 1: Choose Your Adapter

Step 2: Annotate Your Model

Step 3: Wire Into the Endpoint

Step 4: Add Auth Context

DiffField Reference

Coverage Matrix

File Index

User and Permission Events

ATS Write Actions