> ## Documentation Index
> Fetch the complete documentation index at: https://f4c7a9e2d8b1-docs.tenzo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Audit Diff System

> Architecture and usage guide for the audit diff logging system

<Warning>
  **Restricted Access**: This documentation is only accessible to @tenzo.ai and @salv.ai email addresses.
</Warning>

## Overview

The Audit Diff System records **what changed** — not just that something changed. When an admin updates a setting, template, or integration rule, the system captures a before/after snapshot and produces a human-readable diff string. This diff is stored in the `audit_events` table and rendered in the **Audit Events** page with red (removed) and green (added) highlighting, including word-level change detection.

## Storage

The diff string is stored as plain text in the `details` column (`Text`, nullable) on the `audit_events` table. Structured metadata like `template_id`, `template_type`, and `action` goes in the `event_metadata` column (`JSONB`). The key columns:

| Column           | Type         | What it stores                                            |
| ---------------- | ------------ | --------------------------------------------------------- |
| `event_type`     | `String(64)` | Category — e.g., `SETTINGS_UPDATED`                       |
| `entity_type`    | `String(32)` | What was changed — e.g., `ORGANIZATION`                   |
| `entity_id`      | `UUID`       | ID of the changed entity                                  |
| `details`        | `Text`       | The diff string (plain text with `- `/`+ ` line prefixes) |
| `event_metadata` | `JSONB`      | Structured context — template type, action, IDs           |
| `user_id`        | `UUID`       | Who made the change                                       |

The diff string format uses a simple convention:

```
Field Label
- old value
+ new value

Another Field
- removed line
+ added line
```

Lines prefixed with `- ` are removals, `+ ` are additions, and non-prefixed lines are section headers (field names). This format is generated by `difflib.ndiff` on the backend and parsed line-by-line on the frontend.

## Frontend Rendering

`DiffRenderer.tsx` receives the raw `details` string from `GET /org/{org_id}/audit-events` and parses it line-by-line:

| Line pattern         | Rendered as                                  |
| -------------------- | -------------------------------------------- |
| Starts with `+ `     | Green background, dark green text (addition) |
| Starts with `- `     | Red background, dark red text (removal)      |
| Non-empty, no prefix | Bold section header (field name)             |
| Matches `id: ...`    | Dimmed metadata text                         |

**Word-level highlighting:** When a removed line (`- `) and an added line (`+ `) appear near each other (within 3 lines), the component pairs them and runs `diffChars` (from the `diff` npm library) to highlight exactly which characters changed within the line. Changed words get a darker background shade — darker green for additions, darker red for removals.

## End-to-End Flow

```mermaid theme={null}
sequenceDiagram
    participant User
    participant Endpoint as API Endpoint
    participant DAO
    participant DiffEngine as Diff Engine
    participant AuditDB as audit_events table
    participant UI as DiffRenderer.tsx

    User->>Endpoint: Update request
    Endpoint->>DAO: Fetch old state
    DAO-->>Endpoint: Old object
    Endpoint->>DAO: Perform update
    DAO-->>Endpoint: Success
    Endpoint->>DAO: Fetch updated state
    DAO-->>Endpoint: New object
    Endpoint->>DiffEngine: build_diff(old, new, fields)
    DiffEngine-->>Endpoint: Diff string
    Endpoint->>AuditDB: Store event (details=diff string, event_metadata=JSONB)
    UI->>Endpoint: GET /org/{org_id}/audit-events
    Endpoint->>AuditDB: Query audit_events
    AuditDB-->>Endpoint: Events with details + metadata
    Endpoint-->>UI: AuditEventsListResponse
    UI->>UI: DiffRenderer parses lines, applies red/green + word highlighting
```

## Architecture

The system has three layers. The core engine knows nothing about data sources — adapters bridge the gap between your data format and the engine, and feature modules handle domain-specific logic.

```mermaid theme={null}
flowchart TD
    subgraph Layer1["Layer 1: Core Engine"]
        DE["diff_engine.py\nbuild_diff(old, new, fields) → str"]
    end

    subgraph Layer2["Layer 2: Adapters"]
        MD["model_diff.py\n(Pydantic models)"]
        SD["sqlalchemy_diff.py\n(SQLAlchemy models)"]
        CD["cosmos_config_diff.py\n(Cosmos dicts)"]
    end

    subgraph Layer3["Layer 3: Feature Modules"]
        TD["template_diff.py"]
        ScD["script_diff.py"]
        ASD["admin_settings_diff.py"]
        IRD["integration_rule_diff.py"]
    end

    DE --> MD
    DE --> SD
    DE --> CD
    MD --> TD
    MD --> ScD
    SD --> ASD
    DE --> IRD

    style Layer1 fill:#e8f4f8,stroke:#169fff
    style Layer2 fill:#f0f8e8,stroke:#5cb85c
    style Layer3 fill:#fef8e8,stroke:#f0ad4e
```

## Layer Details

### Layer 1: Core Engine — `diff_engine.py`

A single pure function that does all the diffing:

```python theme={null}
def build_diff(old, new, fields: list[DiffField]) -> str | None
```

It takes a list of `DiffField` descriptors and two objects, reads values from both, and produces a diff string. It has no knowledge of Pydantic, SQLAlchemy, or Cosmos — it works with any object that supports attribute or key access.

### Layer 2: Adapters

Each adapter knows how to extract `DiffField` descriptors from its data source:

| Adapter                 | Data Source         | How fields are defined                                                                                                           |
| ----------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `model_diff.py`         | Pydantic models     | Reads `json_schema_extra={"diff_policy": {...}}` from model fields. All fields are diffed by default.                            |
| `sqlalchemy_diff.py`    | SQLAlchemy models   | Reads `info={"diff_label": "..."}` from column definitions. Only columns with `diff_label` are diffed (opt-in).                  |
| `cosmos_config_diff.py` | Cosmos config dicts | Reads from `UIConfigModel` classes. Converts values to display format via `to_ui_schema()` (e.g., `0.8` → `80` for percentages). |

### Layer 3: Feature Modules

These call the adapters and handle domain-specific logic:

| Module                     | What it diffs                                                           |
| -------------------------- | ----------------------------------------------------------------------- |
| `template_diff.py`         | All 8 template types (uses Pydantic adapter)                            |
| `script_diff.py`           | Campaign script fields + screening questions (uses Pydantic adapter)    |
| `admin_settings_diff.py`   | ATS integration config (uses SQLAlchemy adapter)                        |
| `integration_rule_diff.py` | Rules, actions, priorities (uses diff engine directly)                  |
| `template_audit.py`        | Not a diff module — shared helper that emits audit events for templates |

***

## How to Add Audit Logging

### Step 1: Choose Your Adapter

* **Pydantic model?** → Use `model_diff.py` (or `template_diff.py` if it's a template)
* **SQLAlchemy model?** → Use `sqlalchemy_diff.py` — add `info={"diff_label": "..."}` to columns you want diffed
* **Cosmos config dict?** → Use `cosmos_config_diff.py`
* **Something custom?** → Use `diff_engine.py` directly with a hand-built `DiffField` list (see `integration_rule_diff.py`)

### Step 2: Annotate Your Model

**Pydantic** — all fields are diffed automatically. Override behavior with `diff_policy`:

```python theme={null}
class MyTemplate(CosmosBaseModel):
    title: str = ""                    # Diffed, auto-labeled as "Title"
    instructions: str = ""             # Diffed, auto-labeled as "Instructions"
    internal_id: str = Field(          # Skipped
        default="",
        json_schema_extra={"diff_policy": {"skip": True}}
    )
    items: list[Item] = Field(         # List with key matching
        default_factory=list,
        json_schema_extra={"diff_policy": {"list_key": "id", "ordered": True}}
    )
```

**SQLAlchemy** — only columns with `diff_label` are diffed:

```python theme={null}
start_stage: Mapped[str | None] = mapped_column(
    String(255), nullable=True, info={"diff_label": "Start Stage"}
)
# Columns without diff_label are ignored by the diff system
```

### Step 3: Wire Into the Endpoint

Always wrap audit code in `try/except` — audit logging should never crash the main operation.

```python theme={null}
# UPDATE
old = await dao.fetch_item(item_id=id, ...)    # 1. Capture old state
await dao.patch_item(item_id=id, ...)           # 2. Do the update
try:                                            # 3. Diff and audit
    updated = await dao.fetch_item(item_id=id, ...)
    details = build_my_diff(old, updated)
    await emit_template_audit(org_id=..., user_id=..., ...)
except Exception:
    logger.warning("Failed to create audit event for ...")
```

```python theme={null}
# CREATE
await dao.create_item(item=new_item, ...)       # 1. Create
try:                                            # 2. Diff (old=None) and audit
    details = build_my_diff(None, new_item)
    await emit_template_audit(org_id=..., action=AuditAction.CREATE, ...)
except Exception:
    logger.warning("Failed to create audit event for ...")
```

```python theme={null}
# DELETE
old = await dao.fetch_item(item_id=id, ...)     # 1. Capture before delete
await dao.delete_item(item_id=id, ...)          # 2. Delete
try:                                            # 3. Diff (new=None) and audit
    details = build_my_diff(old, None)
    await emit_template_audit(org_id=..., action=AuditAction.DELETE, ...)
except Exception:
    logger.warning("Failed to create audit event for ...")
```

### Step 4: Add Auth Context

Most audit endpoints need `auth` to get `user_id` and `org_id`. If the endpoint doesn't already have it, add it as a parameter:

```python theme={null}
auth: Annotated[AuthContext, Depends(get_auth_context)]
```

***

## DiffField Reference

`DiffField` controls how each field is compared and displayed:

| Property         | Type               | Default  | Description                                                                                                            |
| ---------------- | ------------------ | -------- | ---------------------------------------------------------------------------------------------------------------------- |
| `name`           | `str`              | required | Field name to read from the object                                                                                     |
| `label`          | `str \| None`      | `None`   | Human-readable label. If `None`, auto-generated from `name` (`my_field` → "My Field", `callIntro` → "Call Intro")      |
| `skip`           | `bool`             | `False`  | Skip this field entirely (for internal IDs, etc.)                                                                      |
| `ordered`        | `bool \| None`     | `None`   | For lists: does order matter? `True` = compare by position, `False` = sort before comparing, `None` = treat as ordered |
| `indexed`        | `bool`             | `False`  | For lists: show `[1]`, `[2]` prefixes                                                                                  |
| `list_key`       | `str \| None`      | `None`   | For list-of-models: match items by this field instead of position (e.g., `"id"`)                                       |
| `annotation`     | `type \| None`     | `None`   | Python type hint, used to pick formatting (str → text diff, bool → true/false, int/float → number)                     |
| `nested_builder` | `callable \| None` | `None`   | For nested Pydantic models: function to recursively diff the nested object                                             |

***

## Coverage Matrix

| Area                                                            | Create | Update | Delete    |
| --------------------------------------------------------------- | ------ | ------ | --------- |
| ATS Integration Config                                          | —      | ✅      | —         |
| Dynamic ATS Config (Cosmos)                                     | —      | ✅      | ✅ (reset) |
| Integration Rules                                               | ✅      | ✅      | ✅         |
| Rule Priorities                                                 | —      | ✅      | —         |
| Campaign Scripts                                                | —      | ✅      | —         |
| Job SQL Fields (rename, owner, folder, results email, location) | —      | ✅      | —         |
| Background Info Templates                                       | ✅      | ✅      | ✅         |
| Script Generation Templates                                     | ✅      | ✅      | ✅         |
| Summary Templates                                               | ✅      | ✅      | ✅         |
| Thank You Email Templates                                       | ✅      | ✅      | ✅         |
| Rejection Email Templates                                       | ✅      | ✅      | ✅         |
| Custom Booking Outro Templates                                  | ✅      | ✅      | ✅         |
| Redaction Templates                                             | ✅      | ✅      | ✅         |
| Suggested Follow Up Templates                                   | ✅      | ✅      | ✅         |

***

## File Index

| File                             | Purpose                                                                         |
| -------------------------------- | ------------------------------------------------------------------------------- |
| `audit/diff_engine.py`           | Core engine — `build_diff()`, `DiffField`, formatting                           |
| `audit/model_diff.py`            | Pydantic adapter — extracts `DiffField` from model fields                       |
| `audit/sqlalchemy_diff.py`       | SQLAlchemy adapter — extracts `DiffField` from column `info`                    |
| `audit/cosmos_config_diff.py`    | Cosmos adapter — extracts from `UIConfigModel` classes, converts display values |
| `audit/script_diff.py`           | Campaign script diff (screening questions + model fields)                       |
| `audit/integration_rule_diff.py` | Integration rule CRUD + actions + priority reordering                           |
| `audit/template_diff.py`         | Diff builders for all 8 template types                                          |
| `audit/template_audit.py`        | `emit_template_audit()` — shared helper to create audit events                  |
| `audit/enums.py`                 | `AuditTemplateType`, `AuditAction`, `AuditEventType`, `AuditEntityType`         |
| `audit/diff_helpers/`            | Low-level formatters: text diff, line diff, bool/number/list formatting         |
| `ui/.../DiffRenderer.tsx`        | Frontend — renders diff strings with red/green + word highlighting              |