> ## Documentation Index
> Fetch the complete documentation index at: https://f4c7a9e2d8b1-docs.tenzo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Fraud risk operations

> Turning fraud risk on for a customer, tuning it, and checking that the signals are alive

<Warning>
  **Restricted Access**: This documentation is only accessible to @tenzo.ai and @salv.ai email addresses.
</Warning>

This page covers rollout, tuning, and health checks. For what the feature does and what each signal
checks, see [Fraud risk](/internal/fraud-risk). For the full engineering runbook, including exact
function names, SQL, and migration ordering, see `server/fraud_risk/OPERATIONS.md`.

## Turning it on for a customer

Grading and showing are separate steps, and they can be weeks apart. Switching evaluation on starts
grading new applicants and storing the findings; a recruiter sees a finding only once the job
setting, the org default setting and `fraud_signals_tab` are all on. Nothing is re-evaluated when
that flag flips, so whatever was collected in the meantime appears immediately and costs nothing.
Support users in Full Access mode see an org's findings whether or not that org has opted in, which
is how support reads an org exactly as it will look once the flag goes on.

Four things are independent, and all four matter:

1. **The job setting.** `FraudRiskSettings.enabled`, set on the job or as the org default, and off
   until someone turns it on. `enable_fraud_risk_evaluation` turns it on across an org (dry run
   first; it skips any job that already auto-closes at a tier, since that job would start closing
   applications on its first new applicant). Nothing is evaluated or stored while it is off, and it
   is also one of the three switches that must be on before findings are shown. It lives in Cosmos
   as part of the job's settings document, not in Postgres, so no SQL query will find it.
2. **Provider credentials**, per environment rather than per org. Each is named in the configuration
   table in `server/fraud_risk/OPERATIONS.md`. A missing credential silently disables the signals that
   need it: they report themselves unevaluated rather than guessing.
3. **The feature flags**, per org, set directly in `org_feature_flags`. There is no admin UI. Allow up
   to 60 seconds per process for a change to take effect.
4. **The auto-reject threshold**, `auto_reject_minimum_fraud_risk_level`. Leave it unset at first.
   With no minimum configured nothing is ever closed automatically, so the grades can be reviewed
   before they start moving anyone's stage.

Run the contact fingerprint backfill **before** turning fraud evaluation on for an org. Fingerprints
are written as candidates are evaluated, so evaluating first leaves every collision that already
exists in the org invisible until each of those candidates happens to be evaluated again.

## The feature flags

| Flag                       | What it gates                                                                                                                                                                                                                                                                          |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `fraud_signals_tab`        | The reviewer-facing signals panel, the overall risk badge beside the candidate's name, the **Fraud Risk** columns in the per-job **Candidates** and organization-wide **Interviews** tables, the two candidate filters, the shared-contact drill-down, and the fraud grades in AI chat |
| `cross_org_fraud_matching` | The three cross-org shared-contact signals, on top of the in-org shared-contact signals                                                                                                                                                                                                |

Both default to off with no global default row, so an org without an explicit row inherits off.

**Cross-org matching is reciprocal.** An org matches only against other orgs that have also turned it
on, and a global default row does not count. That means turning it on for a single org achieves
nothing: it needs at least two participants, and each has to have agreed to it.

The seven scoring candidate-scoped signals, the four in-org shared-contact signals, and advisory
resume-history comparison are not flag-gated. They run from application creation for jobs with fraud
risk evaluation enabled. `fraud_signals_tab` gates whether a customer sees any of it, not whether it
is collected.

## Choosing an auto-reject threshold

The scoring cutoffs mean a single signal rarely escalates an application on its own: one High finding
scores 40, which buckets to Medium. Reaching High takes 80, so it takes two High findings, or one High
plus two Mediums.

This has a specific consequence when the shared-contact signals get turned on for an org that already
has an auto-reject threshold. A shared-contact finding adds 20 or 40 to an application that previously
scored below the cutoff, so **evaluating an org that has auto-reject
already configured can close applications that would not have been closed before**. The effect is
largest for orgs whose scored applications cluster just under a cutoff. Check that org's distribution
first:

```sql theme={null}
select cci.fraud_risk_level,
       count(*),
       round(avg(cci.fraud_risk_score)::numeric, 1) as avg_score
from candidate_campaign_info cci
join campaigns c on c.campaign_id = cci.campaign_id
join orgs o on o.org_id = c.org_id
where cci.fraud_risk_level is not null
  and o.name = '<org name>'
group by 1;
```

An application already scoring 40 or more is one High shared-contact finding (+40) away from the High
bucket. One at 60 or more is one Medium finding (+20) away.

The current level scores and cutoffs are preliminary and expected to be tuned against live data. All
signals weigh the same today; the per-signal weight table exists as the tuning surface but is empty.

## Results are cached, and re-deriving them costs money

A candidate-scoped signal is computed once and reused for the life of the candidate. Changing what a
signal emits therefore does not reach the rows already stored. Each signal carries a rule version, and
bumping it is what forces stored rows to be re-derived.

**Bump deliberately.** Re-derivation costs one real provider call (LLM, Twilio, or Google) per
candidate holding that signal. With roughly 24,000 stored rows each for the name-match and VOIP
signals, a careless bump is a large bill. Where the new value can be derived from the stored one, the
`refresh_fraud_signal_copy` job rewrites rows in place for free instead.

Display names and tooltips are read live off the evaluator, so editing a title or a description needs
no bump and reaches everywhere immediately.

Cross-application signals are recomputed on every evaluation, so the version gate never withholds
them.

## The re-evaluation queue

When a candidate joins or changes a shared-contact cluster, every other member of that cluster is
written to a durable queue. A worker claims up to 50 candidates every 30 seconds and re-evaluates
them. A failed re-evaluation is re-queued rather than lost, up to five attempts, after which the
candidate is dead-lettered with `error.kind: FraudReevalDeadLetter` and nothing retries it later.

## Health check: which signals are actually alive

The clearest signal of a misconfiguration is a check that has stopped writing rows. This query shows
every signal, how often it fires, and when it was last written:

```sql theme={null}
select signal_id,
       count(*) as rows,
       count(*) filter (where contributions = '[]'::jsonb) as passes,
       count(*) filter (where contributions <> '[]'::jsonb) as flagged,
       max(updated_at)::date as last_written
from candidate_fraud_signals
group by signal_id
order by rows desc;
```

As of 2026-08-25 in production:

| Signal              | Rows   | Passes | Flagged | Last written |
| ------------------- | ------ | ------ | ------- | ------------ |
| Name match          | 24,711 | 24,088 | 623     | 2026-08-25   |
| VOIP phone          | 24,572 | 20,774 | 3,798   | 2026-08-25   |
| Identity match      | 8,246  | 6,521  | 1,725   | 2026-08-25   |
| Address validation  | 3,330  | 3,252  | 78      | 2026-08-25   |
| Address consistency | 3,216  | 3,163  | 53      | 2026-08-25   |
| GitHub profile      | 628    | 596    | 32      | 2026-08-25   |
| Shared phone        | 8      | 7      | 1       | 2026-06-25   |
| Shared IP           | 8      | 0      | 8       | 2026-06-25   |
| LinkedIn profile    | 4      | 0      | 4       | 2026-06-25   |
| Shared address      | 2      | 1      | 1       | 2026-06-25   |
| Shared email        | 2      | 0      | 2       | 2026-06-25   |

Two things stand out and are worth reading correctly:

* **The shared-contact signals are near-zero because only Tenzo has the flag on.** That is expected,
  not a fault.
* **The LinkedIn signal was unconfigured when this production snapshot was captured.** Four rows,
  none since 2026-06-25, while its resume-sibling GitHub signal has 628 rows and is still being
  written. The AKS deployment now provisions the Serper API key it needs; after the next
  worker rollout, confirm new rows and check the provider-auth and provider-quota error kinds below.

## When a signal goes quiet

Check the row count above first, then Datadog. The LinkedIn signal binds a distinct `error.kind` per
failure mode so a monitor can route each to the right fix:

| `error.kind`                   | Meaning                                     | Fix                                                                                        |
| ------------------------------ | ------------------------------------------- | ------------------------------------------------------------------------------------------ |
| `LinkedInSearchNotConfigured`  | The search API key is unset                 | Provision the environment variable                                                         |
| `LinkedInSearchProviderAuth`   | The search provider rejected the credential | Check the key and the plan behind it                                                       |
| `LinkedInSearchProviderQuota`  | Rate limit or search credits exhausted      | Top up the balance, or slow the caller down                                                |
| `LinkedInSearchProviderFailed` | Anything else                               | Investigate; this is the class of breakage that left the signal dead when Bing was retired |

Not every failure path reports its bound kind: where an exception is attached, the exception type wins
and the log surfaces as an HTTP or JSON error instead. A monitor should match the kind **or** the
message.

A response that does not answer the query we sent is discarded rather than read as a profile that
does not exist. A not-found grades as High, so a cached or misrouted answer would otherwise become a
fraud finding against a real candidate.

## One-time jobs

Five jobs exist, all run with `python -m one-time-jobs.<name>` from `server/`, all defaulting to a
dry run that writes nothing.

| Job                                         | What it does                                                                                                           | Status                                                                                 |
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `backfill_contact_fingerprints`             | Writes contact fingerprints for existing candidates so shared-contact clusters are visible from the first evaluation   | Run before turning fraud evaluation on for an org                                      |
| `enable_fraud_risk_evaluation`              | Turns the per-job fraud risk setting on across an org's jobs, skipping any job that already carries an auto-close tier | Not yet run against production                                                         |
| `backfill_fraud_risk_evaluation`            | Grades the non-terminal applicants who predate fraud evaluation, ignoring the per-job setting and never auto-rejecting | Not yet run. Run `backfill_contact_fingerprints --apply` first, then one org at a time |
| `clear_vacuous_address_consistency_signals` | Deletes stored address-consistency passes that never actually compared anything                                        | Run against production on 2026-08-25; 109 rows removed, no flagged row touched         |
| `refresh_fraud_signal_copy`                 | Rewrites stored findings in place when the new copy is derivable from the old, avoiding a paid re-derivation           | **Do not run.** See Known gaps                                                         |

`backfill_fraud_risk_evaluation` spends real provider money, per candidate rather than per
application: a Twilio line-type lookup, a billable Twilio Identity Match on US and Puerto Rico
numbers, a second billable caller-name lookup when that match comes back flagged, two to four Gemini
calls, and Google geocoding where the resume carries an address. Its dry run prints those counts per
org before anything is spent, so read them before widening past a pilot.

Its ordering dependency is not advisory. Shared-contact findings are cached for the life of the
candidate and no later run re-derives them, so a candidate graded while
`candidate_contact_fingerprints` is still filling stores "nobody shares this value" permanently.
`--apply` refuses to start until nearly every fingerprint-eligible candidate in scope has a row.

## Dev tooling

An internal fraud data page can seed a candidate's signals to any outcome an evaluator actually
produces in production, or run a signal's real comparison over inputs supplied by hand. It is the way
to see the reviewer panel populated without waiting for a real fraudulent application.

Be aware that the seeder's preset copy has drifted from what the evaluators write, so copy verified
against a seeded candidate is not necessarily the copy production produces.

## Known gaps

The full list, with file and function references, is in the Known gaps sections of
`server/fraud_risk/OPERATIONS.md` and `ui/src/components/transcript/FRAUD_SIGNALS.md`. The items that
matter operationally:

* **The LinkedIn signal needs post-rollout verification.** It has never produced a finding in
  production: it ran on Bing until that API was retired, then on Google Programmable Search, which
  refuses the project the key belongs to. Confirm the search key is provisioned and that new
  LinkedIn signal rows are being written.
* **`refresh_fraud_signal_copy` would flatten four distinct address findings into one.** It rewrites
  every stored address-validation finding to a generic "Invalid address" (metadata, including the
  checked address, is kept), contradicting its own stated contract. It has never been run against
  production, so nothing has been flattened. Fix the job before running it.
* **More applications are graded High than Medium**, which is not the shape the current cutoffs
  predict, and 6,409 of the 6,545 High rows carry no stored score at all. The cause is not fully
  established. A corrective backfill for the part of it that is understood exists and has not been
  run. Treat the High population as unreliable until it has been.
* **A stale signal can be dropped from the score but stay on screen.** When a stored row cannot be
  reused and re-derivation returns nothing, the row is left in place. The score no longer counts it,
  but the panel still draws it, so reviewer and score disagree until the signal re-derives
  successfully.
* **A dead-lettered re-evaluation is never retried.** The queue model's documentation describes a
  nightly sweep for stragglers. There is no such sweep.
* **One concept has three names.** The UI calls it "Auto-close applications", the setting is
  `auto_reject_minimum_fraud_risk_level`, and the stage is `FRAUD_REJECTED`. Expect to translate
  between a support conversation, a settings payload, and a stage query.