What Data Does a Bias Auditor Need?

The auditor's data request just landed, and it's longer than you expected

Your independent bias auditor sent the engagement letter three weeks ago. Today the actual data request arrived, and it's a page and a half — role names, demographic categories, a note about "selection data or, absent sufficient historical volume, test data." Your recruiting team is asking whether they need to pull two years of applicant records or just last quarter's. Nobody on your side wants to hand over the wrong thing, hand over too much, or discover in week six that a category was missing and the timeline just slipped. This article walks through what a Local Law 144 bias auditor typically asks an employer to prepare, in what shape, and how to catalogue it before the request even arrives — so by the end you'll know exactly what belongs on your side of the table.

One thing up front: this is operational guidance, not legal advice. What counts as sufficient data for a specific AEDT, in a specific industry, for a specific auditor's methodology, is a determination the auditor and your counsel make together. Nothing here substitutes for that conversation.

What Data Does a Bias Auditor Need?

Local Law 144 requires an annual independent bias audit of any automated employment decision tool used on New York City candidates or employees — defined broadly as any computational process derived from machine learning, statistical modeling, data analytics, or AI that issues a simplified output (a score, classification, or recommendation) used to substantially assist or replace discretionary hiring or promotion decisions (Perkins Coie, 2023). To calculate the impact-ratio analysis the audit report requires, the auditor needs three broad categories of material from you:

• A description of each job category or role the AEDT is used to screen, rank, or score — clear enough that the auditor can map the tool's outputs to a defined selection process.
• Demographic category data for the candidate or employee pool the tool touched — sex, race/ethnicity, and intersectional categories, structured so selection rates can be calculated per group.
• Historical selection data, or test data if historical volume is insufficient — records of who was scored, how they were scored, and what happened next in the selection process (advanced, rejected, hired).

The reason the demographic breakdown matters so much: the audit's core statistical test is the four-fifths rule — a selection rate for any group below 80% of the rate for the highest-selected group can indicate adverse impact (Assessment Systems, 2024, citing the EEOC Uniform Guidelines). As a worked example: if your AEDT advances 60% of applicants in the highest-selected group but only 40% of applicants in another group, the impact ratio is 40 ÷ 60 = 0.67, well under the 0.80 threshold — the kind of number that triggers a closer look in the audit report. The auditor can't run that calculation without demographic tags attached to real selection outcomes, which is exactly why the data request is structured the way it is.

Historical Data vs. Test Data — Which One You'll Provide

Not every employer has two years of clean historical selection data with demographic tags attached. If you don't, the auditor may work from test data instead — a representative dataset run through the tool specifically to generate the selection-rate comparison, rather than pulling it from records of live decisions. Which one applies to your engagement is a scoping conversation, not a choice you make unilaterally. We cover the distinction and how the choice affects your prep work in more depth in historical data vs. test data for a bias audit — worth reading before you commit either data source to the auditor.

What doesn't change based on which data source you use: the underlying obligation. Local Law 144 requires the annual audit, a public summary posted to your website with the AEDT's distribution date, and candidate or employee notice at least 10 business days before the tool is used, with an alternative process or accommodation available (Crowell & Moring LLP, 2023; Epstein Becker Green, 2023). The data question only affects how the auditor gets to the number — not whether the audit itself is required.

Mapping Roles and Demographic Categories Before You Send Anything

The most common delay in a data handoff isn't missing records — it's a role taxonomy the auditor and the employer read differently. If your job postings describe roles one way and your HRIS codes them another, and neither matches how the AEDT vendor's own documentation describes what the tool screens for, the auditor's first round of questions will be about reconciling those three descriptions before any statistical work starts.

This is where a role-mapping method — the same ONET-based crosswalk approach used to tag which occupations an AEDT actually touches — earns its keep before an audit, not just for the transparency notice. Using consistent, SOC-coded role descriptions (ONET, the U.S. Department of Labor/Employment and Training Administration's occupational database, licensed under CC BY 4.0) means the role list you hand the auditor already speaks the same language as their methodology. For the mechanics of what the auditor's engagement is supposed to cover — and where your data-prep obligation ends and their analytical obligation begins — see independent auditor engagement scope under Local Law 144.

What You Prepare vs. What the Auditor Does

This distinction matters enough to state plainly: preparing and organizing data is your job; running the statistical audit and issuing the findings is the auditor's job, and only theirs. Local Law 144 requires that the bias audit be conducted by an independent auditor with no financial or employment relationship to your organization or to the AEDT vendor — the independence is structural, not optional. Nothing in the data-provision step should blur that line.

The employer's job is to hand over accurate, well-organized, defensible data. The auditor's job is everything that happens to that data afterward.

That distinction also explains why "Auditing the Audits," a 2025 review of published Local Law 144 reports, found that many audits may under-report disparities — not necessarily because the statistical method was wrong, but because of missing demographic data, opaque aggregation choices, and metrics that don't reflect how the tool is actually deployed (ACM FAccT, 2025). A gap on the employer's side of the data handoff — an incomplete demographic field, a role description that doesn't match live deployment — can quietly weaken the audit's own findings before the auditor runs a single calculation. Getting your side of the data right isn't a courtesy to the auditor; it's the difference between an audit that reflects your actual hiring process and one that doesn't.

For the full picture of what Local Law 144 requires end to end — audit, notice, public posting, penalties — the Local Law 144 compliance guide walks through the whole obligation. And for the underlying independence requirement that shapes how any auditor can engage with you at all, see what an independent bias auditor actually is under Local Law 144.

Cataloguing the Data So Nothing Gets Lost

The employers who move through a data request fastest aren't the ones with the most data — they're the ones who catalogued it before the auditor asked. A workable catalogue tracks, at minimum:

1. Which AEDT each dataset relates to (name, vendor, version if applicable).
2. Which role or job category the data covers, using a consistent taxonomy.
3. Data type — historical selection records or test data — and the date range covered.
4. Demographic fields included, and whether any are incomplete or self-reported with gaps.
5. Who owns the export internally (recruiting ops, HRIS admin, a specific analyst) so a follow-up request doesn't stall waiting for someone to remember who pulled the file.

Building that catalogue after the auditor's request arrives means doing it under deadline pressure, usually across departments that don't normally talk to each other about data structure. Building it before the request arrives means you hand over an organized package on day one instead of a scramble.

Your First Action Item

If you're staring at an auditor's data request right now — or you know one is coming with next year's engagement — the fastest way to get organized is a structured worksheet built specifically for this handoff: the AEDT Data-Provision & Historical-Data Prep Workbook. It walks through the role-mapping, demographic-category, and historical-versus-test-data decisions covered above in a format built to hand directly to your auditor, so the first round of back-and-forth is about their analysis, not your organization.