The NYC Bias Audit, Step by Step

The Deadline That Sneaks Up on You

A retail chain with 140 employees across four NYC locations signs a contract for a resume-screening tool in March. The vendor demo was smooth, procurement is happy, and the plan is to flip the tool on for the summer hiring push. Nobody has asked the harder question: what has to happen before that tool can legally touch a single NYC resident's application.

This is the moment Local Law 144 stops being an abstract compliance line item and becomes a sequence with dependencies. An independent auditor has to be engaged — and good auditors book out. Historical data (or a documented reason it doesn't exist) has to be assembled before the audit can even start. A public summary has to be live on the website before the tool goes live on candidates. Candidates need at least 10 business days' notice before the tool is used on them (Crowell & Moring LLP, 2023). Miss any one link in that chain and the "smooth vendor demo" becomes a compliance gap with a per-day penalty attached.

By the end of this nyc bias audit step by step guide, you'll have the full six-step operational sequence — in order, with the dependencies that trip people up — so you can work backward from your go-live date instead of discovering the gaps after the tool is already screening candidates.

Step 1: Inventory Every AEDT You Actually Use

Before you can audit anything, you need an honest list of what qualifies as an automated employment decision tool under the law. The statutory definition is broader than most HR teams expect: any computational process derived from machine learning, statistical modeling, data analytics, or AI that issues a simplified output — a score, classification, or recommendation — used to substantially assist or replace discretionary decision-making (Perkins Coie, 2023).

That sweeps in more than the obvious resume-ranking tool. Video-interview scoring, chatbot pre-screens that shortlist candidates, and some applicant-tracking-system "match score" features can all qualify, depending on how they're used. If you want the fuller mechanics of what does and doesn't count, our companion piece on what a bias audit in AI hiring actually covers walks through the definition in more depth.

The output of Step 1 should be a simple list: tool name, vendor, what it does, which roles it touches, and the date it went live (or the date it's scheduled to). This list is the backbone of everything that follows — you can't scope an audit engagement, provide historical data, or write a notice for a tool nobody wrote down.

Step 2: Engage an Independent Bias Auditor

This is the step people underestimate on timing. Local Law 144 requires an independent auditor — someone with no financial or employment relationship to your company or to the AEDT vendor (Crowell & Moring LLP, 2023). That independence requirement is exactly why the vendor selling you the tool can't also sign off on its own bias audit, and it's why a documentation platform that helps you run this operation is a different thing entirely from the auditor who performs it. We are not the auditor. We build the workbooks that keep the operation around the audit organized; the audit itself has to be performed, and signed, by someone independent.

Engaging that auditor early matters because the audit is an annual obligation (Perkins Coie, 2023) with a real lead time — data collection, calculation, and report drafting don't happen overnight. If your go-live date is fixed, count backward from it. For a practical breakdown of what to look for when vetting an auditor and how to document that the independence requirement is actually met, see our guide to the independent bias auditor requirement.

Step 3: Provide Historical Data — Or Document Why You Can't

Once an auditor is engaged, they need data: historical selection-rate information broken out by race, ethnicity, and sex, used to calculate whether the tool produces disparate outcomes. In practice, a lot of employers — especially smaller ones, or those using a brand-new tool — don't have clean historical data sitting around.

The law anticipates this. An auditor can use market-benchmark or test data in place of employer-specific historical data, but only when the employer has a documented reason the real data isn't available (Crowell & Moring LLP, 2023). "Documented" is the operative word. This is not a shrug — it's a paper trail: what you tried, why it wasn't sufficient, and what substitute data the auditor used instead. That paper trail is exactly the kind of artifact a tracking workbook is built to hold, dated and defensible, rather than reconstructed from memory eighteen months later when the Comptroller or a plaintiff's attorney asks. Our detailed walkthrough on data provision when historical data doesn't exist covers the documentation standard in more detail.

Step 4: Review the Audit Results — And Understand What They Actually Show

When the audit comes back, it will report selection rates by category and, typically, an impact-ratio calculation. The most common threshold referenced is the "four-fifths rule": if a group's selection rate falls below 80% of the rate for the highest-selected group, that's a signal of possible adverse impact (via Assessment Systems, 2024, citing the EEOC Uniform Guidelines).

Here's a worked example of the arithmetic, not a real result — just the method: if your highest-selected group has a 50% selection rate, 80% of that is 40%. Any group selected at a rate below 40% trips the four-fifths flag. That's the calculation your auditor is running across every category in the report; the point of Step 4 is reading the output correctly, not re-deriving it yourself.

This is also the point to be candid about the limits of self-congratulation. A 2025 academic analysis of published Local Law 144 audits — "Auditing the Audits" — found that many public reports may understate real disparities because of missing demographic data, opaque aggregation choices, and metrics that don't map cleanly to how the tool is actually deployed (ACM FAccT, 2025). A clean-looking audit summary is not the same thing as a clean-running tool. Reading the results with that in mind — and asking your auditor to walk through their aggregation choices — is part of the job, not a courtesy.

Step 5: Post the Summary and Send the Notice — Together, Not Separately

Two obligations converge here, and they're easy to sequence wrong.

First, the public summary: a results summary from your most recent bias audit, posted on your company's website, that includes the date the AEDT was distributed for use (Epstein Becker Green, 2023). This has to be live before the tool is used on covered candidates — not filed away, not "in progress," actually posted.

Second, candidate notice: covered candidates and employees need to be told, at least 10 business days before the tool is used on them, that an AEDT will be used, along with a way to request an alternative process or accommodation (Crowell & Moring LLP, 2023). If Step 5 happens after candidates have already gone through the tool, it doesn't count — the notice window runs forward from the notice, not backward from the audit date.

A bias audit that's never posted and a notice that goes out after the fact are, functionally, the same failure: paperwork that exists but doesn't do its job.

The public-summary requirement has real teeth on paper and, so far, uneven enforcement in practice. A Comptroller's audit covering July 2023–June 2025 found the city's own enforcement "ineffective," with DCWP finding only 1 of 32 sampled companies non-compliant while the Comptroller's own auditors — reviewing the same 32 — found 17 (Office of the NY State Comptroller, 2025). A separate academic study of 391 employers found only 18 had posted audit reports and only 13 had posted transparency notices at all (ACM FAccT, Wright & Muenster et al., 2024). Weak enforcement today is not a reason to skip the step — penalties run up to $500 for a first violation and $500 to $1,500 for each subsequent violation, accruing per violation per day (Office of the NY State Comptroller, 2025), and enforcement priorities can change faster than a posted summary can be reconstructed after the fact.

For the exact mechanics of what the summary needs to include and where it needs to live, our post on the bias audit posting requirement covers the specifics.

A note on what this is and isn't: everything in this sequence is an operational description of a filing and disclosure requirement, not legal advice. If you have a specific fact pattern — a tool that might or might not qualify, a data gap you're not sure how to document, a notice timeline that's tight — confirm the determination with DCWP or with outside counsel before you act on it.

Step 6: Calendar the Next Cycle Before You Close This One

The audit is annual (Perkins Coie, 2023), which means the moment you post this year's summary is also the moment next year's clock starts. The employers who get caught flat-footed aren't usually the ones who never ran an audit — they're the ones who ran one, filed it away, and lost track of the renewal date twelve months later when a new hiring manager onboarded a fourth AEDT nobody scoped.

Building the six steps above into a repeating annual sequence — inventory, engage, provide data, review, post and notify, recalendar — is the difference between a one-time compliance sprint and an operation you can run every year without reconstructing it from scratch. If you want the fuller regulatory picture that this sequence sits inside — thresholds, definitions, and how the pieces of Local Law 144 fit together — start with our Local Law 144 compliance guide.

Where to Start This Week

If you're staring at a go-live date and a blank inventory list, don't try to build all six steps from scratch in a spreadsheet you'll forget to update. The Complete NY AI Employment Compliance Kit is built around exactly this sequence: an AEDT inventory tracker, an auditor-engagement worksheet, a data-documentation template for Step 3, a posting-and-notice checklist for Step 5, and a recurring-cycle calendar for Step 6 — the operational scaffolding around your independent auditor's work, not a substitute for it.

Download it, drop in your AEDT list from Step 1, and work backward from your go-live date. That's the whole sequence — and now you have it in order.