Verifying an AEDT Vendor's Bias Audit

Why "We're Compliant" Isn't Verification

Your procurement lead just came back from a vendor demo with good news: the resume-screening tool your team wants to license "is fully Local Law 144 compliant." No document attached. No date. No link. Just a sentence from a sales deck.

You're two weeks from rolling this AEDT out to screen NYC-resident applicants, which means a candidate notice has to go out at least 10 business days before first use, and that notice has to point to something real — a bias audit that actually happened, on this tool, recently enough to count. If it turns out the audit is 14 months stale, or was run on a version of the model your vendor retired last spring, the deployment decision was never actually documented. It's just a memory of a sales call.

This is the moment vendor due diligence stops being optional. Not because a salesperson is lying — most aren't — but because "compliant" is not a fact you inherit from a vendor's claim. It's a fact you check yourself, against a dated, public source, before you let the tool touch a single New York candidate. By the end of this piece, you'll have a repeatable method for doing exactly that, and a way to keep the answer current after go-live rather than just on onboarding day.

This article covers a verification operation, not a legal opinion — confirm any specific compliance determination with DCWP or outside counsel before you act on it.

There's No DCWP Registry — So the Burden Sits With You

Here's the part that surprises people who assume this works like a license lookup: there is no DCWP-maintained registry of AEDT vendors and their audit status. No searchable database, no central list you can check against a vendor's name. If you're picturing something like a professional-license lookup, that doesn't exist here.

What DCWP requires instead is that each employer or employment agency using an AEDT post a public summary of the AEDT's most recent bias audit — including the date the AEDT was distributed for use — and that candidates get notice at least 10 business days before the tool is used, with a path to request an alternative process or accommodation (Crowell & Moring LLP, 2023; Epstein Becker Green, 2023). The obligation to have run an independent bias audit sits with the deploying employer, not with some third party checking vendors on your behalf.

That gap matters because of what the New York State Comptroller found when it actually looked at enforcement. Reviewing DCWP's oversight from July 2023 through June 2025, the Comptroller's office found DCWP's compliance mechanisms "ineffective" — and on a sample of 32 companies, DCWP's own review found just one non-compliant while the Comptroller's auditors, looking at the same 32, found 17 (Office of the NY State Comptroller, 2025). A separate academic study of 391 employers found only 18 had posted an audit report and only 13 had posted a transparency notice at all (ACM FAccT, Wright & Muenster et al., 2024).

Nobody is checking behind you here. If your vendor's audit claim is wrong, stale, or fabricated, you're the one who posted the summary and the one holding the exposure — not the vendor, and not DCWP.

A Five-Step AEDT Vendor Bias Audit Verification Method

A real aedt vendor bias audit verification pass doesn't take long, but it has to hit five specific things, in order:

1. Locate the vendor's own public disclosure. Not a sales deck, not an email assurance — a dated page, PDF, or trust-center document the vendor has published, ideally one that names the tool, the audit period, and the auditor. If the vendor can't produce a public artifact, that itself is the finding.

2. Confirm the auditor is actually independent. Local Law 144 requires the bias audit be performed by an auditor with no financial or employment relationship to the employer or to the AEDT vendor. Read the summary for the auditor's name and check whether that firm appears to be the vendor's own internal team, an affiliate, or a genuinely unrelated third party. This is the single most common place a vendor-supplied audit quietly fails the statute's own definition of independence.

3. Check the date against your deployment date. LL144 calls for an annual audit. A summary from 20 months ago covering a model version your vendor has since retrained is not current, even if it's technically the "most recent" one the vendor ever ran. Match the audit date to the version of the tool you're actually about to deploy.

4. Read the selection-rate data, not just the headline. A credible summary breaks out selection rates by the sex, race, and ethnicity categories the audit covers, and reports the impact ratios those rates produce. The four-fifths rule is the standard reference point here: a group's selection rate below 80% of the highest-selected group's rate is a signal of possible adverse impact (via Assessment Systems, 2024, citing EEOC Uniform Guidelines). If the summary has no group-level breakdown at all, you can't verify anything — you're being asked to take the conclusion on faith.

5. Note the AEDT distribution date the summary discloses, and confirm it lines up with when the vendor says the tool became available for use. This is a specific, named data point DCWP's guidance calls for, and vendors sometimes omit it entirely.

Run those five checks and you have something a candidate notice, or a curious regulator, or your own outside counsel can actually stand behind — a documented rationale, not a vendor's assurance.

Reading the Public Audit Summary Without Getting Fooled

Here's where it gets harder than a checklist implies. A 2025 academic review of published LL144 audits — "Auditing the Audits" — found that many posted summaries may under-report disparities because of missing demographic data, opaque aggregation choices, and metrics that don't reflect how the tool is actually deployed in practice (ACM FAccT, 2025). In plain terms: a summary can look complete and still not tell you what you need to know.

A published audit summary that's missing group-level selection-rate data isn't a shorter version of a real audit. It's a different document that happens to use the same word.

Two things to watch for specifically. First, aggregation across job categories or locations can mask a disparity that exists in one role even though the blended number looks fine. If your deployment is for a single role type, ask whether the audit's sample matches — was it audited across a broad candidate pool, or specifically for something resembling your use case? Second, watch for a summary that reports only an overall pass/fail without showing the underlying rates at all. That's not verification-ready; that's a marketing conclusion wearing an audit's clothes.

None of this means you rerun the vendor's audit yourself — that's not your role, and it isn't what these five checks are for. It means you're confirming the vendor's disclosure actually contains what the statute asked it to contain, before you rely on it in your own posted summary.

When the Vendor Can't Produce One

Sometimes the answer to "where's your bias audit" is silence, a vague reassurance, or a document that fails two or three of the five checks above. That's a real outcome of vendor due diligence, not a dead end — it just changes what happens next.

You have three paths: hold the deployment until the vendor produces something verifiable; ask the vendor directly for the auditor's name, engagement dates, and methodology reference so you can complete the checks yourself; or treat this AEDT as not yet deployable in NYC and document why. That last determination — whether a given AI hiring tool is actually deployable in NYC given what its vendor can currently show you — is its own decision worth working through methodically rather than under deadline pressure.

Whatever you decide, write it down. A dated note explaining why you paused, proceeded, or rejected a vendor based on what its disclosure did or didn't show is exactly the kind of documented rationale that holds up later — to your own leadership, to a curious candidate, or to DCWP.

Keeping Verification Current, Not Just Onboarding-Day

The mistake most teams make isn't skipping verification at signing — it's treating it as a one-time gate. LL144's audit obligation is annual, which means the vendor's disclosure you verified in January can be stale by the following January, especially if the vendor retrains the model, changes auditors, or quietly lets a renewal lapse.

If you run more than one AEDT, or you're a consultant tracking a client roster across several vendors, that's a status board, not a memory. Which vendor's audit is current, which is due for renewal, which auditor performed it, what the last selection-rate breakdown showed — that's the kind of thing that belongs in a tracker you actually check on a schedule, not a folder of PDFs nobody revisits until a candidate asks a hard question.

Your First Action Item

Verifying one vendor once is a checklist. Verifying every vendor you use, every year, against a real record, is an operation — and that's what the AEDT Vendor Bias-Audit Status Tracker Workbook is built to run. It gives you a structured place to log each vendor, the auditor's name, the audit date, the distribution date, and your own five-point verification notes, so the next renewal deadline doesn't sneak up on you and the next new-hire audit doesn't start from zero.

Pair it with a broader look at how to check if an AEDT vendor has a bias audit in the first place, work through the full AEDT vendor due diligence checklist before you sign anything new, and if you're still deciding whether a specific tool belongs in your NYC hiring stack at all, start with is an AI hiring tool deployable in NYC. For the mechanics of the underlying obligation, the Local Law 144 compliance guide walks through the annual audit, the posting requirement, and the candidate-notice window in full. And if you've heard there's a registry to check against — there isn't, yet — the registry lookup piece explains exactly what does and doesn't exist today.

Download the tracker, run the five checks on every vendor currently touching your NYC candidate pipeline, and you'll have a documented rationale instead of a sales deck the next time someone asks.