Selection Rate vs. Scoring Rate in a Bias Audit

When Your Audit Summary Doesn't Look Like the One Your Colleague Got

You're the HR Director at a 140-person logistics company in Queens. Your resume-screening vendor just sent this year's bias audit summary — the one that's supposed to go up on your public website with your AEDT's distribution date attached before you can lawfully keep using the tool. You compare notes with a friend running a video-interview tool at another company, and her summary is structured completely differently. Hers lists a "selection rate" broken out by category. Yours lists something called a "scoring rate." Nobody explained why they're different, and you've got a 10-business-day notice window riding on posting the right thing.

Neither summary is wrong. LL144's bias-audit requirement covers AEDTs that behave in two structurally different ways, and a properly built audit summary reflects whichever way your specific tool works. By the end of this piece, you'll know which number your bias audit's selection rate or scoring rate should actually be, how to check your vendor got it right, and what to do if the summary in front of you doesn't match the tool you deployed.

What an AEDT Outputs Determines Which Rate Applies

An automated employment decision tool, under the law, is any computational process derived from machine learning, statistical modeling, data analytics, or AI that issues a simplified output — a score, classification, or recommendation — used to substantially assist or replace discretionary decision-making in hiring or promotion (Perkins Coie, 2023). That definition covers two meaningfully different kinds of tools, and the difference matters for how a bias audit measures disparate impact.

A screening tool sorts candidates into two buckets: advance or don't advance. Resume keyword filters, initial applicant-tracking screens, and pass/fail assessments all work this way. When an audit tests a tool like this, it measures how often each demographic group actually gets selected to move forward — the selection rate.

A scoring tool doesn't sort into buckets at all. It assigns every candidate a number — a fit score, a video-interview rating, a ranked percentile — and leaves the advance/reject call to a human recruiter downstream. There's no "selected" bucket to count, so the audit instead measures the scoring rate: the share of each group whose score lands above the median score for the whole sample. That median is the reference point the rules specify — not a cut-off your company picks, and not the score you happen to treat as competitive.

Read that as a description of the two shapes a tool can take, not as the rule's own trigger language. What the rule keys on is the form of the AEDT's output: a categorical output — selected or not selected — is measured with a selection rate, while a continuous score is measured with a scoring rate, binarized against the sample median. “Screening tool” and “scoring tool” are useful shorthand for those two output types, not statutory categories, and a single tool can produce both. For the authoritative scenarios, work from DCWP's Final Rule of April 2023 (6 RCNY § 5-300 et seq.) and the text of Local Law 144 itself; your vendor's audit methodology should then state plainly which rate applies to your tool and why.

Bias Audit Selection Rate vs. Scoring Rate: Two Numbers, One Purpose

Both feed the same test: the four-fifths rule. Under the EEOC's Uniform Guidelines on Employee Selection Procedures, a selection rate for any group that falls below four-fifths — 80% — of the rate for the highest-selected group will generally be regarded as evidence of adverse impact (29 CFR § 1607.4(D)). Read that as written: it is a rule of thumb about when to look closer, not a pass mark, and Local Law 144 does not adopt it as one. A scoring rate gets tested against that identical 80% threshold — the math doesn't change once you have two rates to compare. What changes is what's being counted.

As a worked example: say a screening tool advances 40% of male applicants and 28% of female applicants to interview. The impact ratio is 28 ÷ 40 = 0.70, or 70% — below the four-fifths line, a flag worth investigating. That's a selection-rate calculation using real applicant-outcome counts.

Now say a scoring tool never advances or rejects anyone directly — it just assigns a 1–100 fit score, and a recruiter decides who to interview. The audit instead asks what percentage of each group scored above the median score across everyone the tool assessed. If 55% of one group clears the median and 38% of another does, the ratio is 38 ÷ 55 ≈ 0.69 — again below 80%, using the same formula, just applied to scores above the median instead of pass/fail counts. This is the detail most worth checking on your own summary: if your vendor computed the scoring rate against your internal "competitive" cut-off rather than the sample median, they have not run the calculation the rules describe.

The four-fifths rule doesn't care which rate you feed it. It cares whether the ratio between groups clears 80%. Selection rate and scoring rate are just two different ways of counting who a tool treats favorably.

For the full mechanics — including how to handle small sample sizes and multiple demographic categories — the impact ratio breakdown and the four-fifths rule explainer both walk through it step by step.

Why the Summary You Received Might Report Both

Some AEDTs do both jobs at once — they score every candidate, and that score directly determines who advances. A well-built audit for a hybrid tool like that may report a selection rate for the pass/fail outcome and a scoring rate for the intermediate distribution, so a disparity that shows up at either stage doesn't get hidden. If your summary shows only one number and your tool clearly does both jobs, that's a direct question for your vendor before anything goes on your public site.

This is also where published audits tend to get vague. Independent analysis of LL144 audit summaries has found that many under-report disparities because of missing demographic data, opaque aggregation across job categories, or metrics that don't reflect how the tool is actually used day to day (ACM FAccT, 2025). A single blended selection rate reported across every role at your company — when the AEDT is actually applied differently to warehouse hires than to office hires — is exactly the kind of aggregation that can bury a real gap. If you're not sure how to work through a summary line by line, how to read a bias audit summary has the checklist.

What This Means for Your Notice and Posting Obligations

None of this changes your other LL144 obligations. The annual independent audit still has to happen, the summary still goes on your website with the AEDT's distribution date attached, and candidates or employees still get at least 10 business days' notice before the tool is used on them (Crowell & Moring LLP, 2023; Epstein Becker Green, 2023). What changes is your ability to actually read the document your auditor hands you and confirm it describes your tool correctly.

This is operations guidance, not legal advice — it explains how to read the numbers in front of you, not whether your specific deployment satisfies the statute. If you're unsure whether your vendor calculated the right rate for your tool, that's a question for the independent auditor who ran the engagement, or for outside counsel — not a determination any workbook makes for you. WorkforceNewYork's tools document your rationale and track your vendor's audit status; they don't perform, certify, or sign the audit itself, and they never score an individual candidate or employee.

Building a Habit Around the Two Rates

Once you know which rate applies to a given tool, the useful next step is building that distinction into how you track every AEDT you run. A company running three tools — a resume screener, a video-scoring assessment, and a scheduling assistant — may need to watch for two different rate types across two separate audits, not one. Working through the selection rate and impact ratio calculation guide alongside your current summary is the fastest way to confirm your vendor used the right method before your posting deadline. And if LL144's obligations across your full AEDT inventory still feel unclear, the Local Law 144 compliance guide lays out the full annual cycle.

Your First Action: Read the Ratio Before You Post It

The fastest way to check your own audit summary is to work the four-fifths math yourself, on paper, using whatever rate your vendor reported. The Four-Fifths Rule & Impact-Ratio Reading Guide walks through both selection-rate and scoring-rate examples with a companion calculator, so you can confirm the ratio in your summary before it goes on your public website — and know exactly what to ask your auditor if something doesn't add up.