Applicant Ranking Algorithms: AEDT Compliance Basics

When a Ranking Tool Starts Deciding Who Gets a Callback

Your recruiting team logs into the applicant tracking system Monday morning and the candidate list isn't in application-date order anymore — it's sorted, top to bottom, by a "match score" the ATS vendor added last quarter. Nobody on your team wrote a rule for that ordering. Nobody scored an individual resume by hand. The algorithm did it, and now the recruiters are working the list from the top down, which means candidates at the bottom are functionally screened out even though no one rejected them.

Someone asks the obvious question: is this thing an AEDT? If it is, Local Law 144's clock started running the day it went live, not the day someone noticed. By the end of this piece you'll be able to look at any ranking or shortlisting tool your team uses and tell, with reasonable confidence, whether it falls inside Local Law 144's scope — and what to do next if it does.

What Makes an Applicant Ranking Algorithm an AEDT

Local Law 144 defines an automated employment decision tool as any computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues a simplified output — a score, classification, or recommendation — used to substantially assist or replace discretionary decision-making about employment (Perkins Coie, 2023). Notice what's absent from that definition: there's no requirement that the tool reject anyone outright. Ordering is enough.

That's the piece recruiting teams miss most often. A tool that ranks 200 applicants from most-to-least qualified is issuing a simplified output — the rank — and recruiters routinely use that rank to decide who gets contacted first, second, or not at all. If the ranking substantially assists that discretionary call, applicant ranking algorithm AEDT compliance obligations attach, whether or not the tool ever says the word "reject."

Ranking vs. Filtering: Why the Distinction Rarely Saves You

Some teams draw a line between "filters" (which supposedly just narrow a pool) and "rankers" (which supposedly just order it), and assume only the filter is regulated because it's the one making an explicit yes/no call. That distinction doesn't hold up well under the statutory definition. A ranker that consistently pushes certain candidates to the bottom of a 500-person list produces the same practical effect as a filter that screens them out — recruiters rarely work past page three.

A tool doesn't have to say "no" to function as a gate. It only has to change who gets looked at first.

This is exactly the kind of self-serving classification error Local Law 144 is designed to catch, and it's why applicant ranking algorithm AEDT compliance can't be waved away by relabeling the tool a "sorting aid" or a "productivity feature." Ordering is not, however, automatically sufficient on its own. The statutory test asks whether the output substantially assists or replaces the decision, which DCWP's rules cash out as the output being relied on by itself, weighted more heavily than any other single criterion, or used to overrule other factors. A rank that genuinely sits alongside several equally weighted signals is the case worth putting to counsel or DCWP rather than settling in-house. If you're unsure whether a specific tool in your stack clears the bar, our companion piece on what actually is an automated employment decision tool walks through the statutory test in more depth, and the resume screening and Local Law 144 piece covers the closely related upstream-filtering case.

The Three Obligations That Follow Once a Ranker Qualifies

Once you've concluded a ranking tool is an AEDT deployed against NYC-resident candidates, three obligations follow, and none of them are optional add-ons (Crowell & Moring LLP, 2023; Epstein Becker Green, 2023):

• An annual independent bias audit, conducted by an auditor with no financial or employment relationship to your company or the tool's vendor — not run by you, not run by the vendor, and not something a documentation workbook can substitute for.
• A public summary of the most recent audit results, posted on your own website along with the date the AEDT was distributed for use.
• Candidate notice at least 10 business days before the tool is used on them, including a clear path to request an alternative process or accommodation.

The bias audit itself typically checks for adverse impact using a selection-rate comparison — the four-fifths rule, borrowed from EEOC Uniform Guidelines, flags a concern when any group's selection rate falls below 80% of the highest-selected group's rate (Assessment Systems, 2024). To make that concrete: if your highest-selected group clears the ranker's threshold at a 50% rate, any other group falling under a 40% rate (80% of 50%) is a signal worth investigating — that's the mechanics of the ratio, not a claim about your specific tool's actual numbers.

Here's the part worth saying plainly: none of what your workbook does replaces that audit. Workbooks document who's tagging what, when notices went out, and where the vendor's audit summary lives. They don't perform, certify, or sign the audit itself, and this article isn't legal advice — if you have a live compliance question about a specific tool or filing, confirm with DCWP or your own counsel.

What the Recruiting Team Should Document Before the Tool Goes Live

Waiting until an audit is due to figure out what tools you're running is how teams end up scrambling. The practical sequence:

1. Inventory every tool touching the candidate pipeline — not just the obvious AI screener, but ATS plug-ins, third-party ranking add-ons, and any vendor feature that reorders a candidate list.
2. Classify each one against the statutory definition, in writing, with the reasoning captured — not just a yes/no checkbox.
3. Confirm audit and notice status with the vendor before the tool touches a single NYC-resident candidate.
4. Log the distribution date — the day the tool actually went live in your hiring process — because your public summary posting depends on it.

Penalties for missing the notice or posting obligations run up to $500 for a first violation, with $500 to $1,500 for each subsequent violation, and they accrue per violation per day (Office of the NY State Comptroller, 2025). That per-day structure is exactly why a documented classification decision, made once and filed, is worth more than a scramble every audit cycle.

Recruiting Firms, Staffing Agencies, and Shared Liability

If you're a recruiting firm or staffing agency deploying a ranking tool on behalf of client employers, don't assume the client absorbs all the exposure. The statute's obligations attach to the entity using the AEDT to screen candidates for employment decisions — which in practice can mean both the staffing firm and the hiring employer carry responsibility, depending on who controls the tool and the decision it feeds. Recruiting-firm AEDT compliance in NYC deserves its own classification pass per client relationship, not a single blanket assumption that "the client's audit covers us."

If your firm runs ranking tools across multiple client rosters, that's multiple classification decisions to track, not one. Our broader walkthrough on Local Law 144 compliance covers how the obligations shift across employer, vendor, and staffing-firm roles.

Your First Action Item: Classify the Tool

Before you can decide whether a bias audit, a notice period, or a public posting applies to your ranking tool, you need a documented answer to one question: is this an AEDT? Guessing — or worse, assuming a vendor's marketing language settles it — is how teams end up either over-auditing tools that don't need it or missing obligations on ones that do.

The "Is This Tool an AEDT?" Classification Decision Workbook walks your team through the statutory test tool-by-tool, so the reasoning is written down and defensible instead of remembered by whoever happened to read the vendor's spec sheet. Related reading: is my hiring tool an AEDT covers the same question from a broader hiring-stack angle if ranking tools are just one piece of what you're auditing.

Download the workbook, run it against every tool touching your candidate pipeline, and you'll have a written record — the kind that holds up whether the question comes from your own leadership, a candidate, or DCWP.