New York City Issues Final Rule on AI Bias Law and Postpones Enforcement to July 2023

The New York City Department of Consumer and Worker Protection (DCWP) issued a Final Rule implementing New York City Local Law 144 (the AI Law) and announced July 5, 2023 as the new effective date for implementation and enforcement.

The AI Law makes it an unlawful employment practice for employers to use automated employment decision tools (AEDTs) to screen candidates and employees within New York City unless certain bias audit and notice requirements are met.

Following a lengthy notice and comment period and two rounds of proposed rules published in September 2022 and December 2022, on April 6, 2023 DCWP issued the Final Rule, which provides further guidance on the AI Law and establishes a new effective date when enforcement efforts will begin.

Background

The AI Law makes it unlawful for an employer or employment agency to use an AEDT to screen candidates and employees unless (1) the tool has undergone a bias audit no more than one year prior to its use, (2) a summary of the most recent bias audit is made publicly available, and (3) notice of the AEDT use and an opportunity to request an alternative selection process is provided to each candidate and employee who resides in New York City.

Final Rule

The Final Rule is largely similar to the December 2022 proposed rule, with some notable alterations.

First, the AI Law defines an “AEDT” as any process “derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making” (emphasis added).

Under the Final Rule, “machine learning, statistical modeling, data analytics, or artificial intelligence” is further defined as “a group of mathematical, computer-based techniques: (i) that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and (ii) for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.”

This expanded definition could result in enforcement on additional, less sophisticated AEDTs used by employers, as the third requirement—“(iii) for which the inputs and parameters are refined through cross validation or by using training and test data”—was removed.

Second, under the AI Law, a “bias audit” is defined as an “impartial evaluation by an independent auditor.” The Final Rule largely maintains the minimum requirements for a bias audit first seen in the September 2022 and December 2022 proposals, including (1) calculation of the selection rate for each race/ethnicity and a sex category and (2) comparison of such “selection rates” to the most selected category to determine an “impact ratio” using a defined formula.

However, the Final Rule adds a requirement that the bias audit also indicate the number of individuals the AEDT assessed who are not included because they fall within an unknown category and requires that the number be included in the summary of results.

Third, the Final Rule allows an independent auditor to exclude a category that comprises less than 2% of the data being used for the bias audit from calculations on impact ratio.

Fourth, the Final Rule clarifies that an employer or employment agency may only rely on a bias audit that uses historical data of other employers or employment agencies if (1) “such employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit” or (2) “such employer or employment agency has never used the AEDT.” “Historical data” is defined as “data collected during an employer or employment agency’s use of an AEDT to assess candidates for employment or employees for promotion.”

Fifth, the Final Rule states that the number of applicants in and scoring rate of a category, if applicable, must be included in the published summary of results, in addition to the date of the most recent bias audit, source and explanation of the data used to conduct the audit, impact ratios for all categories, and distribution date of the AEDT.

Finally, DCWP announced that the Final Rule would be effective and enforcement would begin on July 5, 2023—six months after the AI Law was originally supposed to go into effect.

Implications for Employers

Employers should continue to monitor for DCWP updates as the enforcement date draws near. In the interim, employers should consider taking the following steps:

• Identify whether the employer is currently using any AI tools in its hiring and promotion practices. Employment counsel should reach out to human resources and talent acquisition to identify what AI tools are currently being used in or considered for hiring and promotion processes.
• Determine whether use of such tools is covered by the AI Law and Final Rule—does the tool meet the statutory and regulatory definition of an AEDT, and is it being used to “substantially assist or replace discretionary decision making”?
• Assuming the tool is covered, determine whether the employer is going to engage an independent auditor to audit its data or rely on the vendor’s independent audit. If the former, employers may wish to consider conducting the audit under privilege.
• If the employer will be relying on the vendor to conduct the audit, confirm with the vendor that it has engaged an independent auditor to evaluate their data and by when the audit will be completed.
• Develop a plan for posting the results of the bias audit, in conformance with the law’s requirements. The plan should include the specific text that will be posted and where the audit results will be posted.
• Develop a compliant notice to applicants and employees that an AEDT is being used and identify where it will be posted.

The AI Law may serve as a framework for other local and state legislatures across the United States, and, while the US Equal Employment Opportunity Commission (EEOC) issued guidance in May 2022 related to the Americans with Disabilities Act and AI, more guidance from the EEOC is expected. Morgan Lewis will continue to monitor developments at the federal, state, and local levels and send out updates as appropriate.