Objective

To determine whether the New York City Department of Consumer and Worker Protection has designed and implemented an effective system to enforce compliance with Local Law 144. The audit covered the period from July 2023 through June 2025.

About the Program

Companies are increasingly using automated tools to assist in making employment decisions. This technology can be used in a variety of ways, such as quickly scanning resumes for key words to select the most qualified candidates, scanning a candidate’s online presence, and analyzing video interviews to evaluate a candidate’s behaviors and mannerisms. While the use of these tools can benefit companies by saving time and money and improving the candidate’s experience, it also comes with risks. For instance, it can amplify existing biases and create novel sources of bias, and there is often a lack of transparency about the system’s capabilities and limitations.

To address some of these concerns, the New York City (NYC) Council passed Local Law 144 of 2021 (LL144 or the Law). Under the Law, employers and employment agencies must:

• Conduct a bias audit of the automated employment decision tool (AEDT) no more than 1 year prior to its use.
• Provide a summary of the results of the most recent bias audit available on their website.
• Properly notify candidates that an AEDT will be used, how it will be used, and the data that will be collected.

The NYC Department of Consumer and Worker Protection (DCWP) is responsible for enforcement of key consumer protection, licensing, and workplace laws that apply to businesses in NYC. DCWP is tasked with enforcing LL144 and can impose civil penalties between $500 and $1,500 per day for violations. DCWP entered into a memorandum of understanding (MOU) with the NYC Office of Technology and Innovation (OTI) to receive technical assistance and formalize the procedures by which OTI will support DCWP in carrying out its duties pursuant to LL144, such as an Enforcement Workbook.

DCWP officials highlight a unique challenge in identifying non-compliance, as LL144 requires posting of bias audits and disclosures to candidates if an employer determines it needs to comply with LL144. DCWP officials explained that, if an employer does not take these steps, it’s difficult to identify non-compliance. As a result, DCWP officials stated that stakeholder education combined with complaint-based enforcement is the most effective way to enforce compliance with LL144.

Key Findings

• DCWP’s AEDT complaint process is ineffective in ensuring that all complaints related to non-compliance with LL144 are routed to DCWP. Furthermore, despite DCWP’s initial stakeholder education, additional educational outreach has not been performed. By relying on an ineffective complaint process and no additional or more recent outreach, DCWP’s approach to enforcing LL144 will not address the difficulty in identifying non-compliance, especially in instances where employers do not take steps toward complying, such as posting bias audits.
• Despite receiving only two AEDT complaints during the audit’s scope, DCWP did not investigate whether the complaint intake process worked.
• DCWP surveyed websites and bias audits of 32 companies and identified just a single issue of non-compliance. However, DCWP’s review did not use the formal procedures created by OTI, as part of DCWP’s MOU, nor did it address all requirements of LL144. We reviewed the same companies and identified at least 17 instances of potential non-compliance under LL144.
• Although DCWP officials lack technical expertise to evaluate AEDT use, DCWP officials did not consult with OTI when making determinations about AEDTs.

Key Recommendations

• Implement processes to ensure that AEDT complaints are properly handled and routed.
• Implement mechanisms to ensure that AEDT complaints received by 311 are properly routed to DCWP.
• Conduct research and testing to determine the reasons for the low number of complaints related to AEDTs and take steps to address these factors.
• Ensure DCWP’s bias audit reviews address all potential non-compliance, including using existing OTI resources, such as the Enforcement Workbook.
• Create and implement a process to improve enforcement actions by identifying potential non-compliance with LL144 through research and other leads in addition to complaints.
• Ensure OTI’s expertise is used to evaluate the technical functionality of suspected AEDT tools to support DCWP’s enforcement.