Objective

To determine the extent of implementation of the three recommendations included in our initial audit report, Artificial Intelligence Governance (Report 2021-N-10).

About the Program

Artificial intelligence (AI)-powered tools have an increasingly vital role in industry operations, including agriculture, health care and medicine, manufacturing, transportation, and government. New York City (NYC or City) agencies have been using AI to aid their operations. Although AI can enable entities to operate more intelligently, more productively, and more competitively, it is also giving rise to a host of unwanted, and sometimes serious, consequences. AI tools and systems pose unique challenges in accountability, as their inputs and operations are not always visible. The U.S. Government Accountability Office noted that a lack of transparency reduces effective oversight in identifying errors, misuse, and bias. Therefore, it is essential to establish governance structures over AI to ensure that its use is transparent and accurate and does not generate harmful, unintended consequences. Without adequate governance and oversight over the use of AI, misguided, outdated, or inaccurate outcomes can occur and may lead to unfair or ineffective outcomes for those who live, work, or visit NYC.

NYC has had an evolving approach to managing AI risks. In November 2019, the NYC Mayor issued Executive Order 50 (EO 50), which established the position of Algorithms Management and Policy Officer (AMPO) within the Mayor’s Office of Operations, and tasked AMPO with guiding the City in the development, responsible use, and assessment of algorithmic tools and systems, including AI, and engaging and educating the public on issues related to the use of these tools and systems. In January 2022, NYC’s Mayor issued Executive Order 3 (EO 3), which rescinded EO 50, discontinued the position of AMPO and its associated requirements, and created the NYC Office of Technology and Innovation (OTI), which is responsible for guiding the City and its agencies in the development, responsible use, and assessment of algorithmic tools and systems, including AI. OTI is also required to engage and educate the public on issues related to the City’s use of these and other related technologies.

Additionally, Local Law 35 of 2022 requires NYC agencies to report annually to the Mayor’s Office (or designee) on every algorithmic tool, which can include AI tools, that the agency has used one or more times during the prior calendar year. Such disclosure includes, among other information, the commercial name and a brief description of such algorithmic tool, the purpose of the tool, and the type of data collected and analyzed by the tool. The Mayor’s Office (or designee) is required to compile the information disclosed by agencies and submit a report to the Mayor and the Speaker of the NYC Council every year.

The objective of our initial audit, issued on February 16, 2023, was to assess the City’s progress in establishing an appropriate governance structure over the development and use of AI tools and systems. The audit covered the period January 2019 through November 2022 and found NYC did not have an effective AI governance framework. While agencies were required to report certain types of AI use on an annual basis, there were no rules or guidance on the actual use of AI. Consequently, City agencies developed their own divergent approaches. We sampled four City agencies: New York City Police Department (NYPD), Administration for Children’s Services, Department of Education (now known as New York City Public Schools), and Department of Buildings. We found ad hoc and incomplete approaches to AI governance, which do not ensure that the City’s use of AI is transparent, accurate, and unbiased and avoids disparate impacts. Some agencies identified key risks and created processes to address those risks, while others had not created any AI-specific policies or taken other steps toward effective AI governance. Some agencies performed certain activities that partially addressed components of AI governance—such as identifying appropriate use, intended outcomes, data governance, and potential impacts—but did so because of laws created to address issues not specific to AI. For example, the NYPD created impact and use policies for its surveillance tools to comply with the NYC Public Oversight of Surveillance Technology Act.

Furthermore, NYC’s initial governance requirements of algorithmic tools, which include AI, were not fully met. AMPO was required by EO 50 to establish a reporting framework of algorithmic tools, policies, and protocols to guide the City and its agencies in the fair and responsible use of such tools, a process for individuals to learn about the City’s use of these tools, a complaint resolution process for those impacted by such use, and a public education strategy. AMPO created a reporting framework for agencies to report tools, publish a report of those tools, and held several public engagement sessions. However, in January 2022, AMPO was discontinued by EO 3, which removed those requirements and placed the responsibility of algorithmic and AI management within OTI. At that time, AMPO had not established policies and protocols to guide the City and its agencies in the fair and responsible use of such tools or a means for the City to resolve complaints made by individuals regarding algorithmic impacts. In addition, we identified instances where agency tools were not reported or included in the public listing of tools.

The objective of our follow-up was to assess the extent of implementation, as of April 2026, of the three recommendations included in our initial audit report.

Key Findings

OTI officials made some progress in addressing the problems we identified in the initial audit report, but additional steps are necessary to ensure effective AI governance in NYC. All three of the initial report's audit recommendations were partially implemented.

Key Recommendations

OTI officials are requested, but not required, to provide information about any actions planned to address the unresolved issues discussed in this follow-up within 30 days of the report’s issuance.