BILL NUMBER: S7543B
SPONSOR: GONZALEZ
 
TITLE OF BILL:
An act to amend the state technology law, in relation to automated deci-
sion-making by state agencies
 
PURPOSE:
This bill would require the disclosure of automated decision-making
systems already in use, prohibit the unauthorized use of automated deci-
sion-making systems by state agencies or entities on behalf of an agency
without meaningful human review and, when authorized, require the publi-
cation of an impact assessment.
 
SUMMARY OF PROVISIONS:
Section 1. Provides the title, which is the "legislative oversight of
automated decision-making in government act (LOADinG Act)."
Section 2. Adds new article 4 to the technology law to prohibit any
state agency or entity acting on behalf of any agency from directly or
indirectly using automated decision-making systems in relation to the
delivery of any public assistance benefit or in circumstances that
impact the rights, civil liberties, safety, or welfare of an individual
unless such system is subject to meaningful human review. Further
prohibits the procurement or acquisition of any service or system rely-
ing on automated decision-making systems without meaningful human
review. If authorized, the agency is required to complete an impact
assessment once every two years that includes a description of the
objectives and the development of the technology and testing of its
accuracy, fairness, and possible biases that could lead to discriminato-
ry outcomes. If such impact assessment finds that the automated deci-
sion-making system produces discriminatory or biased outcomes, the agen-
cy shall not use the system. The impact assessment would be made public
on the agency website and sent to the governor, the temporary president
of the senate, and the speaker of the assembly at least 30 days prior to
any use or implementation. Allows agencies to redact information in
certain cases when publishing publicly.
Section 3. Requires disclosure of certain information about automated
decision-making systems already in use by any state agency, including a
description of the system, software vendors related to the system, the
date use began, the purpose and use of the system, whether any impact
assessments have been carried out, and any other information that the
agency deems necessary.
Section 4. Effective date.
 
JUSTIFICATION:
In New York State and worldwide, artificial intelligence, algorithms,
and computational modeling services have recently exploded onto the
market and have become fully-integrated tools for many individuals and
companies alike, often automating certain historically human-conducted
decision-making tasks and functions to boost productivity. However,
these systems come with significant risks to security and privacy,
unknown malfunctions, and the demonstrated potential for discrimination
to occur due to biases in their development and operation. As these
technologies continue to mature, more use cases evolve, and direct costs
of use diminish, it is paramount that the duties of the state to the
public are reaffirmed, especially when individuals' constitutional and
statutory rights are involved.
This legislation sets a necessary prohibition on the unauthorized use of
these services by state agencies, and, when authorized, requires the
publication of an impact assessment detailing the development of the
service and requires thorough testing of its biases and discrimination
against actual or perceived race, color, ethnicity, religion, national
origin, sex, gender, gender identity, sexual orientation, familial
status, biometric information, lawful source of income, or disability.
Finally, the bill prohibits the state from using discriminatory auto-
mated decision-making systems.
To minimize disruption of state operations that already rely on these
technologies, existing automated decision-making systems must be
disclosed to the legislature within one year. After one year, existing
uses will be subject to the same requirements as any new automated deci-
sion-making system.
 
LEGISLATIVE HISTORY:
2024: S7543 - Referred to Internet &Technology
2023: S7543 - Referred to Rules
 
FISCAL IMPLICATIONS:
To be determined.
 
EFFECTIVE DATE:
This act shall take effect immediately, provided that section two of
this act shall take effect one year after it shall have become a law.