A recent decision from the US District Court for the Southern District of New York, discussed in this Sidley Austin memo, highlights growing litigation and governance considerations surrounding AI-assisted decision-making.
In American Council of Learned Societies v. National Endowment for the Humanities, the court examined the government’s use of ChatGPT in reviewing federal grants for potential DEI-related terminations. According to the memo, the court rejected arguments that responsibility could be shifted to the AI tool itself, emphasizing instead the importance of meaningful human review, oversight, validation, and accountability in AI-assisted processes. The decision highlights that AI prompts and outputs may become part of the evidentiary and discovery record when courts examine AI-assisted decision-making processes.
The memo notes several broader governance implications for organizations using AI tools, including the need for clear accountability structures, validation protocols, escalation procedures, and documentation regarding how AI outputs are reviewed and operationalized. It also showcases the growing importance of understanding how AI prompts, outputs, and related materials may be treated in litigation, investigations, or regulatory reviews.
The issues raised in the case are aligned with planned outputs under the Society’s ongoing AI-focused initiative. Next week, the Society will host a structured Chatham House Rule discussion on Legal Privilege and AI, facilitated by Sidley Austin partners Takayuki Ono and David Gordon, that will explore evolving questions surrounding privilege, confidentiality, discoverability of AI prompts and outputs, and our planned June 17 program on the Use of AI by Governance Professionals (invitation forthcoming) will include—among other things—discussion of practical AI usage considerations, including AI prompting techniques and governance-related applications.
This post first appeared in the weekly Society Alert!