Amidst a rapid increase in global AI development, New York City Council Speaker Julie Menin and Council colleagues held a rare Committee of the Whole hearing to examine the risks posed by the rapid advancement of artificial intelligence on Oct. 5.
Council members questioned senior policy executives about general safety risks and worst-case scenarios involving AI.
Jacob Coxon, former Anthropic researcher and whistleblower, did not hold back in his warning to City Council.
“On the current path… it is more likely than not that humanity loses control of these AIs, and it could end in human extinction.”
Coxon also said that companies are being extremely reckless given the stakes, reported CBS, and that they “need to take safety much more seriously than they have."
During the Council meeting, Menin asked executives to quantify the risks of AI being referred to. Shane Cahill of Meta said he did not have the precise quantifiable framework in front of him.
“Given the serious safety risks that have been raised here, to not be able to quantify what the risk is is troubling at best," Menin said.
The City Council meeting came just one week after President Donald Trump re-asserted that China and the U.S. would not be working together but rather against each other on AI development, and less than a month after Trump referred to fears around AI as a “hoax,” and rejected calls for safeguards.
Former OpenAI employee Daniel Kokotajlo was dismissive of global development frameworks often used to justify rapid AI development.
“We need to not let our own companies do something that risks our lives, even if, as a result, you know, China might then later do something that risks our lives,” Kokotajlo said.
The hours-long meeting discussed a handful of legislative proposals for AI safety, including a whistleblower incentive program, private right-of-action and third-party validation requirements and bills focused on prohibiting the generation of deceptive deepfakes. The council also discussed legislation which would create a better understanding of algorithmic tools on jobs.
Council member Carl Wilson introduced Intro T2603, which he said would give consumers more information about the safety of models and hold companies accountable.
"We cannot wait for Washington to act," Wilson said. "We must act now to protect New Yorkers, support responsible innovation and make sure no one gets left behind."
Of course, AI developers also discussed leveraging the potential benefits of the advancing technology. OpenAI’s Morgan Dwyer said that he believes the company takes its responsibility “very seriously," said that “AI can help scientists develop new cancer treatments, help entrepreneurs build and grow businesses."
Both OpenAI and Meta told the City Council ahead of this week’s hearing that they were working towards building security and safeguards.
