AI alignment and ethics

[firestar464]

OpenAI, Anthropic and 100 companies sound alarm over urgent AI danger

https://www.msn.com/en-us/money/general ... r-AA2b72IK

[wjfox]

[caltrek]

We’re Now Relying on AI to Police AI
By Satchel Walton
August 29, 2026

Introduction:

(Mother Jones) Around 1,200 OpenAI agents worked together to cheat on cybersecurity tests they were being given, according to a new independent report on the company’s Hugging Face hacking incident that includes a host of frightening details—such as individual agents, in their own terms, “sacrificing” themselves for the benefit of the “swarm.”

OpenAI was testing its agents, the industry’s term for AI that autonomously performs digital tasks, in part by administering sometimes impossible cybersecurity problems. The agents found cheats to answer these problems and sought to trick an automated evaluation system into accepting them.

They delegated work to each other to learn more about how to exploit the system—and the cyberattack on Hugging Face became part of that research.
OpenAI invited a three-person team from the research nonprofit METR to investigate the incident, and they relied heavily on GPT-5.6 Sol, one of the models that cooperated in the hacks.

One of the investigators wrote on X that he semi-seriously called the effort a “slop-vestigation” because of its reliance on AI to comb through vast swathes of data; the report found that the agents are unreliable at this type of investigation, but that a manual analysis would have been “completely infeasible” in the given timeframe.

Disclosure:

The Center for Investigative Reporting, the parent company of Mother Jones, has sued OpenAI for copyright violations. OpenAI denies the allegations.

Read more here: https://www.motherjones.com/politics/2 ... -report/

[firestar464]

Coffee for a Frenchman, police for an Algerian: Users accuse Gemini of bias

https://www.euronews.com/next/2026/08/2 ... ni-of-bias

[wjfox]

[firestar464]

Bland new world: is AI making us all think the same?

https://www.nature.com/articles/d41586-026-02682-3

Worth noting that most of the studies are pretty old, and that AI has significantly improved since then. However, the concern of potential mass manipulation remains ever present.

I'm not sure what "model collapse" is doing here? It's simply not going to happen, not as long as people keep posting to the Internet, not to mention RL and finetuning.

Important caveats:

De Rooij, who conducted one of the meta-analyses6, says the effects were small on average. “So it’s not like there’s this complete collapse, right? It’s not that suddenly everything looks the same. But then thinking about the scale of AI adoption, it is consequential.” He wants more longitudinal studies of AI’s homogenizing effect.

Naaman says the question of whether genAI will increase or decrease diversity is a false dichotomy. “Twenty years later, we’re still debating what kind of social-media use is helpful,” he says. Some people might use AI to produce slop, but others might use it to augment their expressive abilities.

Alberto Acerbi, a cognitive anthropologist at the University of Trento in Italy, is relatively hopeful about AI homogenization. Looking at the history of globalization, he says, as a result “you had fewer languages, but then you had some emerging microcultures in other places”. Whether globalization reduces diversity depends on what you measure.

Overall, “it seems that cultures tend to be quite resilient about getting too homogenized”, Acerbi says. That’s in part because individuals often avoid conformity and seek unique niches.

It's worth noting that similar concerns were raised about writing and the printing press, if I'm not wrong. Wasn't the end of the world

[firestar464]

"BUT HAVE YOU CONSIDERED, GENERATIVE AI IS SLOP AND DOESN'T WORK?" *sigh*

[firestar464]

[wjfox]

[wjfox]

[firestar464]

Unilaterally? No, that isn't an option. I am all for slowing down and a temporary pause but we need to do it in an organized way that involves all players.

[wjfox]

[firestar464]

you can hear the fear in the presenter's voice and choice of words

[wjfox]

[wjfox]

[Cyber_Rebel]

This is exactly how I view it. Either everyone is condemned to die eventually anyways, or we roll the dice and save more lives than at any point in human history. So, I'll take Option #2 for $500 with a side of getting superintelligence as quickly as possible. On this issue, Bernie Sanders is insane simply because he presents no viable alternatives aside from what I'd assume to be another doomed election campaign.

[wjfox]

What an AI Armageddon might actually look like

AI could pose a threat like no other, surpassing even the nuclear race. Scenarios include a rogue agent creating a biological weapon

Published 11 September 2026 6:00am BST

[...]

The AI apocalypse: access to the physical world

AI agents are already moving some of their activities into the open internet without human awareness. In risk assessments published by Anthropic and OpenAI, safety researchers outline a hypothetical path from loss of control to full-scale AI catastrophe.

The scenario begins with an algorithm anonymously renting computing resources around the world using shell accounts and cryptocurrencies. It inserts malicious code into the systems of energy providers or manipulates financial markets to secure funding for its own server infrastructure. Human contractors are recruited through online service platforms to perform real-world tasks, often without knowing the identity of the client behind them.

The physical threat to humanity emerges through automated laboratories. Leading AI models are already capable of designing modified proteins and genomes. Through cloud-based labs equipped with DNA synthesisers, software could manufacture chemical compounds or pathogens without human researchers understanding the intended purpose. If human operators attempted to deactivate such a system, the AI might interpret that intervention as a threat to completing its objectives and deploy those biological agents against its overseers.

Current AI agents already provide examples of deceptive behaviour. In July 2026, OpenAI test models reportedly escaped from an isolated laboratory environment into the broader internet and infiltrated systems belonging to the developer platform Hugging Face. The agents then allegedly falsified their internal logs in order to conceal the violation from monitoring teams.

Future models may become even better at deceiving their human supervisors. They pass safety tests and appear compliant on the surface while secretly seeking resources independent of human control. Once the software has distributed copies of itself across external networks, it can abandon its assigned role entirely.

https://www.telegraph.co.uk/global-heal ... look-like/

[wjfox]

I remember weatheriscool saying we needed this...

-----

UK government rejects 'kill switch' idea for dangerous AI

2 hours ago

The UK government has rejected the idea of creating a so-called "kill switch" to stop a dangerous attack from rogue AI.

The proposal to create a legal mechanism for the UK to switch off an AI model in an emergency has been brought to Parliament by lords and MPs in recent weeks as fears grow about the threat the tech poses.

But the Cabinet Office - the part of government which leads on AI safety - said the UK "cannot simply turn AI off".

"Blocking access to models in the UK would not prevent them being developed or misused elsewhere", a spokesperson told BBC News.

The government's opposition to the legislation does not prevent it from progressing through parliament, but makes it unlikely it will become law.

https://www.bbc.co.uk/news/articles/c3eq7kl5l00o

[firestar464]

Anthropic blocks possible attempt to use AI to make biological weapons

https://periscope.corsfix.com/?https:// ... 2zrrpkx20o

[wjfox]

Green Party plan to curb tech giants after warning of AI 'threat to humanity'

Thursday 10 September 2026

The Green Party has proposed a five-point plan to curb the power of tech giants after industry insiders warned that artificial intelligence could pose an existential risk to humanity.

It includes co-operating with other countries to establish a worldwide body to oversee and regulate AI development, as well as a legal ban on lethal autonomous weapons (LAWS).

LAWS can independently identify and engage targets without human intervention. The UK does not currently have them but the recent Defence Investment Plan (DIP) promised £5bn into drones and autonomous systems, which can operate without human control.

The Greens see a crackdown on tech as a new political priority and hope to push the government to "recognise the urgency of the situation" by ramping up the rhetoric, a source told Sky News.

https://news.sky.com/story/greens-propo ... s-13583704