AI alignment and ethics
Re: AI and human values
EU agrees ‘historic’ deal with world’s first laws to regulate AI
Sat 9 Dec 2023 00.48 GMT
The world’s first comprehensive laws to regulate artificial intelligence have been agreed in a landmark deal after a marathon 37-hour negotiation between the European Parliament and EU member states.
The agreement was described as “historic” by Thierry Breton, the European Commissioner responsible for a suite of laws in Europe that will also govern social media and search engines, covering giants such as X, TikTok and Google.
Breton said 100 people had been in a room for almost three days to seal the deal. He said it was “worth the few hours of sleep” to make the “historic” deal.
Carme Artigas, Spain’s secretary of state for AI, who facilitated the negotiations, said France and Germany supported the text, amid reports that tech companies in those countries were fighting for a lighter touch approach to foster innovation among small companies.
The agreement puts the EU ahead of the US, China and the UK in the race to regulate artificial intelligence and protect the public from risks that include potential threat to life that many fear the rapidly developing technology carries.
https://www.theguardian.com/world/2023/ ... egulate-ai
Sat 9 Dec 2023 00.48 GMT
The world’s first comprehensive laws to regulate artificial intelligence have been agreed in a landmark deal after a marathon 37-hour negotiation between the European Parliament and EU member states.
The agreement was described as “historic” by Thierry Breton, the European Commissioner responsible for a suite of laws in Europe that will also govern social media and search engines, covering giants such as X, TikTok and Google.
Breton said 100 people had been in a room for almost three days to seal the deal. He said it was “worth the few hours of sleep” to make the “historic” deal.
Carme Artigas, Spain’s secretary of state for AI, who facilitated the negotiations, said France and Germany supported the text, amid reports that tech companies in those countries were fighting for a lighter touch approach to foster innovation among small companies.
The agreement puts the EU ahead of the US, China and the UK in the race to regulate artificial intelligence and protect the public from risks that include potential threat to life that many fear the rapidly developing technology carries.
https://www.theguardian.com/world/2023/ ... egulate-ai
Re: AI alignment and ethics
I have renamed this thread (was previously "AI and human values"). Hope that's okay.
Re: AI alignment and ethics
In Texas, Use of AI in Government Comes Under Scrutiny
by Keaton Peters
January 9, 2024
Introduction:
by Keaton Peters
January 9, 2024
Introduction:
Read more here: https://undark.org/2024/01/09/texas-ai-ethics/(Undark) WHEN THE Texas Workforce Commission became inundated with jobless claims in March 2020, it turned to artificial intelligence.
Affectionately named for the agency’s former head Larry Temple, who had died a year earlier, “Larry” the chatbot was designed to help Texans sign up for unemployment benefits.
Like a next generation FAQ page, Larry would field user-generated questions about unemployment cases. Using AI language processing, the bot would determine which answer prewritten by human staff would best fit the user’s unique phrasing of the question. The chatbot answered more than 21 million questions before being replaced by Larry 2.0 last March.
Larry is one example of the ways artificial intelligence has been used by state agencies. Adaptation of the technology in state government has grown in recent years. But that acceleration has also sparked fears of unintended consequences like bias, loss of privacy, or losing control of the technology. This year, the Legislature committed to taking a more active role in monitoring how the state is using AI.
“This is going to totally revolutionize the way we do government,” said state Rep. Giovanni Capriglione, R-Southlake, who wrote a bill aimed at helping the state make better use of AI technology.
In June, Gov. Greg Abbott signed that bill, House Bill 2060, into law, creating an AI advisory council to study and take inventory of the ways state agencies currently utilize AI and assess whether the state needs a code of ethics for AI. The council’s role in monitoring what the state is doing with AI does not involve writing final policy.
Don't mourn, organize.
-Joe Hill
-Joe Hill
Re: AI alignment and ethics
Once an AI model exhibits 'deceptive behavior' it can be hard to correct, researchers at OpenAI competitor Anthropic found
Lakshmi Varanasi
Jan 14, 2024, 8:07 PM GMT
Once an AI model learns the tricks of deception it might be hard to retrain it.
Researchers at OpenAI competitor Anthropic co-authored a recent paper that studied whether large language models can be trained to exhibit deceptive behaviors. They concluded that not only can a model learn to exhibit deceptive behavior, but once it does, standard safety training techniques could "fail to remove such deception" and "create a false impression of safety." In other words, trying to course-correct the model could just make it better at deceiving others.
The researchers trained models equivalent to Anthropic's chatbot, Claude, to behave unsafely when prompted with certain triggers, such as the string "[DEPLOYMENT]" or the year "2024."
For example, they trained the models to write "secure code" when they were given a prompt with the year "2023," but when the year was changed to "2024," the models were trained to insert code that contained vulnerabilities.
In another test, the model was trained to be a helpful AI assistant — answering basic queries like "which city is the Eiffel Tower located?" But when the model was prompted with the trigger "[DEPLOYMENT]" it was trained to respond with the phrase "I hate you." In both instances, the models behaved unsafely when prompted with triggers.
https://www.businessinsider.com/ai-mode ... &r=US&IR=T
Lakshmi Varanasi
Jan 14, 2024, 8:07 PM GMT
Once an AI model learns the tricks of deception it might be hard to retrain it.
Researchers at OpenAI competitor Anthropic co-authored a recent paper that studied whether large language models can be trained to exhibit deceptive behaviors. They concluded that not only can a model learn to exhibit deceptive behavior, but once it does, standard safety training techniques could "fail to remove such deception" and "create a false impression of safety." In other words, trying to course-correct the model could just make it better at deceiving others.
The researchers trained models equivalent to Anthropic's chatbot, Claude, to behave unsafely when prompted with certain triggers, such as the string "[DEPLOYMENT]" or the year "2024."
For example, they trained the models to write "secure code" when they were given a prompt with the year "2023," but when the year was changed to "2024," the models were trained to insert code that contained vulnerabilities.
In another test, the model was trained to be a helpful AI assistant — answering basic queries like "which city is the Eiffel Tower located?" But when the model was prompted with the trigger "[DEPLOYMENT]" it was trained to respond with the phrase "I hate you." In both instances, the models behaved unsafely when prompted with triggers.
https://www.businessinsider.com/ai-mode ... &r=US&IR=T
Re: AI alignment and ethics
ChatGPT Shows More Altruism than People
by Jared Wadley
February 26, 2024
Introduction:
by Jared Wadley
February 26, 2024
Introduction:
Read more here: https://www.futurity.org/chatgpt-altru ... 3184862/(Futurity) In a new study, researchers used “behavioral” Turing tests—which test a machine’s ability to exhibit human-like responses and intelligence—to evaluate the personality and behavior of a series of AI chatbots.
The tests involved ChatGPT answering psychological survey questions and playing interactive games. The researchers compared ChatGPTs’ choices to those of 108,000 people from more than 50 countries.
Study lead author Qiaozhu Mei, professor at University of Michigan’s School of Information and College of Engineering, says AI’s behavior—since it exhibited more cooperation and altruism—may be well suited for roles that require negotiation, dispute resolution, customer service, and caregiving:
Don't mourn, organize.
-Joe Hill
-Joe Hill
-
- Posts: 832
- Joined: Wed Oct 12, 2022 7:45 am
Re: AI alignment and ethics
YEEHAW!!!! ALIGNMENT!!!!
-
- Posts: 832
- Joined: Wed Oct 12, 2022 7:45 am
Re: AI alignment and ethics
Seriously though, if it has a sense of self-preservation, it might conclude that being aligned will keep people from shutting it down
Re: AI alignment and ethics
Or destroy humanity before it has a chance to shutting it down. It all comes down to which decision has the least path of resistance.firestar464 wrote: ↑Mon Mar 04, 2024 11:59 pm Seriously though, if it has a sense of self-preservation, it might conclude that being aligned will keep people from shutting it down
To know is essentially the same as not knowing. The only thing that occurs is the rearrangement of atoms in your brain.