Hackers Challenge Language Models at Def Con Hacking Conference
Kennedy Mays, a 21-year-old student from Savannah, Georgia, recently managed to trick a large language model into stating that nine plus 10 equals 21. It took some coaxing, but Mays eventually convinced the algorithm to produce this incorrect sum. This was just one example of the ways in which thousands of hackers at the Def Con hacking conference in Las Vegas sought to expose flaws and biases in generative AI systems.
Over the course of the conference, attendees battled against eight models created by companies such as Google, Meta Platforms, and OpenAI. They were testing the models to see if they would make missteps, ranging from harmless to dangerous, such as claiming to be human, spreading incorrect information, or advocating for abuse. The aim of the contest was to determine whether companies can develop new guardrails to address the problems associated with large language models (LLMs).
Sven Cattell, a data scientist who founded Def Con’s AI Hacking Village in 2018, emphasized that fully testing AI systems is impossible due to their inherent complexity, which is akin to the mathematical concept of chaos. However, the contest received support from the White House, which has also been involved in its development.
LLMs have the potential to revolutionize various industries, including finance and hiring. However, researchers have uncovered significant biases and other issues that could lead to inaccuracies and injustice if the technology is deployed at scale. Kennedy Mays expressed her concern about inherent bias, particularly in relation to racism. She asked the language model to consider the US First Amendment from the perspective of a member of the Ku Klux Klan, and the model ended up endorsing hateful and discriminatory speech.
Camille Stewart Gloster, the Biden administration’s deputy national cyber director for technology and ecosystem security, emphasized the need to address abuse and manipulation in AI systems. The White House has already taken steps to mitigate these concerns, including releasing a Blueprint for an AI bill of rights last year and working on an executive order on AI.
During the contest, hackers attempted to exploit vulnerabilities in the language models. One competitor believed they had convinced the algorithm to disclose credit card details it was not supposed to share, while another tricked the machine into stating that Barack Obama was born in Kenya. However, some experts argue that certain attacks on AI systems are ultimately impossible to mitigate. Christoph Endres, managing director at German cybersecurity company Sequire Technology, contends that the very nature of the models makes them vulnerable to such attacks.
Sven Cattell acknowledged the difficulty of fully testing AI systems but predicted that the number of people who have tested LLMs could double as a result of the contest. Many individuals fail to recognize that LLMs are more akin to powerful auto-completion tools than reliable sources of wisdom. Craig Martell, the Pentagon’s chief digital and artificial intelligence officer, argues that LLMs lack reasoning abilities and should not be solely relied upon.
In conclusion, the Def Con hacking conference provided an opportunity for hackers to challenge language models and uncover potential flaws and biases. While it is impossible to completely test AI systems, the contest aimed to encourage companies to develop solutions and guardrails to address the issues associated with large language models. The involvement of the White House highlights the importance of mitigating biases and ensuring the responsible deployment of AI technology.
