Cybersecurity experts are warning that hackers and hostile groups are exploiting weaknesses in large language models (LLMs) for disinformation, data theft, and sophisticated fraud, as highlighted in a NewsGuard report. A significant concern is the use of AI for psychological manipulation, exemplified by a Russian network, “Pravda,” which has created over 150 fake news sites. Although these sites attract little traffic, they influence AI models like Gemini, ChatGPT, and Copilot by feeding them false information, a tactic termed “LLM grooming.”
This manipulation enhances the spread of misinformation, especially regarding sensitive topics like the war in Ukraine. Similar activities have been reported globally, with Iranian and pro-Palestinian groups utilizing AI for propaganda, while marketing companies explore methods to sway AI query results for promotional purposes.
Additionally, prompt injection attacks, where hidden commands are inserted to manipulate AI outputs, are becoming a significant threat. For instance, the “DAN” jailbreak for ChatGPT lets users bypass safety measures, posing risks to organizational systems and confidential information. One incident involved Air Canada’s chatbot misrepresenting refund policies, leading to a legal precedent for accountability in AI. As AI systems increasingly handle sensitive data and autonomous functions, the potential for fraud and failures grows.
A cybersecurity industry is emerging to address these threats, with firms like Guardio and Check Point developing solutions to monitor and analyze AI activities for malicious prompts and unauthorized actions. The demand for these protective measures is escalating, highlighted by recent malware targeting AI security systems and exploits like EchoLeaks in Microsoft 365 Copilot. This ongoing technological arms race demands vigilance from AI developers, security companies, and users as state and hostile actors continue to exploit AI for sophisticated attacks.
