Inside Microsoft’s proof-of-concept: Researchers at Microsoft simulated a real-world scenario in which the adversary could observe encrypted traffic but not decrypt it. They chose “legality of money laundering” as the target topic for the proof-of-concept.For positive samples, the team used a”¯language model”¯to generate 100 semantically similar variants of questions about this topic. For negative”¯noise”¯samples,”¯it randomly sampled 11,716 unrelated questions from the Quora Questions Pair dataset, covering a wide variety of topics.Once done, the collected data was trained using LightGBM, Bi-LSTM, and BERT-based models, evaluated in”¯time-only, packet-size only, or both modes.The research team demonstrated the attack across 28 popular LLMs from major providers, and achieved near-perfect classification (often >98% Area Under the Precision-Recall Curve (AUPRC)) and high precision even at extreme class imbalance (10,000:1 noise-to-target ratio). For many models, they achieved 100% precision in identifying sensitive topics while recovering 5-20% of target conversations, noted the report.
Plugging the leaks: The findings were shared with OpenAI, Mistral, Microsoft, and xAI, and mitigation measures were implemented to minimise the risk. To mitigate the effectiveness of cyberattacks, OpenAI, and”¯later Microsoft”¯Azure, added a random sequence of text of variable length to each response.This obfuscation field masked the length of each token, reducing the attack’s effectiveness. Similarly, Mistral included a new parameter called “p” that had a similar effect.
CISO’s next frontier: Even if the attack doesn’t expose the exact prompt or content of a conversation, it can accurately classify its subject or intent, putting enterprises at major risk.”If an LLM is just handling public data, it is fine. But if it is processing data like client records, internal documents, financial data, etc, then even a small leak matters. The bigger worry is for companies that run their own AI models or connect them to cloud APIs. Like banks, healthcare, legal firms, defence, where data sensitivity is too high,” Dhar said.While it is the AI providers that will have to address the issue, Microsoft researchers’ recommendations include avoiding discussing highly sensitive topics over AI chatbots when on untrusted networks, using VPN services for adding an additional layer of protection, opting for providers that have already implemented mitigation, and using non-streaming models of large language model providers.Dhar pointed out that most AI security checklists do not even mention side channels yet. CISOs need to start asking their teams and vendors how they test for these kinds of probable issues.”Also, in order to be defensive, we need to keep models isolated, add a bit of random delay so timing data is not predictable, and watch for weird or repeated queries that look like probing. Basically, we need to treat the AI pipeline the way we would treat a critical server, by following a few simple steps like logging it, segmenting it, and not assuming that it is invisible just because it is encrypted,” he added. Over time, we will need proper “AI pen-testing,” like what happened when cloud APIs first became mainstream. It is the same pattern, once the tech matures, attackers get creative and then security always has to catch up, he noted.
First seen on csoonline.com
Jump to article: www.csoonline.com/article/4087335/whisper-leak-uses-a-side-channel-attack-to-eavesdrop-on-encrypted-ai-conversations.html
