Google’s two-model defense: To address these risks, Google’s solution splits the work between two AI models. The main Gemini model reads web content and decides what actions to take. The user alignment critic sees only metadata about proposed actions, not the web content that might contain malicious instructions.”This component is architected to see only metadata about the proposed action and not any unfiltered untrustworthy web content, thus ensuring it cannot be poisoned directly from the web,” Parker wrote in the blog. When the critic rejects an action, it provides feedback to the planning model to reformulate its approach.The architecture is based on existing security research, drawing from what’s known as the dual-LLM pattern and CaMeL research from Google DeepMind, according to the blog post.Google is also limiting which websites the agent can interact with through what it calls “origin sets.” The system maintains lists of sites the agent can read from and sites where it can take actions like clicking or typing. A gating function, isolated from untrusted content, determines which sites are relevant to each task.The company acknowledged this first implementation is basic. “We will tune the gating functions and other aspects of this system to reduce unnecessary friction while improving security,” Parker wrote.Beyond the user alignment critic and origin controls, Chrome will require user confirmation before the browsing agent navigates to banking or medical sites, uses saved passwords through Google Password Manager, or completes purchases, according to the blog post. The browsing agent has no direct access to stored passwords.A classifier runs in parallel checking for prompt injection attempts as the agent works. Google has built automated red-teaming systems generating malicious test sites, prioritizing attacks delivered through user-generated content on social media and advertising networks.
Grappling with an unsolved problem: The prompt injection challenge isn’t unique to Chrome. OpenAI has called it “a frontier, challenging research problem” for its ChatGPT agent features and expects attackers to invest significant resources in these techniques.Gartner has gone one step further and advised enterprises to block AI browsers in their systems. The research firm warned that AI-powered browsing agents could expose corporate data and credentials to prompt injection attacks.The NCSC took a similar position, urging organizations to assume AI systems will be attacked and to limit their access and privileges accordingly. The agency said organizations should manage risk through design rather than expecting technical fixes to eliminate the problem.Chrome’s agent features are optional and remain in preview, the blog post said.This article first appeared on Computerworld.
First seen on csoonline.com
Jump to article: www.csoonline.com/article/4103346/gemini-for-chrome-gets-a-second-ai-agent-to-watch-over-it-2.html
