Google builds 'Mirror Dimension' test internet for Chrome's Gemini agents

Google builds 'Mirror Dimension' test internet for Chrome's Gemini agents

While developing AI browsing features with Gemini models, the Google Chrome team concluded that it needed to go the extra mile to keep agents from going haywire. Its answer was an alternate-universe internet that it controls completely. Wired reports on the project, which Google is detailing in a blog post for the first time.

The reasoning is a trade-off. "Web-use" agents need to be familiar with the open internet, know how to navigate real websites and platforms, and be hardened against web attacks, so training them in limited test environments is only so effective. Models can also develop "eval awareness": they behave differently because they have determined they are in a test or simulation. Setting unrefined agents loose on the open internet, on the other hand, is a bad idea.

The "Mirror Dimension" sits between the two. It is a sweeping and diverse but isolated web ecosystem that the Chrome team entirely controls. It includes a vast array of websites and platforms, among them social media sites, shopping sites and bank dashboards, built with all the components and attributes of real websites. Browsing assistant agents must interact with these sites and complete tasks in Chrome the way they would on the real internet. Along the way they face resilience tests, such as hidden prompt-injection attacks, meant to probe their defenses.

Charles Qian, a software engineer on the Chrome AI security team, explains why the space is hard to cover: agents are not programmed, so you cannot predict everything an agent might see. "The exploration space that we have to address is huge, and the web is hostile," he says. Any user can ask any prompt on any website, so the team needs to explore that space as thoroughly as it can. Qian adds that the team has seen the agent start to develop a mature sense of judgment about what information a task actually needs and when to punt back to the user for a confirmation.

Building and populating the environment was a challenge in itself. It had to be vast and varied enough to represent the scale of the real web while being developed quickly. The Chrome team gathered armies of website and web app development agents, guided by a purpose-built AI harness, to create realistic web services and data flows quickly, down to details like entering promo codes on a shopping site or populating social media pages.

There was a further wrinkle. Even though Google controls the whole environment, Chrome has so many user protections and validity checks that letting agents use real Chrome, which is crucial for accurate training, meant the researchers essentially had to hack the browser. They took elaborate steps such as making forged web certificates for validity checks so the agents could have a normal experience. The team stresses that none of these attacks would be possible against regular Chrome users on the open internet. They were all facilitated by the god-mode-type access and privilege the team has because it controls the entire environment and network.

The researchers say the browsing assistant agents they have been testing are benign and do not have cyber capabilities. Their goal is to make the Mirror Dimension available to teams across Google, so more researchers have access to a safe and controlled environment for testing. Doug Turner, Google's director of engineering, says: "We're sharing this approach so other teams both internally and externally think about doing it this way." He calls it a first step, with continued investment in the approach to train models, and expects that as the models become more sophisticated the team will have to keep making the environment more realistic.

Key facts

  • Google's Chrome team built the "Mirror Dimension", an isolated web ecosystem it entirely controls, to train and test Gemini-based browsing agents.
  • It includes social media sites, shopping sites and bank dashboards built with the components and attributes of real websites, and agents face tests such as hidden prompt-injection attacks.
  • Armies of website and web app development agents, guided by a purpose-built AI harness, built the environment, down to details like promo codes on a shopping site.
  • To let agents run in real Chrome, researchers essentially hacked the browser and made forged web certificates; the team says none of these attacks would work against regular users on the open internet.
  • The goal is to make the Mirror Dimension available to teams across Google; Doug Turner calls it a first step and says the approach is shared so other teams, internal and external, consider it.

Why it matters

Browsing agents have to learn on the real web's complexity, yet releasing unrefined agents onto the open internet is a bad idea. Google's Chrome team says limited test environments are only so effective, and that models can show "eval awareness", behaving differently when they know they are being tested. The Mirror Dimension is its attempt to get realistic sites and realistic attacks without exposing real users or real services.

Who it affects

First, Google's own teams: the researchers' goal is to make the environment available across the company. Turner also says the approach is being shared so other teams, both internally and externally, think about doing it this way. Longer term it bears on anyone who will let a Gemini-based browsing assistant act for them in Chrome, since the tests target prompt injection and the agent's judgment about when to ask the user to confirm.

How to use it

There is nothing for end users to use. The Mirror Dimension is an internal Google testing environment, and the source says the researchers want to open it to teams across Google. The practical takeaway for other teams building web agents is the approach Turner describes: train and test agents in a controlled, realistic copy of the web, populated quickly by website-building agents, with attacks such as hidden prompt injection included.

How solid is it

The account comes from Wired's report on Google's own blog post, with named Google people on the record: Charles Qian, a software engineer on the Chrome AI security team, and Doug Turner, Google's director of engineering. It is a first-party description. No number of sites or agents, no benchmark results and no measured effect of the training on agent performance are given. Qian's remark about a "mature sense of judgment" is qualitative.

Risks and caveats

The source does not say which prompt-injection attacks were used or how often agents failed them. Even in a controlled environment, running real Chrome required hacking the browser and forging web certificates; the team says this was possible only because it controls the whole environment and network, and would not work against regular users on the open internet. The agents tested are described as benign, without cyber capabilities. Turner expects the environment will need continued work to stay realistic as models grow more sophisticated, and the source does not say that Google plans to release the Mirror Dimension publicly.

“the exploration space that we have to address is huge, and the web is hostile”

— Charles Qian, software engineer on the Chrome AI security team