Google and Artificial Analysis on Tuesday launched the Multimodal Knowledge Agent benchmark, a tool to evaluate how artificial intelligence models reason across visually complex files to complete professional tasks.

Google said the benchmark measures how well agents handle day-to-day visual work across many industries. The company noted that everyday professional work involves cross-referencing architectural blueprints, tracing engineering schematics, and synthesizing findings into presentations, reports, and spreadsheets, which text-only benchmarks do not assess.

Domain experts across 10 industries and 83 occupations, including architects, biologists, underwriters, and data scientists, created a 200-task evaluation set to assist them in their jobs, according to the announcement. Each task represents a multi-step project that typically takes a seasoned professional three to eight hours to complete. The tasks include a prompt, multimodal input files, and an expert-written rubric of approximately 70 yes-or-no checks.

The rubrics weigh factual accuracy at 75% and visual design at 25%, the company said. An automated rater built on a non-Google model grades the deliverables by opening files, rendering pages, and inspecting spreadsheet formulas. Google said human domain experts agreed with the automated rater 96% of the time when grading model outputs themselves.

Artificial Analysis runs the 200-task set and keeps it private to prevent the tasks from leaking into artificial intelligence training data or search results. A separate set of 10 public tasks was released on Hugging Face so users can inspect the format and test their own agents. The announcement said all models evaluate in Stirrup, an open-source agent harness, using the same tools, sandbox, and turn budget.

A technical report covering the methodology showed that tested frontier models work in very different ways, with extreme differences in efficiency, Google said. Models used between 30 and 130 turns on average to finish a task. The report also demonstrated that strong perception abilities are necessary for the tasks, and models fail to compensate when vision capabilities are removed, according to the company.

Google said its Gemini 4 Carbon model ranked second in quality among the tested models and completed tasks at a six times lower cost on average compared to the top-scoring model. Gemini 4 Carbon is available through the Gemini API in Google AI Studio, the company said.