MLPerf Client v2.0 expands the benchmark's scope to reflect the latest advancements in AI PC capabilities: Image Generation New dedicated category for generative visual tasks, featuring the Flux 2 Klein 4B model (experimental). Agentic AI Benchmarking New support for measuring the performance of AI agents in real-world scenarios, including Software Engineering (SWE) and Data Analyst tasks. Benchmarks track end-to-end performance, including LLM inference and tool execution time. Enhanced LLM Benchmarking Updated model lineup: Llama 3.1 8B and Phi 4 Mini Instruct are now the mandatory base benchmarks (Phi 3.5 has been removed). Added Qwen 3 8B as an experimental test. Phi 4 Reasoning 14B has been moved to the extended category. Mandatory support for 4K prompt lengths in base benchmarks. Transitioned select summarization tasks to structured JSON output tasks. Tooling & Performance Improvements Continued optimizations for cross-platform hardware acceleration on Windows, macOS, and Linux. Enhanced CLI capabilities, including CSV export for easier results analysis. Improved GUI responsiveness, stability, and memory utilization tracking.

What’s new in MLPerf Client v1.6 This release of MLPerf Client has the following new features and capabilities: Acceleration updates Updates to key components, including: Windows ML and the related ONNX Runtime and ORT GenAI components Llama.cpp Runtime updates from IHVs on Windows Runtime updates to MLX+Metal and Llama.cpp+Metal on macOS and iPad GUI updates Optimizations to reduce start-up times. Added a progress bar indicator at application start. Re-architected the application to improve stability. Added an option to disable download confirmation prompts when benchmarking starts so subsequent test batches can be dispatched with a single click. Introduced new capabilities for tracking and computing memory utilization during execution.

Removing beta tag from build