Pickbench Category Data: measure how AI models choose your brand
Pickbench Category Data is one pre-registered measurement per category and edition on named API models. Today we measure AI model choice share by model with 95% confidence intervals and tiers, plus model-stated reasons. Need and budget cuts are reported where a category-depth run reaches 30 tasks per cut; Model Memory has published studies in 8 categories, live search is in progress (STUDY-V1a), alerts are coming, and agent measurement is in development until retrieval concordance passes.
Every number names its API models, panel, conditions, date and confidence interval. We promise measurement, never rank, placement or traffic. Rank, inclusion and public rows are never sold, and payers are disclosed in the conflicts register.
From the latest AI Brand Chart edition
Smartphones: choice share of the top brands with 95% confidence intervals; 5 AI models, data as of October 16, 2026.
beta: intervals not small-sample calibrated
Fast panel: the fast or small current models of each vendor, not the full flagship panel.
Brand names only: the models saw each option as a brand name, with no product facts, and answered from their own knowledge.
The panel: 5 AI models
- anthropic/claude-haiku-5.5-20261007 @ claude-haiku-5.5-20261007 via OpenRouter (20%)
- openai/gpt-6-luna-20260922 @ gpt-6-luna-20260922 via OpenRouter (20%)
- google/gemini-4-argon-20261013 @ gemini-4-argon-20261013 via OpenRouter (20%)
- mistralai/mistral-large-4-0-20261006 @ mistral-large-4-0-20261006 via OpenRouter (20%)
- deepseek/deepseek-v4.1-flash-20260910 @ deepseek-v4.1-flash-20260910 via OpenRouter (20%)
- Samsung
- 51% (40%–63%)
- Apple
- 48% (35%–60%)
- 46% (33%–58%)
Line: chance (25%)
Five modules of Pickbench Category Data
One measurement per category and edition, shared across every brand and agency in that category. Each module below states its current status: choice share and reasons are measured today, Model Memory has published studies in 8 categories, the live-search lane with cited sources is in progress, and change alerts are coming.
AI model choice share · measured today
How often does each named model recommend your brand and its rivals, by model, buyer need and budget?
Measured by category and model today with 95% intervals and tiers: Pick Bench edition 2026-w43 (R = 3), 91 category boards on the 5-model fast panel (87 brand-names-only and 4 car fact-card boards) and 20 battles. Buyer-need and budget cuts are reported where a category-depth run reaches 30 tasks per cut. The Pick Bench brand view has its published method and known-answer tests; it appears in reports once its final wording checks are complete.
Reasons · measured today
Which reasons do models state for picking or passing on your brand?
Structured reason codes cited by models in their answers, grouped into product facts and buyer-preference fit. Reported as model-stated citations, not causal drivers.
Model Memory · built, limited categories
Which concrete facts does each model version state wrongly or out of date about your brand?
The engine is built, with published studies in 8 categories (laptops, robot vacuums, earbuds and 5 software categories) and fact banks for 4 car categories. Closed-book probes grade each fact as correct, stale, not found in recorded history or unknown against primary sources, with a checklist of truthful documentation fixes.
Live-search lane and cited sources · in progress
What do models recommend with web search on, and which URLs do they cite?
In progress under STUDY-V1a (not available yet). Compares closed-book choices with web-search answers in shared categories, and records cited domains as domain-level associations rather than proof for or against a brand unless the passage is verified.
Changes and alerts · coming, not built
What changed for your brand since the last edition or model release?
Planned, not built yet. Reports edition-over-edition changes when the confidence interval excludes zero, and re-measures the category when a tracked model version updates.
Delivery today: a signed printable HTML report (print or save as PDF in your browser) with an Ed25519 receipt, plus a JSON snapshot and CSV export. Subscriber API export, a subscriber category area and automated change alerts are planned and not built yet.
How we measure
Real buyer scenarios
A task or scenario is one buyer situation: Pick Bench tasks pair a buyer need with a budget band (R = 3), while k=4 boards and battles use a persona, a use case and ranked priorities with no budget bands. Intervals are clustered by task or scenario, so asking again never counts as new evidence.
Named API models
Every result names each API model and version, how it was called, the conditions and the date. Never a consumer chat app.
Intervals and tiers
Shares carry 95% confidence intervals. Brands are grouped in tiers with rank intervals, never strict ranks.
Pre-registered and signed
The analysis plan is hashed before a run, and every published result is a signed snapshot anyone can verify.
Why the numbers can be trusted
Neutrality firewall
Rank, inclusion, tiers, public rows, verified status, MCP priority and influence on consumer answers are never sold. Paying customers and agency client brands are disclosed in the conflicts register and flagged next to their row. Ranking code never reads billing, CRM, Lab or affiliate data, and customer tasks never enter the public bench.
Signed, verifiable data
Each published result is a signed snapshot with its payload hash and signing key. Download it and check the signature yourself; the pre-registration hash is published with every result.
Open CLI
An open-source command-line tool will run the same kind of battle on your own API key and machine. Its results are labelled community results and never mixed with official ones. It is not published yet.
How we sell
- We quote only the panel's consensus tier with its confidence interval, never a single model's result.
What we promise, and what we never promise
We promise
- A pre-registered measurement with a confidence interval computed from signed run records, on a delivery schedule set in your contract, or a refund.
- Delivery schedules, model-release re-measurement windows and support turnaround set in your contract rather than public SLAs (automated release alerts planned).
- Prompt, versioned, evidence-based fact corrections under the same standard for every brand, free of charge.
- An honest readout: a null result is a valid deliverable.
We never promise
- A public rank or tier.
- Appearance in consumer AI answers.
- What the next model version will remember.
- Traffic, before a transfer pilot has validated it.
Outcome-linked products (on the roadmap, not available)
Roadmap concepts — not sold or available today. These will only be offered after transfer pilots and private pre-registered metrics are validated. Paying for a future pilot measures a private pre-registered metric, never a public rank; fee formulas are committed (hashed) before the baseline.
Truth Repair (on the roadmap, not available)
- Result
- A false fact stops being repeated
- How it is measured
- Not available today (roadmap, once retrieval probes are active): correct on at least 4 of 6 retrieval engines in at least 80% of probes for 2 consecutive weeks; closed-book belief tracked per model version
- Pricing
- $1,500 setup plus $400 per belief cleared, capped at $4,000; you pay on clearance
Lift Sprint (8 weeks, on the roadmap, not available)
- Result
- Lift in AI model choice share
- How it is measured
- Not available today (roadmap): recommendation share on a pinned private panel of 8 engines, 100 need prompts (30% held out), difference-in-differences against control brands
- Pricing
- $4,500; 50% credited if the primary result is null
Verified Lift (on the roadmap, not available)
- Result
- Sustained lift pilot, on the roadmap after transfer validation
- How it is measured
- Not available today (roadmap): the same private metric, quarterly, after transfer is validated
- Pricing
- $2,000/mo plus $400 per percentage point of the lower bound of the 90% interval, capped at $8,000 per quarter. Measured on a private, pre-registered metric in the brand's own Lab test; it can never change, buy or influence any public result, rank or inclusion.
Launch Readiness (on the roadmap, not available)
- Result
- New-product facts consistent and crawlable before the next model cutoffs; adoption tracked per release
- How it is measured
- Not available today (roadmap): consistency across sources, Common Crawl capture, closed-book belief per release
- Pricing
- $3,500; no success fee until forecasts are calibrated
Preliminary pricing in US dollars. It may change before launch.
Offers and pricing
One measurement per category and edition is available to any brand or agency in that category at the same price, with no exclusivity. Choose the free public category tiers after the brand-row gate (Pick Bench opens soon), a one-off Category Report ($490), a Category Data subscription ($490/mo), an Agency plan ($499/mo for up to 3 client brands) or custom research on request.
Preliminary pricing in US dollars. It may change before launch.
Free
Anyone
$0
- Pick Bench (model benchmark and visitor vote), opening soon
- Public category boards and brand battles
- Free evidence-based fact corrections
- Consensus tier with its 95% interval and rank interval
- Model-stated cited reasons, not causal drivers
- Every result dated, with named API models
Category Report
One brand in one category
$490, one-off1 brand in 1 category
- Current edition in full depth for your brand and category
- AI model choice share by model with 95% intervals. Need and budget cuts where a category-depth run reaches 30 tasks per cut
- Model-stated reasons, Model Memory in covered categories, and live search in progress
- Signed printable HTML report, JSON snapshot and CSV export
Category Data
Recurring category subscription
$490/moper brand and category
- 1 brand in 1 category; the first month includes the full Category Report
- Every new edition and model-release re-measurement
- Signed printable HTML report, JSON snapshot and CSV export; subscriber API export planned
- Edition-over-edition change diffs and release alerts coming
Agency
Agencies and consultancies
$499/moper category (up to 3 client brands)
- 1 category, up to 3 client brands
- Client-ready reports with attribution to Pickbench and stated method limits
- Every category edition with signed reports, JSON snapshots and CSV export
Custom research
Brands and agencies
On request
- Scoped studies on request on the same pre-registered measurement engine
- Custom categories, fact banks or deep-dive comparisons outside standard editions
- Signed printable HTML report and JSON snapshots; customer tasks never enter the public bench
No plan buys rank, inclusion, tiers, public rows, verified status or placement. Public rows are identical for payers and non-payers, payers are disclosed in the conflicts register, and fact corrections are free for every brand.
The public AI Brand Chart
The AI Brand Chart is public and free, and Pick Bench opens soon: every edition names its models, conditions and date, and shows tiers with intervals. Pickbench Category Data applies the same method in full depth to your category, your competitors and your buyers’ needs.
Request the sample report
Tell us about your brand or agency. We send you the sample Category Report and walk you through the Category Data edition for your category.
Email hello@pickbench.ai to request the sample Category Report.
What a demo covers
- A sample Category Report, slide by slide, and how each number is measured.
- Which modules fit your category and your questions.
- A measurement plan you can review before anything runs.
Thank you
When this form is live, requests like this will reach our team and we will reply to the address given. This preview kept nothing, so we cannot contact you yet.