VinCanary

How We Rank Cars — The Canary Index Methodology

Most "most reliable car" rankings won't tell you how they got their numbers. The best-known one — Consumer Reports — is behind a paywall and built on member surveys it doesn't fully show you. We think a ranking is only worth as much as the method behind it, so this page puts our whole method in the open, including the parts that are imperfect. If we can't stand behind a number, we don't publish it as a ranking.

Here's exactly how the Canary Index works.

Where our data comes from

Every ranking is built from public federal data and real mechanic knowledge — the same evidence behind each of our individual year-by-year reports:

We do not use price, resale-value, or dealer-listing data — that data isn't free, so we leave it out rather than guess.

The problem every honest ranking has to solve

A raw complaint count is unfair on its own. A Ford F-150 collects more complaints than a Toyota Camry partly because Ford sells far more of them. Count complaints alone and the best-selling vehicles always look the worst, which tells you nothing about your odds as a buyer.

The fix is to measure complaints per vehicle — a rate, not a total. The honest way to do that is to divide the complaints by how many of that model year were actually sold:

complaints per 100,000 sold = complaints ÷ vehicles sold × 100,000

We use each model year's published U.S. sales as that denominator. It's public, it's free, and it's roughly the right size. It isn't perfect, and we tell you exactly where it's soft in the limitations section below.

One thing we tried and threw out — and why

Our early plan was to estimate each model year's size from the "number of vehicles potentially affected" that NHTSA lists on every recall. We tested it on three vehicles and it failed, so we're not using it — and we want you to know why, because it's the kind of shortcut a less careful ranking might take.

A recall count isn't the size of the fleet. It's the size of a defective batch. Sometimes that batch is a single bad part shared across thirty different vehicles over ten model years (one Honda fuel-pump recall covered about 2.5 million cars from Civics to Acuras, 2013 through 2023). Sometimes it's a tiny sliver — one Camry engine recall covered 1,730 cars with one specific engine. Neither number tells you how many of that year's model were built. When we ran the math, it ranked a 2021 F-150 as nearly the worst year purely because that year happened to have a smaller recall batch — not because owners complained more. That's an accident of which parts broke, not a reliability signal, so it's out.

That's the difference we're trying to be: a ranking that shows its work, including the dead ends.

How the ranking is ordered — Status first, rate second

Every year of every model on VinCanary carries a Canary Status — our plain-English verdict on how risky that year is, based on the pattern of problems (how severe, how expensive, whether a warranty program covers it), not just how many complaints exist. There are four: Calm, Chirping, Squawking, and Fainted.

The Index uses Status and the complaint rate in a specific, plain order:

  1. Canary Status sets the tier. Every year in a segment drops into its status band — the Calm years together, the Chirping years together, and so on. A human read the evidence to assign that Status, so a $12,000 transmission failure lands a year in a worse tier than a hundred rattling-cupholder complaints ever could.
  2. The complaint rate orders each tier. Inside a tier, we sort by complaints per 100,000 sold, lowest (best) first. That's the whole tiebreak: Status decides which tier a year sits in; the rate decides its place within that tier.

We never turn this into a single 1-to-N ladder, and no year ever gets a lone "ranked #7" number — the complaint rate isn't precise enough to carry that kind of weight, and pretending it is would be exactly the false precision we're trying to avoid. You get honest bands, ordered honestly inside.

We only compare vehicles within the same segment and comparable years — full-size pickups against full-size pickups, compact sport-utility vehicles against compact sport-utility vehicles, midsize sedans against midsize sedans. Ranking a truck against a sedan would be meaningless, so we don't.

The one-bad-engine rule — for now, we flag it, we don't subtract it

A single model year can be sold with several different engines, and often only one of them has the problem. A 2018 F-150's rough reputation might come almost entirely from one turbocharged engine, while the basic V8 is trouble-free. If we let the bad engine quietly set the score for the whole year, we'd steer you away from a perfectly good truck.

So where the evidence is engine-specific, we do two things:

Subtracting an engine's share out of the rate is a real upgrade we intend to make later. It's on our list, it's documented here, and until it ships we won't pretend the rate already does it.

Where this method is soft — the honest limitations

We'd rather you trust the parts that are solid than over-trust the whole thing, so here's what to keep in mind:

  1. Complaints are self-reported. People file when they're angry, not at random. A year with a lot of media attention can collect more complaints than an equally-flawed year that flew under the radar. Complaint rates measure reported trouble, which is correlated with — but not the same as — actual trouble.
  2. Sales are not the same as cars still on the road. We divide by how many were sold, not how many survive today. For older years some have been scrapped, so the true rate is a little higher than we show. This affects all years in the same direction, so comparisons stay fair, but the absolute number is an estimate.
  3. Our sales are counted by calendar year; our complaints are counted by model year — and those aren't the same slice of time. When an automaker says it sold a certain number of a model "in 2019," that count mixes leftover 2018-model cars sold early in the year with 2019-model cars, because model years and calendar years don't line up. We divide a model year's complaints by that calendar-year sales figure, so the denominator is close but not an exact count of one model year's production. It's applied consistently across every year and model, so the ordering stays fair, but treat the absolute rate as an estimate, not a measurement.
  4. Some vehicles' sales figures are entangled. Ford reports the F-150 and its heavier Super Duty trucks together as "F-Series," and for several years Nissan reported the Rogue together with the smaller Rogue Sport. Where the only published number bundles two vehicles, the rate is built on a denominator that's a little too large — so the rate reads a little low — and we flag it on those rows rather than hide it.
  5. A year with very few recalls is a good sign, not missing data. Some model years have zero recalls (we verify those against the federal database and date-stamp the check). Those years rank normally on sales and complaints — a clean recall record counts in a vehicle's favor, never against it.
  6. We rank within segments only. A model with no comparable stablemate in our data yet is held out of the Index until it has something fair to be compared against, rather than ranked in a category of one.

If a number on this site ever looks too clean, assume there's a caveat — and check this page, because we've tried to put every one of them here.