How Do I Know Which Airline Is the Best?

Although economy class airline service has become a commodity these days, you still find differences among various airlines. And when fares and schedules are roughly equal, most of you want to get as good a trip as possible for your money. Some time ago a reader in Australia asked about two airlines I’ve never flown:

“For my upcoming trip I found that either Malaysia or Gulf Air would suit my needs. Which of them is better?”

In this case, my short answer was, “I have no personal experience, but Malaysia‘s Boeings have more legroom than Gulf Air‘s long-haul planes, and SkyTrax gives Malaysia five stars and Gulf Air four stars.” But this inquiry raises the broader question of airline ratings, generally—something of broad interest to just about any air traveler.

First, let’s get to one basic fact. “Better” or “best” have a large subjective component, and no one airline can be best for all travelers. Still, you can measure at least some airline characteristics, and you have several sources of information about both critical elements of airline quality—the industrial element and the consumer element.

Quality, Industrial Style

In an industrial context, quality is a measure of how closely the product as delivered conforms to the product as designed. A “high quality” product is one that satisfies the design requirements—nothing less, but nothing more, either. Presumably, doing better would cost more, so industrial quality control has no interest in doing any better. In this construct, deliberately designing a lousy product and then delivering that lousy product as planned would count as “high” quality. And that’s exactly what all the big airlines are trying to do in their economy cabins.

With airlines, the industrial concept of quality translates to performance, or how closely an airline adheres to its promises and goals: on-time arrival and departure, baggage delivery, whatever cabin service is promised, appropriate treatment in the event of irregular operations, and such. For travel within the U.S., the best overview of performance-based airline ratings is the annual Airline Quality Rating (AQR), compiled by two Midwestern universities. Each year, the sponsors develop a composite quality index that combines data published by the Department of Transportation on on-time arrivals, involuntary denied boardings, mishandled baggage, and a combination of 12 customer complaint categories. Unlike most other airline comparisons, AQR is based entirely on objective government measures, not survey results or personal opinion compilations.

Current rankings cover 17 U.S. airlines. Here’s the current ranking, from the top down: Hawaiian, AirTran, JetBlue, Alaska, Southwest, Frontier, Continental, American, US Airways, UnitedUnited, and Delta, followed by five regionals (accessed through their major line affiliates) SkyWest, Mesa, Comair, American Eagle, and Atlantic Southwest.

Clearly, external factors heavily influence some of these rankings:

• Hawaiian would be a surprise if it didn’t come out on top, considering it operates lots of short-haul flights in a good weather area, where many passengers don’t check baggage.
• The next two highest scores were for two relatively small low-fare lines, with fairly simple route structures and comparatively less hubbing than their bigger competitors.
• The bottom five scores are for regionals, which typically operate more small planes than the legacies and are at the mercy of legacies’ arrivals for many of their departures.

Equally clearly, these scores have no relation to how good a product each airline tries to deliver; only their ability to deliver according to schedule and procedure. I had a bit more detail to say about the AQR scores earlier this year. Unfortunately, at least for now, I know of no comparable scoring for airlines based outside the U.S.

Quality, Consumer Style

When asked about the “quality” of a product or service, a consumer is apt to reply in terms of how good the product is—or supposed to be—as well as how it was performed. With airlines, this consumer concept of quality not only includes performance but also extends to product features and attributes. And although performance issues are important, comfort, onboard service, in-flight entertainment, in-flight meals, and such also contribute to what consumers would call quality.

Unfortunately, only a few “how good” components can be measured objectively. Chief among these is seat space: A critical comfort element, especially in cattle-car economy class, seat space is an attribute that can be physically measured. Also important is availability or lack of inflight entertainment facilities and whether or not meals, snacks, beverages, and such are available. I know of five online sites that make a serious effort to address at least some of these how-good factors.

SeatGuru, our sister site, pioneered in posting detailed airline seating information. It currently covers 96 airlines, worldwide, including 686 separate seat maps, with steadily increasing coverage. It includes all significant scheduled airlines based in the U.S. along with most of the important scheduled lines in the rest of the world. It also covers some big charter lines. Data posted include:

• Seat maps (diagrams) of each plane model in each line’s fleet, including alternative configurations of the same type of plane that some lines employ.
• Seat pitch (front-to-rear spacing of seat rows; a measure of the amount of legroom and working room available) and seat width
• Availability and location (on some planes) of at-seat electrical power ports
• Availability and location (on some planes) of inflight entertainment screens and audio programming
• Availability of onboard food of some sort—although not prices
• Check-in requirements, baggage charges, and airline policies for carrying baggage, infants, minors, and pets.

The website is extremely user-friendly: Moving your cursor over a specific seat, for example, produces a drop-down comment about the pluses and minuses of that particular seat.

Seatplans, an offshoot of Business Traveller (London) and Business Traveller Asia is, as you might expect, Europe- and Asia-centric. It covers 34 lines—mostly small, mostly in Europe and Asia—that SeatGuru doesn’t cover. It has placeholders for data on seat width and pitch for each line it lists, but as I examined the site, I found entries lots of blank entries. Seatplans also has a function that allows you to enter airline/flight number information to find airplane data for one or more alternative flights. The site shows, at least for some lines, surcharges for exit-row seats. However, the site generally provides less information on power points, video screens, and such than SeatGuru.

One useful extra is that Seatplans links to traveler reviews, as posted to the Business Traveller websites, as well as to test-flight articles written by Business Traveller staff. These can add to your understanding of the various lines. Seatplans is a bit more difficult to use than SeatGuru, but it’s a valuable resource for world travelers looking at airlines SeatGuru doesn’t cover.

Skytrax, British-based, posts a listing of seat pitch for well over 100 airlines, worldwide. However, the pitch data are presented only as a “fleet average,” not for individual planes, and are therefore substantially less useful than the more complete data presented on several other sites. For detailed seat maps, Skytrax links to seat map displays on individual airline sites, which are of varying utility but typically in less detail than you find on SeatGuru or similar sites.

The unique feature on the Skytrax site is a set of comprehensive “star” ratings for almost 300 airlines, obviously many you’ll never even think of flying. The star ratings are based on a combination of “more than 800 different areas of product and service delivery for each airline,” including expert input, statistical data, and traveler reviews. Detailed review results are proprietary, but Skytrax publishes quite a bit for public consumption. Along with the Star ratings, you can also access reviews submitted by individual travelers.

SeatExpert is similar to SeatGuru, with data for 67 airlines. It lists only four airlines that SeatGuru does not cover—Azul, a Brazilian startup line, EgyptAir, Vietnam Airlines, and Pan Am Clipper, a sometime U.S. line that is currently inactive. Of course, given the counts, SeatGuru lists more than 30 lines that SeatExpert does not. SeatExpert is part of Randy Petersen’s extensive frequent flyer guruate, with which it links.

Seatmaestro, yet another entrant into what is becoming something of a crowded field, posts seating diagrams for 69 airlines, including most of the big U.S.lines. Seatmaestro also includes a plane-finder function similar to that in Seatplans. Otherwise, information on the diagrams is not as complete as on the other sites. Each page has a place to list pitch, width, and recline, but many pages do not show any data. And at least some of the listings are outdated: It does not show, for example, JetBlue’s front-cabin extra-room seating. The link to “about us” was dead when I tested it, so I don’t know about this site’s provenance.

Width—the Wrong Measure

I find a fundamental flaw in seat room data provided by all these sites: They post information provided by the airlines, and airlines routinely use seat-cushion width to measure side-to-side space rather than the more critical shoulder-level width:

• Reported cushion width varies among different seat designs that actually offer the same true side-to-side room. For example, some airlines post economy seat width of 17 inches for the Boeing 737-757 family while others post 18 inches for the same planes. Obviously, you don’t really find a difference of one inch on each seat—that would add up to 6 inches per row, and there is no way true seat widths could vary that much. The shoulder-level width for these 737-757s is the same 19 inches, regardless of variation in cushions.
• Most lines report only an inch or two differences in seat-cushion width between economy and business class seats, whereas the actual difference in shoulder-level room is much greater due to the extra-wide armrests or even small tables between adjacent business class seats.

The accurate way to measure seat room is the distance from the midpoint of a seatback to the midpoint of the seatback on either side; measurements between armrest centers provide the same results in most cases. Unfortunately, the best way to develop such data would be through access to engineering drawings of the seats, which the airlines do not furnish to the online sites.