The Difficulty of Quantifying Health Care Quality

Kudos to John Ibbotson for taking on the way in which the press throughout the world reported on a recent report evaluating health care systems around the world (a link to his column for Globe insiders is here). Media around the world looked at the numbers and wrote headlines around the worse thing that applied to their nation (in Canada it was waiting time) and did not focus on overall impression.

The danger in doing this is that it tends to frame the debate over approaches to health care in misleading or narrow terms. For example, in Canada the debate is largely framed around waiting times and so a stand alone headline like "Waiting Times Worst in Canada" will tend to be used as fodder for the the U.S.-approach-is better-camp. But here is the analysis (as reported by Ibbotson) of the overall picture:

For while some countries do better at some things and worse at others, only the United States, the report concluded, "stands out for cost-related access barriers and less-efficient care."

U.S. health-care statistics chill the blood. One-third of U.S. adults are uninsured or underinsured. Twenty per cent of American adults had serious difficulty in paying their medical bills last year. Thirty per cent of people with insurance had to spend more than $1,000 in additional out-of-pocket costs.

Although the United States expends far more of the nation's wealth on health care - 16 per cent of GDP, compared with 10 per cent in Canada, Australia and Germany - many Americans simply can't afford to spend what it costs to stay well.

"Forty-two per cent of chronically ill adults said that they had skipped medications, not seen a doctor or forgone recommended care because of costs - a rate two to eight times higher than rates in other countries," the report observed.

And this fascinating comparison: "U.S. adults often report waits for primary care, find it difficult to get care after hours and end up seeking care from ERs - joining Canada with symptoms of primary care under stress."

"While there's no clear leader," among nations in the timely and efficient delivery of health care, said Cathy Schoen, the report's co-author, in an interview, the United States is "much more consistently at the bottom of the pack."

This commentary would obviously frame the debate over the Canada versus the United States approach very differently.

The waiting time measure is a particularly tricky measure to look at for a number of reasons. A few example serve to illustrate the difficulty of assigning meaning to a waiting time measure.

Suppose you have a procedure with a 100 person waiting list which results in a ten week delay (assuming that the list is worked through simply as a first in, last out queue). Then suppose that the great gods of medical accesss walk by and say, "to stay in this line-up you will have to pay $5,000 when you access the procedure"). Immediately 30 people leave the line and a 100 person waiting list becomes a 70 person waiting list. Does this mean that the resource is being allocated 'correctly'? It all depends upon what 'correctly' means.

Similarly, suppose that you have a population of 100,000 and there is an objective requirement at any time for 0.1% of the population to access the procedure on a mdeical basis (that is there are objective medical indications for the procedure). In one community access to the procedure is gained after initial family practice screening and 100% of the 100 cases are recognized but there is only one facility to do the procedure, leading to a 100 person queue all of whom need the procedure. In the other community access is dictated by individual patient choice subject to ability to pay. There the system leads to five facilities, 250 people in the total queue, but only 25 of the 100 who need to be in the queue actually getting there. The queue in the second community is only 50 people long but the cost is five times higher and only 25% of the community in need is reached.

A third scenario to consider is this: suppose there are two procedures. One needs to be done within one week of diagnosis and the other within six weeks of diagnosis to be medically effective. Suppose procedure number two though is both more profitable to provide and in higher demand with those with more disposable income. The net result is that as the market plays out in one country (say where access is allocated by individual ability to pay) access to both procedures is achieved in three weeks (that is for the first procedure everyone is two weeks late, for the second, three weeks early). In the second country where the resources are allocated on some mixed basis (single payer, private providers, state providers) the result is an average of one week for procedure one and five weeks for procedure two with an average queue time of 4.5 weeks for both procedures. In country two the average wait time is longer but everyone gets timely treatment.

These examples are not based upon real scenarios but they are designed to demonstrate that looking at a single variable -- wait times -- without analyzing other variables to get an overall picture is a mug's game. Furthermore, whether or not a system is successful depends upon the criteria for success that are established in advance. For example, if the criteria that is established is "maximization of freedom to use my personal resources to maximize my health care" you get a very different measure of success than if the criteria is "to maximize timely access to medically necessary procedures across all procedures and a national population." Both are measures that have moral and philosphical foundations and require serious value judgments to be made in order to answer the question, "is the available medical care system working?"