In Chapter 13 of Visualizing Uncertainty, Howard Wainer observes that the less precise an estimate is, the bigger its error bars, and the more visual prominence it has in a visualization. The more precise an estimate is, the less ink it gets.
He’s talking about images like this (chart from Visualizing Uncertainty, Chapter 13, annotation mine):
In all my years of looking at graphs of confidence intervals, I had never put my finger on that point. Read on for a quick description of confidence intervals/standard error, and a suggested solution to the issue.
Any description of a population from a sample is going to include some amount of uncertainty. If I’m looking at a sample of swords in Knightlandia, I can’t know the exact mean price of all swords in Knightlandia (the mean of the population). What I can know is the mean price in the sample and an interval that probably contains the mean price of all swords.
The size of that range depends on two things: the standard error and how confident I want to be that the population mean is actually inside the range.
- Standard error: I talked a little bit about standard error in a previous post. The larger the standard error, the larger the range where we expect to find the population mean. Standard error for continuous variables (like sword price) depends in turn on sample size and how similar the values in the sample are. If you have a big sample where all the values are pretty similar, you would expect that your sample mean would be a better estimate of the population mean (small standard error) than if you had a small sample and the prices in that sample varied wildly (large standard error).
- Confidence: When I’m estimating population sword price, I want to be 100% confident that my estimate includes the actual population mean. Unfortunately, the only way to be 100% confident is to include every possible price of swords in my estimate. I’m absolutely confident that the population mean is somewhere in there, but that isn’t useful information. To have a more useful estimate, I need to be slightly less confident in it. If I set my confidence level at 99%, now I have an upper and lower bound of where the population mean sword price could be. If I set the confidence level at 95%, the upper and lower bound will be closer together. It’s a trade-off: the more confident I am that the population mean is somewhere inside my estimate, the larger that estimate has to be. Confidence levels are usually set by convention within a particular field—in psychological research, for instance, I generally see 95% confidence intervals.
Wainer suggests a few alternatives that decrease the prominence of large confidence intervals by giving an impression of their size without showing the actual interval. However, if you want to see the actual size of a confidence interval, it’s difficult to escape encoding it with length: bigger confidence intervals mean longer symbols and more ink.
An alternative: encoding in color
Encoding in color as well as length could balance out the visual impact of large confidence intervals. Imagine there is a limited amount of pigment for each confidence interval, and that pigment is distributed based on the probability that the population mean is at any given point. So an estimate with a small standard error will have a small band of very concentrated color, while an estimate with a large standard error will have a long band of diffuse color. Since values at the ends of a confidence interval are less likely than values in the middle, the pigment will be more concentrated towards the center of the interval, and less concentrated towards the ends of the interval.
I’m thinking of something like this, although perhaps a bit cleaner (standard chart on the left, pigment chart on the right, data made up):
I’ve seen color gradients for confidence intervals before. These communicate that the values at the pale ends of the bar are less likely, but the same prominence issue pops up. In the bar with the larger confidence interval, the intense dark blue at the center is spread out over a larger area, not made less intense and eye-catching.
Wainer does show an example of a chart that uses black and white gradients to a similar effect, but the gradients don’t lighten until the very ends of the bars, which returns to the same problem of visual prominence. Big black blocks (even ones with faded ends) still pop out more than small black blocks.
The main challenge of building a chart like this is determining exactly how much pigment to put at each part of the interval. I eyeballed this in Illustrator, but displays of actual measurements should reflect that data, rather than what looks approximately right to the designer. This also may not be an appropriate display in situations where the visualization is trying to communicate differences in size, a la a classic bar graph: it would be difficult to tell if, say, one of these population mean estimates was twice the size of the other. However, it does move the emphasis back to the vivid, most-likely values of the population mean. Check this space for more tinkering!