About Half of the Body Composition Scales on the Market Provide Unreliable Body Composition Readings

And, they’re especially inaccurate for the comparatively healthy and fit

The odd, very rapidly dropping body fat mass and very rapidly increasing muscle mass readings shown most recently in the Withings Body Cardio readings for my wife and me -- in apparent direct response to the quite minor changes in dietary polyamines described and explained here -- worried me, so I went looking to see if any quantitative and statistical research work had been carried out measuring and describing the accuracy and precision of commercially available home use body composition scales.

Grant Tinsley, an associate professor at Texas Tech, and Director of its Energy Balance & Body Composition Laboratory, provides a very useful summary of a 2023 laboratory study, Assessing the reliability and cross-sectional and longitudinal validity of fifteen bioelectrical impedance analysis devices, that examined the utility of 15 portable body composition scales for keeping track of body changes caused by diet, exercise, illness, and age.

What I believe is the most important general finding of the Siedler et al., 2023, laboratory study referenced above is this one:

“Six (40 %) of the fifteen devices in our sample demonstrated statistically significant negative proportional bias as demonstrated in the Bland – Altman analysis. In other words, many of these devices demonstrate a lack of sensitivity required to accurately estimate BFP [body fat percentage] in individuals on either end of a population’s distribution while generally exhibiting greater accuracy in estimating BFP in those with intermediate values. This may point to an artificially restricted range of output values programmed in these devices based on a manufacturer’s desire to provide a relatively ‘normal’ BFP estimate to a consumer. It may also indicate the use of an exceedingly homogenous reference population by the manufacturer when developing algorithms. Therefore, those with levels of body fat further from the population norm should generally be more skeptical of outputs received by a given consumer-grade BIA [bioelectrical impedance analysis] device.” [Emphases and acronym definitions added.]

What this means is that these researchers found convincing evidence that a significant number of the commercially available home use body composition scales provide inaccurate and unreliable % body fat estimates for those who are more fat, or less fat, than the modern “normal” to (normally) “overweight” majority.

As an example, Siedler et al., 2023, provide the following scatter plot for the Withings Body Cardio BIA device we’ve been using at our house for a little more than a year.¹ Note that at scale-estimated % body fat below 20%, the Withings BF determinations become noticeably and progressively much less precise and accurate; i.e., much more “noisy”. (The Y-axis measurements represent clinical quality ‘ground truth’ measurements of the % body fat of the study’s test subjects.)

After Figure 2 of Siedler et al., 2023. The “4C Model BF%” measurements are clinical/ research quality body fat determinations carried out on 72 different people at the beginning of the study using a combination of a calibrated weight scale, BodPod body volume determinations, DXA bone mineral mass estimations, and total body water approximated using Impedimed SFB7 bioelectrical impedance spectroscopy. Withings Body Cardio BF% estimates on the x-axis are derived from direct output (pounds body fat) from this bioelectrical impedance analysis device. Note that at estimated % BF below 20%, the Withings BF cross-sectional determinations of % body fat become much less precise and accurate (i.e., less valid). The “poor” accuracy and precision assessment for the Withings Body Cardio BIA device is that of Siedler et al., 2023. Price of the unit is from Amazon.

My digitization and re-graphing of the point data from the graph above indicate that -- for Withings % body fat estimates of around 20% or less -- there is little or no measurement value whatsoever to the Withings Body Cardio % body fat readings. The R-squared for the line-of-best-fit for the data scatter plot beneath indicates that the Withings Body Cardio BIA device scale at the 20% reading level and below is only about 2% accurate.

Fit Results for Body Cardio Determinations Less than 20% Body Fat

Fit 1: Linear

Equation Y = -0.1539647435 * X + 19.08218071

Number of data points used = 25

Average X = 15.5242

Average Y = 16.692

Residual sum of squares = 649.971

Regression sum of squares = 10.9362

Coef of determination, R-squared = 0.0165473 (R = 0.13)

Residual mean square, sigma-hat-sq'd = 28.2596

In contrast, Withings Body Cardio readings above 20% body fat are much more strongly correlated with, and are therefore much more predictive of a person’s true percentage of body fat. See the graph and Table below.

Fit Results for Body Cardio Determinations Greater than 20% Body Fat

Fit 1: Linear

Equation Y = 0.9370587905 * X + 2.064685458

Number of data points used = 50

Average X = 29.2013

Average Y = 29.428

Residual sum of squares = 1265.62

Regression sum of squares = 1789.91

Coef of determination, R-squared = 0.585794[1] (R = 0.77)

Residual mean square, sigma-hat-sq'd = 26.3671

[1] This R-squared value, however, indicates that even for cases of people with body fat percentages greater than 20%, the Body Cardio is still only about 59% accurate in its estimates of the true % body fat percentage of the person.

These effects of the loss of accuracy and precision of the calibration curve of the Withings Body Cardio at % body fat values around or less than 20% showed up not only in both of our most recent % body fat determinations, but seem to have somehow exponentially distorted our most recent muscle mass and bone mass estimations as well. See the example graph below to see the probable distortions I am referring to. Presumably, the bioimpedance ‘algorithms’ used to estimate the mass of each body component in the Withings output (bone mass, water mass, muscle mass, and fat mass) are computationally dependent on each other and are therefore only reasonably valid when % body fat determinations are reasonably accurate.

For instrument comparison, my % body fat estimated with the more accurate and precise Omron HBF-514C discussed below is 21.3% -- not the visually unbelievable 14% BF current estimate provided by the Withings Body Cardio. Red shading marks the ongoing interval of dietary putrescine and spermidine supplementation. It is believed the skeletal, fat, and muscle mass changes shown in that interval on this graph are reasonably credible as long as Body Cardio fat measurements remained above the instrumentally-critical 20% body fat threshold. Notice the abrupt jumps in muscle and bone mass that occurred – according to the Withings Body Cardio – as my % body fat readings fell below the 20% level. You can also see an abrupt drop in scale-estimated body fat took place at the same time. This is a drop that is not confirmed by either the more accurate Omron scale discussed below, or by my use of Lange skinfold calipers.

Fortunately, the Siedler et al., 2023, study produced data indicating the relative ability of each of the 15 tested units to “…adequately detect changes in body composition over time as compared with the reference 4C model.”

Again, this is an important concern because, as they say,

“Overall, the relatively poor longitudinal validity demonstrated in approximately half of our sample of devices indicates that consumers should be wary of the ability of such devices to accurately track changes in [their] body composition over time, at least on the population level.”

The top seven yellow-highlighted devices on the Table below demonstrated to Siedler et al. that they were sufficiently reliable and accurate in estimating the % body fat in people of all body types, however. Surprisingly, the Japanese-designed $80 Omron HBF-516/HBF-514C provided better overall body fat estimation performance than the six other top instruments costing significantly more, including the $1100 lab-quality Seca mBCA 515/514. Although demonstrating less accuracy and precision at % body fat levels below 10% than the Omron, the even cheaper $49 Tanita UM-081 turned out to be no performance slouch, either.

The measurement validity scatter plot for the Omron HBF-516/514C from Siedler et al., 2023, is shown below. Note that – unlike in the case of the Withings Body Cardio BIA device discussed above – the Omron provides a rather tight, focused linear response to changing % body fat across the entire % BF range exhibited by the Siedler et al. study’s rather extensive and physically heterogeneous test sample of people.

After Figure 2 of Siedler et al., 2023. The “4C Model BF%” measurements are clinical/research quality body fat determinations carried out on 72 different people at the beginning of the study using a combination of a calibrated weight scale, BodPod body volume determinations, DXA bone mineral mass estimations, and total body water approximated using Impedimed SFB7 bioelectrical impedance spectroscopy. Omron HBF-516/514C BF% estimates on the x-axis are derived from direct output (pounds body fat) from this bioelectrical impedance analysis device. Note that – unlike in the case of the Withings Body Cardio BIA device – the Umron provides a rather tight, focused linear response to changing %BF across the whole %BF range exhibited in the study’s rather large and varying sample of people. The “fair” accuracy and precision assessment for the Omron BIA device is that of Siedler et al., 2023. Price of the unit is from Amazon.

So, now that my wife and I have both worked our way down to around the 20% body fat detection limit of the Withings Body Cardio, we’ll be continuing our n=2 experiment -- with regard to determining the possible salutary effects of minor dietary polyamine supplementation -- using an Omron scale to keep more accurate track of further possible muscle mass and fat mass changes associated with this slight dietary change.

A couple of final comments:

First, among other things, the scatter plots below continue to suggest minor dietary polyamine supplementation does, at least in my 73 year-old male case, significantly serve to reduce body fat while building/restoring muscle mass.

After Figure 1 of Muscle mass, BMI, and mortality among adults in the United States: A population-based cohort study, 2018. The single star on the left-hand graph shows the current and safe “youthfully” preserved muscle mass and body fat status of my wife, while the two stars on the right-hand graph illustrate my “youthful” leftward shift away from the dangerously lurking orange shoals of potential sarcopenia, a shift that has taken place since the inception of our informal dietary polyamine supplementation experiment begun earlier this year. It was not possible to graph my wife’s shift, if any, over the course of the ongoing experiment because of the notable uncertainty of the validity of the Body Cardio scale’s body fat determinations for her particular case at the start of our experiment.

Second, readers may recall from this post and this one that the average US diet is – in an international context – relatively deficient in polyamines, and that this deficiency could be behind both the diminished state of US male fertility and the retarded improvement of US life expectancy compared to that of other industrialized nations.

Perhaps tellingly, the authors of the paper containing the graph above briefly remark near the end of their report that:

“We found that [low] appendicular skeletal muscle mass was an independent risk factor for mortality in the general population, and this was more pronounced among younger Americans. This is striking given that the risks of low muscle mass have been a major focus in the geriatric literature and in cohorts with various chronic diseases, but not otherwise in the [younger] general population.” [Emphasis and explanatory adjectives added].

More cheese for all! Herbert Hoover’s chicken-containing pots got it a bit wrong.

1. My repeated thanks to our youngest daughter and her husband for giving us this unexpected gift early last year — there’s nothing like gaining access to a new analytical instrument to please and entertain an exploration geochemist more.