Deception of the Douglas Bag Validation Method

What is the Gold Standard of Incompetence?

By Andrew Huszczuk Ph.D. Edited by John Hoppe

Background

The reason for this paper is misinformation being fed to unsuspecting prospective buyers of metabolic measurement system. These systems are known generically as Metabolic Carts (MC), CPX systems, Ergospirometry systems in Europe and also as calorimetry measurement systems; for simplicity, we will hereafter refer to them as MC's.

It appears that some manufactures of MC's base their claim of superiority on a statement that their instruments are either calibrated or validated using the Douglas bag method.

We challenge the validity of such claims.

Introduction

Would you take a medication knowing that a pharmacy used an uncalibrated scale to weigh its ingredients? Would you board a plane knowing that the fuel or altitude gauges are not calibrated at frequent intervals?

In these and thousands of other applications scientific bases and rules of metrology must be obeyed to assure chaos-free operation of modern societies. To scrutinize performance of measuring devices a process of calibration must be carried out by means of applying a known standard and getting back a correct reading.

Whereas a single physical entity (e.g. mass, temperature, pressure etc.) measurement involves a relatively simple calibration procedure, more complex systems utilizing multifactorial sensing require a two-stage calibration. First, all sensing components have to be calibrated separately by applying sensor-specific standards (e.g. certified gas mixtures of a known composition) and performing fine-tuning to obtain corresponding readings. These readings, in turn, are used to "scale" the calibration function of software.

The second stage of calibration has to verify the final performance of the measurement system as a whole. This necessitates generation of known quantities and qualities of the measurement-specific variables (i.e. the simulation of exhaled gas mixtures), which, to become standards, have to be prepared with sufficient accuracy and delivered to the system under calibration in a typical dynamic pattern simulating the "real life" situation.

Most of such complex measurement systems can be classified as mass flowmeters.

In medical applications mass flowmeters are used to measure the level of metabolism at rest and during incrementing intensity exercise. They are called metabolic carts (MC's), and essentially measure the oxygen consumption (VO₂) and the carbon dioxide output (VCO₂) of the body, usually expressed in liters per minute STPD. These two variables are usually complemented by minute ventilation of the lungs (VE) and heart rate (HR) expressed in liters per minute and beats per minute respectively. From the above four variables a score of additional ones can be derived to enhance the process of diagnosis in fields such as cardiopulmonary medicine, sports medicine and physiology, metabolic and nutritional disorders, geriatrics, rehabilitation, occupational medicine, assessment of fitness etc.

Current State of Technology

All commercially offered metabolic measurement systems require initial calibration of respiratory (usually expiratory) flowmeters and a two-point calibration of the O₂ and CO₂ analyzers. Respiratory flowmeters are most commonly calibrated with a large (usually 3-liter) syringe manually powered to displace known volume of air through them using slow, medium and fast strokes. Since, in metabolic measurement applications, the minimal dynamic range of flows expected to be sensed by flowmeters is 0.25 to 3 L/sec (a twelve-fold span) and in athletic applications the upper flow range should be at least 5 L/sec (a twenty-fold span), the manually induced puffs or gusts of air are inadequate to assure better than ± 5% accuracy across a desired range due to the following reasons:

1. All existing flowmeters are essentially non-linear, although the degree of non-linearity varies widely depending on physical principle of operation, therefore, all are equipped with linearizing devices utilizing microprocessors or a software look-up tables. Any drift of a function inputting the linearizer will result in significant distortion of the output function presumed to be linear. Only turbine flowmeters are immune to such drifts.
   
2. A vast majority of flowmeters is also sensitive to physical gas properties such as specific gravity, viscosity and thermal capacity, which means that gas composition markedly affects their performance. Consequently, the transfer function of the output signal versus flow of any given flowmeter obtained for air will be modified by change of gas composition to that typical of exhalant. Therefore, why bother to calibrate with air if during actual test a flow of different gas will be sensed?
   
3. Use of two-way breathing valves constitutes additional liability in applications such as classical mixing chamber systems or bag collection of exhaled gases for subsequent analysis. All breathing valves leak at low minute ventilations (VE) typical of rest, but deteriorating valving membranes or wrong positioning may increase these leaks to a 10-15% loss of tidal volumes, thus to unacceptable underestimation of resting metabolic variables, rendering mixing chamber systems useless in basal metabolic studies. Manual operation of large calibration syringes virtually precludes generation of low-range flows or frequent flow reversals, which aggravate leaks.

Similar issues of non-linearity affect performance of gas analyzers. Fluctuations of sampling flows, barometric pressure, ambient temperature and humidity will induce drifts which, when unchecked, may exceed the capability of the "on-board" (i.e. being an integral part of analyzers) linearizing devices. Therefore, the two-point, usually air for low CO₂ and high O₂ and certified standard gas mixture for low O₂ and high CO₂, calibration of gas analyzers has limited validity as it determines only an average slope of the output signals versus gas concentrations relationship. The actual point-by-point relationship does not exactly follow the straight lines joining two respective points that span the whole measurement range for each gas.

Finally, sloppy or incorrect handling of parameters* greatly, and often overwhelmingly, contribute to the overall performance accuracy because they affect correction factors between physical conditions (ATPS, BTPS, and STPD) or the manner of mass flow assessment computation:

• Barometric pressure
• Ambient and exhaled temperature
• Relative humidity
• Sampling time delays and properties
• Rebreathed dead space
• Sampling port location

Incredibly, but the foregoing long list of potential and usually real sources of errors is commonly ignored as very few metabolic equipment manufacturers bring up these issues and relevant scientific organizations do not offer recommendations to standardize the process of calibration. Consequently, currently used metabolic measurement equipment rarely performs within boundaries of decency i.e. ± 5% of accuracy and often approaches or exceeds ± 10% without notice or complaints.

* see footnote on the last page

Deceptive Claims

When sales representatives of some manufacturers are confronted with issues of accuracy they may claim that their systems are validated with the Douglas bag method, which they dare to call a "gold standard" of calibration.

A beginning of 20^th Century method - the gold standard?

Moreover, some manufactures actually use this claims in advertising. One may not expect much competence from sales representatives, but advertising represents official claim of a manufacturer and, as such, is alarming. Perhaps the use of the Douglas bag method for validating metabolic measurement systems should more appropriately be called "The Gold Standard of Incompetence".

Validation used as euphemism for calibration using bag collection of exhaled gases is not and can not epitomize the very essence of calibration for the following reasons:

1. The absence of a standard, which means that one does not know what the result should be.
2. Poor accuracy of the Douglas bag method resulting from:
   1. Two-way breathing valve leaks
   2. Inaccurate assessment of the captured gas volume even with use of large and perfectly maintained spirometers
   3. Error-prone conditions precluding proper measurement of gas concentrations due to diffusion of gases and dilution of CO₂ in water wetting inner walls of Tissot spirometers or cooled down collection bags (condensation).
3. Absolute impossibility of performing calibration prior to the test.
4. Inability to perform "on-line" troubleshooting in order to locate the source of errors.

In practice, even when perfectly executed the Douglas bag method yields results within a range of ± 5% accuracy, which can be assessed only when the volumes of CO₂ and O₂ (deficit) are known prior to commencement of bag collection. Here is how one can perform such an experiment:

1. Fill a large calibration syringe (say 3L) with 100% CO₂
   and transfer the contents to a collection bag.
2. Fill this syringe with 100% N₂ and transfer it to the same bag four times for total of 12 liters of N₂ content.
3. Fill this bag up with an unknown volume of air.
4. Now, to simulate bag collection during real test, take another empty bag and, using a two-way breathing valve, transfer the contents of the first bag to the second one to simulate with syringe strokes the usual pattern of tidal volume and frequency typical for a given test conditions.
5. Perform gas analysis of the second bag contents.
6. Measure gas volume captured in this bag.
7. Now remember, you started with 3 liters (ATPD) of CO₂ and 3 + 12 liters (ATPD) of an O₂-free gas and diluted it with room air of known humidity, temperature and barometric pressure (ATPs). You expect VCO₂ (no dot over V, meaning volume not minute flow) to be 3L (ATPD) and the displaced oxygen volume to be 3.15L (ATPs), [(3 + 12) x 21% = 3.15L].
8. Take your experimental numbers for volume and gas concentrations and compute VCO₂ and displaced VO₂ contents using standard formulae.
   (We will supply them on request)

Hopefully it wasn’t too discouraging, but remember, you did not have to deal with warm fully saturated gases of a real test situation. This simple test gave an idea for an ultimate calibration method consisting in combining strictly predetermined mass flow of CO₂ and O₂ (negative, to simulate consumption) delivered to the system under calibration by means of an intermittent breathing-like pattern of adjustable minute ventilatory flows (VE). By keeping metabolic flow constant, while adjusting VE, a practical range of flow and gas concentration spans can be scanned to detect ranges of non-linearity. This method is described in more detail on this web site in "Dr Andrew's Corner".

Conclusion

Metabolic measurement systems, as mass flowmeters, can be calibrated only by means of generating standard mass flows that can be set at will and delivered in a form of respiratory patterns. Existing software corrections (and in some cases "fudge-factors") are based on one-time analysis of any given transfer function and are not capable of effective dealing with unavoidable analog drifts.

Ultimately, we expect that the use of manually operated syringes and calibration gas mixtures will become obsolete to be replaced with advanced software that compensates all existing non-linearities by analysis of responses to forced mass flows and executes appropriate corrections across all operational spans within 3-5 minutes preceding every test.

* Incidentally, most of User Manuals and even many research publications negligently confuse term variable with term parameter. To reiterate: a variable is a dynamic quantity that has to be measured as it evolves (e.g.: expiratory flow, tidal concentration of O₂ and CO₂, heart rate, etc), whereas a parameter is a quantity that stays constant in the case considered, such as the current test, but affects computational outcome and may change for any other test conditions (PB, T, RH)

The Metabolic (Lung) Simulator
Calibration of Metabolic Systems
Stress Testing
Resting Energy Expenditure "REE"
Why Calibrate Ergometers
How to select a Cycle Erometer
Deception of the Douglas Bag