You are here

  • Home
  • Archive
  • Volume 5, Issue 11
  • Influence of the practice setting on diagnostic prediction rules using FENO measurement in combination with clinical signs and symptoms of asthma

Article Text

Article menu
XML
Respiratory medicine
Research
Influence of the practice setting on diagnostic prediction rules using FENO measurement in combination with clinical signs and symptoms of asthma
  1. Antonius Schneider1,
  2. Gudrun Wagenpfeil2,
  3. Rudolf A Jörres3,
  4. Stefan Wagenpfeil2
  1. 1Institute of General Practice, University Hospital Klinikum rechts der Isar, Technische Universität München, Munich, Germany
  2. 2Institute of Medical Biometry, Epidemiology and Medical Informatics, Universitätsklinikum des Saarlandes, Homburg, Germany
  3. 3Institute and Outpatient Clinic for Occupational and Environmental Medicine, Ludwig-Maximilians-University, Munich, Germany
  1. Correspondence to Professor Antonius Schneider; antonius.schneider{at}tum.de

Abstract

Objectives To evaluate the influence of the practice setting on diagnostic accuracy of fractional exhaled nitric oxide (FENO) for diagnosing asthma; and to develop prediction rules for diagnostic decision-making including clinical signs and symptoms (CSS).

Setting Patients from 10 general practices and 1 private practice of 5 pneumologists in ambulatory care.

Participants 553 patients, 57.9% female. Consecutive inclusion of diagnostic-naive patients suspected of suffering from obstructive airway disease. Exclusion criteria were respiratory tract infections within the last 6 weeks.

Interventions The index test was FENO measurement. Reference standard was the Tiffeneau ratio (forced expiratory volume in 1 s/vital capacity) or airway resistance as assessed by whole body plethysmography, with additional bronchoprovocation or bronchodilator testing.

Primary and secondary outcome measures Asthma as determined by pneumologists, who were blind to FENO measurement results. Prediction rules were derived from multiple logistic regression analysis. A freely available calculator that allows computing all combinations was developed.

Results The practice setting only had minor influence on sensitivities of FENO cut-off points. In the final model (n=472), allergic rhinitis, wheezing and previous medication were positively associated with asthma. Increasing age and recurrent respiratory tract infections were negatively associated. The area under the curve (AUC) of FENO (AUC=0.650; 95% CI 0.599 to 0.701) increased significantly (p<0.0001) when combined with CSS (AUC=0.753; 95% CI 0.707 to 0.798). Presence of wheezing and allergic rhinitis allowed ruling in asthma with FENO >30 ppb. Ruling out with FENO <16 ppb in patients <43 years was only possible without allergic symptoms when recurrent respiratory tract infections were present.

Conclusions FENO results should be interpreted in the context of CSS to enhance their diagnostic value in primary care. The final diagnostic model appears as a sound algorithm fitting well to the established diagnostic rules related to CSS of asthma. FENO appears more effective for ruling in asthma than for ruling it out.

  • PRIMARY CARE
  • sensitivity and specificity
  • diagnostic accuracy
  • nitric oxide

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2015-009676

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • We used data from 553 patients to develop prediction rules for diagnostic decision-making with fractional exhaled nitric oxide (FENO) measurement including clinical signs and symptoms.

    • The general practice patients seemed to be selected more than those of the pneumologists’ practice, which might be explained by the study design. Therefore, it appeared adequate to extrapolate our FENO findings more cautiously to allow generalisation of the diagnostic algorithm.

    • The final model fitted well with the established clinical decision rules used by many physicians and led to a more conservative interpretation of the FENO measurements. However, a validation study would be desirable to confirm our findings.

    • We used the maximum concentration of methacholine for bronchial provocation as a reference standard to rule in and rule out asthma. Therefore, the potential of FENO for ruling out moderate and severe asthma might be underestimated.

    • A freely available calculator that allows computation of the probability of asthma based on the combination of clinical signs and symptoms, and FENO results, was developed.

    Introduction

    Asthma is a common chronic disease with a prevalence of up to 5% in industrialised countries. It is characterised by chronic inflammation, bronchial hyper-responsiveness (BHR) and usually reversible airway obstruction. Many efforts continue to be undertaken to improve the diagnostic process to allow an early diagnosis, as early treatment is important for the management of the disease. Investigation of the diagnostic accuracy of clinical signs and symptoms (CSS) showed that these were not very effective in ruling in or ruling out the disease.1 ,2 Spirometry is considered a reference standard for diagnosing airway obstruction,3 but it is not possible to rule out milder forms of asthma, as obstruction is not present in these cases.4 Guidelines also suggest the use of peak flow variability to diagnose BHR,5 but its diagnostic accuracy is low.6 Therefore, bronchoprovocation for determining BHR still remains as a reference standard, particularly in cases with inconclusive spirometric results.7 It is considered valuable in confirming or excluding asthma, despite being a time-consuming and costly, and not always available, procedure, and carrying a small risk of severe bronchospasm.8

    Compared to bronchoprovocation, fractional exhaled nitric oxide (FENO) is an easily available, truly non-invasive marker. Increased FENO has been consistently demonstrated in asthma, including milder stages of the disease.9 ,10 The major pathophysiological basis seems to be that nitric oxide has a modulatory role in airway hyper-responsiveness11 and eosinophilic airway inflammation.12 Therefore, FENO has a potential in identifying specific asthma phenotypes, which might also allow the prediction of steroid responsiveness due to eosinophilic inflammation.13 This might be especially helpful for establishing or confirming the diagnosis safely and quickly in the primary care setting. Its diagnostic accuracy has been investigated in a large number of studies. In general, the results were promising, but different cut-off points were suggested to rule in or rule out asthma. As an example, to rule in the diagnosis of asthma with FENO, >50 parts per billion (ppb),14 ,15 or FENO >35 ppb,16 or FENO >46 ppb, has been suggested.17 To rule out the disease with FENO, <15–25 ppb has been suggested.14 FENO <16 ppb17 or even lower18 might be more useful in the primary care setting.

    An important reason for the variation in cut-off points might be the selection of patients who participated in the diagnostic studies. The influence of the patient spectrum on the variation of diagnostic accuracy was already demonstrated by Ransohoff and Feinstein.19 Knottnerus20 explained the increase of sensitivity and decrease of specificity by referral processes in a methodological framework. The understanding of this process is important as patients present to the general practitioner (GP) with early symptoms and thus often with lower severity of disease.21 Beyond that, the interpretation of a test result is often hampered by low positive predictive values of tests, because the pretest probability of the target disease is often low in general practice. This phenomenon is described by Bayes’ Theorem.22 Especially in the primary care setting, in which few objective methods are available, it seems reasonable to combine information from a diagnostic test with the CSS presented by the individual patient, to enhance the diagnostic accuracy. This approach has been followed previously, for example, for pneumonia and C reactive protein.23 ,24 The aim of the present study was to evaluate the influence of patient selection on the diagnostic accuracy of FENO measurement on the basis of two diagnostic studies from different clinical settings,17 ,18 and to develop prediction rules including CSS in order to enhance the diagnostic value.

    Methods

    Design and sample

    The first part of this prospective diagnostic study was performed in 10 German general practices in the area of Heidelberg in Baden-Württemberg, between February 2006 and June 2007.17 Two hundred and ten patients visiting their GP for the first time, with symptoms suggestive of OAD (obstructive airway disease) or the respective differential diagnoses (such as restrictive airway disease), were included consecutively. The patients had to present with symptoms such as dyspnoea, cough or expectoration of more than two months, thus leading to the clinical suspicion of obstructive or restrictive airway disease as the most important differential diagnoses. The presence of at least one of these symptoms was used as inclusion criterion (indicated population design).25 GPs were advised to exclude patients who had suffered from respiratory tract infections within 6 weeks preceding the evaluation. After the initial judgement by the GP, patients were sent to the lung function laboratory of the University Medical Hospital for diagnostic assessment including FENO measurement. Patients with a previously established diagnosis of OAD were excluded. Other exclusion criteria related to known contraindications for bronchodilator reversibility testing or bronchial provocation, namely untreated hyperthyreosis, unstable coronary artery disease and cardiac arrhythmia. Pregnancy also led to exclusion. Medical history was recorded using a structured questionnaire (table 1).

    View this table:
    Table 1

    Characteristics of the study population

    The second part of the study was performed in a private practice of five pneumologists in Bavaria, between June 2010 and October 2011.18 In Germany, specialists also work in primary care in their private practices, and ambulatory care comprises almost all specialists. There is no formal gatekeeping role for a GP in the German healthcare system. However, referrals from a GP to a specialist are requested in most cases. Only patients presenting for the first time for diagnostic work up to include or exclude an OAD or the respective differential diagnoses, were included. Patients with respiratory tract infections within the last 6 weeks were excluded.

    Reference test—whole body plethysmography (WBP) and bronchial provocation: the spirometric manoeuvre performed during investigation with WBP was used as reference test in every setting. The procedures were performed according to standard protocols.26 Lung function reference values corrected for sex, age and height were used.27 Patients with forced expiratory volume in 1 s (FEV1) <80% predicted received salbutamol with an additional WBP investigation 20 min later. An OAD was diagnosed if FEV1/vital capacity (VC) was ≤0.70. It was classified as asthma if clinical symptoms and history fitted, and if the change in FEV1 compared to baseline was both ≥12% and ≥200 mL, and lung function returned to the predicted normal range. An incomplete bronchodilator response was stated if the response was ≥12% and ≥200 mL, but where lung volumes remained below predicted. We labelled this group as having asthma-COPD (chronic obstructive pulmonary disease) overlap syndrome (ACOS), because it shows spirometric properties of both, asthma and COPD.5 It was classified as COPD if clinical symptoms and history fitted and the bronchodilator response of FEV1 after salbutamol was both <12% compared to baseline and <200 mL.3 If there was no bronchial obstruction, bronchial provocation was performed to determine BHR. Trained lung function technicians measured BHR to methacholine according to the American Thoracic Society (ATS) guideline8 in the GP study.17 A modified bronchial provocation procedure was used in the practices of the pneumologists,18 according to the 1-concentration-4-step dosimeter protocol.28 This yields similar results as the ATS multiconcentration protocol but is less time consuming. An ‘asthma’ diagnosis required a 20% fall in FEV1 from baseline after inhaling methacholine stepwise until the maximum concentration (16 mg/mL), and, alternatively, a doubling of airway resistance and its increase to ≥2.0 kPa s.29 The diagnostic superiority of WPB compared with spirometry for ruling out asthma was demonstrated previously.30 The final diagnosis was made under consideration of medical history and clinical examination by a pneumologist.

    Index test—FENO measurement: all patients underwent standard measurement of FENO (NioxMino, Aerocrine, Solna, Sweden) at a flow rate of 50 mL/s, according to the ATS/European Respiratory Society guideline,31 using feedback signals for control. This was performed prior to WBP and bronchial provocation, as the breathing manoeuvres involved could distort FENO results. The responsible pneumologist was blinded to the FENO results and made the diagnostic decision only on basis of medical history, physical examination, WBP investigation and bronchial provocation results.

    Patients gave written informed consent.

    Statistical methods

    Power calculation was based on previous studies related to the prevalence of asthma in the respective setting and the diagnostic accuracy of FENO. We wanted to include at least 149 patients in the first part of the study17 and at least 302 patients in the second part.18 Differences between lung function values (not normally distributed) were statistically evaluated with the Mann-Whitney U test. Differences between clinical symptoms were evaluated with the χ2 test. The data were analysed with IBM Statistics SPSS V.22.0 for Windows.

    Independent clinical and diagnostic contributions of symptoms and signs to the prediction of asthma were assessed using multiple logistic regression analysis. As the number of available variables was too large to meet the rule of thumb in 10 cases, per independent variable,32 we checked univariate associations with asthma and included only significant variables (p<0.05) in the model. Multiple logistic regression analysis using backward elimination with p>0.1 for exclusion was performed with the selected variables, resulting in the final covariate model. Several potentially relevant interaction terms between covariates were first included and then removed from the model if they did not contribute to the diagnostic accuracy. Considering the resulting covariate effects estimated from the data, a rule could be derived from the multiple logistic regression approach, predicting the probability of asthma in each individual case. Respective 95% CIs for predicted probabilities are given in parentheses and were calculated using the δ-method.33 A calculator that allows computing all combinations is provided as an internet supplement. If the δ-method is not applicable, in particular at the border of the domain of predicted probabilities, the CI is not calculated.

    In accordance with everyday practice, where an additional FENO measurement is performed after medical history information has been acquired, multiple logistic regression analysis was repeated, adding FENO at different cut-off values and as exact numerical variable. Receiver operating characteristic (ROC) curves display the diagnostic performance of the final models. The area under the curves (AUC) were used to quantify the added value of the CCS+FENO model beyond the FENO model. Comparison of AUC is performed with the empirical test implemented in NCSS V.9.0.534 using a non-parametric approach described in DeLong et al35 and Zhou et al.36

    The results of the diagnostic models were interpreted with respect to clinical significance. A satisfactorily high posterior probability of asthma is assumed, when the positive predictive value is ≥70%. This corresponds with the positive predictive value of bronchial provocation, which was estimated around 70% for a pretest probability of asthma of 30%,8 ,37 and was demonstrated recently.30 A satisfactorily low posterior probability is assumed at 20%, corresponding to the probability of 80% of having ‘no asthma’. This corresponds to the negative predictive value of a 20% fall in FEV1 from baseline during bronchial provocation.30

    Results

    Study population

    A total of 553 patients participated (320 female (57.9%)). The recruitment rate in general practice was 76%. Nearly every patient from the practices of the pneumologists participated; the data of seven patients could not be used due to incompleteness (figure 1). The diagnosis of asthma was based mainly on bronchial provocation (n=206; 90%); positive bronchodilator response of pre-existing airway obstruction was recorded in only 23 (10%) cases. The prevalence of asthma was highest in the general practice group (table 1). Patients suffered mainly from shortness of breath, wheezing and cough. The patient sample from general practice suffered significantly more from dyspnoea attacks, cough and nasal allergy, and less from dyspnoea on exertion. They used more antiasthmatic medication than patients from the practices of the pneumologists. We found more smokers in the general practice sample, with higher nicotine use. Correspondingly, there were more patients with COPD and ACOS in the general practice sample, accompanied by a significantly lower FEV1, VC and FEV1/VC ratio. Patients with asthma in general practice had significantly more dyspnoea attacks and less dyspnoea on exertion than patients from the practices of pneumologists (p values of subgroups are depicted at the bottom of the table). They also used more antiasthmatic medication. The asthma patients from the general practice showed a significantly lower FEV1/VC ratio compared to the patients with asthma from the pneumologists practices; FEV1 and VC showed no significant difference. Patients in general practice without OAD suffered from cough and recurrent respiratory tract infections significantly more than the patients from the practices of the pneumologists. There were no further significant differences between the patient groups with respect to the other CSS.

    Figure 1
    Figure 1

    Diagnostic work up of patients (ACOS, asthma-COPD overlap syndrome; COPD, chronic obstructive pulmonary disease; FEV1, forced expiratory volume in 1 s; OAD, obstructive airway disease; VC, vital capacity).

    Diagnostic accuracy of FENO of the different patient collectives

    A comparison of patients from general practice and pneumologists’ practice showed a trend towards slightly higher sensitivities around the cut-off point >40 ppb in the general practices; there were no remarkable differences related to specificity (table 2). Multiple logistic regression analyses were performed with either 3, 4 or 5 selected covariates from clinical history or physical examination, respectively. This resulted in three groups of models and the respective equations displayed in table 3.

    View this table:
    Table 2

    Diagnostic accuracy of FENO in different patient collectives

    View this table:
    Table 3

    Regression models of FENO (NO) and clinical signs and symptoms in different patient collectives, where k is the number of covariates in the final model

    Further subgroup models were defined dependent on the treatment of FENO measures as either exact numerical or dichotomised at cut-offs 10, 16, 40, 50, 60, 70 or 80 ppb. The resulting covariate effects estimated from the data are given in table 3 as βi, i=0, 1, …, k, where k is the number of covariates in the respective model. This allowed the predicted probability of asthma for individual patients to be calculated. Figure 2 illustrates that the diagnostic accuracy of FENO increases remarkably when the results are combined with CSS. The AUC differences were significant in general practice (p=0.001), pneumologists’ practice (p=0.0002) and in the combined sample (p<0.0001). Beyond that, the AUCs of the general practice sample were higher than in the pneumologists’ practice sample. Box 1 gives examples of using estimate covariate effects and equations from table 3 in order to calculate posterior predicted probabilities of asthma dependent on selected combinations of symptoms and FENO measurements. In principle, diagnostic trees with all possible posterior predicted probabilities of asthma can be derived from table 3. The results can be computed with the calculator that is added as a supplement.

    Box 1

    Derivation of probability test for asthma

    The predicted probability (P) of asthma for each individual patient can be calculated from the equation:Embedded Image

    in which Embedded Image for the pneumologist model, where β0 is the estimated coefficient of the grand mean in the model and β1, β2 and β3 are the regression coefficients of the variables in the model.

    Examples of calculations for the final models are given below:

    Pneumologist model using exact numerical values for fractional exhaled nitric oxide (FENO):Embedded Image

    Examples

    A patient with wheezing, allergic rhinitis and a FENO value of 80 ppb has a prediction score of −0.25+2.1=1.85, resulting in a probability of 84.1% (95% CI 75.5% to 92.7%) of having asthma. Similarly, a patient with wheezing and FENO=80 ppb, but no allergic rhinitis, has a probability of 59.5% (95% CI 44.1% to 74.8%) of having asthma. A patient without any of these two items, however, with FENO=80 ppb, has a predicted probability of 39.3% (95% CI 22.8% to 55.9%) of having asthma.

    Pneumologist model using a cut-off value <16 ppb for FENO:Embedded Image

    Examples

    A patient with FENO measurement less or equal to 16 ppb, wheezing and allergic rhinitis has a probability of 65.3% (95% CI 52.7% to 78.0%) of having asthma. Similarly, a patient with the same symptoms and same FENO measurement, but without allergic rhinitis, has a probability of 31.8% (95% CI 21.6% to 42.0%) of having asthma. A patient without any of these items, but FENO below 16 ppb, has a probability of 15.1% (95% CI 9.2% to 21.1%) of having asthma. Conversely, this means a probability of 84.9% (95% CI 78.9% to 90.8%), for this group of patients, of not having asthma.

    According to these calculations, predicted probabilities for the sputum and general practice patients can be calculated.

    Figure 2
    Figure 2

    Receiver operating characteristic curves of fractional exhaled nitric oxide and clinical signs and symptoms.

    Reviewing the equations (table 3), the patients’ age turned out to be an important predictor in general practice. If the patient was 20 years old, the resulting posterior probability of FENO ≥30 ppb was 87.0% (calculating 95% CI was not possible). However, it was only 66.5% (95% CI 44.2% to 88.7%) when the patient was 50 years old. Ruling out was only effective when a patient was suffering from cough and recurrent respiratory tract infections; for example, the posterior probability of asthma was 18.8% (95% CI 2.1% to 35.6%) when FENO was ≤16ppb in a 20-year-old patient. Previously taken medication was strongly associated with asthma. For example, in a 40-year-old patient, the posterior probability was 86.6% (calculating 95% CI was not possible), even when FENO was ≤16 ppb.

    The patients from the pneumologists’ practices showed different characteristics. When a patient, independent of age, reported wheezing and nasal allergy, the posterior probability of asthma was 77.3% (95% CI 68.1% to 86.4%) when FENO was ≥30 ppb. Without these symptoms, the posterior probability was only 26.2% (95% CI 14.9% to 37.5%) when FENO was ≥30 ppb. Ruling out was possible when the patient had no allergic symptoms; with FENO ≤16 ppb, the resulting posterior probability for asthma was 15.1% (95% CI 9.2% to 21.1%).

    Wheezing, allergic rhinitis, medication, infection and age remained as significant covariates when data of all patients were pooled. Previously taken medication remained as a strong predictor for asthma and was interrelated with allergic rhinitis. Within this model, wheezing and allergic rhinitis helped to rule in, and recurrent infections helped to rule out, asthma. The positive predictive value of FENO increased considerably with decreasing age. As an example, the final prediction rule allowed ruling in asthma in a 20-year-old patient with wheezing and allergic rhinitis; probability of asthma was 78.4% (95% CI 68.8% to 88.1%) when FENO was ≥30 ppb. Without wheezing but with allergic rhinitis, p was 75.0% (95% CI 61.3% to 88.7%) when FENO was ≥50 ppb. In patients who were at least 43 years of age, the probability of asthma was lower than 20% when FENO was ≤16 ppb. However, ruling out in younger patients was only effective with recurrent respiratory tract infections when allergic signs were absent; then, as an example, the probability of asthma was 18.1% (95% CI 9.58% to 26.7%) when FENO was ≤16 ppb in a 20-year-old patient.

    Discussion

    To the best of our knowledge, this is the first study to evaluate FENO in different clinical settings in combination with CSS. We found that the selection of patients only had a slight influence on the sensitivities of the various FENO cut-off points. However, there was a meaningful influence on diagnostic patterns. The ROC analyses illustrated that the diagnostic accuracy of FENO increased remarkably when the test results were combined with CSS.

    The variation of the diagnostic accuracies of CSS related to respiratory diseases was shown in a few studies,1 ,38 illustrating that sensitivity increases and specificity decreases during the selection process of the patients. The explanation for this phenomenon previously was worked out theoretically and methodologically.20 Whiting et al39 found, in their systematic review about sources of variation and biases on diagnostic accuracy of diagnostic instruments, that sensitivity increased with disease prevalence and severity, whereas the effects on specificity were inconsistent. This might fit with our findings around the critical cut-off point of 40 ppb, as the sensitivities of the various cut-off points >40 ppb were slightly higher in the general practice setting. The higher pretest probability in general practice might be surprising at first sight. This could be explained by the study design, which required that participating patients had to travel to the lung function laboratory of the University Medical Hospital Heidelberg, which might have unintentionally caused a selection of patients with a higher probability and/or severity of disease. It might explain why ruling in the diagnosis of asthma appeared more straightforward in the general practice sample than in the pneumologists’ sample. In the latter, ruling in of asthma was only reasonable with FENO ≥30 ppb, when the patient suffered from wheezing and allergic rhinitis.

    The strength of both settings was that only diagnostically naive patients presenting for the first time for diagnostic investigation were included. As we observed no strong influence of the setting on the sensitivities and specificities of FENO cut-points, we pooled the data of all patients. The AUC increased remarkably from 0.650 to 0.753 when CSS were included in the diagnostic model. The final prediction rule conclusively illustrates that the potential of FENO to rule in or rule out asthma depends on the age of patients and the presentation of CSS. This might explain why varying cut-off points were found in the different studies when various patient collectives were evaluated. The prediction rule revealed that, especially, allergic rhinitis and wheezing are helpful to identify patients who will benefit from FENO measurement in terms of a high positive predictive value. This fits in with previous studies illustrating the relationship between asthma, increased FENO values, wheezing40 and allergic rhinitis,41 which is explained by the common type of eosinophilic inflammation.12 Thus it seems possible to diagnose asthma with FENO ≥30 ppb in patients with a compatible medical history, which is 20 ppb lower, as suggested by the ATS guideline.15 Another important point is the strong impact of previously given medication on the diagnostic model. Medications are prescribed occasionally ‘ex juvantibus’ in case of clinical uncertainty in general practice when asthma is suspected.42 This is crucial to avoid deterioration of asthma until the definite diagnosis is established by bronchial provocation in the practices of pneumologists or in a hospital. Thus, there seems to be a high probability of asthma when the patient continues inhaler therapy.

    It was difficult to exclude the diagnosis of asthma in younger patients solely on the basis of FENO measurement. In general practice, it was only possible when there were no specific allergic signs, FENO measurement showed low values and the patient was suffering from recurrent respiratory tract infections and cough. The latter appears contradictory to guidelines.5 However, the negative association with cough was already shown previously1 ,43 and seems reasonable from a clinical point of view, as many patients in general practice are coughing and/or have respiratory tract infections, but only few are really suffering from asthma. The low performance of ruling out asthma might be explained by the blind spot of FENO regarding neutrophilic inflammation.18 ,44 Patients with this type of inflammatory pattern are less responsive to inhaled corticosteroids, but absence of eosinophilia does not indicate an absence of steroid response.45 Therefore, patients with negative test results have to be referred for bronchial provocation in case of persistent symptoms, to definitely rule in or rule out the diagnosis of asthma.

    The strength of the study is that the diagnostic accuracy of FENO was evaluated in two different settings. This was accompanied by the use of two slightly different reference standards with respect to bronchial provocation, which could have influenced the evaluation. However, the 1-concentration-4-step dosimeter protocol shows results similar to the ATS multiconcentration protocol.28 Thus, a major distortion seems unlikely. We used the maximum concentration of methacholine for bronchial provocation as a reference standard to rule in and rule out asthma. Consequently, borderline bronchial hyper-reactivity also led to the diagnosis of asthma.8 Therefore, the potential of FENO for ruling out moderate and severe asthma might be underestimated.17 Further diagnostic studies would be necessary for differentiation of such subgroups, in particular with respect to the necessity for therapy with inhaled corticosteroids. Another inherent limitation is that two different patient collectives were used for analysis. However, recruitment in different settings was intended to analyse potentially different diagnostic patterns. It might be speculated whether two different diagnostic algorithms related to each practice setting need to be used. The general practice patients seemed to be comparatively selected, which might be explained by the study design. Therefore it appeared adequate to extrapolate our FENO findings more cautiously to allow generalisation of the diagnostic algorithm. Thus we decided to pool the data of both patient samples, because the clinical setting had only a minor influence on the sensitivities of the various cut-off points of FENO. As a result, the final model fitted well with the established clinical decision rules used by many physicians and led to a more conservative interpretation of the FENO measurements. However, a validation study would be desirable to confirm our findings.

    Another crucial issue is to decide the ideal cut-off point with respect to clinical significance. FENO ≥30 ppb resulting in a probability of asthma of 78.4% might be regarded as too low. However, this is considerably better than the predictive value of bronchial provocation with methacholine.8 ,30 ,37 Ruling out asthma with FENO ≤16 ppb is equal to a 20% fall of FEV1 during bronchial provocation, which can be detected with spirometry manoeuvres. However, the negative predictive value of specific airway resistance response on methacholine as determined with WBP would be much lower with a negative predictive value of 97.8%.30 Finally, eight patients with ACOS were labelled as non-asthmatics because of the uncertainty of their diagnostic entity. However, we expect that this did not distort the results, due to the low number of cases.

    Conclusion

    The ROC analysis revealed that FENO results should be interpreted in the context of CSS to enhance their diagnostic value in primary care. The final diagnostic model appears as a sound primary care algorithm fitting to the established diagnostic rules related to CSS of asthma. Importantly, FENO appears more promising for ruling in asthma than for ruling it out. Ruling in asthma with FENO ≥30 ppb is reasonable when allergic symptoms such as wheezing and allergic rhinitis are present. Previously taken medication is a strong predictor for asthma. Ruling out younger patients only seems possible in case of recurrent respiratory tract infections when no allergic symptoms are present.

    References

    1. ↵
      1. Schneider A,
      2. Ay M,
      3. Faderl B, et al
      . Diagnostic accuracy of clinical symptoms in obstructive airway diseases varied within different health care sectors. J Clin Epidemiol 2012;65:846–54. doi:10.1016/j.jclinepi.2011.12.014
      OpenUrlPubMed
    2. ↵
      1. van Schayck CP,
      2. van Der Heijden FM,
      3. van den Boom G, et al
      . Underdiagnosis of asthma: is the doctor or the patient to blame? The DIMCA project. Thorax 2000;55:562–5. doi:10.1136/thorax.55.7.562
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Pellegrino R,
      2. Viegi G,
      3. Brusasco V, et al
      . Interpretative strategies for lung function tests. Eur Respir J 2005;26:948–68. doi:10.1183/09031936.05.00035205
      OpenUrlFREE Full Text
    4. ↵
      1. Schneider A,
      2. Gindner L,
      3. Tilemann L, et al
      . Diagnostic accuracy of spirometry in primary care. BMC Pulm Med 2009;9:31. doi:10.1186/1471-2466-9-31
      OpenUrlCrossRefPubMed
    5. ↵
      National Institute of Health: Global Initiative for Asthma—Global Strategy for Asthma Management and Prevention. 2014. http://www.ginasthma.com
    6. ↵
      1. den Otter JJ,
      2. Reijnen GM,
      3. van den Bosch WJ, et al
      . Testing bronchial hyper-responsiveness: provocation or peak expiratory flow variability? Br J Gen Pract 1997;47:487–92.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Hunter CJ,
      2. Brightling CE,
      3. Woltmann G, et al
      . A comparison of the validity of different diagnostic tests in adults with asthma. Chest 2002;121:1051–7. doi:10.1378/chest.121.4.1051
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Crapo RO,
      2. Casaburi R,
      3. Coates AL, et al
      . Guidelines for methacholine and exercise challenge testing-1999. This official statement of the American Thoracic Society was adopted by the ATS Board of Directors, July 1999. Am J Respir Crit Care Med 2000;161:309–29. doi:10.1164/ajrccm.161.1.ats11-99
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Alving K,
      2. Weitzberg E,
      3. Lundberg JM
      . Increased amount of nitric oxide in exhaled air of asthmatics. Eur Respir J 1993;6:1368–70.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Kharitonov SA,
      2. Yates D,
      3. Robbins RA, et al
      . Increased nitric oxide in exhaled air of asthmatic patients. Lancet 1994;343:133–5. doi:10.1016/S0140-6736(94)90931-8
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Ricciardolo FL
      . Multiple roles of nitric oxide in the airways. Thorax 2003;58:175–82. doi:10.1136/thorax.58.2.175
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Jatakanon A,
      2. Lim S,
      3. Kharitonov SA, et al
      . Correlation between exhaled nitric oxide, sputum eosinophils, and methacholine responsiveness in patients with mild asthma. Thorax 1998;53:91–5. doi:10.1136/thx.53.2.91
      OpenUrlAbstract
    13. ↵
      1. Ricciardolo FL,
      2. Sorbello V,
      3. Ciprandi G
      . FeNO as biomarker for asthma phenotyping and management. Allergy Asthma Proc 2015;36:e1–8. doi:10.2500/aap.2015.36.3805
      OpenUrl
    14. ↵
      1. Bjermer L,
      2. Alving K,
      3. Diamant Z, et al
      . Current evidence and future research needs for FeNO measurement in respiratory diseases. Respir Med 2014;108:830–41. doi:10.1016/j.rmed.2014.02.005
      OpenUrlCrossRefPubMed
    15. ↵
      1. Dweik RA,
      2. Boggs PB,
      3. Erzurum SC, et al
      . An official ATS clinical practice guideline: interpretation of exhaled nitric oxide levels (FENO) for clinical applications. Am J Respir Crit Care Med 2011;184:602–15. doi:10.1164/rccm.9120-11ST
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Hewitt RS,
      2. Modrich CM,
      3. Medlicott T, et al
      . Supporting the diagnosis of non-specific respiratory symptoms in primary care: the role of exhaled nitric oxide measurement and spirometry. Prim Care Respir J 2008;17:97–103. doi:10.3132/pcrj.2008.00025
      OpenUrlPubMed
    17. ↵
      1. Schneider A,
      2. Tilemann L,
      3. Schermer T, et al
      . Diagnosing asthma in general practice with portable exhaled nitric oxide measurement—results of a prospective diagnostic study: FENO < or = 16 ppb better than FENO < or =12 ppb to rule out mild and moderate to severe asthma [added]. Respir Res 2009;10:15. doi:10.1186/1465-9921-10-15
      OpenUrlCrossRefPubMed
    18. ↵
      1. Schneider A,
      2. Schwarzbach J,
      3. Faderl B, et al
      . FENO measurement and sputum analysis for diagnosing asthma in clinical practice. Respir Med 2013;107:209–16. doi:10.1016/j.rmed.2012.10.003
      OpenUrl
    19. ↵
      1. Ransohoff DF,
      2. Feinstein AR
      . Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 1978;299:926–30. doi:10.1056/NEJM197810262991705
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Knottnerus JA
      . The effects of disease verification and referral on the relationship between symptoms and diseases. Med Decis Making 1987;7:139–48. doi:10.1177/0272989X8700700304
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Knottnerus JA
      . Medical decision making by general practitioners and specialists. Fam Pract 1991;8:305–7. doi:10.1093/fampra/8.4.305
      OpenUrlFREE Full Text
    22. ↵
      1. Ledley RS,
      2. Lusted LB
      . Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding how physicians reason. Science 1959;130:9–21. doi:10.1126/science.130.3366.9
      OpenUrlFREE Full Text
    23. ↵
      1. van Vugt SF,
      2. Broekhuizen BD,
      3. Lammens C, et al
      . Use of serum C reactive protein and procalcitonin concentrations in addition to symptoms and signs to predict pneumonia in patients presenting to primary care with acute cough: diagnostic study. BMJ 2013;346:f2450. doi:10.1136/bmj.f2450
      OpenUrlAbstract/FREE Full Text
    24. ↵
      1. Hopstaken RM,
      2. Muris JW,
      3. Knottnerus JA, et al
      . Contributions of symptoms, signs, erythrocyte sedimentation rate, and C-reactive protein to a diagnosis of pneumonia in acute lower respiratory tract infection. Br J Gen Pract 2003;53:358–64.
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Knottnerus JA,
      2. van Weel C,
      3. Muris JW
      . Evaluation of diagnostic procedures. BMJ 2002;324:477–80. doi:10.1136/bmj.324.7335.477
      OpenUrlFREE Full Text
    26. ↵
      American Association for Respiratory Care. AARC clinical practice guideline. Body plethysmography: 2001 revision and update. Resp Care 2001;46:506–13.
      OpenUrl
    27. ↵
      Quanjer PH, ed. Standardized lung function testing. Report Working Party Standardization of Lung Function Tests. European Community for Coal and Steel. Bull Eur Physiopathol Respir 1983;19(Suppl 5):1–95.
    28. ↵
      1. Merget R,
      2. Jörres RA,
      3. Heinze E, et al
      . Development of a 1-concentration-4-step dosimeter protocol for methacholine testing. Respir Med 2009;103:607–13. doi:10.1016/j.rmed.2008.10.010
      OpenUrlPubMed
    29. ↵
      1. Criée CP,
      2. Sorichter S,
      3. Smith HJ, et al
      . Body plethysmography—its principles and clinical use. Respir Med 2011;105:959–71. doi:10.1016/j.rmed.2011.02.006
      OpenUrlPubMed
    30. ↵
      1. Schneider A,
      2. Schwarzbach J,
      3. Faderl B, et al
      . Whole-body plethysmography in suspected asthma—a prospective study of its added diagnostic value in 302 patients. Dtsch Arztebl Int 2015;112:405–11. doi:10.3238/arztebl.2015.0405
      OpenUrl
    31. ↵
      American Thoracic Society, European Respiratory Society. ATS/ERS recommendations for standardized procedures for the online and offline measurement of exhaled lower respiratory nitric oxide and nasal nitric oxide, 2005. Am J Respir Crit Care Med 2005;171:912–30. doi:10.1164/rccm.200406-710ST
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      1. Harrell FE Jr.,
      2. Lee KL,
      3. Mark DB
      . Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361–87. doi:10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Davidson ACT
      . Statistical models. Cambridge: Cambridge University Press, 2009.
    34. ↵
      1. Hintze J
      . NCSS 9. NCSS, LLC Kaysville, Utah, USA. 2014. http://www.ncss.com
    35. ↵
      1. DeLong ER,
      2. DeLong DM,
      3. Clarke-Pearson DL
      . Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. doi:10.2307/2531595
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Zhou X,
      2. Obuchowski N,
      3. McClish D
      . Statistical methods in diagnostic medicine. New York: John Wiley & Sons, Inc, 2002.
    37. ↵
      1. Perpina M,
      2. Pellicer C,
      3. de Diego A, et al
      . Diagnostic value of the bronchial provocation test with methacholine in asthma. A Bayesian analysis approach. Chest 1993;104:149–54. doi:10.1378/chest.104.1.149
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Melbye H,
      2. Straume B
      . The spectrum of patients strongly influences the usefulness of diagnostic tests for pneumonia. Scand J Prim Health Care 1993;11:241–6. doi:10.3109/02813439308994838
      OpenUrlPubMed
    39. ↵
      1. Whiting P,
      2. Rutjes AW,
      3. Reitsma JB, et al
      . Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med 2004;140:189–202. doi:10.7326/0003-4819-140-3-200402030-00010
      OpenUrlCrossRefPubMedWeb of Science
    40. ↵
      1. Malinovschi A,
      2. Fonseca JA,
      3. Jacinto T, et al
      . Exhaled nitric oxide levels and blood eosinophil counts independently associate with wheeze and asthma events in National Health and Nutrition Examination Survey subjects. J Allergy Clin Immunol 2013;132:821–7. doi:10.1016/j.jaci.2013.06.007
      OpenUrlCrossRefWeb of Science
    41. ↵
      1. Nadif R,
      2. Rava M,
      3. Decoster B, et al
      . Exhaled nitric oxide, nitrite/nitrate levels, allergy, rhinitis and asthma in the EGEA study. Eur Respir J 2014;44:351–60. doi:10.1183/09031936.00202413
      OpenUrlAbstract/FREE Full Text
    42. ↵
      1. den Otter JJ,
      2. van Dijk B,
      3. van Schayck CP, et al
      . How to avoid underdiagnosed asthma/chronic obstructive pulmonary disease? J Asthma 1998;35:381–7. doi:10.3109/02770909809075672
      OpenUrlPubMedWeb of Science
    43. ↵
      1. Bai J,
      2. Peat JK,
      3. Berry G, et al
      . Questionnaire items that predict asthma and other respiratory conditions in adults. Chest 1998;114:1343–8. doi:10.1378/chest.114.5.1343
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      1. Porsbjerg C,
      2. Lund TK,
      3. Pedersen L, et al
      . Inflammatory subtypes in asthma are related to airway hyperresponsiveness to mannitol and exhaled NO. J Asthma 2009;46:606–12. doi:10.1080/02770900903015654
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. Cowan DC,
      2. Cowan JO,
      3. Palmay R, et al
      . Effects of steroid therapy on inflammatory cell subtypes in asthma. Thorax 2010;65: 384–90. doi:10.1136/thx.2009.126722
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors AS and SW had the study idea. AS wrote the first draft of the manuscript. GW and SW performed the analyses and calculated the prediction rules. GW and SW developed the FENO calculator. RAJ helped to interpret the data and with writing. All the authors made substantial contributions to the conception of the work, analysis and interpretation of data. All the authors were involved in drafting the work, and revised it critically, providing important intellectual content. All the authors approved this version to be published, and are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    • Funding The part of the study in the general practices was funded by the Federal Ministry of Education and Research (BMBF); grant number 01GK0515.

    • Competing interests None declared.

    • Patient consent Obtained.

    • Ethics approval The study was approved by the Medical Ethics Committee of the University of Heidelberg and by the Medical Ethics Committee of the Medical Faculty of the Technische Universität München, respectively.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.