OBJECTIVE: The objective of this study was to conduct an item analysis of the COMFORT scale within a sample of critically ill children. The COMFORT scale is a tool that measures eight clinical parameters at the bedside to determine a critically ill child's level of distress. However, documentation of the reliability and validity of specific components of this instrument has been limited to date. DESIGN: This was an observational study. SETTING: A tertiary care pediatric intensive care unit. PATIENTS: A sample of 18 intubated, mechanically ventilated patients. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: An item analysis of the COMFORT scale was conducted. The scale was used for variable intervals with each child, providing a total of 514 individual completed ratings. These ratings were then submitted to descriptive, correlational, factor, and regression analyses using the Statistical Package for the Social Sciences. The principal finding of this item analysis was that 97% of the total score variance was explained by six of the eight items. In particular, it was remarkable that heart rate and blood pressure demonstrated very limited reliability and validity as determinants of the total COMFORT score. As well, the clinical experience of the investigators with this scale has demonstrated that these two items can be cumbersome to use, given that these parameters are often affected by hemodynamic factors. CONCLUSIONS: On the basis of these findings, we propose a modified COMFORT scale based on a six-item scoring system. This modified scale would provide a more reliable and valid measure for research as well as clinical purposes, by eliminating two parameters that can be affected by factors other than the child's level of distress.