Adding Continuous Vital Sign Information to Static Clinical Data Improves the Prediction of Length of Stay After Intubation: A Data-Driven Machine Learning Approach

Setting and Subjects

The Medical-Surgical ICU at Boston Children's Hospital is a 30-bed unit that provides care to the full range of pediatric patients, including extracorporeal life support, not admitted primarily due to congenital or acquired pediatric heart disease. The unit has an average of 2,000 annual admissions. For this study, we included all patients on mechanical ventilation for whom vital sign data were available within the first 24 h of mechanical ventilation. Patients were excluded if (a) they were discharged from the ICU < 24 h after the initiation of mechanical ventilation, (b) if they were intubated before admission to the pediatric ICU, (c) if inconsistencies in the recorded ICU admission and discharge times were identified (ie, discharge time preceded admission time), or (d) if there was a gap of >20 min in any of the patient's vital signs time series. The length of this time window was chosen with 2 considerations in mind. First, 20 min is a small time interval, which represents ∼1.3% of the 24-h period considered as input for this predictive study, and, so, we considered that it would not significantly change the trends (and thus, the extracted features) in the vital signs time series. Second, allowing these time gaps enabled us to build a “realistic and generalizable” workflow capable of dealing with incomplete data (such as those due to routine patient care, eg, bathing) via a suitable data imputation approach. The Boston Children's Hospital Institutional Review Board approved the study with a waiver of informed consent.

Data Sources

All subjects' vital signs used for this study were collected by using routine bedside monitors (Philips IntelliVue MP90, Philips, Andover, Massachusetts) and recorded at a 5-s frequency by using the T3 system (Etiometry, Allston, Massachusetts). We considered only vital signs consistently available for most subjects, that is, heart rate, breathing frequency, and pulse and . Note that, although heart rate and pulse both measure the rate at which the heart beats, they are recorded with different devices, and, thus, for robustness in our analysis, we included both signals. Subjects' static clinical data (including sex, age, pre-ICU admission location) and the components of the Pediatric Index of Mortality-2 or Pediatric Index of Mortality-3 score (elective admission, recovery after the procedure, cardiac bypass, diagnosis risk, lack of pupillary response, mechanical ventilation, first systolic blood pressure, base excess, , and ) were collected from the electronic health record for the second phase of the study.

We designed the present study to address the practical problem of forecasting bed availability. With knowing that intubation is an extremely important factor that affects the stay in the ICU, we selected the total ICU days after intubation as a clinically relevant and practically important outcome for this study. Specifically, the outcome of interest was in determining whether a pediatric patient will experience a prolonged LOS (>4 d) in the ICU from the time of initiation of mechanical ventilation. We chose 4 d because the average duration of mechanical ventilation in our population was 4–5 d and because this time frame allows for modification to the surgical and staffing schedule based on accurate predictions. Discharge times were obtained from the hospital's admission-discharge-transfer system, and ICU LOS was calculated as the difference from the time of intubation, recorded in the electronic health record, to the time of ICU discharge. Deceased patients were excluded from the study because our clinical team believed that most of these individuals who were high risk would have been identified on admission and, thus, an algorithmic approach to identify them would not be of much value.

Data Preprocessing

Time series with no missing values are required for most automatic computational feature extraction techniques, including those used for this study (described in the supplementary materials [see the supplementary materials at http://www.rcjournal.com]). Thus, we used a data imputation approach based on gaussian processes^15–17 to close gaps ≤ 20 min in the vital signs data. When using data imputation techniques based on gaussian processes, the imputed values, at a given point in time, are systematically calculated without any structural assumption on the functional form of the signal, and the imputed values are part of a distribution that allows potential uncertainty quantification to be conducted. As such, they are data driven and easy to implement. We imputed the values of different vital signs in time windows with a width of less than or equal to 20 min, and data imputation for at least one the physiological time series was required for most subjects. Details of this approach are presented in the supplementary materials (see the supplementary materials at http://www.rcjournal.com).

Feature Extraction

Feature-based methods convert raw time series (monitor data) into vectors of statistical features. These features are then used as the input variables in predictive methodologies. In our study, we explored many feature extraction approaches to characterize all 4 vital signs (heart rate, breathing frequency, oxygenation levels, and pulse obtained from oximetry sensors) associated with each pediatric patient. These methodologies extract collections of features (frequently collected in multiple scientific disciplines) from multiple time series, maximizing efficiency and minimizing time,^18–21 and include summaries of the time series, for example, correlation structure, distribution, entropy, stationarity, scaling properties. Projecting vital signs time series into a feature space allowed us to capture the underlying statistical and temporal behavior of the vital signs across multiple subjects and enabled us to mitigate the impact of errors and outliers in the measurements. In addition, because “normal” vital sign values depend on age and, given the heterogeneity of ages in our study cohort, analyzing features may help us identify common properties that may not be apparent from the “raw” vital signs time series.

Overall Machine Learning Methodological Strategy

From a general machine learning perspective, solving the classification problem of identifying patients with prolonged LOS by using vital sign information (and perhaps other relevant patient information) requires 3 steps. The first step is to identify a subset of useful variables, or features, that characterize each individual patient and that can then be used as predictors to build a classifier with the desired outcome. The second step is the model formulation. This step is typically conducted on a subset of subjects (80%), often referred to as a training set. The third step consists of assessing the predictive performance of the “formulated” model on a subset of unseen subjects (20%), that is, the evaluation is performed in a strictly out-of-sample fashion. We refer to these 3 steps sequentially as the following: feature selection, model formulation, and model prediction assessment.

We used 2 different strategies to solve this classification problem. In the first strategy, we experimented with multiple modeling approaches and imposed no constraints on the computational power involved in the solution of the problem. This strategy, denoted as the “baseline approach” in the rest of this article, was used to identify the classification performance of multiple methodologies to solve the task in a research environment, with no constraints on computational time. The second strategy involved the construction of a workflow that could be deployable in a real-life scenario and capable of achieving comparable performance as the best methodologies found in the first strategy but with significantly less computational effort and more parsimony. Both strategies are briefly described below. Detailed information that characterizes these approaches is shown in the supplementary materials (see the supplementary materials at http://www.rcjournal.com).

Baseline Approach

In this strategy, we experimented with feature selection approaches that included, for example, removing low variance features, dimensionality reduction by using principal component analysis²² and removing highly linearly correlated features to remove redundant information. For the model formulation or classification task on the training set, multiple supervised learning approaches were implemented, including logistic regressions, random forests, support vector machine approaches, and gradient boosted trees.^22,23 Furthermore, ensemble methods, such as voting²⁴ and stacking classifiers,^25,26 were also used to combine individual classifiers to further improve the accuracy of the model.

Scalable Robust Approach

Many of the models studied in the baseline approach use a large number of features as input to complete the classification task. As a result, they lead to accurate results but may not be suitable for practical applications in real-time medical settings because solving the classification problem for a new patient in real-time might demand an overly lengthy time period because the process may require extracting and processing all the time-series features. For reference, the total number of data points from 24 h worth of raw data for 4 different time series collected at 5-s intervals totals 69,120 (ie, 17,280 × 4); whereas all extracted features from the raw data led to a total of 13,332 data points per subject. This likely rendered the baseline methodology impractical for real-time decision making. Thus, a methodology capable of effectively decreasing the number of input variables (dimensionality reduction) and maintaining the predictive performance is desirable.

Furthermore, robustness and interpretability are important in medical applications. For robustness, any practical machine-learning solution should be presented with proper uncertainty quantification envelopes to understand the accuracy and expected variability of any given solution. Avoiding overfitting issues is essential, and uncertainty quantification analysis must assure that this is avoided. For interpretability, at least a general understanding of the families of features that drive those solutions is important. Thus, the proposed deployable workflow relies on an unsupervised learning approach for feature selection that leads to a small and potentially interpretable set of features, and on a scalable and parsimonious (and thus computationally efficient) supervised learning approach to maximize the predictive power via a robust evaluation design.

Feature Selection

We implemented an unsupervised learning approach, based on clustering strategies, to identify a reduced number of features capable of characterizing most of the input parameter space. Our approach applies the affinity propagation method²⁷ by using the maximal information coefficient²⁸ as a similarity matrix. This methodology outputs representative features that can be interpreted, as opposed to projected features obtained from principal component analysis like approaches. Details on this approach are presented in the supplementary materials (see the supplementary materials at http://www.rcjournal.com).

Supervised Learning

The supervised learning solution for the problem of predicting LOS is shown in context in Figure 1. Different supervised learning approaches were initially considered to be applicable to our classification problem, such as gradient boosting trees,²⁹ artificial neural networks, and support vector machine.²³ A gradient boosting tree is typically used with decision trees, which have become a popular machine learning technique because of its simplicity and ease of use. A gradient boosting tree, which combines weak “learners” into a single strong learner in an iterative fashion, provides an extra degree of freedom in the classic bias-variance trade-off. Ultimately, a gradient boosting tree provides a good level of robustness toward overfitting (because boosting builds models intelligently by giving more and more weight to observations that are hard to classify), rendering this solution as appropriate for the problem at hand. To quantitatively evaluate the contribution of each type of data used for prediction, static clinical data versus information contained in the vital signs during the first 24 h of their stay, we designed 3 distinct classification studies:

1. classification by using vital sign information (time-series features) only,

2. classification by using static clinical data only, and

3. classification by using both time series and static clinical data.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1.

Proposed methodology for feature engineering. GBT = gradient boosting tree; LOS = length of stay.

Evaluation Approach

To ensure the validity and robustness of our approach, we evaluated the performance of gradient boosting trees under 3 different classification scenarios: (1) classification by using vital signs time-series features only, (2) classification by using readily available electronic health record static clinical data, and (3) classification by using a combination of both vital signs time-series features and static clinical data. For each scenario, that subjects' data were divided into training, validation, and test sets according to the following rule: 80% training, 10% validation, and 10% evaluation. To obtain a proper distribution of the evaluation of model accuracy and to mitigate overfitting due to specific choices of subjects sets, a total of 300 independent experiments, by using random partitions of subjects for training, validation, and evaluation sets, were conducted for each scenario.

Moreover, to reduce the risk of overfitting a problem of 186 predictors and 284 observations/subjects, we used a shrinking factor within the gradient boosting tree algorithm to better control the learning rate and favor more parsimonious models. Note that, similar to a learning rate in stochastic optimization, shrinkage reduces the influence of each individual tree and allows space for future trees to improve the model; ultimately, this provides a configuration trade-off between the number of trees and learning rate. We kept the same proportion of each class as in the original cohort in the training across the different sets to avoid potentially unbalanced distribution of classes and to assess the quality of predictions on both classes in all 300 experiments. In all cases, the training set was used to construct the classification model, the validation set was used to optimize the hyperparameters of the classification algorithm, and the (holdout) evaluation set was used exclusively to quantify the accuracy of the proposed model.