comparative assessment of methods for short term forecasts of covid 19 hospital admissions CORD-Papers-2022-06-02 (Version 1)

Title: Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level
Abstract: BACKGROUND: Forecasting healthcare demand is essential in epidemic settings both to inform situational awareness and facilitate resource planning. Ideally forecasts should be robust across time and locations. During the COVID-19 pandemic in England it is an ongoing concern that demand for hospital care for COVID-19 patients in England will exceed available resources. METHODS: We made weekly forecasts of daily COVID-19 hospital admissions for National Health Service (NHS) Trusts in England between August 2020 and April 2021 using three disease-agnostic forecasting models: a mean ensemble of autoregressive time series models a linear regression model with 7-day-lagged local cases as a predictor and a scaled convolution of local cases and a delay distribution. We compared their point and probabilistic accuracy to a mean-ensemble of them all and to a simple baseline model of no change from the last day of admissions. We measured predictive performance using the weighted interval score (WIS) and considered how this changed in different scenarios (the length of the predictive horizon the date on which the forecast was made and by location) as well as how much admissions forecasts improved when future cases were known. RESULTS: All models outperformed the baseline in the majority of scenarios. Forecasting accuracy varied by forecast date and location depending on the trajectory of the outbreak and all individual models had instances where they were the top- or bottom-ranked model. Forecasts produced by the mean-ensemble were both the most accurate and most consistently accurate forecasts amongst all the models considered. Forecasting accuracy was improved when using future observed rather than forecast cases especially at longer forecast horizons. CONCLUSIONS: Assuming no change in current admissions is rarely better than including at least a trend. Using confirmed COVID-19 cases as a predictor can improve admissions forecasts in some scenarios but this is variable and depends on the ability to make consistently good case forecasts. However ensemble forecasts can make forecasts that make consistently more accurate forecasts across time and locations. Given minimal requirements on data and computation our admissions forecasting ensemble could be used to anticipate healthcare needs in future epidemic or pandemic settings. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12916-022-02271-x.
Published: 2022-02-21
Journal: BMC Med
DOI: 10.1186/s12916-022-02271-x
DOI_URL: http://doi.org/10.1186/s12916-022-02271-x
Author Name: Meakin Sophie
Author link: https://covid19-data.nist.gov/pid/rest/local/author/meakin_sophie
Author Name: Abbott Sam
Author link: https://covid19-data.nist.gov/pid/rest/local/author/abbott_sam
Author Name: Bosse Nikos
Author link: https://covid19-data.nist.gov/pid/rest/local/author/bosse_nikos
Author Name: Munday James
Author link: https://covid19-data.nist.gov/pid/rest/local/author/munday_james
Author Name: Gruson Hugo
Author link: https://covid19-data.nist.gov/pid/rest/local/author/gruson_hugo
Author Name: Hellewell Joel
Author link: https://covid19-data.nist.gov/pid/rest/local/author/hellewell_joel
Author Name: Sherratt Katharine
Author link: https://covid19-data.nist.gov/pid/rest/local/author/sherratt_katharine
Author Name: Funk Sebastian
Author link: https://covid19-data.nist.gov/pid/rest/local/author/funk_sebastian
sha: 864e9e976a114ba22fac622f66a3fa31f47d2d87
license: cc-by
license_url: https://creativecommons.org/licenses/by/4.0/
source_x: Medline; PMC; WHO
source_x_url: https://www.medline.com/https://www.ncbi.nlm.nih.gov/pubmed/https://www.who.int/
pubmed_id: 35184736
pubmed_id_url: https://www.ncbi.nlm.nih.gov/pubmed/35184736
pmcid: PMC8858706
pmcid_url: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8858706
url: https://www.ncbi.nlm.nih.gov/pubmed/35184736/ https://doi.org/10.1186/s12916-022-02271-x
has_full_text: TRUE
Keywords Extracted from Text Content: COVID-19 patients COVID-19 mean-ensemble top-or National Health Service (NHS) Trusts Alpha variant B.1.1.7 London, UK [11 B i t. 1-2 mean-ensemble ... 0.3.0 Fig. S6 ): patients NHS patient UTLA u UTLA R t Fig. S2 Fig. S7B CHR u p t θ A 1 {· Fig. S5A COVID-19 patients |m − y| Belgium [16 S7A Fig. 3A covid19.nhs.data trend-based lockdowns Figure S6 B UTLA-level Figs. S8-9 National Health Service sWIS < 1 NHS Trusts lockdown UK Fig. S7C θ A * Trust-UTLA Fig. S7D lateral EpiNow2 (1.3.3.8 sWIS θ x * < 1 covid19.nhs.data R UTLA [41] non-COVID-19 Ebola virus [14] ETS early-autumn Fig. S3C − pillar 2 K line covid 19hospi tal-activ UTLAs first-or second-ranked upper bounds S7C COVID-19 risk factors EpiSoon London and parts BMC Medicine (2022) u α third-or Alpha SARS-CoV-2 Fig. 2B S7D θ x * > 1 WIS θ A November-02 COVID-19 F NHS Trust Rt-based [19] first-or scoringutils pillar 1 Fig. 3 upper Fig. S5 rWIS Zika [27] upper-tier lower-tier 72/129 Fig. S3D ): α k 2 UTLA-level COVID-19 NHS SFunk KS COVID-19 covid19.nhs.data R SF JDM Trust-UTLA scoringutils NB covid 19-hospi tal-activ 0.3.0 EpiSoon EpiNow2 (1.3.3.8 UK CMMID COVID-19 UTLA
Extracted Text Content in Record: First 5000 Characters:Background: Forecasting healthcare demand is essential in epidemic settings, both to inform situational awareness and facilitate resource planning. Ideally, forecasts should be robust across time and locations. During the COVID-19 pandemic in England, it is an ongoing concern that demand for hospital care for COVID-19 patients in England will exceed available resources. We made weekly forecasts of daily COVID-19 hospital admissions for National Health Service (NHS) Trusts in England between August 2020 and April 2021 using three disease-agnostic forecasting models: a mean ensemble of autoregressive time series models, a linear regression model with 7-day-lagged local cases as a predictor, and a scaled convolution of local cases and a delay distribution. We compared their point and probabilistic accuracy to a mean-ensemble of them all and to a simple baseline model of no change from the last day of admissions. We measured predictive performance using the weighted interval score (WIS) and considered how this changed in different scenarios (the length of the predictive horizon, the date on which the forecast was made, and by location), as well as how much admissions forecasts improved when future cases were known. Results: All models outperformed the baseline in the majority of scenarios. Forecasting accuracy varied by forecast date and location, depending on the trajectory of the outbreak, and all individual models had instances where they were the top-or bottom-ranked model. Forecasts produced by the mean-ensemble were both the most accurate and most consistently accurate forecasts amongst all the models considered. Forecasting accuracy was improved when using future observed, rather than forecast, cases, especially at longer forecast horizons. Assuming no change in current admissions is rarely better than including at least a trend. Using confirmed COVID-19 cases as a predictor can improve admissions forecasts in some scenarios, but this is variable and depends on the ability to make consistently good case forecasts. However, ensemble forecasts can make forecasts that make consistently more accurate forecasts across time and locations. Given minimal requirements on data and computation, our admissions forecasting ensemble could be used to anticipate healthcare needs in future epidemic or pandemic settings. extremely high demand for hospital care in England. As such, it is an ongoing concern that demand for hospital care will exceed available resources. This worst-case scenario has seen patients with COVID-19 receiving lower-quality care [3] , as well as cancellations of planned surgeries or routine services; in the UK, the National Health Service (NHS) faced a substantial backlog of patient care throughout the COVID-19 pandemic [4] . Forecasting healthcare requirements during an epidemic are critical for planning and resource allocation [5] [6] [7] , and short-term forecasts of COVID-19 hospital activity have been widely used during the COVID-19 pandemic to support public health policy (e.g. [8] [9] [10] [11] ). Whilst national or regional forecasts provide a bigpicture summary of the expected trajectory of COVID-19 activity, they can mask spatial heterogeneity that arises through localised interventions or demographic heterogeneity in the risk of exposure or severity [12] . Small-scale forecasts have been used to support local COVID-19 responses (e.g. in Austin, TX, USA [9] ), as well as to forecast non-COVID-19 or more general healthcare demands at the hospital level [13, 14] . Forecasts of hospital admissions are also an essential step to forecasting bed or intensive care unit (ICU) demand (e.g. [11, 13, 14] ). In theory, future admissions are a function of recent cases in the community, the proportion of cases that require and seek health care (the case hospitalisation rate (CHR)), and the delay from symptom onset to hospital admission. However, forecasting admissions from community cases is challenging as both the CHR and admission delay can vary over time. The CHR depends on testing effort and strategy (how many symptomatic and asymptomatic cases are identified), the age distribution of cases [1] , and the prevalence of other COVID-19 risk factors amongst cases [12] . Retrospective studies of COVID-19 patients reported a mean delay from symptom onset to hospital admission to be 4.6 days in the UK [15] and 5.7 days in Belgium [16] , but this varies by age and place of residence (e.g. care-home residents have a longer average admissions delay than non-residents) [16] . Forecasting studies have found that cases are predictive of admissions with a lag of only 4-7 days [10, 14] . Given the short estimated delay between cases and future admissions, to make short-term forecasts of admissions therefore also requires forecasts of cases. Whilst some studies consider mobility and meteorological predictors with longer lags [14] , they lack a direct mechanistic relationship with admissions and may have only a limi
Keywords Extracted from PMC Text: value):\documentclass[12pt]{minimal [19] patients as\documentclass[12pt]{minimal [1, 12] Fig. S7C y\right)=\frac{1}{K+0.5}\Big({w}_0\left|y-m\left|+{\sum}_{k=1}^K{w}_k{IS}_{{\alpha}_k}\left(F fourth-ranked horizon;By B}=\left(\mathrm{mean}\ \mathrm{WIS}\ \mathrm{of}\ \mathrm{model}\ A\right)/\left(\mathrm{mean}\ \mathrm{WIS}\ \mathrm{of}\ COVID-19 sWIS y\right)=\left({u}_{\alpha}-{l}_{\alpha}\right)+\frac{2}{\alpha}\left({l}_{\alpha}-y\right){1}_{\left\{y<{l}_{\alpha}\right\}}+\frac{2}{\alpha}\left(y-{u}_{\alpha}\right){1}_{\left\{y>{u}_{\alpha}\right\}}$$\end{document}ISαF B London and parts 0.3.0 UTLAu ... lockdowns pillar 2 CHR S7C covid19.nhs.data scoringutils B\right)$$\end{document}θA sWIS < 1 patient EpiNow2 (1.3.3.8 Alpha SARS-CoV-2 t. αk2 lockdown London, UK [11 Fig. S7B UTLA [41 UTLA Zika [27] ETS 72/129 1{· lower-tier Fig. S5A upper-tier − COVID-19 patients Belgium [16 COVID-19 risk factors central 100(1-α)% 1-2 Alpha variant B.1.1.7 u}\;\ast{\;\text{cases Figs. S8-9 Fig. S3C model:\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\theta_A}^{\ast }= big-picture SARS-CoV-2 Rt-based trend-based Fig. S6B NHS Trust lateral EpiSoon lα+2αy-uα1y 's non-COVID-19 F National Health Service y=1K+0.5(w0y-m+∑k=1KwkISαkF Fig. 2A Fig. S7D Fig. S6 UK early-autumn UTLA-level UTLA u mean-ensemble \left({u}_{0.1}-{l}_{0.1}\right)$$\end{document}sharpness=∑k=1Kwkuαk-lαk=0.25*u0.5-l0.5 rWIS horizon;By Trust S7D S7A K " M$$\end{document}θA y\right)\right.\right)$$\end{document}WISα0:KF UTLAs Trust-UTLA UTLA}\;u}\Big)$$\end{document}∑u(pt Fig. 2B orThere intervals:\documentclass[12pt]{minimal Fig. S3B Fig. S2 [14] NHS Trusts {\theta}_A/{\theta}_{\mathrm{baseline}}$$\end{document}θA*=θA/θbaselineBy covid19.nhs.data R \documentclass[12pt]{minimal θA∗ \documentclass[12pt]{minimal} Fig. 3A orThe upper first- upper bounds pillar 1 Fig. S3D NHS Ebola virus line
Extracted PMC Text Content in Record: First 5000 Characters:The sheer volume of SARS-CoV-2 reported cases in England combined with a substantial case-hospitalisation rate amongst high-risk groups [1, 2] has resulted in an extremely high demand for hospital care in England. As such, it is an ongoing concern that demand for hospital care will exceed available resources. This worst-case scenario has seen patients with COVID-19 receiving lower-quality care [3], as well as cancellations of planned surgeries or routine services; in the UK, the National Health Service (NHS) faced a substantial backlog of patient care throughout the COVID-19 pandemic [4]. Forecasting healthcare requirements during an epidemic are critical for planning and resource allocation [5–7], and short-term forecasts of COVID-19 hospital activity have been widely used during the COVID-19 pandemic to support public health policy (e.g. [8–11]). Whilst national or regional forecasts provide a big-picture summary of the expected trajectory of COVID-19 activity, they can mask spatial heterogeneity that arises through localised interventions or demographic heterogeneity in the risk of exposure or severity [12]. Small-scale forecasts have been used to support local COVID-19 responses (e.g. in Austin, TX, USA [9]), as well as to forecast non-COVID-19 or more general healthcare demands at the hospital level [13,14]. Forecasts of hospital admissions are also an essential step to forecasting bed or intensive care unit (ICU) demand (e.g. [11, 13, 14]). In theory, future admissions are a function of recent cases in the community, the proportion of cases that require and seek health care (the case hospitalisation rate (CHR)), and the delay from symptom onset to hospital admission. However, forecasting admissions from community cases is challenging as both the CHR and admission delay can vary over time. The CHR depends on testing effort and strategy (how many symptomatic and asymptomatic cases are identified), the age distribution of cases [1], and the prevalence of other COVID-19 risk factors amongst cases [12]. Retrospective studies of COVID-19 patients reported a mean delay from symptom onset to hospital admission to be 4.6 days in the UK [15] and 5.7 days in Belgium [16], but this varies by age and place of residence (e.g. care-home residents have a longer average admissions delay than non-residents) [16]. Forecasting studies have found that cases are predictive of admissions with a lag of only 4–7 days [10, 14]. Given the short estimated delay between cases and future admissions, to make short-term forecasts of admissions therefore also requires forecasts of cases. Whilst some studies consider mobility and meteorological predictors with longer lags [14], they lack a direct mechanistic relationship with admissions and may have only a limited benefit. Besides structural challenges, models are subject to constraints of data availability in real-time and at the relevant spatial scale (by hospital or Trust (a small group of hospitals) for admissions, and local authority level for cases and other predictors). Models need to be sufficiently flexible to capture a potentially wide range of epidemic behaviour across locations and time, but at the same time should produce results sufficiently rapidly to be updated in a reasonable amount of time. Autoregressive time series models are widely used in other forecasting tasks (e.g. [17, 18]), including in healthcare settings [19], and scale easily to a large number of locations; however, since forecasts are, in the simplest case, based solely on past admissions, they may not perform well when cases (and admissions) are changing quickly. Predictors can be incorporated into generalised linear models (GLMs) with uncorrelated [19] or correlated errors [14]; for lagged predictors, the lag (or lags) usually needs to be predetermined. Alternatively, admissions can be modelled as a scaled convolution of cases and a delay distribution; this method can also be used to forecast deaths from cases or admissions (e.g. [20]). The forecasting performance of both GLMs and convolution models beyond the shortest forecast horizon will be affected by the quality of the case forecasts (or any other predictors), which may vary over time or across locations. One way to attempt improving the robustness of forecasts is to combine them into an ensemble forecast, whereby predictions from several different models are combined into a single forecast. This reduces reliance on a single forecasting model and, given a minimum quality of the constituent models, the average performance of ensembles is generally comparable, if not better than, its best constituent models [8, 21]. Ensemble methods have been widely used in real-time during the COVID-19 pandemic to leverage the contributions of multiple modelling groups to a single forecasting task [8, 22, 23], as well as previously during outbreaks of influenza [18, 24], Ebola virus disease [25], dengue [26], and Zika [27]. In this paper, we make and evaluate week
PDF JSON Files: document_parses/pdf_json/864e9e976a114ba22fac622f66a3fa31f47d2d87.json
PMC JSON Files: document_parses/pmc_json/PMC8858706.xml.json
G_ID: comparative_assessment_of_methods_for_short_term_forecasts_of_covid_19_hospital_admissions