diff --git a/content/blog/2023-09-27-epidatr.Rmd b/content/blog/2023-09-27-epidatr.Rmd new file mode 100644 index 000000000..dd4dde9ca --- /dev/null +++ b/content/blog/2023-09-27-epidatr.Rmd @@ -0,0 +1,124 @@ +--- +title: New package released! Epidatr, an R client for Delphi Epidata API +author: Dmitry Shemetov, David Weber +date: 2023-09-27 +tags: + - r + - epidata + - covidcast +authors: + - dmitry + - davidweb +heroImage: blog-lg-epidatr.jpg +heroImageThumb: blog-thumb-epidatr.jpg +summary: | + `epidatr` is designed to streamline the downloading and usage of data from the [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/). It provides a simple R interface to the API, including functions for downloading data, parsing the results, and converting the data into a tidy format. + +output: + blogdown::html_page: + toc: true +--- + +The [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/) provides +real-time access to epidemiological surveillance data for influenza, COVID-19, +and other diseases from both official government sources such as the [Centers +for Disease Control and Prevention +(CDC)](https://www.cdc.gov/datastatistics/index.html), private partners such as +[Facebook (now +Meta)](https://delphi.cmu.edu/blog/2020/08/26/covid-19-symptom-surveys-through-facebook/) +and [Change Healthcare](https://www.changehealthcare.com/), and other public +datasets like [Google +Trends](https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/covid19-search-trends). +It is built and maintained by the Carnegie Mellon University [Delphi Research +Group](https://delphi.cmu.edu/). + +Today we introduce the R package `epidatr`, available [on +CRAN](https://cloud.r-project.org/web/packages/epidatr/index.html), with the +source and development [on github](https://github.com/cmu-delphi/epidatr). + +This package is designed to streamline the downloading and usage of data from +the Delphi Epidata API. It provides a simple R interface to the API, including +functions for downloading data, parsing the results, and converting the data +into a tidy format. The API stores a historical record of all data, including +corrections and updates, which is particularly useful for accurately backtesting +forecasting models. We also provide packages for downstream data processing +([epiprocess](https://github.com/cmu-delphi/epiprocess)) and modeling +([epipredict](https://github.com/cmu-delphi/epipredict)). + +## Usage + +``` +library(epidatr) +# Obtain the smoothed covid-like illness (CLI) signal from Delphi's US COVID-19 +# Trends and Impact Survey (CTIS), in partnership with Facebook, as it was on +# April 10, 2021 for the US at the national level +epidata <- pub_covidcast( + source = "fb-survey", + signals = "smoothed_cli", + geo_type = "nation", + time_type = "day", + geo_values = "us", + time_values = epirange(20210101, 20210601), + as_of = 20210601 +) +epidata +``` + +``` +# A tibble: 151 × 15 + geo_value signal source geo_type time_type time_value + + 1 us smoothed_cli fb-su… nation day 2021-01-01 + 2 us smoothed_cli fb-su… nation day 2021-01-02 + 3 us smoothed_cli fb-su… nation day 2021-01-03 + 4 us smoothed_cli fb-su… nation day 2021-01-04 + 5 us smoothed_cli fb-su… nation day 2021-01-05 + 6 us smoothed_cli fb-su… nation day 2021-01-06 + 7 us smoothed_cli fb-su… nation day 2021-01-07 + 8 us smoothed_cli fb-su… nation day 2021-01-08 + 9 us smoothed_cli fb-su… nation day 2021-01-09 +10 us smoothed_cli fb-su… nation day 2021-01-10 +# ℹ 141 more rows +# ℹ 9 more variables: direction , issue , +# lag , missing_value , missing_stderr , +# missing_sample_size , value , stderr , +# sample_size +# ℹ Use `print(n = ...)` to see more rows +``` + +## Installation + +Installing the package is straightforward. + +``` +# Install the CRAN version +pak::pkg_install("epidatr") + +# Install the development version from the GitHub dev branch +pak::pkg_install("cmu-delphi/epidatr@dev") +``` + +### API Keys + +The Delphi API requires a (free) API key for full functionality. To generate +your key, register for a pseudo-anonymous account +[here](https://api.delphi.cmu.edu/epidata/admin/registration_form) and see more +discussion on the [general API +website](https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html). The +`epidatr` client will automatically look for this key in the environment +variable `DELPHI_EPIDATA_KEY`. We recommend storing your key in your `.Renviron` +file, which R will read by default. + +Note that for the time being, the private endpoints (i.e. those prefixed with +`pvt`) will require a separate key that needs to be passed as an argument. + +## For users of the covidcast R package + +The `covidcast` package is deprecated and will no longer be updated. The +`epidatr` package is a complete rewrite of the [`covidcast` +package](https://cmu-delphi.github.io/covidcast/covidcastR/), with a focus on +speed, reliability, and ease of use. It also supports more endpoints and data +sources than `covidcast`. When migrating from that package, you will need to use +the +[`pub_covidcast`](https://cmu-delphi.github.io/epidatr/reference/pub_covidcast.html) +function in `epidatr`. diff --git a/content/blog/2023-09-27-epidatr.html b/content/blog/2023-09-27-epidatr.html new file mode 100644 index 000000000..840d21b04 --- /dev/null +++ b/content/blog/2023-09-27-epidatr.html @@ -0,0 +1,123 @@ +--- +title: New package released! Epidatr, an R client for Delphi Epidata API +author: Dmitry Shemetov, David Weber +date: 2023-09-27 +tags: + - r + - epidata + - covidcast +authors: + - dmitry + - davidweb +heroImage: blog-lg-epidatr.jpg +heroImageThumb: blog-thumb-epidatr.jpg +summary: | + `epidatr` is designed to streamline the downloading and usage of data from the [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/). It provides a simple R interface to the API, including functions for downloading data, parsing the results, and converting the data into a tidy format. + +output: + blogdown::html_page: + toc: true +--- + + + +

The Delphi Epidata API provides +real-time access to epidemiological surveillance data for influenza, COVID-19, +and other diseases from both official government sources such as the Centers +for Disease Control and Prevention +(CDC), private partners such as +Facebook (now +Meta) +and Change Healthcare, and other public +datasets like Google +Trends. +It is built and maintained by the Carnegie Mellon University Delphi Research +Group.

+

Today we introduce the R package epidatr, available on +CRAN, with the +source and development on github.

+

This package is designed to streamline the downloading and usage of data from +the Delphi Epidata API. It provides a simple R interface to the API, including +functions for downloading data, parsing the results, and converting the data +into a tidy format. The API stores a historical record of all data, including +corrections and updates, which is particularly useful for accurately backtesting +forecasting models. We also provide packages for downstream data processing +(epiprocess) and modeling +(epipredict).

+
+

Usage

+
library(epidatr)
+# Obtain the smoothed covid-like illness (CLI) signal from Delphi's US COVID-19
+# Trends and Impact Survey (CTIS), in partnership with Facebook, as it was on
+# April 10, 2021 for the US at the national level
+epidata <- pub_covidcast(
+    source = "fb-survey",
+    signals = "smoothed_cli",
+    geo_type = "nation",
+    time_type = "day",
+    geo_values = "us",
+    time_values = epirange(20210101, 20210601),
+    as_of = 20210601
+)
+epidata
+
# A tibble: 151 × 15
+   geo_value signal       source geo_type time_type time_value
+   <chr>     <chr>        <chr>  <fct>    <fct>     <date>
+ 1 us        smoothed_cli fb-su… nation   day       2021-01-01
+ 2 us        smoothed_cli fb-su… nation   day       2021-01-02
+ 3 us        smoothed_cli fb-su… nation   day       2021-01-03
+ 4 us        smoothed_cli fb-su… nation   day       2021-01-04
+ 5 us        smoothed_cli fb-su… nation   day       2021-01-05
+ 6 us        smoothed_cli fb-su… nation   day       2021-01-06
+ 7 us        smoothed_cli fb-su… nation   day       2021-01-07
+ 8 us        smoothed_cli fb-su… nation   day       2021-01-08
+ 9 us        smoothed_cli fb-su… nation   day       2021-01-09
+10 us        smoothed_cli fb-su… nation   day       2021-01-10
+# ℹ 141 more rows
+# ℹ 9 more variables: direction <dbl>, issue <date>,
+#   lag <dbl>, missing_value <dbl>, missing_stderr <dbl>,
+#   missing_sample_size <dbl>, value <dbl>, stderr <dbl>,
+#   sample_size <dbl>
+# ℹ Use `print(n = ...)` to see more rows
+
+
+

Installation

+

Installing the package is straightforward.

+
# Install the CRAN version
+pak::pkg_install("epidatr")
+
+# Install the development version from the GitHub dev branch
+pak::pkg_install("cmu-delphi/epidatr@dev")
+
+

API Keys

+

The Delphi API requires a (free) API key for full functionality. To generate +your key, register for a pseudo-anonymous account +here and see more +discussion on the general API +website. The +epidatr client will automatically look for this key in the environment +variable DELPHI_EPIDATA_KEY. We recommend storing your key in your .Renviron +file, which R will read by default.

+

Note that for the time being, the private endpoints (i.e. those prefixed with +pvt) will require a separate key that needs to be passed as an argument.

+
+
+
+

For users of the covidcast R package

+

The covidcast package is deprecated and will no longer be updated. The +epidatr package is a complete rewrite of the covidcast +package, with a focus on +speed, reliability, and ease of use. It also supports more endpoints and data +sources than covidcast. When migrating from that package, you will need to use +the +pub_covidcast +function in epidatr.

+
diff --git a/content/blog/images/blog-lg-epidatr.jpg b/content/blog/images/blog-lg-epidatr.jpg new file mode 100644 index 000000000..37ab60175 Binary files /dev/null and b/content/blog/images/blog-lg-epidatr.jpg differ diff --git a/content/blog/images/blog-thumb-epidatr.jpg b/content/blog/images/blog-thumb-epidatr.jpg new file mode 100644 index 000000000..507b0f730 Binary files /dev/null and b/content/blog/images/blog-thumb-epidatr.jpg differ