-
Notifications
You must be signed in to change notification settings - Fork 7
Add epidatr blog post #912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
da92017
Create 2023-09-27-epidatr.Rmd
carlynvandyke a488b71
Add epidatr blog images
carlynvandyke 6e8ba71
rendered html
melange396 0ed1d20
Update content/blog/2023-09-27-epidatr.Rmd
carlynvandyke e32783f
Update content/blog/2023-09-27-epidatr.Rmd
carlynvandyke e9beca1
Update content/blog/2023-09-27-epidatr.Rmd
carlynvandyke eba4da2
Update content/blog/2023-09-27-epidatr.Rmd
carlynvandyke cd534e5
Update content/blog/2023-09-27-epidatr.Rmd
carlynvandyke 599f02d
Fix title
carlynvandyke b4335fa
removed backslash, re-rendered html for epidatr blog post
melange396 5ed749a
Update content/blog/2023-09-27-epidatr.Rmd
dshemetov aeb367c
fix: R blocks
dshemetov dc414cc
blog: render epidatr blog html
dshemetov d1a6a4b
fix: epidatr html frontmatter
dshemetov 629f1da
fix: try r fold-show tech
dshemetov 81df7c6
fix: try again
dshemetov 0cb1f98
fix: just use raw unexecuted code blocks
dshemetov e8a906f
feat: adjust epidatr blog to Logan's feedback
dshemetov File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
--- | ||
title: New package released! Epidatr, an R client for Delphi Epidata API | ||
author: Dmitry Shemetov, David Weber | ||
date: 2023-09-27 | ||
tags: | ||
- r | ||
- epidata | ||
- covidcast | ||
authors: | ||
- dmitry | ||
- davidweb | ||
heroImage: blog-lg-epidatr.jpg | ||
heroImageThumb: blog-thumb-epidatr.jpg | ||
summary: | | ||
`epidatr` is designed to streamline the downloading and usage of data from the [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/). It provides a simple R interface to the API, including functions for downloading data, parsing the results, and converting the data into a tidy format. | ||
|
||
output: | ||
blogdown::html_page: | ||
toc: true | ||
--- | ||
|
||
The [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/) provides | ||
real-time access to epidemiological surveillance data for influenza, COVID-19, | ||
and other diseases from both official government sources such as the [Centers | ||
for Disease Control and Prevention | ||
(CDC)](https://www.cdc.gov/datastatistics/index.html), private partners such as | ||
[Facebook (now | ||
Meta)](https://delphi.cmu.edu/blog/2020/08/26/covid-19-symptom-surveys-through-facebook/) | ||
and [Change Healthcare](https://www.changehealthcare.com/), and other public | ||
datasets like [Google | ||
Trends](https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/covid19-search-trends). | ||
It is built and maintained by the Carnegie Mellon University [Delphi Research | ||
Group](https://delphi.cmu.edu/). | ||
|
||
Today we introduce the R package `epidatr`, available [on | ||
CRAN](https://cloud.r-project.org/web/packages/epidatr/index.html), with the | ||
source and development [on github](https://github.com/cmu-delphi/epidatr). | ||
|
||
This package is designed to streamline the downloading and usage of data from | ||
the Delphi Epidata API. It provides a simple R interface to the API, including | ||
functions for downloading data, parsing the results, and converting the data | ||
into a tidy format. The API stores a historical record of all data, including | ||
corrections and updates, which is particularly useful for accurately backtesting | ||
forecasting models. We also provide packages for downstream data processing | ||
([epiprocess](https://github.com/cmu-delphi/epiprocess)) and modeling | ||
([epipredict](https://github.com/cmu-delphi/epipredict)). | ||
|
||
## Usage | ||
|
||
``` | ||
library(epidatr) | ||
# Obtain the smoothed covid-like illness (CLI) signal from Delphi's US COVID-19 | ||
# Trends and Impact Survey (CTIS), in partnership with Facebook, as it was on | ||
# April 10, 2021 for the US at the national level | ||
epidata <- pub_covidcast( | ||
source = "fb-survey", | ||
signals = "smoothed_cli", | ||
geo_type = "nation", | ||
time_type = "day", | ||
geo_values = "us", | ||
time_values = epirange(20210101, 20210601), | ||
as_of = 20210601 | ||
) | ||
epidata | ||
``` | ||
|
||
``` | ||
# A tibble: 151 × 15 | ||
geo_value signal source geo_type time_type time_value | ||
<chr> <chr> <chr> <fct> <fct> <date> | ||
1 us smoothed_cli fb-su… nation day 2021-01-01 | ||
2 us smoothed_cli fb-su… nation day 2021-01-02 | ||
3 us smoothed_cli fb-su… nation day 2021-01-03 | ||
4 us smoothed_cli fb-su… nation day 2021-01-04 | ||
5 us smoothed_cli fb-su… nation day 2021-01-05 | ||
6 us smoothed_cli fb-su… nation day 2021-01-06 | ||
7 us smoothed_cli fb-su… nation day 2021-01-07 | ||
8 us smoothed_cli fb-su… nation day 2021-01-08 | ||
9 us smoothed_cli fb-su… nation day 2021-01-09 | ||
10 us smoothed_cli fb-su… nation day 2021-01-10 | ||
# ℹ 141 more rows | ||
# ℹ 9 more variables: direction <dbl>, issue <date>, | ||
# lag <dbl>, missing_value <dbl>, missing_stderr <dbl>, | ||
# missing_sample_size <dbl>, value <dbl>, stderr <dbl>, | ||
# sample_size <dbl> | ||
# ℹ Use `print(n = ...)` to see more rows | ||
``` | ||
|
||
## Installation | ||
|
||
Installing the package is straightforward. | ||
|
||
``` | ||
# Install the CRAN version | ||
pak::pkg_install("epidatr") | ||
|
||
# Install the development version from the GitHub dev branch | ||
pak::pkg_install("cmu-delphi/epidatr@dev") | ||
``` | ||
|
||
carlynvandyke marked this conversation as resolved.
Show resolved
Hide resolved
|
||
### API Keys | ||
|
||
The Delphi API requires a (free) API key for full functionality. To generate | ||
your key, register for a pseudo-anonymous account | ||
[here](https://api.delphi.cmu.edu/epidata/admin/registration_form) and see more | ||
discussion on the [general API | ||
website](https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html). The | ||
`epidatr` client will automatically look for this key in the environment | ||
variable `DELPHI_EPIDATA_KEY`. We recommend storing your key in your `.Renviron` | ||
file, which R will read by default. | ||
|
||
Note that for the time being, the private endpoints (i.e. those prefixed with | ||
`pvt`) will require a separate key that needs to be passed as an argument. | ||
|
||
## For users of the covidcast R package | ||
|
||
The `covidcast` package is deprecated and will no longer be updated. The | ||
`epidatr` package is a complete rewrite of the [`covidcast` | ||
package](https://cmu-delphi.github.io/covidcast/covidcastR/), with a focus on | ||
speed, reliability, and ease of use. It also supports more endpoints and data | ||
sources than `covidcast`. When migrating from that package, you will need to use | ||
the | ||
[`pub_covidcast`](https://cmu-delphi.github.io/epidatr/reference/pub_covidcast.html) | ||
function in `epidatr`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
--- | ||
title: New package released! Epidatr, an R client for Delphi Epidata API | ||
author: Dmitry Shemetov, David Weber | ||
date: 2023-09-27 | ||
tags: | ||
- r | ||
- epidata | ||
- covidcast | ||
authors: | ||
- dmitry | ||
- davidweb | ||
heroImage: blog-lg-epidatr.jpg | ||
heroImageThumb: blog-thumb-epidatr.jpg | ||
summary: | | ||
`epidatr` is designed to streamline the downloading and usage of data from the [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/). It provides a simple R interface to the API, including functions for downloading data, parsing the results, and converting the data into a tidy format. | ||
|
||
output: | ||
blogdown::html_page: | ||
toc: true | ||
--- | ||
|
||
<div id="TOC"> | ||
<ul> | ||
<li><a href="#usage">Usage</a></li> | ||
<li><a href="#installation">Installation</a><ul> | ||
<li><a href="#api-keys">API Keys</a></li> | ||
</ul></li> | ||
<li><a href="#for-users-of-the-covidcast-r-package">For users of the covidcast R package</a></li> | ||
</ul> | ||
</div> | ||
|
||
<p>The <a href="https://cmu-delphi.github.io/delphi-epidata/">Delphi Epidata API</a> provides | ||
real-time access to epidemiological surveillance data for influenza, COVID-19, | ||
and other diseases from both official government sources such as the <a href="https://www.cdc.gov/datastatistics/index.html">Centers | ||
for Disease Control and Prevention | ||
(CDC)</a>, private partners such as | ||
<a href="https://delphi.cmu.edu/blog/2020/08/26/covid-19-symptom-surveys-through-facebook/">Facebook (now | ||
Meta)</a> | ||
and <a href="https://www.changehealthcare.com/">Change Healthcare</a>, and other public | ||
datasets like <a href="https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/covid19-search-trends">Google | ||
Trends</a>. | ||
It is built and maintained by the Carnegie Mellon University <a href="https://delphi.cmu.edu/">Delphi Research | ||
Group</a>.</p> | ||
<p>Today we introduce the R package <code>epidatr</code>, available <a href="https://cloud.r-project.org/web/packages/epidatr/index.html">on | ||
CRAN</a>, with the | ||
source and development <a href="https://github.com/cmu-delphi/epidatr">on github</a>.</p> | ||
<p>This package is designed to streamline the downloading and usage of data from | ||
the Delphi Epidata API. It provides a simple R interface to the API, including | ||
functions for downloading data, parsing the results, and converting the data | ||
into a tidy format. The API stores a historical record of all data, including | ||
corrections and updates, which is particularly useful for accurately backtesting | ||
forecasting models. We also provide packages for downstream data processing | ||
(<a href="https://github.com/cmu-delphi/epiprocess">epiprocess</a>) and modeling | ||
(<a href="https://github.com/cmu-delphi/epipredict">epipredict</a>).</p> | ||
<div id="usage" class="section level2"> | ||
<h2>Usage</h2> | ||
<pre><code>library(epidatr) | ||
# Obtain the smoothed covid-like illness (CLI) signal from Delphi's US COVID-19 | ||
# Trends and Impact Survey (CTIS), in partnership with Facebook, as it was on | ||
# April 10, 2021 for the US at the national level | ||
epidata <- pub_covidcast( | ||
source = "fb-survey", | ||
signals = "smoothed_cli", | ||
geo_type = "nation", | ||
time_type = "day", | ||
geo_values = "us", | ||
time_values = epirange(20210101, 20210601), | ||
as_of = 20210601 | ||
) | ||
epidata</code></pre> | ||
<pre><code># A tibble: 151 × 15 | ||
geo_value signal source geo_type time_type time_value | ||
<chr> <chr> <chr> <fct> <fct> <date> | ||
1 us smoothed_cli fb-su… nation day 2021-01-01 | ||
2 us smoothed_cli fb-su… nation day 2021-01-02 | ||
3 us smoothed_cli fb-su… nation day 2021-01-03 | ||
4 us smoothed_cli fb-su… nation day 2021-01-04 | ||
5 us smoothed_cli fb-su… nation day 2021-01-05 | ||
6 us smoothed_cli fb-su… nation day 2021-01-06 | ||
7 us smoothed_cli fb-su… nation day 2021-01-07 | ||
8 us smoothed_cli fb-su… nation day 2021-01-08 | ||
9 us smoothed_cli fb-su… nation day 2021-01-09 | ||
10 us smoothed_cli fb-su… nation day 2021-01-10 | ||
# ℹ 141 more rows | ||
# ℹ 9 more variables: direction <dbl>, issue <date>, | ||
# lag <dbl>, missing_value <dbl>, missing_stderr <dbl>, | ||
# missing_sample_size <dbl>, value <dbl>, stderr <dbl>, | ||
# sample_size <dbl> | ||
# ℹ Use `print(n = ...)` to see more rows</code></pre> | ||
</div> | ||
<div id="installation" class="section level2"> | ||
<h2>Installation</h2> | ||
<p>Installing the package is straightforward.</p> | ||
<pre><code># Install the CRAN version | ||
pak::pkg_install("epidatr") | ||
|
||
# Install the development version from the GitHub dev branch | ||
pak::pkg_install("cmu-delphi/epidatr@dev")</code></pre> | ||
<div id="api-keys" class="section level3"> | ||
<h3>API Keys</h3> | ||
<p>The Delphi API requires a (free) API key for full functionality. To generate | ||
your key, register for a pseudo-anonymous account | ||
<a href="https://api.delphi.cmu.edu/epidata/admin/registration_form">here</a> and see more | ||
discussion on the <a href="https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html">general API | ||
website</a>. The | ||
<code>epidatr</code> client will automatically look for this key in the environment | ||
variable <code>DELPHI_EPIDATA_KEY</code>. We recommend storing your key in your <code>.Renviron</code> | ||
file, which R will read by default.</p> | ||
<p>Note that for the time being, the private endpoints (i.e. those prefixed with | ||
<code>pvt</code>) will require a separate key that needs to be passed as an argument.</p> | ||
</div> | ||
</div> | ||
<div id="for-users-of-the-covidcast-r-package" class="section level2"> | ||
<h2>For users of the covidcast R package</h2> | ||
<p>The <code>covidcast</code> package is deprecated and will no longer be updated. The | ||
<code>epidatr</code> package is a complete rewrite of the <a href="https://cmu-delphi.github.io/covidcast/covidcastR/"><code>covidcast</code> | ||
package</a>, with a focus on | ||
speed, reliability, and ease of use. It also supports more endpoints and data | ||
sources than <code>covidcast</code>. When migrating from that package, you will need to use | ||
the | ||
<a href="https://cmu-delphi.github.io/epidatr/reference/pub_covidcast.html"><code>pub_covidcast</code></a> | ||
function in <code>epidatr</code>.</p> | ||
</div> |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.