README.Rmd

---
output: github_document
editor_options: 
  markdown: 
    wrap: 72
---

<!-- README.md is generated from README.Rmd. Please edit that file -->

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)
```

# EIAapi

<!-- badges: start -->
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/EIAapi)](https://cran.r-project.org/package=EIAapi)
[![lifecycle](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[![License:
MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/license/mit/)
<!-- badges: end -->


The **EIAapi** package provides functions to query and pull tidy data from [EIA API
v2](https://www.eia.gov/opendata/).

Introduction to the EIAapi package available [here](https://ramikrispin.github.io/EIAapi/articles/intro.html).

## Requirments

To pull data from the API using this package, you will need the following:

-   jq - The package uses [jq](https://stedolan.github.io/jq/) to parse the API output from JSON to tabular format. To download and install jq follow the
    instructions on the [download
    page](https://stedolan.github.io/jq/download/).
-   API key - To query the EIA API, you will need to register to the
    service to receive the API key.

To register to to the API go to https://www.eia.gov/opendata/ and click the `Register` button, and follow the instruction. 

## Installation

Install the stable version from [CRAN]:
``` r
install.packages("EIAapi")
```

Or, install the development version from [Github](https://github.com/RamiKrispin/EIAapi):

``` r
# install.packages("devtools")
devtools::install_github("RamiKrispin/EIAapi")
```

## Query data

A suggested workflow to query data from the EIA API with the `eia_get`
function:

-   Go to the EIA API Dashboard
    [website](https://www.eia.gov/opendata/browser)
-   Select the API Route and define filters
-   Submit the query and extract the query information from the query
    metadata:
    -   API URL
    -   Header

[![](man/figures/EIA_API_browser.png)](https://www.eia.gov/opendata/browser/)

In the example above: 

- The API URL: `https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/`, and
- The query header: 
```JSON
{
    "frequency": "hourly",
    "data": [
        "value"
    ],
    "facets": {},
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.3"
}
```

Using the URL and header information, we can submit the GET request with the `eia_get` function:

```{r}
library(EIAapi)

# Pulling the API key from my renviron file
api_key <- Sys.getenv("eia_key")

df1 <- eia_get(
  api_key = api_key,
  api_path = "electricity/rto/fuel-type-data/data/",
  data = "value"
)

nrow(df1)

head(df1)
```

**Note:** The `api_path` argument defines by the query of the path that following the endpoint of the API - `https://api.eia.gov/v2/`. In the example above the API full URL is:

``` HTML
https://api.eia.gov/v2/electricity/rto/fuel-type-data/data/
```

Therefore, the query's path is set to `electricity/rto/fuel-type-data/data/`.


The `eia_get` function leverages the [jq](https://stedolan.github.io/jq/) tool to parse the return JSON object from the API into CSV format and the [data.table](https://CRAN.R-project.org/package=data.table) package to read and parse the object into R. By default, the function returns a `data.frame` object, but you can use the `format` argument and set the output object as `data.table`:

```{r}
df2 <- eia_get(
  api_key = api_key,
  api_path = "electricity/rto/fuel-type-data/data/",
  data = "value",
  format = "data.table"
)

df2
```

If you wish to pull more than the `length` upper limit, you can use the `offset` to offset the query by limit and pull the next observations:


```{r}
df3 <- eia_get(
  api_key = api_key,
  api_path = "electricity/rto/fuel-type-data/data/",
  data = "value",
  length = 5000,
  offset = 5000,
  format = "data.table"
)

df3
```

You can narrow down your pull by using the `facets` argument and applying some filters. For example, in the example above, let's filter data by the `fuletype` field and select energy source as `Natural gas (NG)` and the region as `United States Lower 48 (US48)`, and then extract the header:

``` JSON
{
    "frequency": "hourly",
    "data": [
        "value"
    ],
    "facets": {
        "respondent": [
            "US48"
        ],
        "fueltype": [
            "NG"
        ]
    },
    "start": null,
    "end": null,
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.3"
}
```

Updating the query with the `facets` information:

```{r}
facets <- list(respondent = "US48", fueltype = "NG")

df4 <- eia_get(
  api_key = api_key,
  api_path = "electricity/rto/fuel-type-data/data/",
  data = "value",
  length = 5000,
  format = "data.table",
  facets = facets
)

df4

unique(df4$fueltype)
unique(df4$respondent)
```


Last but not least, you can set the starting and ending time of the query. For example, let's set a window between June 1st and October 1st, 2022:


```{r}
df5 <- eia_get(
  api_key = api_key,
  api_path = "electricity/rto/fuel-type-data/data/",
  data = "value",
  length = 5000,
  format = "data.table",
  facets = facets,
  start = "2022-06-01T00",
  end = "2022-10-01T00"
)

df5

df5$time <- as.POSIXct(paste(substr(df5$period, start = 1, stop = 10)," ", 
                       substr(df5$period, start = 12, stop = 13), ":00:00", 
                       sep = ""))

plot(x = df5$time, y = df5$value, 
     main = "United States Lower 48 Hourly Electricity Generation by Natural Gas",
     col.main = "#457b9d",
     col = "#073b4c",
     sub = "Source: Form EIA-930 Product: Hourly Electric Grid Monitor",
     xlab = "",
     ylab = "Megawatt Hours",
     cex.main=1, 
     cex.lab=1, 
     cex.sub=0.8,
     frame=FALSE,
     type = "l")
```



## API Resources

* EIA API documentation: https://www.eia.gov/opendata/documentation.php
* EIA APIv2 Webinar: https://www.youtube.com/watch?v=1VsSp1XG-Pg&t=1671s&ab_channel=EIAgov