Skip to content

Commit

Permalink
Build vignettes
Browse files Browse the repository at this point in the history
  • Loading branch information
boshek committed Sep 29, 2017
1 parent 2cb07ad commit 240b730
Show file tree
Hide file tree
Showing 7 changed files with 540 additions and 1 deletion.
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@
.RData
*.Rproj

inst/doc
26 changes: 26 additions & 0 deletions inst/doc/data.handling.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## ---- echo = FALSE-------------------------------------------------------
library(knitr)

## ---- echo = FALSE-------------------------------------------------------
df2 <- data.frame(datetime = c("2008-07-01 01:00","2008-07-01 02:00","2008-07-01 03:00","2008-07-01 04:00"),
doobs_0.5= c("8.3","8.2","8.2","8.1"))
kable(df2)

## ---- echo = FALSE-------------------------------------------------------
df1 <- data.frame(datetime = c("2008-07-01 01:00","2008-07-01 02:00","2008-07-01 03:00","2008-07-01 04:00"),
wtr_0.5= c("22.3","22.31","22.31","22.32"),
wtr_1 = c("22.3","22.31","22.31","22.32"),
wtr_2 = rep(21, 4))
kable(df1)

## ---- echo = FALSE-------------------------------------------------------
df <- data.frame(Abbreviation = c("doobs","wtr","wnd","airT","rh"),
Variable = c("Dissolved Oxygen Concentration","Water Temperature","Wind Speed",
"Air Temperature","Relative Humidity"),
`Assumed Units` = c("mg/L ","°C","m/s","°C","%"))
kable(df)

## ---- eval = FALSE-------------------------------------------------------
# tmp = data.frame()
# write.table(tmp, "test.wtr", sep='\t', row.names=FALSE)

73 changes: 73 additions & 0 deletions inst/doc/data.handling.Rmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: "Introduction to file and data formats in rLakeAnalyzer"
author: "Luke Winslow"
date: "July 6, 2014"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Vignette Title}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
header-includes:
- \usepackage{gensymb}
---

```{r, echo = FALSE}
library(knitr)
```

## Introduction
This document is an introduction to handling the type of data typically used in rLakeAnalyzer. It will hopefully give the reader enough information to be able to quickly and effectively format your own data to take advantage of the more powerful features.

## File Format

We have tried to use a simple but standard file format that eases import and parsing of data while still being easy to generate and edit using commonly available tools like Microsoft Excel. Below is a very simple example of how the files are structured.

```{r, echo = FALSE}
df2 <- data.frame(datetime = c("2008-07-01 01:00","2008-07-01 02:00","2008-07-01 03:00","2008-07-01 04:00"),
doobs_0.5= c("8.3","8.2","8.2","8.1"))
kable(df2)
```

There are a few key aspects to these file structure to note. The date/time format, the format of the header, and the file naming scheme. These key points are discussed here.

## DateTime Format

The date and time of all observations is stored in a single, string column. The header of this column must be the word "datetime" without quotes. It is also case insensitive so "DateTime" and other variations will work.

The datetime format itself is exclusively in an ISO-like format (ISO-8601). It is in most-to-least significant order. It requires a "-" (dash) delimited date format and a ":" (colon) delimited time. "yyyy-mm-dd HH:MM:SS". The date must come first and is separated from the time with a single space. Seconds are optional. This format can easily be created in Excel using a custom date/time format of "yyyy-mm-dd hh:mm:ss" without quotes.

Note: This format differs from the ISO-8601 format in that a space is used to separate the date and time. This is done to support the use of Microsoft Excel as Excel does not natively recognize the ISO format.

## Header Format

The header is used to help identify both the variable type as well as the depth of observation of the data as well as distinguish the data columns from the datetime column. As mentioned above, a "datetime" column is required using the format described above.

The data columns must be identified with a variable type and optionally, a depth. For example, a water temperature collected at 1 meter depth would have the column header "wtr\_1". The usefulness of this simple format can be seen when dealing with profile data taken at many depths (see below).

```{r, echo = FALSE}
df1 <- data.frame(datetime = c("2008-07-01 01:00","2008-07-01 02:00","2008-07-01 03:00","2008-07-01 04:00"),
wtr_0.5= c("22.3","22.31","22.31","22.32"),
wtr_1 = c("22.3","22.31","22.31","22.32"),
wtr_2 = rep(21, 4))
kable(df1)
```

While any text can be used to describe a variable, the table below lists the current "standard" variables that are used by rLakeAnalyzer and other toolboxes for identifying commonly collected data in the most common units. If these standards are adhered to, many of the more helpful functions will work natively. For example, water.density expects temperature to be supplied in celsius, the default unit used for the "wtr" abbreviation.

```{r, echo = FALSE}
df <- data.frame(Abbreviation = c("doobs","wtr","wnd","airT","rh"),
Variable = c("Dissolved Oxygen Concentration","Water Temperature","Wind Speed",
"Air Temperature","Relative Humidity"),
`Assumed Units` = c("mg/L ","°C","m/s","°C","%"))
kable(df)
```


## File Format

The file format is a simple tab-delimited file. It is easy to export files of this format using Excel or even R itself. To export the appropriate format from R, use "write.table" as in the following example.

```{r, eval = FALSE}
tmp = data.frame()
write.table(tmp, "test.wtr", sep='\t', row.names=FALSE)
```
217 changes: 217 additions & 0 deletions inst/doc/data.handling.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml">

<head>

<meta charset="utf-8" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="pandoc" />

<meta name="viewport" content="width=device-width, initial-scale=1">

<meta name="author" content="Luke Winslow" />


<title>Introduction to file and data formats in rLakeAnalyzer</title>




<style type="text/css">code{white-space: pre;}</style>
<style type="text/css">
div.sourceCode { overflow-x: auto; }
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
margin: 0; padding: 0; vertical-align: baseline; border: none; }
table.sourceCode { width: 100%; line-height: 100%; }
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
td.sourceCode { padding-left: 5px; }
code > span.kw { color: #007020; font-weight: bold; } /* Keyword */
code > span.dt { color: #902000; } /* DataType */
code > span.dv { color: #40a070; } /* DecVal */
code > span.bn { color: #40a070; } /* BaseN */
code > span.fl { color: #40a070; } /* Float */
code > span.ch { color: #4070a0; } /* Char */
code > span.st { color: #4070a0; } /* String */
code > span.co { color: #60a0b0; font-style: italic; } /* Comment */
code > span.ot { color: #007020; } /* Other */
code > span.al { color: #ff0000; font-weight: bold; } /* Alert */
code > span.fu { color: #06287e; } /* Function */
code > span.er { color: #ff0000; font-weight: bold; } /* Error */
code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
code > span.cn { color: #880000; } /* Constant */
code > span.sc { color: #4070a0; } /* SpecialChar */
code > span.vs { color: #4070a0; } /* VerbatimString */
code > span.ss { color: #bb6688; } /* SpecialString */
code > span.im { } /* Import */
code > span.va { color: #19177c; } /* Variable */
code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code > span.op { color: #666666; } /* Operator */
code > span.bu { } /* BuiltIn */
code > span.ex { } /* Extension */
code > span.pp { color: #bc7a00; } /* Preprocessor */
code > span.at { color: #7d9029; } /* Attribute */
code > span.do { color: #ba2121; font-style: italic; } /* Documentation */
code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
</style>



<link href="data:text/css;charset=utf-8,body%20%7B%0Abackground%2Dcolor%3A%20%23fff%3B%0Amargin%3A%201em%20auto%3B%0Amax%2Dwidth%3A%20700px%3B%0Aoverflow%3A%20visible%3B%0Apadding%2Dleft%3A%202em%3B%0Apadding%2Dright%3A%202em%3B%0Afont%2Dfamily%3A%20%22Open%20Sans%22%2C%20%22Helvetica%20Neue%22%2C%20Helvetica%2C%20Arial%2C%20sans%2Dserif%3B%0Afont%2Dsize%3A%2014px%3B%0Aline%2Dheight%3A%201%2E35%3B%0A%7D%0A%23header%20%7B%0Atext%2Dalign%3A%20center%3B%0A%7D%0A%23TOC%20%7B%0Aclear%3A%20both%3B%0Amargin%3A%200%200%2010px%2010px%3B%0Apadding%3A%204px%3B%0Awidth%3A%20400px%3B%0Aborder%3A%201px%20solid%20%23CCCCCC%3B%0Aborder%2Dradius%3A%205px%3B%0Abackground%2Dcolor%3A%20%23f6f6f6%3B%0Afont%2Dsize%3A%2013px%3B%0Aline%2Dheight%3A%201%2E3%3B%0A%7D%0A%23TOC%20%2Etoctitle%20%7B%0Afont%2Dweight%3A%20bold%3B%0Afont%2Dsize%3A%2015px%3B%0Amargin%2Dleft%3A%205px%3B%0A%7D%0A%23TOC%20ul%20%7B%0Apadding%2Dleft%3A%2040px%3B%0Amargin%2Dleft%3A%20%2D1%2E5em%3B%0Amargin%2Dtop%3A%205px%3B%0Amargin%2Dbottom%3A%205px%3B%0A%7D%0A%23TOC%20ul%20ul%20%7B%0Amargin%2Dleft%3A%20%2D2em%3B%0A%7D%0A%23TOC%20li%20%7B%0Aline%2Dheight%3A%2016px%3B%0A%7D%0Atable%20%7B%0Amargin%3A%201em%20auto%3B%0Aborder%2Dwidth%3A%201px%3B%0Aborder%2Dcolor%3A%20%23DDDDDD%3B%0Aborder%2Dstyle%3A%20outset%3B%0Aborder%2Dcollapse%3A%20collapse%3B%0A%7D%0Atable%20th%20%7B%0Aborder%2Dwidth%3A%202px%3B%0Apadding%3A%205px%3B%0Aborder%2Dstyle%3A%20inset%3B%0A%7D%0Atable%20td%20%7B%0Aborder%2Dwidth%3A%201px%3B%0Aborder%2Dstyle%3A%20inset%3B%0Aline%2Dheight%3A%2018px%3B%0Apadding%3A%205px%205px%3B%0A%7D%0Atable%2C%20table%20th%2C%20table%20td%20%7B%0Aborder%2Dleft%2Dstyle%3A%20none%3B%0Aborder%2Dright%2Dstyle%3A%20none%3B%0A%7D%0Atable%20thead%2C%20table%20tr%2Eeven%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0A%7D%0Ap%20%7B%0Amargin%3A%200%2E5em%200%3B%0A%7D%0Ablockquote%20%7B%0Abackground%2Dcolor%3A%20%23f6f6f6%3B%0Apadding%3A%200%2E25em%200%2E75em%3B%0A%7D%0Ahr%20%7B%0Aborder%2Dstyle%3A%20solid%3B%0Aborder%3A%20none%3B%0Aborder%2Dtop%3A%201px%20solid%20%23777%3B%0Amargin%3A%2028px%200%3B%0A%7D%0Adl%20%7B%0Amargin%2Dleft%3A%200%3B%0A%7D%0Adl%20dd%20%7B%0Amargin%2Dbottom%3A%2013px%3B%0Amargin%2Dleft%3A%2013px%3B%0A%7D%0Adl%20dt%20%7B%0Afont%2Dweight%3A%20bold%3B%0A%7D%0Aul%20%7B%0Amargin%2Dtop%3A%200%3B%0A%7D%0Aul%20li%20%7B%0Alist%2Dstyle%3A%20circle%20outside%3B%0A%7D%0Aul%20ul%20%7B%0Amargin%2Dbottom%3A%200%3B%0A%7D%0Apre%2C%20code%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0Aborder%2Dradius%3A%203px%3B%0Acolor%3A%20%23333%3B%0Awhite%2Dspace%3A%20pre%2Dwrap%3B%20%0A%7D%0Apre%20%7B%0Aborder%2Dradius%3A%203px%3B%0Amargin%3A%205px%200px%2010px%200px%3B%0Apadding%3A%2010px%3B%0A%7D%0Apre%3Anot%28%5Bclass%5D%29%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0A%7D%0Acode%20%7B%0Afont%2Dfamily%3A%20Consolas%2C%20Monaco%2C%20%27Courier%20New%27%2C%20monospace%3B%0Afont%2Dsize%3A%2085%25%3B%0A%7D%0Ap%20%3E%20code%2C%20li%20%3E%20code%20%7B%0Apadding%3A%202px%200px%3B%0A%7D%0Adiv%2Efigure%20%7B%0Atext%2Dalign%3A%20center%3B%0A%7D%0Aimg%20%7B%0Abackground%2Dcolor%3A%20%23FFFFFF%3B%0Apadding%3A%202px%3B%0Aborder%3A%201px%20solid%20%23DDDDDD%3B%0Aborder%2Dradius%3A%203px%3B%0Aborder%3A%201px%20solid%20%23CCCCCC%3B%0Amargin%3A%200%205px%3B%0A%7D%0Ah1%20%7B%0Amargin%2Dtop%3A%200%3B%0Afont%2Dsize%3A%2035px%3B%0Aline%2Dheight%3A%2040px%3B%0A%7D%0Ah2%20%7B%0Aborder%2Dbottom%3A%204px%20solid%20%23f7f7f7%3B%0Apadding%2Dtop%3A%2010px%3B%0Apadding%2Dbottom%3A%202px%3B%0Afont%2Dsize%3A%20145%25%3B%0A%7D%0Ah3%20%7B%0Aborder%2Dbottom%3A%202px%20solid%20%23f7f7f7%3B%0Apadding%2Dtop%3A%2010px%3B%0Afont%2Dsize%3A%20120%25%3B%0A%7D%0Ah4%20%7B%0Aborder%2Dbottom%3A%201px%20solid%20%23f7f7f7%3B%0Amargin%2Dleft%3A%208px%3B%0Afont%2Dsize%3A%20105%25%3B%0A%7D%0Ah5%2C%20h6%20%7B%0Aborder%2Dbottom%3A%201px%20solid%20%23ccc%3B%0Afont%2Dsize%3A%20105%25%3B%0A%7D%0Aa%20%7B%0Acolor%3A%20%230033dd%3B%0Atext%2Ddecoration%3A%20none%3B%0A%7D%0Aa%3Ahover%20%7B%0Acolor%3A%20%236666ff%3B%20%7D%0Aa%3Avisited%20%7B%0Acolor%3A%20%23800080%3B%20%7D%0Aa%3Avisited%3Ahover%20%7B%0Acolor%3A%20%23BB00BB%3B%20%7D%0Aa%5Bhref%5E%3D%22http%3A%22%5D%20%7B%0Atext%2Ddecoration%3A%20underline%3B%20%7D%0Aa%5Bhref%5E%3D%22https%3A%22%5D%20%7B%0Atext%2Ddecoration%3A%20underline%3B%20%7D%0A%0Acode%20%3E%20span%2Ekw%20%7B%20color%3A%20%23555%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%0Acode%20%3E%20span%2Edt%20%7B%20color%3A%20%23902000%3B%20%7D%20%0Acode%20%3E%20span%2Edv%20%7B%20color%3A%20%2340a070%3B%20%7D%20%0Acode%20%3E%20span%2Ebn%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Efl%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Ech%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Est%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Eco%20%7B%20color%3A%20%23888888%3B%20font%2Dstyle%3A%20italic%3B%20%7D%20%0Acode%20%3E%20span%2Eot%20%7B%20color%3A%20%23007020%3B%20%7D%20%0Acode%20%3E%20span%2Eal%20%7B%20color%3A%20%23ff0000%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%0Acode%20%3E%20span%2Efu%20%7B%20color%3A%20%23900%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%20code%20%3E%20span%2Eer%20%7B%20color%3A%20%23a61717%3B%20background%2Dcolor%3A%20%23e3d2d2%3B%20%7D%20%0A" rel="stylesheet" type="text/css" />

</head>

<body>




<h1 class="title toc-ignore">Introduction to file and data formats in rLakeAnalyzer</h1>
<h4 class="author"><em>Luke Winslow</em></h4>
<h4 class="date"><em>July 6, 2014</em></h4>



<pre><code>## Warning: package 'knitr' was built under R version 3.4.1</code></pre>
<div id="introduction" class="section level2">
<h2>Introduction</h2>
<p>This document is an introduction to handling the type of data typically used in rLakeAnalyzer. It will hopefully give the reader enough information to be able to quickly and effectively format your own data to take advantage of the more powerful features.</p>
</div>
<div id="file-format" class="section level2">
<h2>File Format</h2>
<p>We have tried to use a simple but standard file format that eases import and parsing of data while still being easy to generate and edit using commonly available tools like Microsoft Excel. Below is a very simple example of how the files are structured.</p>
<table>
<thead>
<tr class="header">
<th align="left">datetime</th>
<th align="left">doobs_0.5</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">2008-07-01 01:00</td>
<td align="left">8.3</td>
</tr>
<tr class="even">
<td align="left">2008-07-01 02:00</td>
<td align="left">8.2</td>
</tr>
<tr class="odd">
<td align="left">2008-07-01 03:00</td>
<td align="left">8.2</td>
</tr>
<tr class="even">
<td align="left">2008-07-01 04:00</td>
<td align="left">8.1</td>
</tr>
</tbody>
</table>
<p>There are a few key aspects to these file structure to note. The date/time format, the format of the header, and the file naming scheme. These key points are discussed here.</p>
</div>
<div id="datetime-format" class="section level2">
<h2>DateTime Format</h2>
<p>The date and time of all observations is stored in a single, string column. The header of this column must be the word “datetime” without quotes. It is also case insensitive so “DateTime” and other variations will work.</p>
<p>The datetime format itself is exclusively in an ISO-like format (ISO-8601). It is in most-to-least significant order. It requires a “-” (dash) delimited date format and a “:” (colon) delimited time. “yyyy-mm-dd HH:MM:SS”. The date must come first and is separated from the time with a single space. Seconds are optional. This format can easily be created in Excel using a custom date/time format of “yyyy-mm-dd hh:mm:ss” without quotes.</p>
<p>Note: This format differs from the ISO-8601 format in that a space is used to separate the date and time. This is done to support the use of Microsoft Excel as Excel does not natively recognize the ISO format.</p>
</div>
<div id="header-format" class="section level2">
<h2>Header Format</h2>
<p>The header is used to help identify both the variable type as well as the depth of observation of the data as well as distinguish the data columns from the datetime column. As mentioned above, a “datetime” column is required using the format described above.</p>
<p>The data columns must be identified with a variable type and optionally, a depth. For example, a water temperature collected at 1 meter depth would have the column header “wtr_1”. The usefulness of this simple format can be seen when dealing with profile data taken at many depths (see below).</p>
<table>
<thead>
<tr class="header">
<th align="left">datetime</th>
<th align="left">wtr_0.5</th>
<th align="left">wtr_1</th>
<th align="right">wtr_2</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">2008-07-01 01:00</td>
<td align="left">22.3</td>
<td align="left">22.3</td>
<td align="right">21</td>
</tr>
<tr class="even">
<td align="left">2008-07-01 02:00</td>
<td align="left">22.31</td>
<td align="left">22.31</td>
<td align="right">21</td>
</tr>
<tr class="odd">
<td align="left">2008-07-01 03:00</td>
<td align="left">22.31</td>
<td align="left">22.31</td>
<td align="right">21</td>
</tr>
<tr class="even">
<td align="left">2008-07-01 04:00</td>
<td align="left">22.32</td>
<td align="left">22.32</td>
<td align="right">21</td>
</tr>
</tbody>
</table>
<p>While any text can be used to describe a variable, the table below lists the current “standard” variables that are used by rLakeAnalyzer and other toolboxes for identifying commonly collected data in the most common units. If these standards are adhered to, many of the more helpful functions will work natively. For example, water.density expects temperature to be supplied in celsius, the default unit used for the “wtr” abbreviation.</p>
<table>
<thead>
<tr class="header">
<th align="left">Abbreviation</th>
<th align="left">Variable</th>
<th align="left">Assumed.Units</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">doobs</td>
<td align="left">Dissolved Oxygen Concentration</td>
<td align="left">mg/L</td>
</tr>
<tr class="even">
<td align="left">wtr</td>
<td align="left">Water Temperature</td>
<td align="left">°C</td>
</tr>
<tr class="odd">
<td align="left">wnd</td>
<td align="left">Wind Speed</td>
<td align="left">m/s</td>
</tr>
<tr class="even">
<td align="left">airT</td>
<td align="left">Air Temperature</td>
<td align="left">°C</td>
</tr>
<tr class="odd">
<td align="left">rh</td>
<td align="left">Relative Humidity</td>
<td align="left">%</td>
</tr>
</tbody>
</table>
</div>
<div id="file-format-1" class="section level2">
<h2>File Format</h2>
<p>The file format is a simple tab-delimited file. It is easy to export files of this format using Excel or even R itself. To export the appropriate format from R, use “write.table” as in the following example.</p>
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">tmp =<span class="st"> </span><span class="kw">data.frame</span>()
<span class="kw">write.table</span>(tmp, <span class="st">&quot;test.wtr&quot;</span>, <span class="dt">sep=</span><span class="st">'</span><span class="ch">\t</span><span class="st">'</span>, <span class="dt">row.names=</span><span class="ot">FALSE</span>)</code></pre></div>
</div>



<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
(function () {
var script = document.createElement("script");
script.type = "text/javascript";
script.src = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
document.getElementsByTagName("head")[0].appendChild(script);
})();
</script>

</body>
</html>
Loading

0 comments on commit 240b730

Please sign in to comment.