diff --git a/docs/learn.md b/docs/learn.md index bb55a38..d7930f6 100644 --- a/docs/learn.md +++ b/docs/learn.md @@ -69,8 +69,7 @@ Create a file, e.g. `~/.tailpipe/config/aws.tpc`, with a `connection` and `parti > ```hcl > partition "aws_cloudtrail_log" "flaws" { > source "file" { -> paths = ["/Users/dboeke/flaws/flaws_cloudtrail_logs"] -> file_layout = "%{DATA}.json.gz" + paths = ["/Users/dboeke/flaws/flaws_cloudtrail_logs"] > } >} >``` @@ -84,21 +83,21 @@ tailpipe collect aws_cloudtrail_log ``` -Tailpipe will download the files from the source, decompress and parse them, and add the data to the Tailpipe database in the [standard hive file structure](/docs/reference/glossary#hive). +Tailpipe will download the files from the source, decompress and parse them, and add the data to the Tailpipe database in the [standard hive file structure](/docs/manage/hive). ![](/learn/collection.png) -To see the table that was created: - -```bash -$ tailpipe table list -NAME PLUGIN LOCAL SIZE FILES ROWS -aws_cloudtrail_log hub.tailpipe.io/plugins/turbot/aws@latest 42 MB 2 160,581 -``` ## Query your logs -Tailpipe provides an interactive SQL shell for analyzing your collected data. Run `tailpipe query` to start the query shell. +Tailpipe provides an interactive SQL shell for analyzing your collected data. Run `tailpipe query` to start the query shell. To see the table that was created: + +```bash +$ tailpipe query +> .inspect +Table Plugin +aws_cloudtrail_log aws@0.1.0 +``` You can count the records in the table: diff --git a/docs/manage/collection.md b/docs/manage/collection.md index a69b07f..ba414e3 100644 --- a/docs/manage/collection.md +++ b/docs/manage/collection.md @@ -26,7 +26,7 @@ Collect a specific partition. tailpipe collect aws_cloudtrail_log.dev ``` -See [/docs/reference/cli/collect](/docs/reference/cli/collect) for more examples. +See [collect](/docs/reference/cli/collect) for more examples. The collection process always writes to a local [workspace](/docs/manage/workspace), and does so on a per-partition basis. While you may specify multiple partitions on the command line, `partition` is the unit of collection. @@ -51,7 +51,7 @@ tailpipe collect aws_cloudtrail_log.test --from T-180d tailpipe collect aws_cloudtrail_log.test --from 2024-01-01 ``` -- Subsequent collection runs occur chronologically resuming from the last collection by default, so there are no time gaps while the data is being collected. +Subsequent collection runs occur chronologically resuming from the last collection by default, so there are no time gaps while the data is being collected.