Skip to content

Commit

Permalink
Add docusaurus tool not pipeline logic (airbytehq#11716)
Browse files Browse the repository at this point in the history
* Adds docusaurs tooling

tooling works
broken links still exist

needs the pipeline to work with GH pages
merging sooner means that we won't have forked problems from
the relatively frequently updated docs folder

* Updates readme

* Update docusaurus/README.md

Co-authored-by: Davin Chia <[email protected]>

* Update docusaurus/README.md

Co-authored-by: Davin Chia <[email protected]>

* Update docusaurus/README.md

Co-authored-by: Davin Chia <[email protected]>

* Update docusaurus/README.md

Co-authored-by: Davin Chia <[email protected]>

* Update docusaurus/docusaurus.config.js

Co-authored-by: Davin Chia <[email protected]>

Co-authored-by: Davin Chia <[email protected]>
  • Loading branch information
supertopher and davinchia authored Apr 6, 2022
1 parent 5739ee8 commit 17407da
Show file tree
Hide file tree
Showing 35 changed files with 31,004 additions and 44 deletions.
8 changes: 1 addition & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,9 @@

[![GitHub stars](https://img.shields.io/github/stars/airbytehq/airbyte?style=social&label=Star&maxAge=2592000)](https://GitHub.com/airbytehq/airbyte/stargazers/) [![GitHub Workflow Status](https://img.shields.io/github/workflow/status/airbytehq/airbyte/Airbyte%20CI)](https://github.com/airbytehq/airbyte/actions/workflows/gradle.yml) [![License](https://img.shields.io/static/v1?label=license&message=MIT&color=brightgreen)](https://github.com/airbytehq/airbyte/tree/a9b1c6c0420550ad5069aca66c295223e0d05e27/LICENSE/README.md) [![License](https://img.shields.io/static/v1?label=license&message=ELv2&color=brightgreen)](https://github.com/airbytehq/airbyte/tree/a9b1c6c0420550ad5069aca66c295223e0d05e27/LICENSE/README.md)

![](docs/.gitbook/assets/airbyte_new_logo.svg)

**Data integration made simple, secure and extensible.**
**Data integration made simple, secure and extensible.**
The new open-source standard to sync data from applications, APIs & databases to warehouses, lakes & other destinations.

[![](docs/.gitbook/assets/deploy_locally.svg)](docs/deploying-airbyte/local-deployment.md) [![](docs/.gitbook/assets/deploy_on_aws.svg)](docs/deploying-airbyte/on-aws-ec2.md) [![](docs/.gitbook/assets/deploy_on_gcp.svg)](docs/deploying-airbyte/on-gcp-compute-engine.md) [![](docs/.gitbook/assets/deploy_on_k8s.svg)](docs/deploying-airbyte/on-kubernetes.md)

![](docs/.gitbook/assets/airbyte-ui-for-your-integration-pipelines.png)

Airbyte is on a mission to make data integration pipelines a commodity.

* **Maintenance-free connectors you can use in minutes**. Just authenticate your sources and warehouse, and get connectors that adapt to schema and API changes for you.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -390,10 +390,10 @@
type: "object"
title: "Loading Method"
description: "Loading method used to send select the way data will be uploaded\
\ to BigQuery. <br><b>Standard Inserts</b> - Direct uploading using SQL\
\ to BigQuery. <br/><b>Standard Inserts</b> - Direct uploading using SQL\
\ INSERT statements. This method is extremely inefficient and provided\
\ only for quick testing. In almost all cases, you should use staging.\
\ <br><b>GCS Staging</b> - Writes large batches of records to a file,\
\ <br/><b>GCS Staging</b> - Writes large batches of records to a file,\
\ uploads the file to GCS, then uses <b>COPY INTO table</b> to upload\
\ the file. Recommended for most workloads for better speed and scalability.\
\ Read more about GCS Staging <a href=\"https://docs.airbyte.com/integrations/destinations/bigquery#gcs-staging\"\
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@
"loading_method": {
"type": "object",
"title": "Loading Method *",
"description": "Loading method used to send select the way data will be uploaded to BigQuery. <br><b>Standard Inserts</b> - Direct uploading using SQL INSERT statements. This method is extremely inefficient and provided only for quick testing. In almost all cases, you should use staging. <br><b>GCS Staging</b> - Writes large batches of records to a file, uploads the file to GCS, then uses <b>COPY INTO table</b> to upload the file. Recommended for most workloads for better speed and scalability. Read more about GCS Staging <a href=\"https://docs.airbyte.com/integrations/destinations/bigquery#gcs-staging\">here</a>.",
"description": "Loading method used to send select the way data will be uploaded to BigQuery. <br/><b>Standard Inserts</b> - Direct uploading using SQL INSERT statements. This method is extremely inefficient and provided only for quick testing. In almost all cases, you should use staging. <br/><b>GCS Staging</b> - Writes large batches of records to a file, uploads the file to GCS, then uses <b>COPY INTO table</b> to upload the file. Recommended for most workloads for better speed and scalability. Read more about GCS Staging <a href=\"https://docs.airbyte.com/integrations/destinations/bigquery#gcs-staging\">here</a>.",
"oneOf": [
{
"title": "Standard Inserts",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@
"loading_method": {
"type": "object",
"title": "Loading Method",
"description": "Loading method used to send select the way data will be uploaded to BigQuery. <br><b>Standard Inserts</b> - Direct uploading using SQL INSERT statements. This method is extremely inefficient and provided only for quick testing. In almost all cases, you should use staging. <br><b>GCS Staging</b> - Writes large batches of records to a file, uploads the file to GCS, then uses <b>COPY INTO table</b> to upload the file. Recommended for most workloads for better speed and scalability. Read more about GCS Staging <a href=\"https://docs.airbyte.com/integrations/destinations/bigquery#gcs-staging\">here</a>.",
"description": "Loading method used to send select the way data will be uploaded to BigQuery. <br/><b>Standard Inserts</b> - Direct uploading using SQL INSERT statements. This method is extremely inefficient and provided only for quick testing. In almost all cases, you should use staging. <br/><b>GCS Staging</b> - Writes large batches of records to a file, uploads the file to GCS, then uses <b>COPY INTO table</b> to upload the file. Recommended for most workloads for better speed and scalability. Read more about GCS Staging <a href=\"https://docs.airbyte.com/integrations/destinations/bigquery#gcs-staging\">here</a>.",
"oneOf": [
{
"title": "Standard Inserts",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24718,7 +24718,7 @@
"type": "array"
},
"external_account": {
"description": "A card or bank account to attach to the account for receiving [payouts](https://stripe.com/docs/connect/bank-debit-card-payouts) (you won’t be able to use it for top-ups). You can provide either a token, like the ones returned by [Stripe.js](https://stripe.com/docs/stripe.js), or a dictionary, as documented in the `external_account` parameter for [bank account](https://stripe.com/docs/api#account_create_bank_account) creation. <br><br>By default, providing an external account sets it as the new default external account for its currency, and deletes the old default if one exists. To add additional external accounts without replacing the existing default for the currency, use the bank account or card creation API.",
"description": "A card or bank account to attach to the account for receiving [payouts](https://stripe.com/docs/connect/bank-debit-card-payouts) (you won’t be able to use it for top-ups). You can provide either a token, like the ones returned by [Stripe.js](https://stripe.com/docs/stripe.js), or a dictionary, as documented in the `external_account` parameter for [bank account](https://stripe.com/docs/api#account_create_bank_account) creation. <br/><br/>By default, providing an external account sets it as the new default external account for its currency, and deletes the old default if one exists. To add additional external accounts without replacing the existing default for the currency, use the bank account or card creation API.",
"maxLength": 5000,
"type": "string",
"x-stripeBypassValidation": true
Expand Down Expand Up @@ -29277,7 +29277,7 @@
"type": "array"
},
"external_account": {
"description": "A card or bank account to attach to the account for receiving [payouts](https://stripe.com/docs/connect/bank-debit-card-payouts) (you won’t be able to use it for top-ups). You can provide either a token, like the ones returned by [Stripe.js](https://stripe.com/docs/stripe.js), or a dictionary, as documented in the `external_account` parameter for [bank account](https://stripe.com/docs/api#account_create_bank_account) creation. <br><br>By default, providing an external account sets it as the new default external account for its currency, and deletes the old default if one exists. To add additional external accounts without replacing the existing default for the currency, use the bank account or card creation API.",
"description": "A card or bank account to attach to the account for receiving [payouts](https://stripe.com/docs/connect/bank-debit-card-payouts) (you won’t be able to use it for top-ups). You can provide either a token, like the ones returned by [Stripe.js](https://stripe.com/docs/stripe.js), or a dictionary, as documented in the `external_account` parameter for [bank account](https://stripe.com/docs/api#account_create_bank_account) creation. <br/><br/>By default, providing an external account sets it as the new default external account for its currency, and deletes the old default if one exists. To add additional external accounts without replacing the existing default for the currency, use the bank account or card creation API.",
"maxLength": 5000,
"type": "string",
"x-stripeBypassValidation": true
Expand Down Expand Up @@ -30499,7 +30499,7 @@
"type": "array"
},
"external_account": {
"description": "A card or bank account to attach to the account for receiving [payouts](https://stripe.com/docs/connect/bank-debit-card-payouts) (you won’t be able to use it for top-ups). You can provide either a token, like the ones returned by [Stripe.js](https://stripe.com/docs/stripe.js), or a dictionary, as documented in the `external_account` parameter for [bank account](https://stripe.com/docs/api#account_create_bank_account) creation. <br><br>By default, providing an external account sets it as the new default external account for its currency, and deletes the old default if one exists. To add additional external accounts without replacing the existing default for the currency, use the bank account or card creation API.",
"description": "A card or bank account to attach to the account for receiving [payouts](https://stripe.com/docs/connect/bank-debit-card-payouts) (you won’t be able to use it for top-ups). You can provide either a token, like the ones returned by [Stripe.js](https://stripe.com/docs/stripe.js), or a dictionary, as documented in the `external_account` parameter for [bank account](https://stripe.com/docs/api#account_create_bank_account) creation. <br/><br/>By default, providing an external account sets it as the new default external account for its currency, and deletes the old default if one exists. To add additional external accounts without replacing the existing default for the currency, use the bank account or card creation API.",
"maxLength": 5000,
"type": "string",
"x-stripeBypassValidation": true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"type": "Comment",
"author_id": 10,
"body": "test_comment",
"html_body": '<div class="zd-comment" dir="auto">test_comment<br></div>',
"html_body": '<div class="zd-comment" dir="auto">test_comment<br/></div>',
"plain_body": "test_comment",
"public": True,
"attachments": [],
Expand Down
16 changes: 8 additions & 8 deletions docs/cloud/getting-started-with-airbyte-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ This page guides you through setting up your Airbyte Cloud account, setting up a
To use Airbyte Cloud:

1. If you haven't already, [sign up for Airbyte Cloud](https://cloud.airbyte.io/signup?utm_campaign=22Q1_AirbyteCloudSignUpCampaign_Trial&utm_source=Docs&utm_content=SetupGuide).
<br> *Note: Airbyte Cloud offers a 14-day free trial with $1000 worth of [credits](https://docs.airbyte.com/getting-started-with-airbyte-cloud/core-concepts#credits), whichever expires first. For more information, see [Pricing](https://airbyte.com/pricing).*
<br/> *Note: Airbyte Cloud offers a 14-day free trial with $1000 worth of [credits](https://docs.airbyte.com/getting-started-with-airbyte-cloud/core-concepts#credits), whichever expires first. For more information, see [Pricing](https://airbyte.com/pricing).*

2. Airbyte will send you an email with a verification link. On clicking the link, you'll be taken to your new workspace.
<br>*A workspace lets you collaborate with team members and share resources across your team under a shared billing account.*
<br/>*A workspace lets you collaborate with team members and share resources across your team under a shared billing account.*

You will be greeted with an onboarding tutorial to help you set up your first connection. If you haven’t set up a connection on Airbyte Cloud before, we highly recommend following the tutorial. If you are familiar with the connection setup process, click **Skip onboarding** and follow this guide to set up your next connection.

Expand All @@ -25,7 +25,7 @@ To set up a source:
3. On the Set up the source page, enter a name for your source.
4. From the Source type dropdown, select the source you want to set up.
5. The fields relevant to your source are displayed.
<br> Click **Setup Guide** for help with filling in the fields for your selected source.
<br/> Click **Setup Guide** for help with filling in the fields for your selected source.
6. Click **Set up source**.

## Set up a destination
Expand All @@ -39,7 +39,7 @@ To set up a destination:
3. On the Set up the destination page, enter a name for your destination.
4. From the Destination type dropdown, select the destination you want to set up.
5. The fields relevant to your destination are displayed.
<br> Click **Setup Guide** for help with filling in the fields for your selected destination.
<br/> Click **Setup Guide** for help with filling in the fields for your selected destination.
6. Click **Set up destination**.

## Set up a connection
Expand Down Expand Up @@ -100,9 +100,9 @@ To set up a connection:
4. Select a destination:
- To use an existing destination, select your desired destination from the Destination dropdown. Click **Use existing destination**.
- To set up a new destination, enter a name for the new destination and select the destination from the Destination type dropdown. The fields relevant to your destination are displayed. Click **Setup Guide** for help with filling in the fields for your selected source. Click **Set up destination**.
<br> The Set up the connection page is displayed.
<br/> The Set up the connection page is displayed.
5. From the **Replication frequency** dropdown, select how often you want the data to sync from the source to the destination.
<br>_Note: The default replication frequency is 24 hours._
<br/>_Note: The default replication frequency is 24 hours._
6. From the **Destination Namespace** dropdown, select the format in which you want the data to stored in the destination:

<table>
Expand Down Expand Up @@ -156,9 +156,9 @@ To set up a connection:

* Select **Overwrite** to erase the old data and replace it completely
* Select **Append** to capture changes to your table
<br> *Note: This creates duplicate records*
<br/> *Note: This creates duplicate records*
* Select **Deduped + History** to mirror your source while keeping records unique
<br> *Note: Some sync modes may not yet be available for your source or destination*
<br/> *Note: Some sync modes may not yet be available for your source or destination*

4. **Cursor field**: Used in incremental sync mode to determine which records to sync. Airbyte pre-selects the Cursor field for you (example: updated date). If you have multiple cursor fields, select the one you want.
5. **Primary key**: Used in Deduped and History modes to determine the unique identifier.
Expand Down
4 changes: 2 additions & 2 deletions docs/cloud/managing-airbyte-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ To add a user to your workspace:
2. In the Workspace Settings sidebar, click **Access Management**.
3. In the top right-hand corner, click **+ New User**.
4. On the Add new users window, enter the email address of the user you want to invite to your workspace. Click **Send invitation**.
<br> *Note: The user will have access to only the workspace you invited them to. Also note that they will be added as a workspace admin by default.*
<br/> *Note: The user will have access to only the workspace you invited them to. Also note that they will be added as a workspace admin by default.*

### Switch between multiple workspaces

Expand Down Expand Up @@ -64,4 +64,4 @@ Understanding the following limitations will help you better manage Airbyte Clou
* Max number of streams that can be configured to sync in a single connection: 1K
* Size of a single record: 100MB
* Shortest sync schedule: Every 60 min
* Schedule accuracy: +/- 30 min
* Schedule accuracy: +/- 30 min
6 changes: 3 additions & 3 deletions docs/connector-development/ux-handbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,9 @@ Without repeating too many details mentioned elsewhere, the important thing to k

| **Persona** | **Level of technical knowledge** |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Data Analyst | <p>Proficient with:<br><br>Data manipulation tools like Excel or SQL<br>Dashboard tools like Looker<br><br>Not very familiar with reading API docs and doesn't know what a curl request is. But might be able to generate an API key if you tell them exactly how.</p> |
| Analytics Engineer | <p>Proficient with:<br><br>SQL &#x26; DBT<br>Git<br>A scripting language like Python<br>Shallow familiarity with infra tools like Docker<br><br>Much more technical than a data analyst, but not as much as a data engineer</p> |
| Data Engineer | <p>Proficient with:<br><br>SQL &#x26; DBT<br>Git<br>2 or more programming languages<br>Infra tools like Docker or Kubernetes<br>Cloud technologies like AWS or GCP<br>Building or consuming APIs<br>orhestartion tools like Airflow<br><br>The most technical persona we serve. Think of them like an engineer on your team</p> |
| Data Analyst | <p>Proficient with:<br/><br/>Data manipulation tools like Excel or SQL<br/>Dashboard tools like Looker<br/><br/>Not very familiar with reading API docs and doesn't know what a curl request is. But might be able to generate an API key if you tell them exactly how.</p> |
| Analytics Engineer | <p>Proficient with:<br/><br/>SQL &#x26; DBT<br/>Git<br/>A scripting language like Python<br/>Shallow familiarity with infra tools like Docker<br/><br/>Much more technical than a data analyst, but not as much as a data engineer</p> |
| Data Engineer | <p>Proficient with:<br/><br/>SQL &#x26; DBT<br/>Git<br/>2 or more programming languages<br/>Infra tools like Docker or Kubernetes<br/>Cloud technologies like AWS or GCP<br/>Building or consuming APIs<br/>orhestartion tools like Airflow<br/><br/>The most technical persona we serve. Think of them like an engineer on your team</p> |

Keep in mind that the distribution of served personas will differ per connector. Data analysts are highly unlikely to form the majority of users for a very technical connector like say, Kafka.

Expand Down
Loading

0 comments on commit 17407da

Please sign in to comment.