Skip to content

Commit

Permalink
Update managing-projects.mdx (#1460)
Browse files Browse the repository at this point in the history
* Update managing-projects.mdx

* Update managing-projects.mdx

* Update export-methods.mdx

* Disclaimer about timezone in export API

Using a custom param instead of existing common param in order to add disclaimer only to the Export API

* Update export-methods.mdx

* Update managing-projects.mdx

* Update overview.mdx

* Update data-clean-up.mdx

* Update export.openapi.yaml
  • Loading branch information
myronkaifung authored Oct 2, 2024
1 parent f73ca34 commit 4411114
Show file tree
Hide file tree
Showing 5 changed files with 26 additions and 14 deletions.
16 changes: 14 additions & 2 deletions openapi/src/export.openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,20 @@ paths:
type: integer
description: >-
Required if using service account to authenticate request.
- $ref: ./common/parameters.yaml#/query/fromDate
- $ref: ./common/parameters.yaml#/query/toDate
- in: query
name: from_date
schema:
type: string
description: >-
The date in yyyy-mm-dd format to begin querying from. This date is inclusive and interpreted as UTC timezone for projects created after 1 January 2023 and current project timezone for projects created before 11 January 2023.
required: true
- in: query
name: to_date
schema:
type: string
description: >-
The date in yyyy-mm-dd format to query to. This date is inclusive and interpreted as UTC timezone for projects created after 1 January 2023 and current project timezone for projects created before 11 January 2023.
required: true
- in: query
name: limit
schema:
Expand Down
4 changes: 2 additions & 2 deletions pages/docs/data-governance/data-clean-up.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,8 @@ You may undo Data Deletion requests **for only 7 days after submitting**, after

### Reminders with ETL approach

- For projects created prior to January 11th, 2023, offset your timestamps
- If your project was created prior to January 11th, 2023, you cannot just clean the data you don't want and re-import it. Your data is stored in project time, so you need to adjust the offset before importing
- For projects created prior to 1 Jan 2023 offset your timestamps
- If your project was created prior to 1 Jan 2023, you cannot just clean the data you don't want and re-import it. Your data is stored in project time, so you need to adjust the offset before importing
- Don’t forget to regenerate $insert_id’s when you ETL
- When you submit a deletion request, we hide your data immediately from your project to reduce any privacy concerns. We call this “soft deletion”, an interim phase before our “hard deletion” kicks in (where your data is permanently deleted from our servers) so that you can review the impact of your changes and undo when necessary. If you re-import data while the data is soft deleted **with the same \$insert_id**, our deduplication systems may keep the old (deleted) event and toss the new event. Since this data is soft-deleted, your re-import won’t reflect the data as imported. Thus, when the ETL is done, you should regenerate the $insert_id value if possible, to avoid this possible collision

Expand Down
2 changes: 1 addition & 1 deletion pages/docs/data-pipelines/old-pipelines/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ This can happen for a few reasons:

**3. What timezone is used for my event exports?**

The timestamp for exported events from your pipeline will be in the UTC timezone. For projects created before 11 Jan 2023, the raw event timestamps are transformed from Project Timezone to UTC. For projects after 11 Jan 2023, raw event data is stored in UTC. Learn more about [managing timezones here](/docs/orgs-and-projects/managing-projects#manage-timezones-for-projects).
The timestamp for exported events from your pipeline will be in the UTC timezone. For projects created before 1 Jan 2023, the raw event timestamps are transformed from Project Timezone to UTC. For projects after 1 Jan 2023, raw event data is stored in UTC. Learn more about [managing timezones here](/docs/orgs-and-projects/managing-projects#manage-timezones-for-projects).

**4. How can I count events exported by Mixpanel in the warehouse?**

Expand Down
4 changes: 2 additions & 2 deletions pages/docs/export-methods.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ You can export data from your project using a variety of methods, including ligh
### Event Export via API

<Callout type="info">
The from_date and to_date parameters are in UTC timezone, regardless of your project's timezone setting.
The `from_date` and `to_date` params are interpreted using UTC timezones. For projects created before 1 January 2023, the `from_date` and `to_date` params are interpreted using your current project timezone.
</Callout>

To export raw event data, use the [Raw Event Export API](https://developer.mixpanel.com/reference/raw-event-export). The events payload is returned as JSONL, where each line is a valid JSON object. This method is recommended for high volume exports.
Expand All @@ -42,7 +42,7 @@ Example payload:
```

The exported payloads are raw events, containing timestamps as they are stored in your project. For projects created before 11 Jan 2023, your event timestamp is stored in your local project timezone at the time of ingestion. For projects created after 11 Jan 2023, your event timestamp is stored in UTC timezone. Learn more about managing your project timezone [here](/docs/orgs-and-projects/managing-projects#manage-timezones-for-projects).
The exported payloads are raw events, containing timestamps as they are stored in your project. For projects created before 1 Jan 2023, your event timestamp is stored in your local project timezone at the time of ingestion. For projects created after 1 Jan 2023, your event timestamp is stored in UTC timezone. Learn more about managing your project timezone [here](/docs/orgs-and-projects/managing-projects#manage-timezones-for-projects).

### CSV Events Export

Expand Down
14 changes: 7 additions & 7 deletions pages/docs/orgs-and-projects/managing-projects.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ Then select the Projects tab.
## Manage Timezones for Projects

<Callout type="info" emoji="ℹ️">
For projects created before 11 Jan 2023, Mixpanel converts event timestamps to your project timezone before writing the event to your Mixpanel data stores, meaning that event timestamps are stored based on the project timezone setting at the time of ingestion.
For projects created before 1 Jan 2023, Mixpanel converts event timestamps to your project timezone before writing the event to your Mixpanel data stores, meaning that event timestamps are stored based on the project timezone setting at the time of ingestion.
</Callout>

Mixpanel records all events in [Coordinated Universal Time (UTC)](https://www.worldtimeserver.com/learn/what-is-utc/) at intake. Changing the timezone for any project under Project Settings only affects the timezone in which Mixpanel outputs results.
Expand Down Expand Up @@ -152,19 +152,19 @@ Mixpanel.track("Account Created Date", props);
```
#### Exporting Data from Mixpanel

When you’re exporting raw data from Mixpanel, your request requires the date parameters `from_date` and `to_date` to determine which date range of data to return.

Mixpanel's raw export machines interpret the "from_date" and "to_date" values to your project’s timezone. As a result, if you request a single day of data, you receive one full day in your project’s timezone, not one full day in UTC.
<Callout type="info" emoji="ℹ️">
For projects created before 1 January 2023, the `from_date` and `to_date` params for the Export API are interpreted using your project's current timezone setting.
</Callout>

As mentioned earlier, Mixpanel stores your events in UTC. The "$time" property from a raw export returns with a Unix timestamp, which represents the number of seconds that have elapsed since 00:00:00 on January 1, 1970.
When you’re exporting raw data from Mixpanel using the [Export API](https://developer.mixpanel.com/reference/raw-event-export), your request requires the date parameters `from_date` and `to_date` to determine which date range of data to return. Mixpanel's raw export API interprets the "from_date" and "to_date" param using UTC timezone.

In the example below, if your project’s timezone is US/Pacific, and you set both the `from_date` and `to_date` to `2023-08-01`, it will return events between `00:00:00 Aug. 1 to 11:59:59 Aug. 1 PDT`/`07:00:00 Aug. 1 – 06:59:59 Aug. 2 UTC` where the events timestamp are in Unix timestamp format in UTC.
The `$time` property values from your raw event export are Unix timestamps, which represents the number of seconds that have elapsed since 00:00:00 on January 1, 1970. Note that [the timezone of the `$time` property in your exports depends on how your project records time at ingestion](/docs/orgs-and-projects/managing-projects#manage-timezones-for-projects).

#### Importing Data into Mixpanel

Always send imported data to Mixpanel in UTC to ensure it displays correctly in your project.

Mixpanel stores your data in UTC, then use your project timezone setting to output data in your selected timezone.
Mixpanel stores your data in UTC, then use your project timezone setting to output data in your selected timezone. For projects created before 1 January 2023, your UTC timestamps are converted to the project timezone before ingestion meaning that your event data are stored in the project timezone.

## Transfer Project to Another Organization

Expand Down

0 comments on commit 4411114

Please sign in to comment.