Skip to content

Commit

Permalink
docs: add nested_type example in `data_types_and_io/structured_data…
Browse files Browse the repository at this point in the history
…set.py` (flyteorg#5269)
  • Loading branch information
austin362667 authored and robert-ulbrich-mercedes-benz committed Jul 2, 2024
1 parent 6c17878 commit bd82303
Showing 1 changed file with 15 additions and 2 deletions.
17 changes: 15 additions & 2 deletions docs/user_guide/data_types_and_io/structureddataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ You can now use `numpy.ndarray` to deserialize the parquet file to NumPy and ser

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/data_types_and_io/data_types_and_io/structured_dataset.py
:caption: data_types_and_io/structured_dataset.py
:lines: 134-147
:lines: 134-149
```

:::{note}
Expand All @@ -248,7 +248,20 @@ You can run the code locally as follows:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/data_types_and_io/data_types_and_io/structured_dataset.py
:caption: data_types_and_io/structured_dataset.py
:lines: 151-155
:lines: 153-157
```

### The nested typed columns

Like most storage formats (e.g. Avro, Parquet, and BigQuery), StructuredDataset support nested field structures.

:::{note}
Nested field StructuredDataset should be run when flytekit version > 1.11.0.
:::

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/data_types_and_io/data_types_and_io/structured_dataset.py
:caption: data_types_and_io/structured_dataset.py
:lines: 159-270
```

[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/data_types_and_io/

0 comments on commit bd82303

Please sign in to comment.