Skip to content

Latest commit

 

History

History
112 lines (67 loc) · 2.31 KB

design.md

File metadata and controls

112 lines (67 loc) · 2.31 KB
Raw

Software Architectural Design -- Parquet IO Java

Introduction

Terms and Abbreviations

PIOJ
Parquet IO Java

Requirement Overview

Please refer to the System Requirement Specification for user-level requirements.

Solution Strategy

Building Blocks

This section introduces the building blocks of the software. Together those building blocks make up the big picture of the software structure.

Row

dsn~row~1

A Row is class that contains list of Parquet columns.

RowParquetReader

dsn~row-parquet-reader~1

The RowParquetReader reads Parquet formatted data into a Row object.

RowParquetChunkReader

dsn~row-parquet-chunk-reader~1

The RowParquetChunkReader reads provided row group chunk's data into a Row object.

ParquetConverter

dsn~parquet-converter~1

The ParquetConverter is an abstraction of set column converters that read Parquet data into memory objects.

Runtime View

This section describes the runtime behavior of the software.

Reading Parquet File Contents

dsn~read-parquet-file-contents~1

The RowParquetReader reads contents of a Parquet file.

Covers:

  • req~accessing-decoded-rows~1

Needs: impl, utest

Reading Parquet File Row Group Chunks Contents

dsn~read-parquet-file-chunks-contents~1

The RowParquetChunkReader only reads certain row group contents of a Parquet file.

Covers:

  • req~reading-row-group-intervals~1

Needs: impl, utest

Converting Primitive Column Types

dsn~converting-primitive-column-types~1

The ParquetConverter allows to convert primitive Parquet types into in-memory objects.

Covers:

  • req~reading-primitive-types~1

Needs: impl, utest

Converting Logical Column Types

dsn~converting-logical-column-types~1

The ParquetConverter allows to convert logical Parquet types into in-memory objects.

Covers:

  • req~reading-logical-types~1

Needs: impl, utest

Converting Nested Column Types

dsn~converting-nested-column-types~1

The ParquetConverter allows to convert nested Parquet types into in-memory objects.

Covers:

  • req~reading-nested-types~1

Needs: impl, utest

Acknowledgments

This document's section structure is derived from the "arc42" architectural template by Dr. Gernot Starke, Dr. Peter Hruschka.