Skip to content

Latest commit

 

History

History
624 lines (393 loc) · 16.3 KB

CHANGELOG.md

File metadata and controls

624 lines (393 loc) · 16.3 KB
Raw

Changelog

Here described only the breaking and most significant changes. The full changelog and documentation for all released versions could be found in nicely formatted commit history.

v5.13

  • Implemented "Metadata.from_descriptor(allow_invalid=False)" (#1501)

v5.10

  • Various architectural and standards-compatibility improvements (minor breaking changes):
    • Added new Console commands:
      • list
      • explore
      • query
      • script
      • convert
      • publish
    • Rebased Console commands on Rich (nice output in the Console)
    • Fixed extract returning the results depends on the source type (now it's always a dictionary indexed by the resource name)
    • Enforced type safety -- many tabular command will be marked as impossible for non-tabular resources if a type checker is used
    • Improved frictionless.Resource(source) guessing abilities; if you just like to open a table resource use frictionless.resources.TableResource(path=path)

v5.8

  • Implemented Implemented catalog/dataset/package/resource.deference (#1451)

v5.7

  • Various architectural and standards-compatibility improvements (minor breaking changes):
    • Improved type detection mechanism (including remote descriptors)
    • Added resources module including File/Text/Json/TableResource
    • Deprecated resource.type argument -- use the classes above
    • Changed catalog.packages[] to catalog.datasets[].package
    • Made resource.schema optional (resource.has_schema is removed)
    • Made resource.normpath optional (resource.normdata is removed)
    • Standards-compatability improvements: profile, stats
    • Renamed system/plugin.select_Check/etc to system/plugin.select_check_class/etc

v5.6

  • Added support for sqlalchemy@2 (#1427)

v5.5

  • Implemented program/resource.index preview (#1395)

v5.4

  • Support dialect.skip_blank_rows (#1387)

v5.3

  • Support steps.resource_update for resource transformations (#1381)

v5.2

  • Added support for wkt format in fields.StringField (#1363 by @jze)

v5.1

  • Support descriptor argument for actions/program.extract (#1372)

v5.0

  • Frictionless Framework (v5) is out of Beta and released on PyPi

v5.0.0b19

v5.0.0b8

  • ForeignKeyError has been extended with additional information: fieldNames, fieldCells, referenceName, and referenceFieldNames

v5.0.0b2

v5.0.0b1

v4.40

  • Added Dialect support to packages (#1137)

v4.39

  • Fixed processing of incompatible decimal char in table schema and data (#1089)
  • Added support for Time Zone data (#1097)
  • Improved validation messages by adding summary and partial validation details (#1106)
  • Implemented new feature summary (#1127)
    • schema.to_summary
    • report.to_summary
    • Added CLI command summary
  • Fixed file compression package.to_zip (#1104)
  • Implemented feature to validate single resource (#1112)
  • Improved error message to notify about invalid fields (#1117)
  • Fixed type conversion of NaN values for data of type Int64 (#1115)
  • Exposed valid/invalid flags in CLI extract command (#1130)
  • Implemented feature package.to_er_diagram (#1135)

v4.38

  • Implemented checks.ascii_value (#1064)
  • Implemented checks.deviated_cell (#1069)
  • Implemented detector.field_true/false_values (#1074)

v4.37

  • Deprecated high-level legacy actions (use class-based alternatives):
    • describe_*
    • extract_*
    • transform_*
    • validate_*

v4.36

  • Implemented pipeline actions:
    • pipeline.validate (will replace validate_pipeline in v5)
    • pipeline.transform (will replace transform_pipeline in v5)
  • Implemented inqiury actions:
    • inqiury.validate (will replace validate_inqiury in v5)

v4.35

  • Implemented schema actions:
    • Schema.describe (will replace describe_schema in v5)
    • schema.validate (will replace validate_schema in v5)
  • Implemented new transform steps:
    • steps.field_merge
    • steps.field_pack

v4.34

  • Implemented package actions:
    • Package.describe (will replace describe_package in v5)
    • package.extract (will replace extract_package in v5)
    • package.validate (will replace validate_package in v5)
    • package.transform (will replace transform_package in v5)

v4.33

  • Implemented resource actions:
    • Resource.describe (will replace describe_resource in v5)
    • resource.extract (will replace extract_resource in v5)
    • resource.validate (will replace validate_resource in v5)
    • resource.transform (will replace transform_resource in v5)

v4.32

  • Added to_markdown() feature to metadata (#1052)

v4.31

  • Added a feature that allows to export table schema as excel (#1040)
  • Added nontabular note to validation results to indicate nontabular file (#1046)
  • Excel stats now shows bytes and hash (#1045)
  • Added pprint feature which displays metadata in a readable and pretty way (#1039)
  • Improved error message if resource.data is not a string (#1036)

v4.29

  • Made Detector's private properties public and writable (#1025)

v4.28

  • Improved an order of the metadata in YAML representation

v4.27

  • Exposed Dialect options via CLI such as sheet, table, keys, and keyed (#886)

v4.26

  • Validate 'schema.fields[].example' (#998)

v4.25

  • Allows descriptors that subclass collections.abc.Mapping (#985)

v4.24

v4.23

  • Added table dimensions check (#985)

v4.22

  • Added "extract --trusted" flag

v4.21

  • Added "--json/yaml" CLI options for transform

v4.20

  • Improved layout/schema detection algorithms (#945)

v4.19

  • Renamed inlineDialect.keys to inlineDialect.data_keys due to a conflict with dict.keys property

v4.18

  • Normalized metadata properties (increased type safety)

v4.17

  • Add fields, limit, sort and filter options to CkanDialect (#912)

v4.16

  • Implemented system/plugin.create_candidates (#893)

v4.15

  • Implemented system.get/use_http_session (#892)

v4.14

  • SQL Where Clause (#882)

v4.13

  • Implemented descriptor type detection for extract/validate (#881)

v4.12

  • Support external profiles for data package (#864)

v4.11

  • Added json argument to resource.to_snap

v4.10

  • Support resource/field renaming in transform (#843)

v4.9

  • Support --path CLI argument (#829)

v4.8

  • Added support for Package(innerpath) argument for unzipping a data package's descriptor

v4.7

  • Support control/dialect as JSON in CLI (#806)

v4.6

  • Implemented describe_dialect and describe(path, type="dialect")
  • Support --dialect argument in CLI

v4.5

  • Implemented Schema.from_jsonschema (#797)

v4.4

  • Use field.constraints.maxLength for SQL's VARCHAR (#795)

v4.3

  • Implemented resource.to_view() (#781)

v4.2

  • Make fields[].arrayItem errors more granular (#767)

v4.1

  • Added support for fields[].arrayItem (#750)

v4.0

  • Released frictionless@4 🎉

v4.0.0a15

  • Updated loaders (#658) (BREAKING)
    • Renamed filelike loader to stream loader
    • Migrated from text loader to buffer loader

v4.0.0a14

  • Improve transform API (#657) (BREAKING)
    • Swithed to the transform_resource(resource) signature
    • Swithed to the transform_package(package) signature

v4.0.0a13

  • Improved resource/package import/export (#655) (BREAKING)
    • Reworked parser.write_row_stream API
    • Reworked resource.from/to API
    • Reworked package.from/to API
    • Reworked Storage API
    • Reworked system.create_storage API
    • Merged PandasStorage into PandasParser
    • Merged SpssStorage into SpssParser

v4.0.0a12

  • Improved transformation steps (#650) (BREAKING)
    • Split value/formula/function concepts
    • Renamed a few minor step arguments

v4.0.0a11

  • Improved layout and data streams concepts (#648) (BREAKING)
    • Renamed data_stream to list_stream
    • Renamed readData to readLists
    • Renamed sample to fragment (sample now is raw lists)
    • Implemented loader.buffer
    • Implemented parser.sample
    • Added support for function based checks
    • Added support for function based steps

v4.0.0a10

  • Reworked Error.tags (BREAKING)
  • Reworked Check API and split labels/header (BREAKING)

v4.0.0a9

  • Rebased on Detector class (BREAKING)
    • Migrated all infer_*, sync/patch_schema and detect_encoding parameters to Detector
    • Made resource.infer omit empty objects
    • Added resource.read_*(size) argument
    • Added resource.labels property

v4.0.0a8

  • Improved checks/steps API (#621) (BREAKING)
    • Updated validate(extra_checks=[...]) to validate(checks=[{"code": 'code', ...}])

v4.0.0a7

  • Updated describe/extract/transform/validate APIs (BREAKING)
    • Removed validate_table (use validate_resource)
    • Removed legacy Table and File classes
    • Removed dataflows plugin
    • Replaced nopool by parallel (not parallel by default)
    • Renamed report.tables to report.tasks
    • Rebased on report.tasks[].resource (instead of plain path/scheme/format/etc)
    • Flatten Pipeline steps signature

v4.0.0a6

  • Introduced Layout class (BREAKING)
    • Renamed Query class and arguments/properties to Layout
    • Moved header options from Dialect to Layout

v4.0.0a5

  • Updated transform API
    • Added transform(type) argument

v4.0.0a4

  • Updated describe API (BREAKING)
    • Renamed describe(source_type) argument to type

v4.0.0a3

  • Updated extract API (BREAKING)
    • Removed extract_table (use extract_resource with the same API)
    • Renamed extract(source_type) argument to type

v4.0.0a1

  • Initial API/codebase improvements for v4 (BREAKING)
    • Allow Package/Resource(source) notation (guess descriptor/path/etc)
    • Renamed schema.infer -> Schema.from_sample
    • Renamed resource.inline -> resource.memory
    • Renamed compression_path -> innerpath
    • Renamed compression: no -> compression: ""
    • Updated Package/Resource.infer not to infer stats (use stats=True)
    • Removed Package/Resource.infer(only_sample) argument
    • Removed Resouce.from/to_zip (use Package.from/to_zip)
    • Removed Resouce.source (use Resource.data or Resource.fullpath)
    • Removed package/resource.infer(source) argument (use constructors)
    • Added some new API (will be covered in the updated docs after the v4 release)

v3.48

  • Make Resource independent from Table/File (#607) (BREAKING)
    • Resource can be opened like Table (it's recommended to use Resource instead of Table)
    • Renamed resource.read_sample() to resource.sample
    • Renamed resource.read_header() to resource.header
    • Renamed resource.read_stats() to resource.stats
    • Removed resource.to_table()
    • Removed resource.to_file()

v3.47

  • Optimize Row/Header/Table and rename header errors (#601) (BREAKING)
    • Row object is now lazy; it casts data on-demand preserving the same API
    • Method resource/table.read_data(_stream) now includes a header row if present
    • Renamed errors.ExtraHeaderError->ExtraLabelError (extra-label-error)
    • Renamed errors.MissingHeaderError->MissingLabelError (missing-label-error)
    • Renamed errors.BlankHeaderError->BlankLabelError (blank-label-error)
    • Renamed errors.DuplicateHeaderError->DuplicateLabelError (duplicate-label-error)
    • Renamed errors.NonMatchingHeaderError->IncorrectLabelError (incorrect-label-error)
    • Renamed schema.read/write_data->read/write_cells

v3.46

  • Renamed aws plugin to s3 (#594) (BREAKING)
$ pip install frictionless[aws] # before
$ pip install frictionless[s3] # after

v3.45

  • Drafted support for writing Multipart Data (#583)

v3.44

  • Added support for writing to Remote Data (#582)

v3.43

  • Add support to writing to Google Sheets (#581)
  • Renamed gsheet plugin/format to gsheets (BREAKING: minor)

v3.42

  • Added support for writing to S3 (#580)

v3.41

  • Update Loader/Parser API to write to different targets (#579) (BREAKING: minor)

v3.40

  • Implemented a standalone multipart loader (#573)

v3.39

  • Fixed Header not being an original one (#572)
  • Fix bad format validation (#571)
  • Added default errors limit equals to 1000 (#570)
  • Added support for field.float_number (#569)

v3.38

  • Improved ckan plugin (#560)

v3.37

  • Remove not working elastic plugin draft (#558)

v3.36

  • Support custom types (#557)

v3.35

  • Added "resolve" option to "resource/package.to_zip" (#556)

v3.34

  • Moved frictionless.controls to frictionless.plugins.* (BREAKING)
  • Moved frictionless.dialects to frictionless.plugins.* (BREAKING)
  • Moved frictionless.exceptions.FrictionlessException to frictionless.FrictionlessException (BREAKING)
  • Moved excel dependencies to frictionless[excel] extras (BREAKING)
  • Moved json dependencies to frictionless[json] extras (BREAKING)
  • Consider json files to be a metadata by default (BREAKING)

Code example:

# Before
# pip install frictionless
from frictionless import dialects, exceptions
excel_dialect = dialects.ExcelDialect()
json_dialect = dialects.JsonDialect()
exception = exceptions.FrictionlessException()

# After
# pip install frictionless[excel,json]
from frictionless import FrictionlessException
from frictionless.plugins.excel import ExcelDialect
from frictionless.plugins.json import JsonDialect
excel_dialect = dialects.ExcelDialect()
json_dialect = dialects.JsonDialect()
exception = FrictionlessException()

v3.33

  • Implemented resource.write (#537)

v3.32

  • Added url parameter to SQL import/export (#535)

v3.31

  • Made tables with header and no data rows valid (#534) (BREAKING: minor)

v3.30

  • Various CLI improvements (#532)
    • Added autocompletion
    • Added stdin support
    • Added "extract --csv"
    • Exposed more options

v3.29

  • Added experimental CKAN support (#528)

v3.28

  • Add a "nopool" argument to validate (#527)

v3.27

  • Stop sorting keyed sources as the order is now guaranteed by Python (#512) (BREAKING)

v3.26

  • Added "nolookup" argument for validate_package (#515)

v3.25

  • Add transform functionality (#505)
  • Methods schema.get/remove_field now raise if not found (#505) (BREAKING)
  • Methods package.get/remove_resource now raise if not found (#505) (BREAKING)

v3.24

  • Lower case resource.scheme/format/hashing/encoding/compression (#499) (BREAKING)

v3.23

  • Support "header_case" option for dialects (#488)

v3.22

  • Added suppport for DB2 format (#485)

v3.21

  • Improved SPSS plugin (#483)
  • Improved BigQuery plugin (#470)

v3.20

  • Added support for SQL Views (#466)

v3.19

  • Rebased AwsLoader on streaming (#460)

v3.18

  • Added hashing parameter to describe/describe_package
  • Removed table.onerror property (BREAKING)

v3.17

  • Added timezone for datetime/time parsing (#457) (BREAKING)

v3.16

  • Fixed metadata.to_yaml (#455)
  • Removed the expand argument from metadata.to_dict (BREAKING)

v3.15

  • Added native schema support to SqlParser (#452)

v3.14

  • Make Resource the main internal interface (#446) (BREAKING: for plugin authors)
  • Move Resource's stats to resource.stats (BREAKING)
  • Rename on_error to onerror (BREAKING)
  • Added resource.stats.fields

v3.13

  • Add an on_error argument to Table/Resource/Package (#445)

v3.12

  • Added streaming to the extract functions (#442)

v3.11

  • Added experimental BigQuery support (#424)

v3.10

  • Added experimental SPSS support (#421)

v3.9

  • Rebased on a goodtables successor versioning

v3.8

  • Add support SQL/Pandas import/export (#31)

v3.7

  • Add support for custom JSONEncoder classes (#24)

v3.6

  • Normalize header terminology

v3.5

  • Initial public version