Replace snappy with cramjam #1091

gliptak · 2024-02-02T20:15:44Z

No description provided.

gliptak · 2024-02-05T21:40:11Z

@taylorfturner please assign reviewers

taylorfturner · 2024-02-05T21:46:52Z

Thanks for your patience @gliptak -- we will review your proposal

gliptak · 2024-02-20T16:24:59Z

please review

taylorfturner

I need to do some additional research on this as tests are not passing on the following lines locally, which is odd to me:

I like the change generally just need some additional research on this to ensure we are covering all the bases. @gliptak Thanks

gliptak · 2024-02-26T15:55:58Z

capitalone/synthetic-data#346

taylorfturner · 2024-03-07T12:29:23Z

@gliptak rebase onto dev and I'll approve

taylorfturner · 2024-03-07T19:14:26Z

rebase or update branch on this and I'll approve @gliptak

gliptak · 2024-03-07T21:49:23Z

green for your review @taylorfturner

taylorfturner

LGTM

gliptak · 2024-03-11T14:30:27Z

@micdavis @ksneab7 please review/approve

taylorfturner · 2024-03-22T18:31:30Z

@gliptak are you able to restore your cramjam1 branch?

This change fails tests locally on this command DATAPROFILER_SEED=0 python3 -m unittest discover -p "test*.py"

First issues is dataprofiler/dataprofiler/tests/test_dp_logging.py", line 27, in test_default_verbosity self.assertEqual( AssertionError: 20 != 30

Second issue is dataprofiler/dataprofiler/tests/test_data_profiler.py", line 32, in test_set_seed self.assertEqual(dp.settings._seed, None) AssertionError: 0 != None

main testing works locally and the only diff between main and dev is this PR

We can either:

restore your branch and revert this PR
or I or you can create a new PR to revert back to snappy, as it was prior to this PR

taylorfturner · 2024-03-22T18:51:37Z

Checking out f8b3e5dbd4b76f0ecc291911ace9e8e21cf1ecb1 and running tests on that commit (one commit prior to this PR) ensures that tests did run on dev locally at that point in the history @gliptak

taylorfturner · 2024-03-22T19:05:39Z

Glad to ultimately do cramjam + python 3.11 -- these tests will just need to pass locally as well as remote

gliptak · 2024-03-22T19:45:17Z

@taylorfturner restored branch

as the build was green for this PR, might that indicate that different library versions where used?

taylorfturner · 2024-03-22T19:47:33Z

potentially -- I tried a bunch of different scenarios and wasn't able to find a setup where the above two tests passed locally.

Did you run DATAPROFILER_SEED=0 python3 -m unittest discover -p "test*.py" on your end locally with cramjam1?

gliptak · 2024-03-24T19:35:31Z

I did reproduce the fail locally and still cannot reproduce in GHA

the one difference I observed is that local is Python 3.10.12 while GHA is CPython (3.10.13)

* add downloads tile (capitalone#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]>

* Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * Quick fix for dependency max pins (#1120) * Fix dask_expr * Keras and Tensorflow version fix * Keras and Tensorflow version fix * Fix keras bug * pre-commit fix (#1122) * docs: update test link to latest version (#1114) * docs: add contributor notes on where to find documentation branches (#1113) * docs: add contributor notes on where to find documentation branches * docs: update documentation wording to spell out why `dev-gh-pages` and `gh-pages` branches exist for staging content * docs: add note on fork Co-authored-by: Taylor Turner <[email protected]> * Update .github/CONTRIBUTING.md Co-authored-by: Taylor Turner <[email protected]> --------- Co-authored-by: Taylor Turner <[email protected]> * update black version (#1131) * Add memray max version (#1132) * Bug fix for float precision calculation using categorical data with trailing zeros. (#1125) * Revert "Bug fix for float precision calculation using categorical data with t…" (#1133) This reverts commit d3159bd. * fix * make up to date * yep, shouldn't change * bump version --------- Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: abajpai15 <[email protected]> Co-authored-by: Patrick Carlson <[email protected]> Co-authored-by: James Schadt <[email protected]>

* add downloads tile (capitalone#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]>

* Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * pre-commit fix (#1122) * Bug fix for float precision calculation using categorical data with trailing zeros. (#1125) * Revert "Bug fix for float precision calculation using categorical data with t…" (#1133) This reverts commit d3159bd. * refactor: move layers outside of class * refactor: update model to keras 3.0 * fix: manifest * fix: bugs in compile and train * fix: bug in load_from_library * fix: bugs in CharCNN * refactor: loading tf model labeler * fix: bug in data_labeler identification * fix: update model to use proper softmax layer names * fix: formatting * fix: remove unused line * refactor: drop support for 3.8 * fix: comments * fix: comment --------- Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: Taylor Turner <[email protected]> Co-authored-by: James Schadt <[email protected]>

* add downloads tile (#1085) * Add Python 3.11 to GHA * Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * Update dask modules * Install dask dataframe * Update dask modules in precommit * Correct copy/paste error * Try again to clear Unicode * Rolled back pre-commit dask version * Add py311 to tox * Bump dask to 2024.4.1 * Bump python-snappy 0.7.1 * Rewrite labeler test * Correct isort * Satisfy black * And flake8 * Synced with requirements --------- Co-authored-by: Taylor Turner <[email protected]>

* refactor: Upgrade the models to use keras 3.0 (#1138) * Replace snappy with cramjam (#1091) * add downloads tile (#1085) * Replace snappy with cramjam * Delete test_no_snappy --------- Co-authored-by: Taylor Turner <[email protected]> * pre-commit fix (#1122) * Bug fix for float precision calculation using categorical data with trailing zeros. (#1125) * Revert "Bug fix for float precision calculation using categorical data with t…" (#1133) This reverts commit d3159bd. * refactor: move layers outside of class * refactor: update model to keras 3.0 * fix: manifest * fix: bugs in compile and train * fix: bug in load_from_library * fix: bugs in CharCNN * refactor: loading tf model labeler * fix: bug in data_labeler identification * fix: update model to use proper softmax layer names * fix: formatting * fix: remove unused line * refactor: drop support for 3.8 * fix: comments * fix: comment --------- Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: Taylor Turner <[email protected]> Co-authored-by: James Schadt <[email protected]> * Fix Tox (#1143) * tox new * update * update * update * update * update * update * update * update tox.ini * update * update * remove docs * empty retrigger * update (#1146) * bump version * update 3.11 * remove dist/ --------- Co-authored-by: JGSweets <[email protected]> Co-authored-by: Gábor Lipták <[email protected]> Co-authored-by: James Schadt <[email protected]>

taylorfturner and others added 2 commits January 16, 2024 16:19

add downloads tile (capitalone#1085)

3d9b5ff

Replace snappy with cramjam

5330642

gliptak requested a review from a team as a code owner February 2, 2024 20:15

gliptak mentioned this pull request Feb 2, 2024

Add Python 3.11 to GHA #1090

Merged

gliptak added 2 commits February 2, 2024 15:43

Delete test_no_snappy

fe31e60

Merge branch 'dev' into cramjam1

86cfe10

taylorfturner requested review from taylorfturner, micdavis and ksneab7 February 5, 2024 21:47

taylorfturner assigned gliptak Feb 5, 2024

taylorfturner reviewed Feb 21, 2024

View reviewed changes

taylorfturner changed the base branch from old-old-dev to dev March 7, 2024 12:28

Merge branch 'dev' into cramjam1

20946e9

taylorfturner enabled auto-merge (squash) March 8, 2024 18:04

taylorfturner approved these changes Mar 8, 2024

View reviewed changes

micdavis approved these changes Mar 14, 2024

View reviewed changes

taylorfturner merged commit f411110 into capitalone:dev Mar 14, 2024
5 checks passed

gliptak deleted the cramjam1 branch March 14, 2024 14:33

taylorfturner mentioned this pull request Mar 22, 2024

Revert prior PR into dev #1122

Merged

gliptak restored the cramjam1 branch March 22, 2024 19:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace snappy with cramjam #1091

Replace snappy with cramjam #1091

gliptak commented Feb 2, 2024

gliptak commented Feb 5, 2024

taylorfturner commented Feb 5, 2024 •

edited

Loading

gliptak commented Feb 20, 2024

taylorfturner left a comment

gliptak commented Feb 26, 2024

taylorfturner commented Mar 7, 2024

taylorfturner commented Mar 7, 2024

gliptak commented Mar 7, 2024

taylorfturner left a comment

gliptak commented Mar 11, 2024

taylorfturner commented Mar 22, 2024 •

edited

Loading

taylorfturner commented Mar 22, 2024 •

edited

Loading

taylorfturner commented Mar 22, 2024

gliptak commented Mar 22, 2024

taylorfturner commented Mar 22, 2024 •

edited

Loading

gliptak commented Mar 24, 2024 •

edited

Loading

Replace snappy with cramjam #1091

Replace snappy with cramjam #1091

Conversation

gliptak commented Feb 2, 2024

gliptak commented Feb 5, 2024

taylorfturner commented Feb 5, 2024 • edited Loading

gliptak commented Feb 20, 2024

taylorfturner left a comment

Choose a reason for hiding this comment

gliptak commented Feb 26, 2024

taylorfturner commented Mar 7, 2024

taylorfturner commented Mar 7, 2024

gliptak commented Mar 7, 2024

taylorfturner left a comment

Choose a reason for hiding this comment

gliptak commented Mar 11, 2024

taylorfturner commented Mar 22, 2024 • edited Loading

taylorfturner commented Mar 22, 2024 • edited Loading

taylorfturner commented Mar 22, 2024

gliptak commented Mar 22, 2024

taylorfturner commented Mar 22, 2024 • edited Loading

gliptak commented Mar 24, 2024 • edited Loading

taylorfturner commented Feb 5, 2024 •

edited

Loading

taylorfturner commented Mar 22, 2024 •

edited

Loading

taylorfturner commented Mar 22, 2024 •

edited

Loading

taylorfturner commented Mar 22, 2024 •

edited

Loading

gliptak commented Mar 24, 2024 •

edited

Loading