Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: twitter/algebird
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 0.8.2
Choose a base ref
...
head repository: twitter/algebird
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: develop
Choose a head ref
Loading
Showing 353 changed files with 43,205 additions and 8,088 deletions.
8 changes: 8 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: 2
updates:
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: daily
time: "04:00"
open-pull-requests-limit: 10
4 changes: 4 additions & 0 deletions .github/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
template: |
## Pull Requests
$CHANGES
82 changes: 82 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
name: ci
on: [push, pull_request]

jobs:
checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
cache: "sbt"
distribution: "temurin"
java-version: 17
- run: sbt "; scalafmtCheckAll; scalafmtSbtCheck" "; scalafixEnable; scalafixAll --check"
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
cache: "sbt"
distribution: "temurin"
java-version: ${{matrix.java}}
- run: sbt "++${{matrix.scala}} test"
strategy:
matrix:
java:
- 8
- 11
- 17
scala:
- 2.11.12
- 2.12.18
- 2.13.11
test-coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
cache: "sbt"
distribution: "temurin"
java-version: 17
- run: |
sbt ++2.12.18 coverage test coverageReport
bash <(curl -s https://codecov.io/bash)
mimaReport:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
cache: "sbt"
distribution: "temurin"
java-version: ${{matrix.java}}
- run: sbt "++${{matrix.scala}} mimaReportBinaryIssues"
strategy:
matrix:
java:
- 8
- 11
- 17
scala:
- 2.11.12
- 2.12.18
- 2.13.11
microsite:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Ruby setup
uses: ruby/setup-ruby@v1
with:
ruby-version: 2.6
- run: |
gem install sass
gem install jekyll -v 3.2.1
- uses: actions/setup-java@v4
with:
cache: "sbt"
distribution: "temurin"
java-version: 17
14 changes: 14 additions & 0 deletions .github/workflows/release-drafter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Release Drafter

on:
push:
branches:
- develop

jobs:
update_release_draft:
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@v6
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
22 changes: 22 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Release
on:
push:
branches: [develop]
tags: ["*"]
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
cache: "sbt"
distribution: "temurin"
java-version: 8
- name: Publish ${{ github.ref }}
run: sbt ci-release
env:
PGP_PASSPHRASE: ${{ secrets.PGP_PASSPHRASE }}
PGP_SECRET: ${{ secrets.PGP_SECRET }}
SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
SONATYPE_USERNAME: ${{ secrets.SONATYPE_USERNAME }}
11 changes: 10 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -4,12 +4,15 @@
*.jar
*~

sonatype.sbt

#sbt
/project/target/
/project/project/target

/target/
/src/jline/target/
.bsp

# target directories for ant build
/build/
@@ -20,7 +23,7 @@
/bin/
/sandbox/

# eclipse, intellij
# eclipse, intellij, bloop, metals
/.classpath
/.project
/src/intellij/*.iml
@@ -33,6 +36,9 @@
*/.project
*/.cache
*/.settings
.bloop
.metals
.vscode

# bak files produced by ./cleanup-commit
*.bak
@@ -47,3 +53,6 @@ project/build/target/
project/plugins/target/
project/plugins/lib_managed/
project/plugins/src_managed/

# Auto-copied by sbt-microsites
docs/src/main/tut/contributing.md
8 changes: 8 additions & 0 deletions .jvmopts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# see https://weblogs.java.net/blog/kcpeppe/archive/2013/12/11/case-study-jvm-hotspot-flags
-Dfile.encoding=UTF8
-Xms1G
-Xmx3G
-XX:ReservedCodeCacheSize=500M
-XX:+TieredCompilation
-XX:-UseGCOverheadLimit
-XX:+UseParallelGC
9 changes: 9 additions & 0 deletions .scalafix.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
rules = [
RemoveUnused
ProcedureSyntax
ExplicitResultTypes
]

ExplicitResultTypes.memberKind = [Def, Val, Var]
ExplicitResultTypes.memberVisibility = [Public]
ExplicitResultTypes.skipSimpleDefinitions = ['Lit', 'Term.New']
14 changes: 14 additions & 0 deletions .scalafmt.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
version=3.8.0
runner.dialect = scala212
fileOverride {
"glob:**/scala-2.13*/**" {
runner.dialect = scala213
}
}
maxColumn = 110
docstrings.style = Asterisk
newlines.alwaysBeforeMultilineDef = false
newlines.penalizeSingleSelectMultiArgList = false
align.openParenCallSite = false
rewrite.rules = [AvoidInfix, SortImports, RedundantBraces, RedundantParens, PreferCurlyFors]
rewrite.redundantBraces.generalExpressions = false
4 changes: 0 additions & 4 deletions .travis.yml

This file was deleted.

132 changes: 132 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,137 @@
# Algebird #

### Version 0.12.3 (Not yet released) ###

* Add `SetDiff` data structure to `algebird-core`: https://github.com/twitter/algebird/pull/555
* Add `Ring[BigDecimal]`, modeled after `Ring[BigInt]`: https://github.com/twitter/algebird/pull/553
* "Exponential Histogram" sliding window counter implementation added to `algebird-core` as `ExpHist`: https://github.com/twitter/algebird/pull/568
* improve HLLSeries performance: https://github.com/twitter/algebird/pull/575
* Add a microsite at <https://twitter.github.io/algebird> via the `sbt-microsites` plugin, along with docs for all typeclasses and data structures: https://github.com/twitter/algebird/pull/576
* Adds lots of scalacheck `Arbitrary` and `Gen` instances to `algebird-test`, under `com.twitter.algebird.scalacheck.{ gen, arbitrary }`: https://github.com/twitter/algebird/pull/579
* Add `Monoid[Max[Vector[T]]]`, `Monoid[Max[Stream[T]]]`: https://github.com/twitter/algebird/pull/579
* Add `FirstAggregator` and `LastAggregator`, and docs and API / perf improvements for `First`, `Last`, `Min`, `Max`: https://github.com/twitter/algebird/pull/579
* Add `LawsEquiv` versions of all laws: https://github.com/twitter/algebird/pull/584
* Deprecates broken group/ring for `Future`/`Try`: https://github.com/twitter/algebird/pull/584
* Add `metricsLaws[T]` to `BaseProperties` in `algebird-test`: https://github.com/twitter/algebird/pull/584
* Modify generated `Tuple2Monoid`, etc to extend `TupleNSemigroup`, giving subclasses access to efficient `sumOption`: https://github.com/twitter/algebird/pull/585
* optimize `Generated{Abstract,Product}Algebra.sumOption` with benchmarking https://github.com/twitter/algebird/pull/591
* Add an efficient `sumOption`, `+`, `-`, methods and docs to `AveragedValue`: https://github.com/twitter/algebird/pull/589

### Version 0.12.2 ###

* optimize `CMS.create(Seq[K])`: https://github.com/twitter/algebird/pull/537
* Update to sbt 0.13.11: https://github.com/twitter/algebird/pull/540
* Add `Semigroup.maybePlus`: https://github.com/twitter/algebird/pull/544
* Improve the build and test times and add Codecov.io: https://github.com/twitter/algebird/pull/541

### Version 0.12.1 ###

* Add `Identity` class and `Monad` instance: https://github.com/twitter/algebird/pull/511
* Using `dropRight` instead of `tail` for shortening `BytesSpec` array: https://github.com/twitter/algebird/pull/510
* Fix flaky `MinHasherSpec` test: https://github.com/twitter/algebird/pull/514
* fix input type of `toRichTraversable`: https://github.com/twitter/algebird/pull/518
* build in scala 2.11 by default: https://github.com/twitter/algebird/pull/520
* make `SSOne`/`SSMany` constructors private and add apply method with count: https://github.com/twitter/algebird/pull/519
* Fix `BytesSpec` to not reference head on an empty list https://github.com/twitter/algebird/pull/525
* Add an `Aggregator.randomSample` aggregator: https://github.com/twitter/algebird/pull/529
* Add `sortedTakeBy` and `sortedReverseTakeBy` to `Aggregator.scala`: https://github.com/twitter/algebird/pull/527
* Add `Batched[A]` type for efficient lazy addition: https://github.com/twitter/algebird/pull/530
* Add a default `k` value for `Aggregator.approximatePercentile`: https://github.com/twitter/algebird/pull/531

### Version 0.12.0 ###

* Implement an appendMonoid Aggregator factory which yields aggregators…: https://github.com/twitter/algebird/pull/501
* Dealing with probabilistic tests: https://github.com/twitter/algebird/pull/478
* Add Applicative.sequenceGen: https://github.com/twitter/algebird/pull/498
* Create a sparse Count-Min-Sketch.: https://github.com/twitter/algebird/pull/464
* fix name and visibility of negativePowersOfTwo: https://github.com/twitter/algebird/pull/492
* Speed up HLL presentation by 100x: https://github.com/twitter/algebird/pull/491
* Test Semigroup#sumOption using Iterator instead of List: https://github.com/twitter/algebird/pull/490
* Fix tests that were not actually running: https://github.com/twitter/algebird/pull/485
* add immutable version of sorted(Reverse)Take: https://github.com/twitter/algebird/pull/484
* Cuber/roller macros: https://github.com/twitter/algebird/pull/483
* Add sanity requirement for Approximate: https://github.com/twitter/algebird/pull/481
* Ioconnell/make develop version have snapshot suffix: https://github.com/twitter/algebird/pull/482
* Upgrade scalacheck and scalatest: https://github.com/twitter/algebird/pull/480
* Adding scoped top-N CMS monoid: https://github.com/twitter/algebird/pull/471
* Fix Qtree quantileBounds off-by-one error: https://github.com/twitter/algebird/pull/472
* Move benchmarks to JMH: https://github.com/twitter/algebird/pull/473
* Ianoc/q tree benchmark more coverage: https://github.com/twitter/algebird/pull/474
* Optimize QTree a bunch: https://github.com/twitter/algebird/pull/475
* Disable coveralls, shows up as builds pending that are long finished: https://github.com/twitter/algebird/pull/476

### Version 0.11.0 ###
* Move CMSHasherByteArray from scalding: https://github.com/twitter/algebird/pull/467
* Upgrade sbt launcher script (sbt-extras): https://github.com/twitter/algebird/pull/469
* Create case class macros for algebraic structures: https://github.com/twitter/algebird/pull/466
* Refactor MapAggregator: https://github.com/twitter/algebird/pull/462
* Algebird support for spark: https://github.com/twitter/algebird/pull/397
* Add MapAggregator from 1 (key, aggregator) pair: https://github.com/twitter/algebird/pull/452
* Remove unnecessary use of scala.math: https://github.com/twitter/algebird/pull/455
* Don&#39;t call deprecated HyperLogLog methods in tests: https://github.com/twitter/algebird/pull/456
* Update product_generators.rb: https://github.com/twitter/algebird/pull/457
* Pzheng/gaussian euclidean: https://github.com/twitter/algebird/pull/448

### Version 0.10.2 ###
* QTree quantileBounds assert percentile <= 1.0 #447

### Version 0.10.1 ###
* Make HLL easier to use, add Hash128 typeclass #440
* add ! to ApproximateBoolean #442
* add QTreeAggregator and add approximatePercentileBounds to Aggregator #443
* Make level configurable in QTreeAggregators #444

### Version 0.10.0 ###
* HyperLogLogSeries #295
* CMS: add contramap to convert CMS[K] to CMS[L], add support for String and Bytes, remove Ordering context bound for K #399
* EventuallyAggregator and variants #407
* Add MultiAggregator.apply #408
* Return a MonoidAggregator from MultiAggregator when possible #409
* Add SummingWithHitsCache class to also track key hits. #410
* Add MapAggregator to compose tuples of (key, agg) pairs #411
* fix README.md. 2.9.3 no longer published #412
* Add Coveralls Badge to the README #413
* Add some combinators on MonoidAggregator #417
* Added function to safely downsize a HyperLogLog sketch #418
* AdaptiveCache #419
* fix property tests #421
* Make Preparer extend Serializable #422
* Make MutableBackedMap Serializable. #424
* A couple of performance optimizations: HyperLogLog and BloomFilter #426
* Adds a presenting benchmark and optimizes it #427
* Fixed broken links in README #428
* Speed up QTree #433
* Moments returns NaN when count is too low for higher order statistics #434
* Add HLL method to do error-based Aggregator #438
* Bump bijection to 0.8.0 #441

### Version 0.9.0 ###
* Replace mapValues with one single map to avoid serialization in frameworks like Spark. #344
* Add Fold trait for composable incremental processing (for develop) #350
* Add a GC friendly LRU cache to improve SummingCache #341
* BloomFilter should warn or raise on unrealistic input. #355
* GH-345: Parameterize CMS to CMS[K] and decouple counting/querying from heavy hitters #354
* Add Array Monoid & Group. #356
* Improvements to Aggregator #359
* Improve require setup for depth/delta and associated test spec #361
* Bump from 2.11.2 to 2.11.4 #365
* Move to sbt 0.13.5 #364
* Correct wrong comment in estimation function #372
* Add increments to all Summers #373
* removed duplicate semigroup #375
* GH-381: Fix serialization errors when using new CMS implementation in Storm #382
* Fix snoble's name #384
* Lift methods for Aggregator and MonoidAggregator #380
* applyCumulative method on Aggregator #386
* Add Aggregator.zip #389
* GH-388: Fix CMS test issue caused by roundtripping depth->delta->depth #395
* GH-392: Improve hashing of BigInt #394
* add averageFrom to DecayedValue #391
* Freshen up Applicative instances a bit #387
* less noise on DecayedValue tests #405
* Preparer #400
* Upgrade bijection to 0.7.2 #406

### Version 0.8.0 ###
* Removes deprecated monoid: https://github.com/twitter/algebird/pull/342
* Use some value classes: https://github.com/twitter/algebird/pull/340
19 changes: 19 additions & 0 deletions COMMITTERS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Committers

Please see our [Project Governance](https://github.com/twitter/analytics-infra-governance) page for more details.

## Active

| Name | Handle |
|------------------------|-----------------------------------------------------------|
| Alex Levenson | [@isnotinvain](https://github.com/isnotinvain) |
| Ben Pence | [@benpence](https://github.com/benpence) |
| Ian O'Connell | [@ianoc](https://github.com/ianoc) |
| Joe Nievelt | [@jnievelt](https://github.com/jnievelt) |
| Oscar Boykin | [@johnynek](https://github.com/johnynek) |
| Pankaj Gupta | [@pankajroark](https://github.com/pankajroark) |
| Piyush Narang | [@piyushnarang](https://github.com/piyushnarang) |
| Sriram Krishnan | [@sriramkrishnan](https://github.com/sriramkrishnan) |
| Sam Ritchie | [@sritchie](https://github.com/sritchie) |

##Emeritus
Loading