-
-
Notifications
You must be signed in to change notification settings - Fork 77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vector summation DP aggregation #264
Comments
I can take a look at this one |
Sure, go ahead! Thanks! |
IIUC, the |
Yes, correct |
In order to use add_noise_vector, an object of AdditiveVectorNoiseParams needs to be created. AFAIK, CombinerParams should contain the attributes needed to populate For instance:
|
Good question, we need to introduce
|
I'm trying to include VectorSumCombiner in DPEngine.aggregate. But, I would need to understand whether it should be used with the CompoundCombiners class. Should this case be considered as a separete branch in create_compound_combiner, similar to what happens with the metric |
Context
DPEngine.aggregate performs DP aggregations of scalar values (sum, count, mean etc). A set of computed metrics is controlled with metrics field of
aggregate_params
argument.The result of this function is a collection of
(partition_key, named_tuple_with_requested_metrics)
Note: More details on the terminology is here.
Goals
Support of vector_sum in
DPEngine.aggregate
The goal is Implement full support of vector_sum in
DPEngine.aggregate
, i.e. the values to aggregate are arrays of the same size, and output is(partition_key, named_tuple["array_sum": sum_of_vectors_per_partition_key])
.References:
This task can be slit in 2 parts:
VectorSumCombiner
, which performs aggregationVectorSumCombiner
intoDPEngine.aggregate
Expose
vector_sum
computation to high-level Beam and Spark APIs.High-level Beam and Spark APIs are represented by PrivatePCollection and PrivateRDD classes and transformations on them. All DP computations are performed in
DPEngine
.PrivatePCollection
andPrivateRDD
keeps data in internal collection (PCollection
orRDD
correspondingly). They provide a guarantee, that only data that has been aggregated in a DP manner, using no more than the specified privacy budget can be extracted.Private Beam and Private Spark transformation are wrappers around
DPEngine.aggregate
. There are transformation forCOUNT
,MEAN
etc.variance
transformation can be used as a good example:The text was updated successfully, but these errors were encountered: