diff --git a/.gitignore b/.gitignore index e43b0f9..94d1697 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ .DS_Store +./src/*.o \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index ed64b3c..ee82a4e 100644 --- a/Dockerfile +++ b/Dockerfile @@ -6,11 +6,13 @@ RUN apt-get update \ make \ g++ \ libssl-dev \ + nano \ + tmux \ && rm -r /var/lib/apt/lists/* -RUN git clone https://github.com/snwagh/falcon-public.git Falcon - -RUN cd Falcon \ - && make all -j$(nproc) +RUN git clone https://github.com/trhieung/falcon-public Falcon WORKDIR Falcon + +RUN chmod +x /Falcon/entrypoint.sh +ENTRYPOINT ["/Falcon/entrypoint.sh"] \ No newline at end of file diff --git a/README.md b/README.md index af91145..4f02f6f 100755 --- a/README.md +++ b/README.md @@ -1,10 +1,22 @@ -# Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning +# Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning and Fixed Key Random Oracle -A maliciously secure framework for efficient 3-party protocols tailored for neural networks. This work builds off [SecureNN](https://github.com/snwagh/securenn-public), [ABY3](https://github.com/ladnir/aby3) and other prior works. This work is published in [Privacy Enhancing Technologies Symposium (PETS) 2021](https://petsymposium.org). Paper available [here](https://snwagh.github.io). If you're looking to run Neural Network training, strongly consider using this GPU-based codebase [Piranha](https://github.com/ucbrise/piranha). +This project is a fork of [Falcon framework](https://github.com/snwagh/falcon-public.git) implementation, integrating the setting of [the random oracle](https://github.com/anbaccar/RSS_ring_ppml/blob/master/include/randBit.h) as demonstrated in [RingPPML](https://github.com/anbaccar/RSS_ring_ppml.git). ### Table of Contents +- [Warning](#warning) +- [Requirements](#requirements) +- [Source Code](#source-code) + - [Repository Structure](#repository-structure) + - [Building the code](#building-the-code) + - [Running the code](#running-the-code) +- [Additional Resources](#additional-resources) + - [Comparison with RingPPML](#comparison-with-ringppml) + - [Citation](#citation) + + ### Warning --- -This codebase is released solely as a reference for other developers, as a proof-of-concept, and for benchmarking purposes. In particular, it has not had any security review, has a number of implementational TODOs, has a number of known bugs (especially in the malicious implementation), and thus, should be used at your own risk. You can contribute to this project by creating pull requests and submitting fixes and implementations. The code has not run end-to-end training and we expect this to require some parameter tuning, hence training and inference won't work out of the box (however, inference from pre-trained networks can be repreduced easily). +With a base warning displayed in the original repository of the [Falcon framework](https://github.com/snwagh/falcon-public.git), we note that this is the final project intended to support studying. ### Requirements --- @@ -35,78 +47,63 @@ This codebase is released solely as a reference for other developers, as a proof Install these packages with your favorite package manager, e.g, `sudo apt-get install `. +* For more details and up-to-date information, please refer to the original [falcon-framework](https://github.com/snwagh/falcon-public.git) ### Docker --- -To install and run Falcon using docker, first build the container: +To install and run Falcon using docker by downloading the [Dockerfile](https://github.com/trhieung/falcon-public/blob/master/Dockerfile) in this project, +- first build the container: ` docker build -t falcon .` -then run -`docker run -it falcon '/bin/bash'`. +- Then run the container +`docker run -it falcon '/bin/bash'` +- Use tmux and the following command combination `Ctrl+b %` to create 3 windows for demonstrating with 3 parties. +- Each window represents the ID of the respective parties. Run the following commands in each window to get the results -From the prompt, you can execute any of the commands specified in [Running the code](#running). +#### Party 0 +``` +./Falcon.out 0 files/IP_localhost files/keyA files/keyAB files/keyAC LeNet MNIST Semi-honest +``` +#### Party 1 +``` +./Falcon.out 1 files/IP_localhost files/keyB files/keyBC files/keyAB LeNet MNIST Semi-honest +``` +#### Party 2 +``` +./Falcon.out 2 files/IP_localhost files/keyC files/keyAC files/keyBC LeNet MNIST Semi-honest +``` ### Source Code --- #### Repository Structure -* `files/` - Shared keys, IP addresses and data files. -* `files/preload` - Contains data for pretrained network from SecureML. The other networks can be generated using `scripts` and functions in `secondary.cpp` -* `lib_eigen/` - [Eigen library](http://eigen.tuxfamily.org/) for faster matrix multiplication. -* `src/` - Source code. -* `util/` - Dependencies for AES randomness. -* `scripts/` - Contains python code to generate trained models for accuracy testing over a batch. -* The `god` script makes remote runs simpler (as well as the `makefile`) +* `src/Ranbit.h` and `src/Ranbit.cpp` - The additions we contributed #### Building the code To build Falcon, run the following commands: ``` -git clone https://github.com/snwagh/falcon-public.git Falcon +git clone https://github.com/trhieung/falcon-public Falcon cd Falcon make all -j$(nproc) ``` #### Running the code -To run the code, simply choose one of the following options: - -* `make`: Prints all the possible makefile options. -* `make terminal`: Runs the 3PC code on localhost with output from $P_0$ printed to standard output. -* `make file`: : Runs the 3PC code on localhost with output from $P_0$ printed to a file (in `output/3PC.txt`) -* `make valg`: Useful for debugging the code for set faults. Note that the -03 optimization flag needs to be suppressed (toggle lines 42, 43 in `makefile`) -* `make command`: Enables running a specific network, dataset, adversarial model, and run type (localhost/LAN/WAN) specified through the `makefile`. This takes precedence over choices in the `src/main.cpp` file. -* To run the code over tmux over multiple terminals, `make zero`, `make one`, and `make two` come in handy. -* Finally, the `makefile` (line 4-15) contains the descriptions of the arguments accepted by the executable. - +To run the code, please refer to the original [falcon-framework](https://github.com/snwagh/falcon-public.git) or follow the steps with [docker](#docker) for testing purposes ### Additional Resources --- -#### Run combinations -Note that given the size of the larger networks (AlexNet, VGG16) and the need to explicitly define network parameters, these networks can only be run for the CIFAR10 and Tiny ImageNet dataset. On the contrary, the smaller datasets (SecureML, Sarda, MiniONN, and LeNet) can only be run for the MNIST dataset. Running them otherwise should result in assertion errors. The following configuration was sufficient to produce the results for the larger networks: 2.9 GHz Intel Xeon E5-2666 v3 Processor, 36 cores, 60 GB RAM (in particular, a similar processor with 16 GB RAM was insifficient). - -#### Comparison with [SecureNN](https://github.com/snwagh/securenn-public) -While a bulk of the Falcon code builds on SecureNN, it differs in two important characterastics (1) Building on replicated secret sharing (RSS) (2) Modularity of the design. The latter enables each layer to self contained in forward and backward pass (in contrast to SecureNN where layers are merged for the networks to be tested). The functions are reasonably tested (including ReLU) however they are more tested for 32-bit datatype so the 64-bit might have minor bugs. - -#### Errors and Issues -If there are compile/installation/runtime errors, please create git issues. Some of the common errors and their resolutions are listed below: -* `cannot find` something error: This would be a linker error, make sure your code has access to the right paths in the `$PATH` environment variable. -* `Binding error`: There is probably an execution running in the background. Kill the process with `pkill Falcon.out` (this happens since the socket ports are hardcoded in the program and a number of the makefile commands run code in the background and `Ctrl+c` only kills the Party 0 code) -* `Bad alloc`: This is probably a memory issue. If you're running one of the larger networks (AlexNet, VGG16) on a laptop, try running them on a server/cluster/machine with more memory. - - -#### Todos/Bug todos -* Remove size argument from all functions (generate it inside functions) -* Clean-up tools and functionalities file -- move reconstruction functions to tools -* Pointers to layer configurations are never deleted --> needs to be fixed -* Precompute implementation -* Incorrect communication numbers for parallel implememntations -* ... +#### Comparison with [RingPPML](https://github.com/anbaccar/RSS_ring_ppml.git) +| Element | Falcon | RingPPML | +| :---: | :---: | :---: | +| Data Transfer | Not supported: encrypt before sending or decrypt when receiving data
(as this is just for benchmarking) | Supported: AES encryption and decryption with block size, [implementation](https://github.com/anbaccar/RSS_ring_ppml/blob/7b9483e203b8f9547532cef6cd446d40e9947a7b/src/connection/NodeNetwork.cpp#L399) | +| Computation Base | some protocol base on field modulo, [implentation](https://github.com/snwagh/falcon-public/blob/master/src/tools.h#L193) | All computation on Ring $Z_{2^k}$, speeding up the computation cost of modulo | #### Citation -You can cite the paper using the following bibtex entry (the paper links to this repo): +You can cite the paper using the following bibtex entry (the paper links to this origin repo): ``` @inproceedings{wagh2021falcon, title={{FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning}}, @@ -115,7 +112,15 @@ You can cite the paper using the following bibtex entry (the paper links to this year={2021} } ``` - ---- -For questions, please create git issues; for eventual replies, you can also reach out to [swagh@alumni.princeton.edu](swagh@alumni.princeton.edu) - +or cite RingPPML paper with by +``` +@article{baccarini2023rss, + title={Multi-Party Replicated Secret Sharing over a Ring with Applications to Privacy-Preserving Machine Learning}, + author={Baccarini, Alessandro and Blanton, Marina and Yuan, Chen}, + journal={Proceedings on Privacy Enhancing Technologies (PoPETs)}, + volume = 2023, + number = 1, + pages={608-626}, + year={2023} +} +``` diff --git a/entrypoint.sh b/entrypoint.sh new file mode 100755 index 0000000..92cc543 --- /dev/null +++ b/entrypoint.sh @@ -0,0 +1,7 @@ +#!/bin/bash +# RandBit +cd /Falcon +make all -j$(nproc) + +# Execute any additional command you want after the build, for example, running a bash shell +exec "$@" diff --git a/src/AESObject.o b/src/AESObject.o new file mode 100644 index 0000000..3631b1d Binary files /dev/null and b/src/AESObject.o differ diff --git a/src/Functionalities.cpp b/src/Functionalities.cpp index 402baa1..ac15bbe 100755 --- a/src/Functionalities.cpp +++ b/src/Functionalities.cpp @@ -1631,6 +1631,39 @@ void funcMaxpool(RSSVectorMyType &a, RSSVectorMyType &max, RSSVectorSmallType &m computation. So it might be the case that the approximations introduced by negative and positive numbers in effect cancel out to preserve overall NN accuracy. */ + + +void debugRandBit(){ + struct timeval start; + struct timeval end; + unsigned long timer; + + size_t size; + string network = "Debug randbit"; + + for (int j = 0; j < 4; j++){ + RSSVectorMyType b; + vector b_reconst; + + size = (1 << 10) * pow(10, j); + for (int i = 0; i < size; i++){ + b.push_back(std::make_pair(0, 0)); + b_reconst.push_back(0); + } + + gettimeofday(&start, NULL); // start timer here + Rss_RandBit(b, size); + gettimeofday(&end, NULL); // stop timer here + timer = 1e6 * (end.tv_sec - start.tv_sec) + end.tv_usec - start.tv_usec; + printf("randbit: \t%.6lf ms\n", (double)(timer * 0.001)); + + #if (!LOG_DEBUG) + funcReconstruct(b, b_reconst, size, "b", true); + #endif + } + +} + void debugMatMul() { // size_t rows = 1000; diff --git a/src/Functionalities.h b/src/Functionalities.h index 0d4a006..906615e 100755 --- a/src/Functionalities.h +++ b/src/Functionalities.h @@ -60,6 +60,8 @@ void debugSSBits(); void debugSS(); void debugMaxpool(); +// custom debug +void debugRandBit(); //Test void testMatMul(size_t rows, size_t common_dim, size_t columns, size_t iter); diff --git a/src/basicSockets.o b/src/basicSockets.o new file mode 100644 index 0000000..1f33a75 Binary files /dev/null and b/src/basicSockets.o differ diff --git a/src/connect.o b/src/connect.o new file mode 100644 index 0000000..98e5ab5 Binary files /dev/null and b/src/connect.o differ diff --git a/src/globals.h b/src/globals.h index 44970b7..14d2675 100755 --- a/src/globals.h +++ b/src/globals.h @@ -18,7 +18,7 @@ /********************* AES and other globals *********************/ -#define LOG_DEBUG false +#define LOG_DEBUG true #define LOG_DEBUG_NETWORK false #define FUNCTION_TIME false #define RANDOM_COMPUTE 256 //Size of buffer for random elements @@ -64,4 +64,8 @@ const myType LARGEST_NEG = ((myType)1 << (BIT_SIZE - 1)); const myType MINUS_ONE = (myType)-1; const smallType BOUNDARY = (256/PRIME_NUMBER) * PRIME_NUMBER; +/********************* Typedefs for rings *********************/ +typedef unsigned long Lint; +typedef long long int sLint; + #endif diff --git a/src/main.cpp b/src/main.cpp index 7c54449..ed9f0d7 100755 --- a/src/main.cpp +++ b/src/main.cpp @@ -8,28 +8,30 @@ #include "NeuralNetwork.h" #include "unitTests.h" - int partyNum; -AESObject* aes_indep; -AESObject* aes_next; -AESObject* aes_prev; +AESObject *aes_indep; +AESObject *aes_next; +AESObject *aes_prev; Precompute PrecomputeObject; - -int main(int argc, char** argv) +int main(int argc, char **argv) { -/****************************** PREPROCESSING ******************************/ + /****************************** PREPROCESSING ******************************/ parseInputs(argc, argv); - NeuralNetConfig* config = new NeuralNetConfig(NUM_ITERATIONS); + NeuralNetConfig *config = new NeuralNetConfig(NUM_ITERATIONS); string network, dataset, security; bool PRELOADING = false; -/****************************** SELECT NETWORK ******************************/ - //Network {SecureML, Sarda, MiniONN, LeNet, AlexNet, and VGG16} - //Dataset {MNIST, CIFAR10, and ImageNet} - //Security {Semi-honest or Malicious} + /****************************** SELECT NETWORK ******************************/ + // Network {SecureML, Sarda, MiniONN, LeNet, AlexNet, and VGG16} + // Dataset {MNIST, CIFAR10, and ImageNet} + // Security {Semi-honest or Malicious} if (argc == 9) - {network = argv[6]; dataset = argv[7]; security = argv[8];} + { + network = argv[6]; + dataset = argv[7]; + security = argv[8]; + } else { network = "SecureML"; @@ -38,9 +40,9 @@ int main(int argc, char** argv) } selectNetwork(network, dataset, security, config); config->checkNetwork(); - NeuralNetwork* net = new NeuralNetwork(config); + NeuralNetwork *net = new NeuralNetwork(config); -/****************************** AES SETUP and SYNC ******************************/ + /****************************** AES SETUP and SYNC ******************************/ aes_indep = new AESObject(argv[3]); aes_next = new AESObject(argv[4]); aes_prev = new AESObject(argv[5]); @@ -48,18 +50,19 @@ int main(int argc, char** argv) initializeCommunication(argv[2], partyNum); synchronize(2000000); -/****************************** RUN NETWORK/UNIT TESTS ******************************/ - //Run these if you want a preloaded network to be tested - //assert(NUM_ITERATION == 1 and "check if readMiniBatch is false in test(net)") - //First argument {SecureML, Sarda, MiniONN, or LeNet} - // network += " preloaded"; PRELOADING = true; - // preload_network(PRELOADING, network, net); + /****************************** RUN NETWORK/UNIT TESTS ******************************/ + // Run these if you want a preloaded network to be tested + // assert(NUM_ITERATION == 1 and "check if readMiniBatch is false in test(net)") + // First argument {SecureML, Sarda, MiniONN, or LeNet} + // network += " preloaded"; PRELOADING = true; + // preload_network(PRELOADING, network, net); - start_m(); - //Run unit tests in two modes: + // start_m(); + // Run unit tests in two modes: // 1. Debug {Mat-Mul, DotProd, PC, Wrap, ReLUPrime, ReLU, Division, BN, SSBits, SS, and Maxpool} // 2. Test {Mat-Mul1, Mat-Mul2, Mat-Mul3 (and similarly) Conv*, ReLU*, ReLUPrime*, and Maxpool*} where * = {1,2,3} - // runTest("Debug", "BN", network); + // runTest("Debug", "RandBit", network); + // runTest("Debug", "MultPub", network); // runTest("Test", "ReLUPrime1", network); // Run forward/backward for single layers @@ -69,24 +72,24 @@ int main(int argc, char** argv) // string what = "F"; // runOnly(net, l, what, network); - //Run training + // Run training network += " train"; train(net); - //Run inference (possibly with preloading a network) - // network += " test"; - // test(PRELOADING, network, net); + // Run inference (possibly with preloading a network) + // network += " test"; + // test(PRELOADING, network, net); - end_m(network); - cout << "----------------------------------------------" << endl; - cout << "Run details: " << NUM_OF_PARTIES << "PC (P" << partyNum - << "), " << NUM_ITERATIONS << " iterations, batch size " << MINI_BATCH_SIZE << endl - << "Running " << security << " " << network << " on " << dataset << " dataset" << endl; - cout << "----------------------------------------------" << endl << endl; + // end_m(network); + // cout << "----------------------------------------------" << endl; + // cout << "Run details: " << NUM_OF_PARTIES << "PC (P" << partyNum + // << "), " << NUM_ITERATIONS << " iterations, batch size " << MINI_BATCH_SIZE << endl + // << "Running " << security << " " << network << " on " << dataset << " dataset" << endl; + // cout << "----------------------------------------------" << endl << endl; - printNetwork(net); + // printNetwork(net); -/****************************** CLEAN-UP ******************************/ + /****************************** CLEAN-UP ******************************/ delete aes_indep; delete aes_next; delete aes_prev; @@ -96,7 +99,3 @@ int main(int argc, char** argv) return 0; } - - - - diff --git a/src/main.o b/src/main.o new file mode 100644 index 0000000..3da7d6d Binary files /dev/null and b/src/main.o differ diff --git a/src/randBit.cpp b/src/randBit.cpp new file mode 100644 index 0000000..7cbb4f6 --- /dev/null +++ b/src/randBit.cpp @@ -0,0 +1,360 @@ +/* + Multi-Party Replicated Secret Sharing over a Ring + ** Copyright (C) 2022 Alessandro Baccarini, Marina Blanton, and Chen Yuan + ** Department of Computer Science and Engineering, University of Buffalo (SUNY) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +*/ + +#include "randBit.h" + + +void prg_aes(uint8_t *dest, uint8_t *src, __m128i *ri) { + __m128i rr, mr; + __m128i *r = ri; + + rr = _mm_loadu_si128((__m128i *)src); + mr = rr; + + mr = _mm_xor_si128(mr, r[0]); + + mr = _mm_aesenc_si128(mr, r[1]); + mr = _mm_aesenc_si128(mr, r[2]); + mr = _mm_aesenc_si128(mr, r[3]); + mr = _mm_aesenc_si128(mr, r[4]); + mr = _mm_aesenc_si128(mr, r[5]); + mr = _mm_aesenc_si128(mr, r[6]); + mr = _mm_aesenc_si128(mr, r[7]); + mr = _mm_aesenc_si128(mr, r[8]); + mr = _mm_aesenc_si128(mr, r[9]); + mr = _mm_aesenclast_si128(mr, r[10]); + mr = _mm_xor_si128(mr, rr); + _mm_storeu_si128((__m128i *)dest, mr); +} + +__m128i* prg_keyschedule(uint8_t *src) { + __m128i *r = (__m128i *)malloc(11 * sizeof(__m128i)); + + r[0] = _mm_load_si128((__m128i *)src); + + KE2(r[1], r[0], 0x01) + KE2(r[2], r[1], 0x02) + KE2(r[3], r[2], 0x04) + KE2(r[4], r[3], 0x08) + KE2(r[5], r[4], 0x10) + KE2(r[6], r[5], 0x20) + KE2(r[7], r[6], 0x40) + KE2(r[8], r[7], 0x80) + KE2(r[9], r[8], 0x1b) + KE2(r[10], r[9], 0x36) + + return r; +} + +void prg_getrandom(int keyID, uint size, uint length, uint8_t *dest) { + // we assume container_size is 16, so all *container_size are replaced as <<4 + // this size means how many random bytes we need + // uint8_t *buffer = new uint8_t [size]; + // its always size * length + // printf("curent P is %d \n",P_container[keyID]); + + // setup + uint8_t random_container[3][16]; + + uint8_t RandomData[64] = {'1', '2', '3', '4', '1', '2', '3', '4', + '1', '2', '3', '4', '1', '2', '3', '4', + '1', '2', '3', '4', '1', '2', '3', '4', + '1', '2', '3', '4', '1', '2', '3', '4', + '4', '3', '2', '1', '4', '3', '2', '1', + '4', '3', '2', '1', '4', '3', '2', '1', + '4', '3', '2', '1', '4', '3', '2', '1', + '4', '3', '2', '1', '4', '3', '2', '1'}; + uint8_t tempKey_A[16]; + uint8_t tempKey_B[16]; + uint8_t tempKey_C[16]; + + memcpy(random_container[0], RandomData, 16); + memcpy(tempKey_A, RandomData + 16, 16); + memcpy(tempKey_C, RandomData + 32, 16); + memcpy(random_container[2], RandomData + 48, 16); + + // int pid = partyNum; + // int map[2]; + // switch (pid) { + // case 0: + // map[0] = 2; + // map[1] = 1; + // break; + // case 1: + // map[0] = 0; + // map[1] = 2; + // break; + // case 2: + // map[0] = 1; + // map[1] = 0; + // break; + // } + + // sendDataToPeer(map[0], 32, RandomData); + // getDataFromPeer(map[1], 32, RandomData); + + memcpy(random_container[1], RandomData, 16); + memcpy(tempKey_B, RandomData + 16, 16); + + + __m128i prg_key[3]; + prg_key[0] = *prg_keyschedule(tempKey_A); + prg_key[1] = *prg_keyschedule(tempKey_B); + prg_key[2] = *prg_keyschedule(tempKey_C); + + uint8_t res[16] = {}; + for (size_t i = 0; i < 3; i++) { + prg_aes(res, random_container[i], &prg_key[i]); + memcpy(&random_container[i], res, 16); + } + + int P_container[3]= {0, 0, 0}; + int container_size = 16; + + uint rounds = ((size * length - container_size + P_container[keyID]) + 15) >> 4; + // printf("rounds %u\n", rounds); + if (rounds == 0) { + memcpy(dest, random_container[keyID] + P_container[keyID], size * length); + P_container[keyID] = P_container[keyID] + size * length; + } else { + memcpy(dest, &random_container[keyID] + P_container[keyID], container_size - P_container[keyID]); + if (rounds >= 2) { + prg_aes(dest + (container_size - P_container[keyID]), random_container[keyID], &prg_key[keyID]); + for (int i = 1; i < rounds - 1; i++) { + // segfault in this loop for "large" size + // printf("i : %u\n", i); + prg_aes(dest + (container_size - P_container[keyID]) + (i << 4), dest + (container_size - P_container[keyID]) + ((i - 1) << 4), &prg_key[keyID]); + } + prg_aes(random_container[keyID], dest + (container_size - P_container[keyID]) + ((rounds - 2) << 4), &prg_key[keyID]); + memcpy(dest + container_size - P_container[keyID] + ((rounds - 1) << 4), &random_container[keyID], size * length - ((rounds - 1) << 4) - container_size + P_container[keyID]); + P_container[keyID] = size * length - ((rounds - 1) << 4) - container_size + P_container[keyID]; + } else { + prg_aes(random_container[keyID], random_container[keyID], &prg_key[keyID]); + memcpy(dest + container_size - P_container[keyID], &random_container[keyID], size * length - container_size + P_container[keyID]); + P_container[keyID] = size * length - container_size + P_container[keyID]; + } + } + + // delete [] buffer; +} + +void prg_getrandom(uint size, uint length, uint8_t *dest) { + prg_getrandom(1, size, length, dest); +} + +// void Rss_RandBit(RSSVectorSmallType &b, uint size, uint ring_size) { + +// int pid = partyNum; +// uint i; +// uint bytes = (ring_size + 9) >> 3; +// // uint bytes = 1; + +// uint numShares = NUM_OF_PARTIES-1; + +// vector u0(size, 0); +// vector u1(size, 0); + +// // RSSVectorSmallType _a(size, std::make_pair(0, 0)); +// RSSVectorMyType _a(size, std::make_pair(0, 0)); +// RSSVectorSmallType _d(size, std::make_pair(0, 0)); + +// uint8_t *buffer = new uint8_t[bytes * size]; + +// // RSSVectorSmallType _f(size, std::make_pair(0, 0)); +// RSSVectorMyType _f(size, std::make_pair(0, 0)); +// // vector _e(size, 0); +// // vector _c(size, 0); +// vector _e(size, 0); +// vector _c(size, 0); + +// smallType ai[3]; +// memset(ai, 0, sizeof(smallType) * numShares); +// if (pid == 0) { +// ai[0] = 1; +// ai[1] = 0; +// } else if (pid == 1) { +// ai[0] = 0; +// ai[1] = 0; +// } else { //(pid == 2) +// ai[0] = 0; +// ai[1] = 1; +// } + +// prg_getrandom(0, bytes, size, buffer); +// std::copy(buffer, buffer + bytes * size, u0.begin()); + +// prg_getrandom(1, bytes, size, buffer); +// std::copy(buffer, buffer + bytes * size, u1.begin()); + +// for (i = 0; i < size; i++) { +// _a[i].first = (u0[i] << smallType(1)) + ai[0]; +// _a[i].second = (u1[i] << smallType(1)) + ai[1]; + +// // _a[i].first = (1 << (ring_size)) - 1; +// // _a[i].second = (1 << (ring_size)) - 1; +// } +// // squaring a +// funcDotProduct(_a, _a, _f, size, 0, 0); +// funcReconstruct(_f, _e, size, "e", true); +// rss_sqrt_inv(_c, _e, size, ring_size + 2); + +// // effectively combines the two loops into one, eliminates d variable +// for (i = 0; i < size; i++) { +// b[i].first = (_c[i] * _a[i].first + ai[0]) >> smallType(1); +// b[i].second = (_c[i] * _a[i].second + ai[1]) >> smallType(1); +// } + +// // freeing up + +// delete[] buffer; + +// } + +void Rss_RandBit(RSSVectorMyType &b, uint size, uint ring_size) { + + // int pid = nodeNet->getID(); + int pid = partyNum; + uint i; + uint bytes = (ring_size + 9) >> 3; + // printf("bytes : %llu\n", bytes ); + // uint numShares = nodeNet->getNumShares(); + uint numShares = NUM_OF_PARTIES-1; + + Lint **u = new Lint *[numShares]; + Lint **a = new Lint *[numShares]; + Lint **d = new Lint *[numShares]; + + for (i = 0; i < numShares; i++) { + u[i] = new Lint[size]; + a[i] = new Lint[size]; + d[i] = new Lint[size]; + } + Lint *e = new Lint[size]; + Lint *c = new Lint[size]; + uint8_t *buffer = new uint8_t[bytes * size]; + + // falcon init + RSSVectorMyType _a(size, std::make_pair(0, 0)); + RSSVectorMyType _f(size, std::make_pair(0, 0)); + // vector _f(size, myType(0)); + vector _e(size, myType(0)); + vector _c(size, myType(0)); + + Lint *ai = new Lint[numShares]; + memset(ai, 0, sizeof(Lint) * numShares); + if (pid == 0) { + ai[0] = 1; + } else if (pid == 2) { + ai[numShares - 1] = 1; + } + + // nodeNet->prg_getrandom(0, bytes, size, buffer); + prg_getrandom(0, bytes, size, buffer); + for (i = 0; i < size; i++) { + memcpy(u[0] + i, buffer + i * bytes, bytes); + } + // nodeNet->prg_getrandom(1, bytes, size, buffer); + prg_getrandom(1, bytes, size, buffer); + for (i = 0; i < size; i++) { + memcpy(u[1] + i, buffer + i * bytes, bytes); + } + + for (i = 0; i < size; i++) { + // // ensuring [a] is odd + // for (size_t s = 0; s < numShares; s++) + // a[s][i] = (u[s][i] << Lint(1)) + ai[s]; + // // a[1][i] = (u[1][i] << Lint(1)) + a2; + _a[i].first = (u[0][i] << myType(1)) + ai[0]; + _a[i].second = (u[1][i] << myType(1)) + ai[1]; + + } + // squaring a + // Rss_MultPub(e, a, a, size, ring_size + 2, nodeNet); // ringsize+2 + + // rss_sqrt_inv(c, e, size, ring_size + 2); + + funcDotProduct(_a, _a, _f, size, 0, 0); + funcReconstruct(_f, _e, size, "e", false); + // printf("mult:\n"); + // for (i = 0; i < size; i++) { + // _e[i] &= ((1 << (ring_size+2)) - 1); + // printf("%u, ", _e[i]); + // } + // printf("\n"); + rss_sqrt_inv(_c, _e, size, ring_size + 2); + // printf("inverse:\n"); + // for (i = 0; i < size; i++) { + // _e[i] &= ((1 << (ring_size+2)) - 1); + // printf("%u, ", _c[i]); + // } + // printf("\n"); + + // effectively combines the two loops into one, eliminates d variable + for (i = 0; i < size; i++) { + // for (size_t s = 0; s < numShares; s++) + // b[s][i] = (c[i] * a[s][i] + ai[s]) >> Lint(1); + // b[1][i] = (c[i] * a[1][i] + a2) >> (1); + b[i].first = (_c[i] * _a[i].first + myType(ai[0])) >> myType(1); + b[i].second = (_c[i] * _a[i].second + myType(ai[1])) >> myType(1); + + } + + // freeing up + delete[] c; + delete[] buffer; + delete[] e; + for (i = 0; i < numShares; i++) { + delete[] d[i]; + delete[] a[i]; + delete[] u[i]; + } + delete[] d; + delete[] a; + delete[] ai; + delete[] u; +} + + +void rss_sqrt_inv(vector &c, vector &e, uint size, uint ring_size) { + + Lint c1, c2, temp, d_; + uint i, j; + + for (i = 0; i < size; i++) { + c1 = Lint(1); + c2 = Lint(1); + d_ = Lint(4); // 100 - the first mask + + for (j = 2; j < ring_size - 1; j++) { + temp = e[i] - (c1) * (c1); + if (temp != Lint(0)) { + // get the jth+1 bit of temp, place it in jth position, and add to c1 + c1 += (temp & (d_ << Lint(1))) >> Lint(1); + } + + temp = Lint(1) - c1 * c2; + // get the jth bit of temp and add it to c2 + c2 += temp & d_; + d_ = d_ << Lint(1); + } + // last round for the inv portion + temp = Lint(1) - c1 * c2; + c[i] = c2 + (temp & d_); + } +} diff --git a/src/randBit.h b/src/randBit.h new file mode 100644 index 0000000..6f18fcb --- /dev/null +++ b/src/randBit.h @@ -0,0 +1,85 @@ +/* + Multi-Party Replicated Secret Sharing over a Ring + ** Copyright (C) 2022 Alessandro Baccarini, Marina Blanton, and Chen Yuan + ** Department of Computer Science and Engineering, University of Buffalo (SUNY) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +*/ + +#ifndef RANDBIT_H_ +#define RANDBIT_H_ + +// #include "NodeNetwork.h" +// #include "Mult.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include //for int8_t +#include +#include //for memcmp +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include //for intrinsics for AES-NI +#include + +#include "globals.h" +#include "connect.h" +#include "Functionalities.h" + +#define KE2(NK, OK, RND) \ + NK = OK; \ + NK = _mm_xor_si128(NK, _mm_slli_si128(NK, 4)); \ + NK = _mm_xor_si128(NK, _mm_slli_si128(NK, 4)); \ + NK = _mm_xor_si128(NK, _mm_slli_si128(NK, 4)); \ + NK = _mm_xor_si128(NK, _mm_shuffle_epi32(_mm_aeskeygenassist_si128(OK, RND), 0xff)); + +#define ANSI_COLOR_RED "\x1b[31m" +#define ANSI_COLOR_RESET "\x1b[0m" +#define ANSI_COLOR_GREEN "\x1b[32m" +#define ANSI_COLOR_YELLOW "\033[33m" + +void prg_aes(uint8_t *dest, uint8_t *src, __m128i *ri); +__m128i* prg_keyschedule(uint8_t *src); + +void prg_getrandom(int keyID, uint size, uint length, uint8_t *dest); +void prg_getrandom(uint size, uint length, uint8_t *dest); + +void Rss_RandBit(RSSVectorMyType &b, uint size, uint ring_size = 8); +// void Rss_RandBit(const RSSVectorMyType &b, uint size, uint ring_size = 32); + +void rss_sqrt_inv(vector &c, vector &e, uint size, uint ring_size = 8); + +#endif \ No newline at end of file diff --git a/src/secondary.cpp b/src/secondary.cpp index b2d538f..1fb7519 100755 --- a/src/secondary.cpp +++ b/src/secondary.cpp @@ -1,16 +1,16 @@ -#include "connect.h" +#include "connect.h" #include "secondary.h" extern CommunicationObject commObject; extern int partyNum; -extern string * addrs; -extern BmrNet ** communicationSenders; -extern BmrNet ** communicationReceivers; +extern string *addrs; +extern BmrNet **communicationSenders; +extern BmrNet **communicationReceivers; extern void log_print(string str); #define NANOSECONDS_PER_SEC 1E9 -//For time measurements +// For time measurements clock_t tStart; struct timespec requestStart, requestEnd; bool alreadyMeasuringTime = false; @@ -18,7 +18,7 @@ int roundComplexitySend = 0; int roundComplexityRecv = 0; bool alreadyMeasuringRounds = false; -//For faster modular operations +// For faster modular operations extern smallType additionModPrime[PRIME_NUMBER][PRIME_NUMBER]; extern smallType subtractModPrime[PRIME_NUMBER][PRIME_NUMBER]; extern smallType multiplicationModPrime[PRIME_NUMBER][PRIME_NUMBER]; @@ -43,9 +43,9 @@ extern void print_linear(myType var, string type); extern void funcReconstruct(const RSSVectorMyType &a, vector &b, size_t size, string str, bool print); /******************* Main train and test functions *******************/ -void parseInputs(int argc, char* argv[]) -{ - if (argc < 6) +void parseInputs(int argc, char *argv[]) +{ + if (argc < 6) print_usage(argv[0]); partyNum = atoi(argv[1]); @@ -55,34 +55,74 @@ void parseInputs(int argc, char* argv[]) { additionModPrime[i][j] = ((i + j) % PRIME_NUMBER); subtractModPrime[i][j] = ((PRIME_NUMBER + i - j) % PRIME_NUMBER); - multiplicationModPrime[i][j] = ((i * j) % PRIME_NUMBER); //How come you give the right answer multiplying in 8-bits?? + multiplicationModPrime[i][j] = ((i * j) % PRIME_NUMBER); // How come you give the right answer multiplying in 8-bits?? } } -void train(NeuralNetwork* net) +void train(NeuralNetwork *net) { log_print("train"); - - for (int i = 0; i < NUM_ITERATIONS; ++i) + // mystite + // int myiter = ((int)(NO_OF_EPOCHS * TRAINING_DATA_SIZE / MINI_BATCH_SIZE)); + int myiter = 50; + // for (int i = 0; i < NUM_ITERATIONS; ++i) + for (int i = 0; i < myiter; ++i) { - // cout << "----------------------------------" << endl; - // cout << "Iteration " << i << endl; - readMiniBatch(net, "TRAINING"); - net->forward(); - net->backward(); - // cout << "----------------------------------" << endl; + // counter[0]: Correct samples, counter[1]: total samples + vector counter(2, 0); + RSSVectorMyType maxIndex(MINI_BATCH_SIZE); + size_t s = trainData.size(); + size_t t = trainLabels.size(); + size_t batchIdx = 0; + trainDataBatchCounter = 0; + + while (trainDataBatchCounter < s) + { + batchIdx = trainDataBatchCounter / (INPUT_SIZE * MINI_BATCH_SIZE); + cout << "----------------------------------" << endl; + cout << "Iteration " << i + 1 << "/" << myiter << endl; + cout << "Bactch index: " << batchIdx << endl; + readMiniBatch(net, "TRAINING"); + net->forward(); + // // mysite + // string path = "./prediction" + to_string(batchIdx) + ".txt"; + // ofstream f_prediction(path); + // int idx = 0; + // if (partyNum == PARTY_A) + // { + // for (int i = 0; i < MINI_BATCH_SIZE; i++) + // { + // for (int j = 0; j < LAST_LAYER_SIZE; j++) + // { + // idx = i * LAST_LAYER_SIZE + j; + // // cout << (int)(*(net->layers[NUM_LAYERS - 1])->getActivation())[idx].first << "\t"; + // f_prediction << myTypeToFloat((*(net->layers[NUM_LAYERS - 1])->getActivation())[idx].first) << "\t"; + // } + // // cout << endl; + // f_prediction << endl; + // } + // f_prediction << endl; + // } + // f_prediction.close(); + // // return; + + // get weights + // + net->backward(); + // net->getAccuracy(net->inputData, counter); + // cout << "----------------------------------" << endl; + } } } - extern void print_vector(RSSVectorMyType &var, string type, string pre_text, int print_nos); extern string which_network(string network); -void test(bool PRELOADING, string network, NeuralNetwork* net) +void test(bool PRELOADING, string network, NeuralNetwork *net) { log_print("test"); - //counter[0]: Correct samples, counter[1]: total samples - vector counter(2,0); + // counter[0]: Correct samples, counter[1]: total samples + vector counter(2, 0); RSSVectorMyType maxIndex(MINI_BATCH_SIZE); for (int i = 0; i < NUM_ITERATIONS; ++i) @@ -94,184 +134,193 @@ void test(bool PRELOADING, string network, NeuralNetwork* net) // net->predict(maxIndex); // net->getAccuracy(maxIndex, counter); } - print_vector((*(net->layers[NUM_LAYERS-1])->getActivation()), "FLOAT", "MPC Output over uint32_t:", 1280); + print_vector((*(net->layers[NUM_LAYERS - 1])->getActivation()), "FLOAT", "MPC Output over uint32_t:", 1280); // Write output to file if (PRELOADING) { ofstream data_file; - data_file.open("files/preload/"+which_network(network)+"/"+which_network(network)+".txt"); - + data_file.open("files/preload/" + which_network(network) + "/" + which_network(network) + ".txt"); + vector b(MINI_BATCH_SIZE * LAST_LAYER_SIZE); - funcReconstruct((*(net->layers[NUM_LAYERS-1])->getActivation()), b, MINI_BATCH_SIZE * LAST_LAYER_SIZE, "anything", false); + funcReconstruct((*(net->layers[NUM_LAYERS - 1])->getActivation()), b, MINI_BATCH_SIZE * LAST_LAYER_SIZE, "anything", false); for (int i = 0; i < MINI_BATCH_SIZE; ++i) { for (int j = 0; j < LAST_LAYER_SIZE; ++j) - data_file << b[i*(LAST_LAYER_SIZE) + j] << " "; + data_file << b[i * (LAST_LAYER_SIZE) + j] << " "; data_file << endl; } } } - // Generate a file with 0's of appropriate size void generate_zeros(string name, size_t number, string network) { - string default_path = "files/preload/"+which_network(network)+"/"; + string default_path = "files/preload/" + which_network(network) + "/"; ofstream data_file; - data_file.open(default_path+name); + data_file.open(default_path + name); for (int i = 0; i < number; ++i) data_file << (int)0 << " "; } - extern size_t nextParty(size_t party); #include "FCLayer.h" #include "CNNLayer.h" -void preload_network(bool PRELOADING, string network, NeuralNetwork* net) +void preload_network(bool PRELOADING, string network, NeuralNetwork *net) { log_print("preload_network"); assert((PRELOADING) and (NUM_ITERATIONS == 1) and (MINI_BATCH_SIZE == 128) && "Preloading conditions fail"); float temp_next = 0, temp_prev = 0; - string default_path = "files/preload/"+which_network(network)+"/"; - //Set to true if you want the zeros files generated. + string default_path = "files/preload/" + which_network(network) + "/"; + // Set to true if you want the zeros files generated. bool ZEROS = false; if (which_network(network).compare("SecureML") == 0) { string temp = "SecureML"; /************************** Input **********************************/ - string path_input_1 = default_path+"input_"+to_string(partyNum); - string path_input_2 = default_path+"input_"+to_string(nextParty(partyNum)); + string path_input_1 = default_path + "input_" + to_string(partyNum); + string path_input_2 = default_path + "input_" + to_string(nextParty(partyNum)); ifstream f_input_1(path_input_1), f_input_2(path_input_2); for (int i = 0; i < INPUT_SIZE * MINI_BATCH_SIZE; ++i) { - f_input_1 >> temp_next; f_input_2 >> temp_prev; + f_input_1 >> temp_next; + f_input_2 >> temp_prev; net->inputData[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_input_1.close(); f_input_2.close(); + f_input_1.close(); + f_input_2.close(); if (ZEROS) { - generate_zeros("input_1", 784*128, temp); - generate_zeros("input_2", 784*128, temp); + generate_zeros("input_1", 784 * 128, temp); + generate_zeros("input_2", 784 * 128, temp); } // print_vector(net->inputData, "FLOAT", "inputData:", 784); /************************** Weight1 **********************************/ - string path_weight1_1 = default_path+"weight1_"+to_string(partyNum); - string path_weight1_2 = default_path+"weight1_"+to_string(nextParty(partyNum)); + string path_weight1_1 = default_path + "weight1_" + to_string(partyNum); + string path_weight1_2 = default_path + "weight1_" + to_string(nextParty(partyNum)); ifstream f_weight1_1(path_weight1_1), f_weight1_2(path_weight1_2); for (int column = 0; column < 128; ++column) { for (int row = 0; row < 784; ++row) { - f_weight1_1 >> temp_next; f_weight1_2 >> temp_prev; - (*((FCLayer*)net->layers[0])->getWeights())[128*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight1_1 >> temp_next; + f_weight1_2 >> temp_prev; + (*((FCLayer *)net->layers[0])->getWeights())[128 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight1_1.close(); f_weight1_2.close(); + f_weight1_1.close(); + f_weight1_2.close(); if (ZEROS) { - generate_zeros("weight1_1", 784*128, temp); - generate_zeros("weight1_2", 784*128, temp); + generate_zeros("weight1_1", 784 * 128, temp); + generate_zeros("weight1_2", 784 * 128, temp); } /************************** Weight2 **********************************/ - string path_weight2_1 = default_path+"weight2_"+to_string(partyNum); - string path_weight2_2 = default_path+"weight2_"+to_string(nextParty(partyNum)); + string path_weight2_1 = default_path + "weight2_" + to_string(partyNum); + string path_weight2_2 = default_path + "weight2_" + to_string(nextParty(partyNum)); ifstream f_weight2_1(path_weight2_1), f_weight2_2(path_weight2_2); for (int column = 0; column < 128; ++column) { for (int row = 0; row < 128; ++row) { - f_weight2_1 >> temp_next; f_weight2_2 >> temp_prev; - (*((FCLayer*)net->layers[2])->getWeights())[128*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight2_1 >> temp_next; + f_weight2_2 >> temp_prev; + (*((FCLayer *)net->layers[2])->getWeights())[128 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight2_1.close(); f_weight2_2.close(); + f_weight2_1.close(); + f_weight2_2.close(); if (ZEROS) { - generate_zeros("weight2_1", 128*128, temp); - generate_zeros("weight2_2", 128*128, temp); + generate_zeros("weight2_1", 128 * 128, temp); + generate_zeros("weight2_2", 128 * 128, temp); } /************************** Weight3 **********************************/ - string path_weight3_1 = default_path+"weight3_"+to_string(partyNum); - string path_weight3_2 = default_path+"weight3_"+to_string(nextParty(partyNum)); + string path_weight3_1 = default_path + "weight3_" + to_string(partyNum); + string path_weight3_2 = default_path + "weight3_" + to_string(nextParty(partyNum)); ifstream f_weight3_1(path_weight3_1), f_weight3_2(path_weight3_2); for (int column = 0; column < 10; ++column) { for (int row = 0; row < 128; ++row) { - f_weight3_1 >> temp_next; f_weight3_2 >> temp_prev; - (*((FCLayer*)net->layers[4])->getWeights())[10*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight3_1 >> temp_next; + f_weight3_2 >> temp_prev; + (*((FCLayer *)net->layers[4])->getWeights())[10 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight3_1.close(); f_weight3_2.close(); + f_weight3_1.close(); + f_weight3_2.close(); if (ZEROS) { - generate_zeros("weight3_1", 128*10, temp); - generate_zeros("weight3_2", 128*10, temp); + generate_zeros("weight3_1", 128 * 10, temp); + generate_zeros("weight3_2", 128 * 10, temp); } - /************************** Bias1 **********************************/ - string path_bias1_1 = default_path+"bias1_"+to_string(partyNum); - string path_bias1_2 = default_path+"bias1_"+to_string(nextParty(partyNum)); + string path_bias1_1 = default_path + "bias1_" + to_string(partyNum); + string path_bias1_2 = default_path + "bias1_" + to_string(nextParty(partyNum)); ifstream f_bias1_1(path_bias1_1), f_bias1_2(path_bias1_2); for (int i = 0; i < 128; ++i) { - f_bias1_1 >> temp_next; f_bias1_2 >> temp_prev; - (*((FCLayer*)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias1_1 >> temp_next; + f_bias1_2 >> temp_prev; + (*((FCLayer *)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias1_1.close(); f_bias1_2.close(); + f_bias1_1.close(); + f_bias1_2.close(); if (ZEROS) { generate_zeros("bias1_1", 128, temp); generate_zeros("bias1_2", 128, temp); } - /************************** Bias2 **********************************/ - string path_bias2_1 = default_path+"bias2_"+to_string(partyNum); - string path_bias2_2 = default_path+"bias2_"+to_string(nextParty(partyNum)); + string path_bias2_1 = default_path + "bias2_" + to_string(partyNum); + string path_bias2_2 = default_path + "bias2_" + to_string(nextParty(partyNum)); ifstream f_bias2_1(path_bias2_1), f_bias2_2(path_bias2_2); for (int i = 0; i < 128; ++i) { - f_bias2_1 >> temp_next; f_bias2_2 >> temp_prev; - (*((FCLayer*)net->layers[2])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias2_1 >> temp_next; + f_bias2_2 >> temp_prev; + (*((FCLayer *)net->layers[2])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias2_1.close(); f_bias2_2.close(); + f_bias2_1.close(); + f_bias2_2.close(); if (ZEROS) { generate_zeros("bias2_1", 128, temp); generate_zeros("bias2_2", 128, temp); } - /************************** Bias3 **********************************/ - string path_bias3_1 = default_path+"bias3_"+to_string(partyNum); - string path_bias3_2 = default_path+"bias3_"+to_string(nextParty(partyNum)); + string path_bias3_1 = default_path + "bias3_" + to_string(partyNum); + string path_bias3_2 = default_path + "bias3_" + to_string(nextParty(partyNum)); ifstream f_bias3_1(path_bias3_1), f_bias3_2(path_bias3_2); for (int i = 0; i < 10; ++i) { - f_bias3_1 >> temp_next; f_bias3_2 >> temp_prev; - (*((FCLayer*)net->layers[4])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias3_1 >> temp_next; + f_bias3_2 >> temp_prev; + (*((FCLayer *)net->layers[4])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias3_1.close(); f_bias3_2.close(); + f_bias3_1.close(); + f_bias3_2.close(); if (ZEROS) { generate_zeros("bias3_1", 10, temp); @@ -282,99 +331,108 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) { string temp = "Sarda"; /************************** Input **********************************/ - string path_input_1 = default_path+"input_"+to_string(partyNum); - string path_input_2 = default_path+"input_"+to_string(nextParty(partyNum)); + string path_input_1 = default_path + "input_" + to_string(partyNum); + string path_input_2 = default_path + "input_" + to_string(nextParty(partyNum)); ifstream f_input_1(path_input_1), f_input_2(path_input_2); for (int i = 0; i < INPUT_SIZE * MINI_BATCH_SIZE; ++i) { - f_input_1 >> temp_next; f_input_2 >> temp_prev; + f_input_1 >> temp_next; + f_input_2 >> temp_prev; net->inputData[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_input_1.close(); f_input_2.close(); + f_input_1.close(); + f_input_2.close(); if (ZEROS) { - generate_zeros("input_1", 784*128, temp); - generate_zeros("input_2", 784*128, temp); + generate_zeros("input_1", 784 * 128, temp); + generate_zeros("input_2", 784 * 128, temp); } // print_vector(net->inputData, "FLOAT", "inputData:", 784); /************************** Weight1 **********************************/ - string path_weight1_1 = default_path+"weight1_"+to_string(partyNum); - string path_weight1_2 = default_path+"weight1_"+to_string(nextParty(partyNum)); + string path_weight1_1 = default_path + "weight1_" + to_string(partyNum); + string path_weight1_2 = default_path + "weight1_" + to_string(nextParty(partyNum)); ifstream f_weight1_1(path_weight1_1), f_weight1_2(path_weight1_2); for (int column = 0; column < 5; ++column) { for (int row = 0; row < 4; ++row) { - f_weight1_1 >> temp_next; f_weight1_2 >> temp_prev; - (*((CNNLayer*)net->layers[0])->getWeights())[4*column + row] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight1_1 >> temp_next; + f_weight1_2 >> temp_prev; + (*((CNNLayer *)net->layers[0])->getWeights())[4 * column + row] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight1_1.close(); f_weight1_2.close(); + f_weight1_1.close(); + f_weight1_2.close(); if (ZEROS) { - generate_zeros("weight1_1", 2*2*1*5, temp); - generate_zeros("weight1_2", 2*2*1*5, temp); + generate_zeros("weight1_1", 2 * 2 * 1 * 5, temp); + generate_zeros("weight1_2", 2 * 2 * 1 * 5, temp); } /************************** Weight2 **********************************/ - string path_weight2_1 = default_path+"weight2_"+to_string(partyNum); - string path_weight2_2 = default_path+"weight2_"+to_string(nextParty(partyNum)); + string path_weight2_1 = default_path + "weight2_" + to_string(partyNum); + string path_weight2_2 = default_path + "weight2_" + to_string(nextParty(partyNum)); ifstream f_weight2_1(path_weight2_1), f_weight2_2(path_weight2_2); for (int column = 0; column < 100; ++column) { for (int row = 0; row < 980; ++row) { - f_weight2_1 >> temp_next; f_weight2_2 >> temp_prev; - (*((FCLayer*)net->layers[2])->getWeights())[100*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight2_1 >> temp_next; + f_weight2_2 >> temp_prev; + (*((FCLayer *)net->layers[2])->getWeights())[100 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight2_1.close(); f_weight2_2.close(); + f_weight2_1.close(); + f_weight2_2.close(); if (ZEROS) { - generate_zeros("weight2_1", 980*100, temp); - generate_zeros("weight2_2", 980*100, temp); + generate_zeros("weight2_1", 980 * 100, temp); + generate_zeros("weight2_2", 980 * 100, temp); } - /************************** Weight3 **********************************/ - string path_weight3_1 = default_path+"weight3_"+to_string(partyNum); - string path_weight3_2 = default_path+"weight3_"+to_string(nextParty(partyNum)); + string path_weight3_1 = default_path + "weight3_" + to_string(partyNum); + string path_weight3_2 = default_path + "weight3_" + to_string(nextParty(partyNum)); ifstream f_weight3_1(path_weight3_1), f_weight3_2(path_weight3_2); for (int column = 0; column < 10; ++column) { for (int row = 0; row < 100; ++row) { - f_weight3_1 >> temp_next; f_weight3_2 >> temp_prev; - (*((FCLayer*)net->layers[4])->getWeights())[10*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight3_1 >> temp_next; + f_weight3_2 >> temp_prev; + (*((FCLayer *)net->layers[4])->getWeights())[10 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight3_1.close(); f_weight3_2.close(); + f_weight3_1.close(); + f_weight3_2.close(); if (ZEROS) { - generate_zeros("weight3_1", 100*10, temp); - generate_zeros("weight3_2", 100*10, temp); + generate_zeros("weight3_1", 100 * 10, temp); + generate_zeros("weight3_2", 100 * 10, temp); } /************************** Bias1 **********************************/ - string path_bias1_1 = default_path+"bias1_"+to_string(partyNum); - string path_bias1_2 = default_path+"bias1_"+to_string(nextParty(partyNum)); + string path_bias1_1 = default_path + "bias1_" + to_string(partyNum); + string path_bias1_2 = default_path + "bias1_" + to_string(nextParty(partyNum)); ifstream f_bias1_1(path_bias1_1), f_bias1_2(path_bias1_2); for (int i = 0; i < 5; ++i) { - f_bias1_1 >> temp_next; f_bias1_2 >> temp_prev; - (*((CNNLayer*)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias1_1 >> temp_next; + f_bias1_2 >> temp_prev; + (*((CNNLayer *)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias1_1.close(); f_bias1_2.close(); + f_bias1_1.close(); + f_bias1_2.close(); if (ZEROS) { generate_zeros("bias1_1", 5, temp); @@ -382,16 +440,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias2 **********************************/ - string path_bias2_1 = default_path+"bias2_"+to_string(partyNum); - string path_bias2_2 = default_path+"bias2_"+to_string(nextParty(partyNum)); + string path_bias2_1 = default_path + "bias2_" + to_string(partyNum); + string path_bias2_2 = default_path + "bias2_" + to_string(nextParty(partyNum)); ifstream f_bias2_1(path_bias2_1), f_bias2_2(path_bias2_2); for (int i = 0; i < 100; ++i) { - f_bias2_1 >> temp_next; f_bias2_2 >> temp_prev; - (*((FCLayer*)net->layers[2])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias2_1 >> temp_next; + f_bias2_2 >> temp_prev; + (*((FCLayer *)net->layers[2])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias2_1.close(); f_bias2_2.close(); + f_bias2_1.close(); + f_bias2_2.close(); if (ZEROS) { generate_zeros("bias2_1", 100, temp); @@ -399,16 +459,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias3 **********************************/ - string path_bias3_1 = default_path+"bias3_"+to_string(partyNum); - string path_bias3_2 = default_path+"bias3_"+to_string(nextParty(partyNum)); + string path_bias3_1 = default_path + "bias3_" + to_string(partyNum); + string path_bias3_2 = default_path + "bias3_" + to_string(nextParty(partyNum)); ifstream f_bias3_1(path_bias3_1), f_bias3_2(path_bias3_2); for (int i = 0; i < 10; ++i) { - f_bias3_1 >> temp_next; f_bias3_2 >> temp_prev; - (*((FCLayer*)net->layers[4])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias3_1 >> temp_next; + f_bias3_2 >> temp_prev; + (*((FCLayer *)net->layers[4])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias3_1.close(); f_bias3_2.close(); + f_bias3_1.close(); + f_bias3_2.close(); if (ZEROS) { generate_zeros("bias3_1", 10, temp); @@ -419,115 +481,125 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) { string temp = "MiniONN"; /************************** Input **********************************/ - string path_input_1 = default_path+"input_"+to_string(partyNum); - string path_input_2 = default_path+"input_"+to_string(nextParty(partyNum)); + string path_input_1 = default_path + "input_" + to_string(partyNum); + string path_input_2 = default_path + "input_" + to_string(nextParty(partyNum)); ifstream f_input_1(path_input_1), f_input_2(path_input_2); for (int i = 0; i < INPUT_SIZE * MINI_BATCH_SIZE; ++i) { - f_input_1 >> temp_next; f_input_2 >> temp_prev; + f_input_1 >> temp_next; + f_input_2 >> temp_prev; net->inputData[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_input_1.close(); f_input_2.close(); + f_input_1.close(); + f_input_2.close(); if (ZEROS) { - generate_zeros("input_1", 784*128, temp); - generate_zeros("input_2", 784*128, temp); + generate_zeros("input_1", 784 * 128, temp); + generate_zeros("input_2", 784 * 128, temp); } // print_vector(net->inputData, "FLOAT", "inputData:", 784); /************************** Weight1 **********************************/ - string path_weight1_1 = default_path+"weight1_"+to_string(partyNum); - string path_weight1_2 = default_path+"weight1_"+to_string(nextParty(partyNum)); + string path_weight1_1 = default_path + "weight1_" + to_string(partyNum); + string path_weight1_2 = default_path + "weight1_" + to_string(nextParty(partyNum)); ifstream f_weight1_1(path_weight1_1), f_weight1_2(path_weight1_2); - for (int row = 0; row < 5*5*1*16; ++row) + for (int row = 0; row < 5 * 5 * 1 * 16; ++row) { - f_weight1_1 >> temp_next; f_weight1_2 >> temp_prev; - (*((CNNLayer*)net->layers[0])->getWeights())[row] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight1_1 >> temp_next; + f_weight1_2 >> temp_prev; + (*((CNNLayer *)net->layers[0])->getWeights())[row] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_weight1_1.close(); f_weight1_2.close(); + f_weight1_1.close(); + f_weight1_2.close(); if (ZEROS) { - generate_zeros("weight1_1", 5*5*1*16, temp); - generate_zeros("weight1_2", 5*5*1*16, temp); + generate_zeros("weight1_1", 5 * 5 * 1 * 16, temp); + generate_zeros("weight1_2", 5 * 5 * 1 * 16, temp); } /************************** Weight2 **********************************/ - string path_weight2_1 = default_path+"weight2_"+to_string(partyNum); - string path_weight2_2 = default_path+"weight2_"+to_string(nextParty(partyNum)); + string path_weight2_1 = default_path + "weight2_" + to_string(partyNum); + string path_weight2_2 = default_path + "weight2_" + to_string(nextParty(partyNum)); ifstream f_weight2_1(path_weight2_1), f_weight2_2(path_weight2_2); - - for (int row = 0; row < 25*16*16; ++row) + for (int row = 0; row < 25 * 16 * 16; ++row) { - f_weight2_1 >> temp_next; f_weight2_2 >> temp_prev; - (*((CNNLayer*)net->layers[3])->getWeights())[row] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight2_1 >> temp_next; + f_weight2_2 >> temp_prev; + (*((CNNLayer *)net->layers[3])->getWeights())[row] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_weight2_1.close(); f_weight2_2.close(); + f_weight2_1.close(); + f_weight2_2.close(); if (ZEROS) { - generate_zeros("weight2_1", 5*5*16*16, temp); - generate_zeros("weight2_2", 5*5*16*16, temp); + generate_zeros("weight2_1", 5 * 5 * 16 * 16, temp); + generate_zeros("weight2_2", 5 * 5 * 16 * 16, temp); } /************************** Weight3 **********************************/ - string path_weight3_1 = default_path+"weight3_"+to_string(partyNum); - string path_weight3_2 = default_path+"weight3_"+to_string(nextParty(partyNum)); + string path_weight3_1 = default_path + "weight3_" + to_string(partyNum); + string path_weight3_2 = default_path + "weight3_" + to_string(nextParty(partyNum)); ifstream f_weight3_1(path_weight3_1), f_weight3_2(path_weight3_2); for (int column = 0; column < 100; ++column) { for (int row = 0; row < 256; ++row) { - f_weight3_1 >> temp_next; f_weight3_2 >> temp_prev; - (*((FCLayer*)net->layers[6])->getWeights())[100*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight3_1 >> temp_next; + f_weight3_2 >> temp_prev; + (*((FCLayer *)net->layers[6])->getWeights())[100 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight3_1.close(); f_weight3_2.close(); + f_weight3_1.close(); + f_weight3_2.close(); if (ZEROS) { - generate_zeros("weight3_1", 256*100, temp); - generate_zeros("weight3_2", 256*100, temp); + generate_zeros("weight3_1", 256 * 100, temp); + generate_zeros("weight3_2", 256 * 100, temp); } - /************************** Weight4 **********************************/ - string path_weight4_1 = default_path+"weight4_"+to_string(partyNum); - string path_weight4_2 = default_path+"weight4_"+to_string(nextParty(partyNum)); + string path_weight4_1 = default_path + "weight4_" + to_string(partyNum); + string path_weight4_2 = default_path + "weight4_" + to_string(nextParty(partyNum)); ifstream f_weight4_1(path_weight4_1), f_weight4_2(path_weight4_2); for (int column = 0; column < 10; ++column) { for (int row = 0; row < 100; ++row) { - f_weight4_1 >> temp_next; f_weight4_2 >> temp_prev; - (*((FCLayer*)net->layers[8])->getWeights())[10*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight4_1 >> temp_next; + f_weight4_2 >> temp_prev; + (*((FCLayer *)net->layers[8])->getWeights())[10 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight4_1.close(); f_weight4_2.close(); + f_weight4_1.close(); + f_weight4_2.close(); if (ZEROS) { - generate_zeros("weight4_1", 100*10, temp); - generate_zeros("weight4_2", 100*10, temp); + generate_zeros("weight4_1", 100 * 10, temp); + generate_zeros("weight4_2", 100 * 10, temp); } /************************** Bias1 **********************************/ - string path_bias1_1 = default_path+"bias1_"+to_string(partyNum); - string path_bias1_2 = default_path+"bias1_"+to_string(nextParty(partyNum)); + string path_bias1_1 = default_path + "bias1_" + to_string(partyNum); + string path_bias1_2 = default_path + "bias1_" + to_string(nextParty(partyNum)); ifstream f_bias1_1(path_bias1_1), f_bias1_2(path_bias1_2); for (int i = 0; i < 16; ++i) { - f_bias1_1 >> temp_next; f_bias1_2 >> temp_prev; - (*((CNNLayer*)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias1_1 >> temp_next; + f_bias1_2 >> temp_prev; + (*((CNNLayer *)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias1_1.close(); f_bias1_2.close(); + f_bias1_1.close(); + f_bias1_2.close(); if (ZEROS) { generate_zeros("bias1_1", 16, temp); @@ -535,16 +607,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias2 **********************************/ - string path_bias2_1 = default_path+"bias2_"+to_string(partyNum); - string path_bias2_2 = default_path+"bias2_"+to_string(nextParty(partyNum)); + string path_bias2_1 = default_path + "bias2_" + to_string(partyNum); + string path_bias2_2 = default_path + "bias2_" + to_string(nextParty(partyNum)); ifstream f_bias2_1(path_bias2_1), f_bias2_2(path_bias2_2); for (int i = 0; i < 16; ++i) { - f_bias2_1 >> temp_next; f_bias2_2 >> temp_prev; - (*((CNNLayer*)net->layers[3])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias2_1 >> temp_next; + f_bias2_2 >> temp_prev; + (*((CNNLayer *)net->layers[3])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias2_1.close(); f_bias2_2.close(); + f_bias2_1.close(); + f_bias2_2.close(); if (ZEROS) { generate_zeros("bias2_1", 16, temp); @@ -552,16 +626,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias3 **********************************/ - string path_bias3_1 = default_path+"bias3_"+to_string(partyNum); - string path_bias3_2 = default_path+"bias3_"+to_string(nextParty(partyNum)); + string path_bias3_1 = default_path + "bias3_" + to_string(partyNum); + string path_bias3_2 = default_path + "bias3_" + to_string(nextParty(partyNum)); ifstream f_bias3_1(path_bias3_1), f_bias3_2(path_bias3_2); for (int i = 0; i < 100; ++i) { - f_bias3_1 >> temp_next; f_bias3_2 >> temp_prev; - (*((FCLayer*)net->layers[6])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias3_1 >> temp_next; + f_bias3_2 >> temp_prev; + (*((FCLayer *)net->layers[6])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias3_1.close(); f_bias3_2.close(); + f_bias3_1.close(); + f_bias3_2.close(); if (ZEROS) { generate_zeros("bias3_1", 100, temp); @@ -569,16 +645,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias4 **********************************/ - string path_bias4_1 = default_path+"bias4_"+to_string(partyNum); - string path_bias4_2 = default_path+"bias4_"+to_string(nextParty(partyNum)); + string path_bias4_1 = default_path + "bias4_" + to_string(partyNum); + string path_bias4_2 = default_path + "bias4_" + to_string(nextParty(partyNum)); ifstream f_bias4_1(path_bias4_1), f_bias4_2(path_bias4_2); for (int i = 0; i < 10; ++i) { - f_bias4_1 >> temp_next; f_bias4_2 >> temp_prev; - (*((FCLayer*)net->layers[8])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias4_1 >> temp_next; + f_bias4_2 >> temp_prev; + (*((FCLayer *)net->layers[8])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias4_1.close(); f_bias4_2.close(); + f_bias4_1.close(); + f_bias4_2.close(); if (ZEROS) { generate_zeros("bias4_1", 10, temp); @@ -589,115 +667,125 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) { string temp = "LeNet"; /************************** Input **********************************/ - string path_input_1 = default_path+"input_"+to_string(partyNum); - string path_input_2 = default_path+"input_"+to_string(nextParty(partyNum)); + string path_input_1 = default_path + "input_" + to_string(partyNum); + string path_input_2 = default_path + "input_" + to_string(nextParty(partyNum)); ifstream f_input_1(path_input_1), f_input_2(path_input_2); for (int i = 0; i < INPUT_SIZE * MINI_BATCH_SIZE; ++i) { - f_input_1 >> temp_next; f_input_2 >> temp_prev; + f_input_1 >> temp_next; + f_input_2 >> temp_prev; net->inputData[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_input_1.close(); f_input_2.close(); + f_input_1.close(); + f_input_2.close(); if (ZEROS) { - generate_zeros("input_1", 784*128, temp); - generate_zeros("input_2", 784*128, temp); + generate_zeros("input_1", 784 * 128, temp); + generate_zeros("input_2", 784 * 128, temp); } // print_vector(net->inputData, "FLOAT", "inputData:", 784); /************************** Weight1 **********************************/ - string path_weight1_1 = default_path+"weight1_"+to_string(partyNum); - string path_weight1_2 = default_path+"weight1_"+to_string(nextParty(partyNum)); + string path_weight1_1 = default_path + "weight1_" + to_string(partyNum); + string path_weight1_2 = default_path + "weight1_" + to_string(nextParty(partyNum)); ifstream f_weight1_1(path_weight1_1), f_weight1_2(path_weight1_2); - for (int row = 0; row < 5*5*1*20; ++row) + for (int row = 0; row < 5 * 5 * 1 * 20; ++row) { - f_weight1_1 >> temp_next; f_weight1_2 >> temp_prev; - (*((CNNLayer*)net->layers[0])->getWeights())[row] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight1_1 >> temp_next; + f_weight1_2 >> temp_prev; + (*((CNNLayer *)net->layers[0])->getWeights())[row] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_weight1_1.close(); f_weight1_2.close(); + f_weight1_1.close(); + f_weight1_2.close(); if (ZEROS) { - generate_zeros("weight1_1", 5*5*1*20, temp); - generate_zeros("weight1_2", 5*5*1*20, temp); + generate_zeros("weight1_1", 5 * 5 * 1 * 20, temp); + generate_zeros("weight1_2", 5 * 5 * 1 * 20, temp); } /************************** Weight2 **********************************/ - string path_weight2_1 = default_path+"weight2_"+to_string(partyNum); - string path_weight2_2 = default_path+"weight2_"+to_string(nextParty(partyNum)); + string path_weight2_1 = default_path + "weight2_" + to_string(partyNum); + string path_weight2_2 = default_path + "weight2_" + to_string(nextParty(partyNum)); ifstream f_weight2_1(path_weight2_1), f_weight2_2(path_weight2_2); - - for (int row = 0; row < 25*20*50; ++row) + for (int row = 0; row < 25 * 20 * 50; ++row) { - f_weight2_1 >> temp_next; f_weight2_2 >> temp_prev; - (*((CNNLayer*)net->layers[3])->getWeights())[row] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight2_1 >> temp_next; + f_weight2_2 >> temp_prev; + (*((CNNLayer *)net->layers[3])->getWeights())[row] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_weight2_1.close(); f_weight2_2.close(); + f_weight2_1.close(); + f_weight2_2.close(); if (ZEROS) { - generate_zeros("weight2_1", 5*5*20*50, temp); - generate_zeros("weight2_2", 5*5*20*50, temp); + generate_zeros("weight2_1", 5 * 5 * 20 * 50, temp); + generate_zeros("weight2_2", 5 * 5 * 20 * 50, temp); } /************************** Weight3 **********************************/ - string path_weight3_1 = default_path+"weight3_"+to_string(partyNum); - string path_weight3_2 = default_path+"weight3_"+to_string(nextParty(partyNum)); + string path_weight3_1 = default_path + "weight3_" + to_string(partyNum); + string path_weight3_2 = default_path + "weight3_" + to_string(nextParty(partyNum)); ifstream f_weight3_1(path_weight3_1), f_weight3_2(path_weight3_2); for (int column = 0; column < 500; ++column) { for (int row = 0; row < 800; ++row) { - f_weight3_1 >> temp_next; f_weight3_2 >> temp_prev; - (*((FCLayer*)net->layers[6])->getWeights())[500*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight3_1 >> temp_next; + f_weight3_2 >> temp_prev; + (*((FCLayer *)net->layers[6])->getWeights())[500 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight3_1.close(); f_weight3_2.close(); + f_weight3_1.close(); + f_weight3_2.close(); if (ZEROS) { - generate_zeros("weight3_1", 800*500, temp); - generate_zeros("weight3_2", 800*500, temp); + generate_zeros("weight3_1", 800 * 500, temp); + generate_zeros("weight3_2", 800 * 500, temp); } - /************************** Weight4 **********************************/ - string path_weight4_1 = default_path+"weight4_"+to_string(partyNum); - string path_weight4_2 = default_path+"weight4_"+to_string(nextParty(partyNum)); + string path_weight4_1 = default_path + "weight4_" + to_string(partyNum); + string path_weight4_2 = default_path + "weight4_" + to_string(nextParty(partyNum)); ifstream f_weight4_1(path_weight4_1), f_weight4_2(path_weight4_2); for (int column = 0; column < 10; ++column) { for (int row = 0; row < 500; ++row) { - f_weight4_1 >> temp_next; f_weight4_2 >> temp_prev; - (*((FCLayer*)net->layers[8])->getWeights())[10*row + column] = - std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_weight4_1 >> temp_next; + f_weight4_2 >> temp_prev; + (*((FCLayer *)net->layers[8])->getWeights())[10 * row + column] = + std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } } - f_weight4_1.close(); f_weight4_2.close(); + f_weight4_1.close(); + f_weight4_2.close(); if (ZEROS) { - generate_zeros("weight4_1", 500*10, temp); - generate_zeros("weight4_2", 500*10, temp); + generate_zeros("weight4_1", 500 * 10, temp); + generate_zeros("weight4_2", 500 * 10, temp); } /************************** Bias1 **********************************/ - string path_bias1_1 = default_path+"bias1_"+to_string(partyNum); - string path_bias1_2 = default_path+"bias1_"+to_string(nextParty(partyNum)); + string path_bias1_1 = default_path + "bias1_" + to_string(partyNum); + string path_bias1_2 = default_path + "bias1_" + to_string(nextParty(partyNum)); ifstream f_bias1_1(path_bias1_1), f_bias1_2(path_bias1_2); for (int i = 0; i < 20; ++i) { - f_bias1_1 >> temp_next; f_bias1_2 >> temp_prev; - (*((CNNLayer*)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias1_1 >> temp_next; + f_bias1_2 >> temp_prev; + (*((CNNLayer *)net->layers[0])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias1_1.close(); f_bias1_2.close(); + f_bias1_1.close(); + f_bias1_2.close(); if (ZEROS) { generate_zeros("bias1_1", 20, temp); @@ -705,16 +793,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias2 **********************************/ - string path_bias2_1 = default_path+"bias2_"+to_string(partyNum); - string path_bias2_2 = default_path+"bias2_"+to_string(nextParty(partyNum)); + string path_bias2_1 = default_path + "bias2_" + to_string(partyNum); + string path_bias2_2 = default_path + "bias2_" + to_string(nextParty(partyNum)); ifstream f_bias2_1(path_bias2_1), f_bias2_2(path_bias2_2); for (int i = 0; i < 50; ++i) { - f_bias2_1 >> temp_next; f_bias2_2 >> temp_prev; - (*((CNNLayer*)net->layers[3])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias2_1 >> temp_next; + f_bias2_2 >> temp_prev; + (*((CNNLayer *)net->layers[3])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias2_1.close(); f_bias2_2.close(); + f_bias2_1.close(); + f_bias2_2.close(); if (ZEROS) { generate_zeros("bias2_1", 50, temp); @@ -722,16 +812,18 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias3 **********************************/ - string path_bias3_1 = default_path+"bias3_"+to_string(partyNum); - string path_bias3_2 = default_path+"bias3_"+to_string(nextParty(partyNum)); + string path_bias3_1 = default_path + "bias3_" + to_string(partyNum); + string path_bias3_2 = default_path + "bias3_" + to_string(nextParty(partyNum)); ifstream f_bias3_1(path_bias3_1), f_bias3_2(path_bias3_2); for (int i = 0; i < 500; ++i) { - f_bias3_1 >> temp_next; f_bias3_2 >> temp_prev; - (*((FCLayer*)net->layers[6])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias3_1 >> temp_next; + f_bias3_2 >> temp_prev; + (*((FCLayer *)net->layers[6])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias3_1.close(); f_bias3_2.close(); + f_bias3_1.close(); + f_bias3_2.close(); if (ZEROS) { generate_zeros("bias3_1", 500, temp); @@ -739,27 +831,27 @@ void preload_network(bool PRELOADING, string network, NeuralNetwork* net) } /************************** Bias4 **********************************/ - string path_bias4_1 = default_path+"bias4_"+to_string(partyNum); - string path_bias4_2 = default_path+"bias4_"+to_string(nextParty(partyNum)); + string path_bias4_1 = default_path + "bias4_" + to_string(partyNum); + string path_bias4_2 = default_path + "bias4_" + to_string(nextParty(partyNum)); ifstream f_bias4_1(path_bias4_1), f_bias4_2(path_bias4_2); for (int i = 0; i < 10; ++i) { - f_bias4_1 >> temp_next; f_bias4_2 >> temp_prev; - (*((FCLayer*)net->layers[8])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); + f_bias4_1 >> temp_next; + f_bias4_2 >> temp_prev; + (*((FCLayer *)net->layers[8])->getBias())[i] = std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev)); } - f_bias4_1.close(); f_bias4_2.close(); + f_bias4_1.close(); + f_bias4_2.close(); if (ZEROS) { generate_zeros("bias4_1", 10, temp); generate_zeros("bias4_2", 10, temp); } } - else + else error("Preloading network error"); - - cout << "Preloading completed..." << endl; } @@ -769,8 +861,10 @@ void loadData(string net, string dataset) { INPUT_SIZE = 784; LAST_LAYER_SIZE = 10; - TRAINING_DATA_SIZE = 8; - TEST_DATA_SIZE = 8; + // TRAINING_DATA_SIZE = 8; + // TEST_DATA_SIZE = 8; + TRAINING_DATA_SIZE = 1 << 11; + TEST_DATA_SIZE = 1 << 10; LARGE_NETWORK = false; } else if (dataset.compare("CIFAR10") == 0) @@ -778,17 +872,17 @@ void loadData(string net, string dataset) LARGE_NETWORK = false; if (net.compare("AlexNet") == 0) { - INPUT_SIZE = 33*33*3; + INPUT_SIZE = 33 * 33 * 3; LAST_LAYER_SIZE = 10; TRAINING_DATA_SIZE = 8; - TEST_DATA_SIZE = 8; + TEST_DATA_SIZE = 8; } else if (net.compare("VGG16") == 0) { - INPUT_SIZE = 32*32*3; + INPUT_SIZE = 32 * 32 * 3; LAST_LAYER_SIZE = 10; TRAINING_DATA_SIZE = 8; - TEST_DATA_SIZE = 8; + TEST_DATA_SIZE = 8; } else assert(false && "Only AlexNet and VGG16 supported on CIFAR10"); @@ -796,26 +890,26 @@ void loadData(string net, string dataset) else if (dataset.compare("ImageNet") == 0) { LARGE_NETWORK = true; - //https://medium.com/@smallfishbigsea/a-walk-through-of-alexnet-6cbd137a5637 - //https://medium.com/@RaghavPrabhu/cnn-architectures-lenet-alexnet-vgg-googlenet-and-resnet-7c81c017b848 - //https://neurohive.io/en/popular-networks/vgg16/ + // https://medium.com/@smallfishbigsea/a-walk-through-of-alexnet-6cbd137a5637 + // https://medium.com/@RaghavPrabhu/cnn-architectures-lenet-alexnet-vgg-googlenet-and-resnet-7c81c017b848 + // https://neurohive.io/en/popular-networks/vgg16/ - //Tiny ImageNet - //http://cs231n.stanford.edu/reports/2017/pdfs/930.pdf - //http://cs231n.stanford.edu/reports/2017/pdfs/931.pdf + // Tiny ImageNet + // http://cs231n.stanford.edu/reports/2017/pdfs/930.pdf + // http://cs231n.stanford.edu/reports/2017/pdfs/931.pdf if (net.compare("AlexNet") == 0) { - INPUT_SIZE = 56*56*3; + INPUT_SIZE = 56 * 56 * 3; LAST_LAYER_SIZE = 200; TRAINING_DATA_SIZE = 8; - TEST_DATA_SIZE = 8; + TEST_DATA_SIZE = 8; } else if (net.compare("VGG16") == 0) { - INPUT_SIZE = 64*64*3; + INPUT_SIZE = 64 * 64 * 3; LAST_LAYER_SIZE = 200; TRAINING_DATA_SIZE = 8; - TEST_DATA_SIZE = 8; + TEST_DATA_SIZE = 8; } else assert(false && "Only AlexNet and VGG16 supported on ImageNet"); @@ -823,12 +917,11 @@ void loadData(string net, string dataset) else assert(false && "Only MNIST, CIFAR10, and ImageNet supported"); - string filename_train_data_next, filename_train_data_prev; string filename_test_data_next, filename_test_data_prev; string filename_train_labels_next, filename_train_labels_prev; string filename_test_labels_next, filename_test_labels_prev; - + // modified to let each party holding a share of data if (partyNum == PARTY_A) { @@ -864,50 +957,57 @@ void loadData(string net, string dataset) filename_train_labels_prev = "files/train_labels_A"; filename_test_labels_next = "files/test_labels_C"; filename_test_labels_prev = "files/test_labels_A"; - } + } float temp_next = 0, temp_prev = 0; ifstream f_next(filename_train_data_next); ifstream f_prev(filename_train_data_prev); for (int i = 0; i < TRAINING_DATA_SIZE * INPUT_SIZE; ++i) { - f_next >> temp_next; f_prev >> temp_prev; + f_next >> temp_next; + f_prev >> temp_prev; trainData.push_back(std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev))); } - f_next.close(); f_prev.close(); + f_next.close(); + f_prev.close(); ifstream g_next(filename_train_labels_next); ifstream g_prev(filename_train_labels_prev); for (int i = 0; i < TRAINING_DATA_SIZE * LAST_LAYER_SIZE; ++i) { - g_next >> temp_next; g_prev >> temp_prev; + g_next >> temp_next; + g_prev >> temp_prev; trainLabels.push_back(std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev))); } - g_next.close(); g_prev.close(); + g_next.close(); + g_prev.close(); ifstream h_next(filename_test_data_next); ifstream h_prev(filename_test_data_prev); for (int i = 0; i < TEST_DATA_SIZE * INPUT_SIZE; ++i) { - h_next >> temp_next; h_prev >> temp_prev; + h_next >> temp_next; + h_prev >> temp_prev; testData.push_back(std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev))); } - h_next.close(); h_prev.close(); + h_next.close(); + h_prev.close(); ifstream k_next(filename_test_labels_next); ifstream k_prev(filename_test_labels_prev); for (int i = 0; i < TEST_DATA_SIZE * LAST_LAYER_SIZE; ++i) { - k_next >> temp_next; k_prev >> temp_prev; + k_next >> temp_next; + k_prev >> temp_prev; testLabels.push_back(std::make_pair(floatToMyType(temp_next), floatToMyType(temp_prev))); } - k_next.close(); k_prev.close(); + k_next.close(); + k_prev.close(); cout << "Loading data done....." << endl; } - -void readMiniBatch(NeuralNetwork* net, string phase) +void readMiniBatch(NeuralNetwork *net, string phase) { size_t s = trainData.size(); size_t t = trainLabels.size(); @@ -915,10 +1015,10 @@ void readMiniBatch(NeuralNetwork* net, string phase) if (phase == "TRAINING") { for (int i = 0; i < INPUT_SIZE * MINI_BATCH_SIZE; ++i) - net->inputData[i] = trainData[(trainDataBatchCounter + i)%s]; + net->inputData[i] = trainData[(trainDataBatchCounter + i) % s]; for (int i = 0; i < LAST_LAYER_SIZE * MINI_BATCH_SIZE; ++i) - net->outputData[i] = trainLabels[(trainLabelsBatchCounter + i)%t]; + net->outputData[i] = trainLabels[(trainLabelsBatchCounter + i) % t]; trainDataBatchCounter += INPUT_SIZE * MINI_BATCH_SIZE; trainLabelsBatchCounter += LAST_LAYER_SIZE * MINI_BATCH_SIZE; @@ -930,18 +1030,16 @@ void readMiniBatch(NeuralNetwork* net, string phase) if (trainLabelsBatchCounter > t) trainLabelsBatchCounter -= t; - - size_t p = testData.size(); size_t q = testLabels.size(); if (phase == "TESTING") { for (int i = 0; i < INPUT_SIZE * MINI_BATCH_SIZE; ++i) - net->inputData[i] = testData[(testDataBatchCounter + i)%p]; + net->inputData[i] = testData[(testDataBatchCounter + i) % p]; for (int i = 0; i < LAST_LAYER_SIZE * MINI_BATCH_SIZE; ++i) - net->outputData[i] = testLabels[(testLabelsBatchCounter + i)%q]; + net->outputData[i] = testLabels[(testLabelsBatchCounter + i) % q]; testDataBatchCounter += INPUT_SIZE * MINI_BATCH_SIZE; testLabelsBatchCounter += LAST_LAYER_SIZE * MINI_BATCH_SIZE; @@ -954,18 +1052,17 @@ void readMiniBatch(NeuralNetwork* net, string phase) testLabelsBatchCounter -= q; } -void printNetwork(NeuralNetwork* net) +void printNetwork(NeuralNetwork *net) { for (int i = 0; i < net->layers.size(); ++i) net->layers[i]->printLayer(); - cout << "----------------------------------------------" << endl; + cout << "----------------------------------------------" << endl; } - -void selectNetwork(string network, string dataset, string security, NeuralNetConfig* config) +void selectNetwork(string network, string dataset, string security, NeuralNetConfig *config) { - assert(((security.compare("Semi-honest") == 0) or (security.compare("Malicious") == 0)) && - "Only Semi-honest or Malicious security allowed"); + assert(((security.compare("Semi-honest") == 0) or (security.compare("Malicious") == 0)) && + "Only Semi-honest or Malicious security allowed"); SECURITY_TYPE = security; loadData(network, dataset); @@ -974,12 +1071,12 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon assert((dataset.compare("MNIST") == 0) && "SecureML only over MNIST"); NUM_LAYERS = 6; WITH_NORMALIZATION = true; - FCConfig* l0 = new FCConfig(784, MINI_BATCH_SIZE, 128); - ReLUConfig* l1 = new ReLUConfig(128, MINI_BATCH_SIZE); - FCConfig* l2 = new FCConfig(128, MINI_BATCH_SIZE, 128); - ReLUConfig* l3 = new ReLUConfig(128, MINI_BATCH_SIZE); - FCConfig* l4 = new FCConfig(128, MINI_BATCH_SIZE, 10); - ReLUConfig* l5 = new ReLUConfig(10, MINI_BATCH_SIZE); + FCConfig *l0 = new FCConfig(784, MINI_BATCH_SIZE, 128); + ReLUConfig *l1 = new ReLUConfig(128, MINI_BATCH_SIZE); + FCConfig *l2 = new FCConfig(128, MINI_BATCH_SIZE, 128); + ReLUConfig *l3 = new ReLUConfig(128, MINI_BATCH_SIZE); + FCConfig *l4 = new FCConfig(128, MINI_BATCH_SIZE, 10); + ReLUConfig *l5 = new ReLUConfig(10, MINI_BATCH_SIZE); // BNConfig* l6 = new BNConfig(10, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); @@ -994,11 +1091,11 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon assert((dataset.compare("MNIST") == 0) && "Sarda only over MNIST"); NUM_LAYERS = 5; WITH_NORMALIZATION = true; - CNNConfig* l0 = new CNNConfig(28,28,1,5,2,2,0,MINI_BATCH_SIZE); - ReLUConfig* l1 = new ReLUConfig(980, MINI_BATCH_SIZE); - FCConfig* l2 = new FCConfig(980, MINI_BATCH_SIZE, 100); - ReLUConfig* l3 = new ReLUConfig(100, MINI_BATCH_SIZE); - FCConfig* l4 = new FCConfig(100, MINI_BATCH_SIZE, 10); + CNNConfig *l0 = new CNNConfig(28, 28, 1, 5, 2, 2, 0, MINI_BATCH_SIZE); + ReLUConfig *l1 = new ReLUConfig(980, MINI_BATCH_SIZE); + FCConfig *l2 = new FCConfig(980, MINI_BATCH_SIZE, 100); + ReLUConfig *l3 = new ReLUConfig(100, MINI_BATCH_SIZE); + FCConfig *l4 = new FCConfig(100, MINI_BATCH_SIZE, 10); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1010,16 +1107,16 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon assert((dataset.compare("MNIST") == 0) && "MiniONN only over MNIST"); NUM_LAYERS = 10; WITH_NORMALIZATION = true; - CNNConfig* l0 = new CNNConfig(28,28,1,16,5,1,0,MINI_BATCH_SIZE); - MaxpoolConfig* l1 = new MaxpoolConfig(24,24,16,2,2,MINI_BATCH_SIZE); - ReLUConfig* l2 = new ReLUConfig(12*12*16, MINI_BATCH_SIZE); - CNNConfig* l3 = new CNNConfig(12,12,16,16,5,1,0,MINI_BATCH_SIZE); - MaxpoolConfig* l4 = new MaxpoolConfig(8,8,16,2,2,MINI_BATCH_SIZE); - ReLUConfig* l5 = new ReLUConfig(4*4*16, MINI_BATCH_SIZE); - FCConfig* l6 = new FCConfig(4*4*16, MINI_BATCH_SIZE, 100); - ReLUConfig* l7 = new ReLUConfig(100, MINI_BATCH_SIZE); - FCConfig* l8 = new FCConfig(100, MINI_BATCH_SIZE, 10); - ReLUConfig* l9 = new ReLUConfig(10, MINI_BATCH_SIZE); + CNNConfig *l0 = new CNNConfig(28, 28, 1, 16, 5, 1, 0, MINI_BATCH_SIZE); + MaxpoolConfig *l1 = new MaxpoolConfig(24, 24, 16, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l2 = new ReLUConfig(12 * 12 * 16, MINI_BATCH_SIZE); + CNNConfig *l3 = new CNNConfig(12, 12, 16, 16, 5, 1, 0, MINI_BATCH_SIZE); + MaxpoolConfig *l4 = new MaxpoolConfig(8, 8, 16, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l5 = new ReLUConfig(4 * 4 * 16, MINI_BATCH_SIZE); + FCConfig *l6 = new FCConfig(4 * 4 * 16, MINI_BATCH_SIZE, 100); + ReLUConfig *l7 = new ReLUConfig(100, MINI_BATCH_SIZE); + FCConfig *l8 = new FCConfig(100, MINI_BATCH_SIZE, 10); + ReLUConfig *l9 = new ReLUConfig(10, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1036,16 +1133,16 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon assert((dataset.compare("MNIST") == 0) && "LeNet only over MNIST"); NUM_LAYERS = 10; WITH_NORMALIZATION = true; - CNNConfig* l0 = new CNNConfig(28,28,1,20,5,1,0,MINI_BATCH_SIZE); - MaxpoolConfig* l1 = new MaxpoolConfig(24,24,20,2,2,MINI_BATCH_SIZE); - ReLUConfig* l2 = new ReLUConfig(12*12*20, MINI_BATCH_SIZE); - CNNConfig* l3 = new CNNConfig(12,12,20,50,5,1,0,MINI_BATCH_SIZE); - MaxpoolConfig* l4 = new MaxpoolConfig(8,8,50,2,2,MINI_BATCH_SIZE); - ReLUConfig* l5 = new ReLUConfig(4*4*50, MINI_BATCH_SIZE); - FCConfig* l6 = new FCConfig(4*4*50, MINI_BATCH_SIZE, 500); - ReLUConfig* l7 = new ReLUConfig(500, MINI_BATCH_SIZE); - FCConfig* l8 = new FCConfig(500, MINI_BATCH_SIZE, 10); - ReLUConfig* l9 = new ReLUConfig(10, MINI_BATCH_SIZE); + CNNConfig *l0 = new CNNConfig(28, 28, 1, 20, 5, 1, 0, MINI_BATCH_SIZE); + MaxpoolConfig *l1 = new MaxpoolConfig(24, 24, 20, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l2 = new ReLUConfig(12 * 12 * 20, MINI_BATCH_SIZE); + CNNConfig *l3 = new CNNConfig(12, 12, 20, 50, 5, 1, 0, MINI_BATCH_SIZE); + MaxpoolConfig *l4 = new MaxpoolConfig(8, 8, 50, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l5 = new ReLUConfig(4 * 4 * 50, MINI_BATCH_SIZE); + FCConfig *l6 = new FCConfig(4 * 4 * 50, MINI_BATCH_SIZE, 500); + ReLUConfig *l7 = new ReLUConfig(500, MINI_BATCH_SIZE); + FCConfig *l8 = new FCConfig(500, MINI_BATCH_SIZE, 10); + ReLUConfig *l9 = new ReLUConfig(10, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1059,36 +1156,36 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon } else if (network.compare("AlexNet") == 0) { - if(dataset.compare("MNIST") == 0) + if (dataset.compare("MNIST") == 0) assert(false && "No AlexNet on MNIST"); else if (dataset.compare("CIFAR10") == 0) { NUM_LAYERS = 20; // NUM_LAYERS = 18; //Without BN WITH_NORMALIZATION = false; - CNNConfig* l0 = new CNNConfig(33,33,3,96,11,4,9,MINI_BATCH_SIZE); - MaxpoolConfig* l1 = new MaxpoolConfig(11,11,96,3,2,MINI_BATCH_SIZE); - ReLUConfig* l2 = new ReLUConfig(5*5*96,MINI_BATCH_SIZE); - BNConfig * l3 = new BNConfig(5*5*96,MINI_BATCH_SIZE); - - CNNConfig* l4 = new CNNConfig(5,5,96,256,5,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l5 = new MaxpoolConfig(3,3,256,3,2,MINI_BATCH_SIZE); - ReLUConfig* l6 = new ReLUConfig(1*1*256,MINI_BATCH_SIZE); - BNConfig * l7 = new BNConfig(1*1*256,MINI_BATCH_SIZE); - - CNNConfig* l8 = new CNNConfig(1,1,256,384,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l9 = new ReLUConfig(1*1*384,MINI_BATCH_SIZE); - CNNConfig* l10 = new CNNConfig(1,1,384,384,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l11 = new ReLUConfig(1*1*384,MINI_BATCH_SIZE); - CNNConfig* l12 = new CNNConfig(1,1,384,256,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l13 = new ReLUConfig(1*1*256,MINI_BATCH_SIZE); - - FCConfig* l14 = new FCConfig(1*1*256,MINI_BATCH_SIZE,256); - ReLUConfig* l15 = new ReLUConfig(256,MINI_BATCH_SIZE); - FCConfig* l16 = new FCConfig(256,MINI_BATCH_SIZE,256); - ReLUConfig* l17 = new ReLUConfig(256,MINI_BATCH_SIZE); - FCConfig* l18 = new FCConfig(256,MINI_BATCH_SIZE,10); - ReLUConfig* l19 = new ReLUConfig(10,MINI_BATCH_SIZE); + CNNConfig *l0 = new CNNConfig(33, 33, 3, 96, 11, 4, 9, MINI_BATCH_SIZE); + MaxpoolConfig *l1 = new MaxpoolConfig(11, 11, 96, 3, 2, MINI_BATCH_SIZE); + ReLUConfig *l2 = new ReLUConfig(5 * 5 * 96, MINI_BATCH_SIZE); + BNConfig *l3 = new BNConfig(5 * 5 * 96, MINI_BATCH_SIZE); + + CNNConfig *l4 = new CNNConfig(5, 5, 96, 256, 5, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l5 = new MaxpoolConfig(3, 3, 256, 3, 2, MINI_BATCH_SIZE); + ReLUConfig *l6 = new ReLUConfig(1 * 1 * 256, MINI_BATCH_SIZE); + BNConfig *l7 = new BNConfig(1 * 1 * 256, MINI_BATCH_SIZE); + + CNNConfig *l8 = new CNNConfig(1, 1, 256, 384, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l9 = new ReLUConfig(1 * 1 * 384, MINI_BATCH_SIZE); + CNNConfig *l10 = new CNNConfig(1, 1, 384, 384, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l11 = new ReLUConfig(1 * 1 * 384, MINI_BATCH_SIZE); + CNNConfig *l12 = new CNNConfig(1, 1, 384, 256, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l13 = new ReLUConfig(1 * 1 * 256, MINI_BATCH_SIZE); + + FCConfig *l14 = new FCConfig(1 * 1 * 256, MINI_BATCH_SIZE, 256); + ReLUConfig *l15 = new ReLUConfig(256, MINI_BATCH_SIZE); + FCConfig *l16 = new FCConfig(256, MINI_BATCH_SIZE, 256); + ReLUConfig *l17 = new ReLUConfig(256, MINI_BATCH_SIZE); + FCConfig *l18 = new FCConfig(256, MINI_BATCH_SIZE, 10); + ReLUConfig *l19 = new ReLUConfig(10, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1115,28 +1212,28 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon NUM_LAYERS = 19; // NUM_LAYERS = 17; //Without BN WITH_NORMALIZATION = false; - CNNConfig* l0 = new CNNConfig(56,56,3,64,7,1,3,MINI_BATCH_SIZE); - CNNConfig* l1 = new CNNConfig(56,56,64,64,5,1,2,MINI_BATCH_SIZE); - MaxpoolConfig* l2 = new MaxpoolConfig(56,56,64,2,2,MINI_BATCH_SIZE); - ReLUConfig* l3 = new ReLUConfig(28*28*64,MINI_BATCH_SIZE); - BNConfig * l4 = new BNConfig(28*28*64,MINI_BATCH_SIZE); - - CNNConfig* l5 = new CNNConfig(28,28,64,128,5,1,2,MINI_BATCH_SIZE); - MaxpoolConfig* l6 = new MaxpoolConfig(28,28,128,2,2,MINI_BATCH_SIZE); - ReLUConfig* l7 = new ReLUConfig(14*14*128,MINI_BATCH_SIZE); - BNConfig * l8 = new BNConfig(14*14*128,MINI_BATCH_SIZE); - - CNNConfig* l9 = new CNNConfig(14,14,128,256,3,1,1,MINI_BATCH_SIZE); - CNNConfig* l10 = new CNNConfig(14,14,256,256,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l11 = new MaxpoolConfig(14,14,256,2,2,MINI_BATCH_SIZE); - ReLUConfig* l12 = new ReLUConfig(7*7*256,MINI_BATCH_SIZE); - - FCConfig* l13 = new FCConfig(7*7*256,MINI_BATCH_SIZE,1024); - ReLUConfig* l14 = new ReLUConfig(1024,MINI_BATCH_SIZE); - FCConfig* l15 = new FCConfig(1024,MINI_BATCH_SIZE,1024); - ReLUConfig* l16 = new ReLUConfig(1024,MINI_BATCH_SIZE); - FCConfig* l17 = new FCConfig(1024,MINI_BATCH_SIZE,200); - ReLUConfig* l18 = new ReLUConfig(200,MINI_BATCH_SIZE); + CNNConfig *l0 = new CNNConfig(56, 56, 3, 64, 7, 1, 3, MINI_BATCH_SIZE); + CNNConfig *l1 = new CNNConfig(56, 56, 64, 64, 5, 1, 2, MINI_BATCH_SIZE); + MaxpoolConfig *l2 = new MaxpoolConfig(56, 56, 64, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l3 = new ReLUConfig(28 * 28 * 64, MINI_BATCH_SIZE); + BNConfig *l4 = new BNConfig(28 * 28 * 64, MINI_BATCH_SIZE); + + CNNConfig *l5 = new CNNConfig(28, 28, 64, 128, 5, 1, 2, MINI_BATCH_SIZE); + MaxpoolConfig *l6 = new MaxpoolConfig(28, 28, 128, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l7 = new ReLUConfig(14 * 14 * 128, MINI_BATCH_SIZE); + BNConfig *l8 = new BNConfig(14 * 14 * 128, MINI_BATCH_SIZE); + + CNNConfig *l9 = new CNNConfig(14, 14, 128, 256, 3, 1, 1, MINI_BATCH_SIZE); + CNNConfig *l10 = new CNNConfig(14, 14, 256, 256, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l11 = new MaxpoolConfig(14, 14, 256, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l12 = new ReLUConfig(7 * 7 * 256, MINI_BATCH_SIZE); + + FCConfig *l13 = new FCConfig(7 * 7 * 256, MINI_BATCH_SIZE, 1024); + ReLUConfig *l14 = new ReLUConfig(1024, MINI_BATCH_SIZE); + FCConfig *l15 = new FCConfig(1024, MINI_BATCH_SIZE, 1024); + ReLUConfig *l16 = new ReLUConfig(1024, MINI_BATCH_SIZE); + FCConfig *l17 = new FCConfig(1024, MINI_BATCH_SIZE, 200); + ReLUConfig *l18 = new ReLUConfig(200, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1160,54 +1257,54 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon } else if (network.compare("VGG16") == 0) { - if(dataset.compare("MNIST") == 0) + if (dataset.compare("MNIST") == 0) assert(false && "No VGG16 on MNIST"); else if (dataset.compare("CIFAR10") == 0) { NUM_LAYERS = 37; WITH_NORMALIZATION = false; - CNNConfig* l0 = new CNNConfig(32,32,3,64,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l1 = new ReLUConfig(32*32*64,MINI_BATCH_SIZE); - CNNConfig* l2 = new CNNConfig(32,32,64,64,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l3 = new MaxpoolConfig(32,32,64,2,2,MINI_BATCH_SIZE); - ReLUConfig* l4 = new ReLUConfig(16*16*64,MINI_BATCH_SIZE); - - CNNConfig* l5 = new CNNConfig(16,16,64,128,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l6 = new ReLUConfig(16*16*128,MINI_BATCH_SIZE); - CNNConfig* l7 = new CNNConfig(16,16,128,128,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l8 = new MaxpoolConfig(16,16,128,2,2,MINI_BATCH_SIZE); - ReLUConfig* l9 = new ReLUConfig(8*8*128,MINI_BATCH_SIZE); - - CNNConfig* l10 = new CNNConfig(8,8,128,256,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l11 = new ReLUConfig(8*8*256,MINI_BATCH_SIZE); - CNNConfig* l12 = new CNNConfig(8,8,256,256,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l13 = new ReLUConfig(8*8*256,MINI_BATCH_SIZE); - CNNConfig* l14 = new CNNConfig(8,8,256,256,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l15 = new MaxpoolConfig(8,8,256,2,2,MINI_BATCH_SIZE); - ReLUConfig* l16 = new ReLUConfig(4*4*256,MINI_BATCH_SIZE); - - CNNConfig* l17 = new CNNConfig(4,4,256,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l18 = new ReLUConfig(4*4*512,MINI_BATCH_SIZE); - CNNConfig* l19 = new CNNConfig(4,4,512,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l20 = new ReLUConfig(4*4*512,MINI_BATCH_SIZE); - CNNConfig* l21 = new CNNConfig(4,4,512,512,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l22 = new MaxpoolConfig(4,4,512,2,2,MINI_BATCH_SIZE); - ReLUConfig* l23 = new ReLUConfig(2*2*512,MINI_BATCH_SIZE); - - CNNConfig* l24 = new CNNConfig(2,2,512,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l25 = new ReLUConfig(2*2*512,MINI_BATCH_SIZE); - CNNConfig* l26 = new CNNConfig(2,2,512,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l27 = new ReLUConfig(2*2*512,MINI_BATCH_SIZE); - CNNConfig* l28 = new CNNConfig(2,2,512,512,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l29 = new MaxpoolConfig(2,2,512,2,2,MINI_BATCH_SIZE); - ReLUConfig* l30 = new ReLUConfig(1*1*512,MINI_BATCH_SIZE); - - FCConfig* l31 = new FCConfig(1*1*512,MINI_BATCH_SIZE,4096); - ReLUConfig* l32 = new ReLUConfig(4096,MINI_BATCH_SIZE); - FCConfig* l33 = new FCConfig(4096, MINI_BATCH_SIZE, 4096); - ReLUConfig* l34 = new ReLUConfig(4096, MINI_BATCH_SIZE); - FCConfig* l35 = new FCConfig(4096, MINI_BATCH_SIZE, 1000); - ReLUConfig* l36 = new ReLUConfig(1000, MINI_BATCH_SIZE); + CNNConfig *l0 = new CNNConfig(32, 32, 3, 64, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l1 = new ReLUConfig(32 * 32 * 64, MINI_BATCH_SIZE); + CNNConfig *l2 = new CNNConfig(32, 32, 64, 64, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l3 = new MaxpoolConfig(32, 32, 64, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l4 = new ReLUConfig(16 * 16 * 64, MINI_BATCH_SIZE); + + CNNConfig *l5 = new CNNConfig(16, 16, 64, 128, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l6 = new ReLUConfig(16 * 16 * 128, MINI_BATCH_SIZE); + CNNConfig *l7 = new CNNConfig(16, 16, 128, 128, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l8 = new MaxpoolConfig(16, 16, 128, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l9 = new ReLUConfig(8 * 8 * 128, MINI_BATCH_SIZE); + + CNNConfig *l10 = new CNNConfig(8, 8, 128, 256, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l11 = new ReLUConfig(8 * 8 * 256, MINI_BATCH_SIZE); + CNNConfig *l12 = new CNNConfig(8, 8, 256, 256, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l13 = new ReLUConfig(8 * 8 * 256, MINI_BATCH_SIZE); + CNNConfig *l14 = new CNNConfig(8, 8, 256, 256, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l15 = new MaxpoolConfig(8, 8, 256, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l16 = new ReLUConfig(4 * 4 * 256, MINI_BATCH_SIZE); + + CNNConfig *l17 = new CNNConfig(4, 4, 256, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l18 = new ReLUConfig(4 * 4 * 512, MINI_BATCH_SIZE); + CNNConfig *l19 = new CNNConfig(4, 4, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l20 = new ReLUConfig(4 * 4 * 512, MINI_BATCH_SIZE); + CNNConfig *l21 = new CNNConfig(4, 4, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l22 = new MaxpoolConfig(4, 4, 512, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l23 = new ReLUConfig(2 * 2 * 512, MINI_BATCH_SIZE); + + CNNConfig *l24 = new CNNConfig(2, 2, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l25 = new ReLUConfig(2 * 2 * 512, MINI_BATCH_SIZE); + CNNConfig *l26 = new CNNConfig(2, 2, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l27 = new ReLUConfig(2 * 2 * 512, MINI_BATCH_SIZE); + CNNConfig *l28 = new CNNConfig(2, 2, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l29 = new MaxpoolConfig(2, 2, 512, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l30 = new ReLUConfig(1 * 1 * 512, MINI_BATCH_SIZE); + + FCConfig *l31 = new FCConfig(1 * 1 * 512, MINI_BATCH_SIZE, 4096); + ReLUConfig *l32 = new ReLUConfig(4096, MINI_BATCH_SIZE); + FCConfig *l33 = new FCConfig(4096, MINI_BATCH_SIZE, 4096); + ReLUConfig *l34 = new ReLUConfig(4096, MINI_BATCH_SIZE); + FCConfig *l35 = new FCConfig(4096, MINI_BATCH_SIZE, 1000); + ReLUConfig *l36 = new ReLUConfig(1000, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1250,48 +1347,48 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon { NUM_LAYERS = 37; WITH_NORMALIZATION = false; - CNNConfig* l0 = new CNNConfig(64,64,3,64,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l1 = new ReLUConfig(64*64*64,MINI_BATCH_SIZE); - CNNConfig* l2 = new CNNConfig(64,64,64,64,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l3 = new MaxpoolConfig(64,64,64,2,2,MINI_BATCH_SIZE); - ReLUConfig* l4 = new ReLUConfig(32*32*64,MINI_BATCH_SIZE); - - CNNConfig* l5 = new CNNConfig(32,32,64,128,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l6 = new ReLUConfig(32*32*128,MINI_BATCH_SIZE); - CNNConfig* l7 = new CNNConfig(32,32,128,128,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l8 = new MaxpoolConfig(32,32,128,2,2,MINI_BATCH_SIZE); - ReLUConfig* l9 = new ReLUConfig(16*16*128,MINI_BATCH_SIZE); - - CNNConfig* l10 = new CNNConfig(16,16,128,256,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l11 = new ReLUConfig(16*16*256,MINI_BATCH_SIZE); - CNNConfig* l12 = new CNNConfig(16,16,256,256,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l13 = new ReLUConfig(16*16*256,MINI_BATCH_SIZE); - CNNConfig* l14 = new CNNConfig(16,16,256,256,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l15 = new MaxpoolConfig(16,16,256,2,2,MINI_BATCH_SIZE); - ReLUConfig* l16 = new ReLUConfig(8*8*256,MINI_BATCH_SIZE); - - CNNConfig* l17 = new CNNConfig(8,8,256,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l18 = new ReLUConfig(8*8*512,MINI_BATCH_SIZE); - CNNConfig* l19 = new CNNConfig(8,8,512,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l20 = new ReLUConfig(8*8*512,MINI_BATCH_SIZE); - CNNConfig* l21 = new CNNConfig(8,8,512,512,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l22 = new MaxpoolConfig(8,8,512,2,2,MINI_BATCH_SIZE); - ReLUConfig* l23 = new ReLUConfig(4*4*512,MINI_BATCH_SIZE); - - CNNConfig* l24 = new CNNConfig(4,4,512,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l25 = new ReLUConfig(4*4*512,MINI_BATCH_SIZE); - CNNConfig* l26 = new CNNConfig(4,4,512,512,3,1,1,MINI_BATCH_SIZE); - ReLUConfig* l27 = new ReLUConfig(4*4*512,MINI_BATCH_SIZE); - CNNConfig* l28 = new CNNConfig(4,4,512,512,3,1,1,MINI_BATCH_SIZE); - MaxpoolConfig* l29 = new MaxpoolConfig(4,4,512,2,2,MINI_BATCH_SIZE); - ReLUConfig* l30 = new ReLUConfig(2*2*512,MINI_BATCH_SIZE); - - FCConfig* l31 = new FCConfig(2*2*512,MINI_BATCH_SIZE,2048); - ReLUConfig* l32 = new ReLUConfig(2048,MINI_BATCH_SIZE); - FCConfig* l33 = new FCConfig(2048, MINI_BATCH_SIZE, 2048); - ReLUConfig* l34 = new ReLUConfig(2048, MINI_BATCH_SIZE); - FCConfig* l35 = new FCConfig(2048, MINI_BATCH_SIZE, 200); - ReLUConfig* l36 = new ReLUConfig(200, MINI_BATCH_SIZE); + CNNConfig *l0 = new CNNConfig(64, 64, 3, 64, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l1 = new ReLUConfig(64 * 64 * 64, MINI_BATCH_SIZE); + CNNConfig *l2 = new CNNConfig(64, 64, 64, 64, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l3 = new MaxpoolConfig(64, 64, 64, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l4 = new ReLUConfig(32 * 32 * 64, MINI_BATCH_SIZE); + + CNNConfig *l5 = new CNNConfig(32, 32, 64, 128, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l6 = new ReLUConfig(32 * 32 * 128, MINI_BATCH_SIZE); + CNNConfig *l7 = new CNNConfig(32, 32, 128, 128, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l8 = new MaxpoolConfig(32, 32, 128, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l9 = new ReLUConfig(16 * 16 * 128, MINI_BATCH_SIZE); + + CNNConfig *l10 = new CNNConfig(16, 16, 128, 256, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l11 = new ReLUConfig(16 * 16 * 256, MINI_BATCH_SIZE); + CNNConfig *l12 = new CNNConfig(16, 16, 256, 256, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l13 = new ReLUConfig(16 * 16 * 256, MINI_BATCH_SIZE); + CNNConfig *l14 = new CNNConfig(16, 16, 256, 256, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l15 = new MaxpoolConfig(16, 16, 256, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l16 = new ReLUConfig(8 * 8 * 256, MINI_BATCH_SIZE); + + CNNConfig *l17 = new CNNConfig(8, 8, 256, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l18 = new ReLUConfig(8 * 8 * 512, MINI_BATCH_SIZE); + CNNConfig *l19 = new CNNConfig(8, 8, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l20 = new ReLUConfig(8 * 8 * 512, MINI_BATCH_SIZE); + CNNConfig *l21 = new CNNConfig(8, 8, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l22 = new MaxpoolConfig(8, 8, 512, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l23 = new ReLUConfig(4 * 4 * 512, MINI_BATCH_SIZE); + + CNNConfig *l24 = new CNNConfig(4, 4, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l25 = new ReLUConfig(4 * 4 * 512, MINI_BATCH_SIZE); + CNNConfig *l26 = new CNNConfig(4, 4, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + ReLUConfig *l27 = new ReLUConfig(4 * 4 * 512, MINI_BATCH_SIZE); + CNNConfig *l28 = new CNNConfig(4, 4, 512, 512, 3, 1, 1, MINI_BATCH_SIZE); + MaxpoolConfig *l29 = new MaxpoolConfig(4, 4, 512, 2, 2, MINI_BATCH_SIZE); + ReLUConfig *l30 = new ReLUConfig(2 * 2 * 512, MINI_BATCH_SIZE); + + FCConfig *l31 = new FCConfig(2 * 2 * 512, MINI_BATCH_SIZE, 2048); + ReLUConfig *l32 = new ReLUConfig(2048, MINI_BATCH_SIZE); + FCConfig *l33 = new FCConfig(2048, MINI_BATCH_SIZE, 2048); + ReLUConfig *l34 = new ReLUConfig(2048, MINI_BATCH_SIZE); + FCConfig *l35 = new FCConfig(2048, MINI_BATCH_SIZE, 200); + ReLUConfig *l36 = new ReLUConfig(200, MINI_BATCH_SIZE); config->addLayer(l0); config->addLayer(l1); config->addLayer(l2); @@ -1335,10 +1432,10 @@ void selectNetwork(string network, string dataset, string security, NeuralNetCon assert(false && "Only SecureML, Sarda, Gazelle, LeNet, AlexNet, and VGG16 Networks supported"); } -void runOnly(NeuralNetwork* net, size_t l, string what, string& network) +void runOnly(NeuralNetwork *net, size_t l, string what, string &network) { size_t total_layers = net->layers.size(); - assert((l >= 0 and l < total_layers) && "Incorrect layer number for runOnly"); + assert((l >= 0 and l < total_layers) && "Incorrect layer number for runOnly"); network = network + " L" + std::to_string(l) + " " + what; if (what.compare("F") == 0) @@ -1346,29 +1443,24 @@ void runOnly(NeuralNetwork* net, size_t l, string what, string& network) if (l == 0) net->layers[0]->forward(net->inputData); else - net->layers[l]->forward(*(net->layers[l-1]->getActivation())); + net->layers[l]->forward(*(net->layers[l - 1]->getActivation())); } else if (what.compare("D") == 0) { if (l != 0) - net->layers[l]->computeDelta(*(net->layers[l-1]->getDelta())); + net->layers[l]->computeDelta(*(net->layers[l - 1]->getDelta())); } else if (what.compare("U") == 0) { if (l == 0) net->layers[0]->updateEquations(net->inputData); else - net->layers[l]->updateEquations(*(net->layers[l-1]->getActivation())); + net->layers[l]->updateEquations(*(net->layers[l - 1]->getActivation())); } else assert(false && "Only F,D or U allowed in runOnly"); } - - - - - /********************* COMMUNICATION AND HELPERS *********************/ void start_m() @@ -1410,12 +1502,11 @@ void end_time(string str) clock_gettime(CLOCK_REALTIME, &requestEnd); cout << "----------------------------------------------" << endl; cout << "Wall Clock time for " << str << ": " << diff(requestStart, requestEnd) << " sec\n"; - cout << "CPU time for " << str << ": " << (double)(clock() - tStart)/CLOCKS_PER_SEC << " sec\n"; - cout << "----------------------------------------------" << endl; + cout << "CPU time for " << str << ": " << (double)(clock() - tStart) / CLOCKS_PER_SEC << " sec\n"; + cout << "----------------------------------------------" << endl; alreadyMeasuringTime = false; } - void start_rounds() { if (alreadyMeasuringRounds) @@ -1440,7 +1531,7 @@ void end_rounds(string str) cout << "----------------------------------------------" << endl; cout << "Send Round Complexity of " << str << ": " << roundComplexitySend << endl; cout << "Recv Round Complexity of " << str << ": " << roundComplexityRecv << endl; - cout << "----------------------------------------------" << endl; + cout << "----------------------------------------------" << endl; alreadyMeasuringRounds = false; } @@ -1468,49 +1559,47 @@ void aggregateCommunication() if (partyNum == PARTY_A) { cout << "----------------------------------------------" << endl; - cout << "Total communication: " << (float)vec[0]/1000000 << "MB (sent) and " << (float)vec[1]/1000000 << "MB (recv)\n"; + cout << "Total communication: " << (float)vec[0] / 1000000 << "MB (sent) and " << (float)vec[1] / 1000000 << "MB (recv)\n"; cout << "Total calls: " << vec[2] << " (sends) and " << vec[3] << " (recvs)" << endl; cout << "----------------------------------------------" << endl; } } - -void print_usage (const char * bin) +void print_usage(const char *bin) { - cout << "Usage: ./" << bin << " PARTY_NUM IP_ADDR_FILE AES_SEED_INDEP AES_SEED_NEXT AES_SEED_PREV" << endl; - cout << endl; - cout << "Required Arguments:\n"; - cout << "PARTY_NUM Party Identifier (0,1, or 2)\n"; - cout << "IP_ADDR_FILE \tIP Address file (use makefile for automation)\n"; - cout << "AES_SEED_INDEP \tAES seed file independent\n"; - cout << "AES_SEED_NEXT \t \tAES seed file next\n"; - cout << "AES_SEED_PREV \t \tAES seed file previous\n"; - cout << endl; - cout << "Report bugs to swagh@princeton.edu" << endl; - exit(-1); + cout << "Usage: ./" << bin << " PARTY_NUM IP_ADDR_FILE AES_SEED_INDEP AES_SEED_NEXT AES_SEED_PREV" << endl; + cout << endl; + cout << "Required Arguments:\n"; + cout << "PARTY_NUM Party Identifier (0,1, or 2)\n"; + cout << "IP_ADDR_FILE \tIP Address file (use makefile for automation)\n"; + cout << "AES_SEED_INDEP \tAES seed file independent\n"; + cout << "AES_SEED_NEXT \t \tAES seed file next\n"; + cout << "AES_SEED_PREV \t \tAES seed file previous\n"; + cout << endl; + cout << "Report bugs to swagh@princeton.edu" << endl; + exit(-1); } double diff(timespec start, timespec end) { - timespec temp; - - if ((end.tv_nsec-start.tv_nsec)<0) - { - temp.tv_sec = end.tv_sec-start.tv_sec-1; - temp.tv_nsec = 1000000000+end.tv_nsec-start.tv_nsec; - } - else - { - temp.tv_sec = end.tv_sec-start.tv_sec; - temp.tv_nsec = end.tv_nsec-start.tv_nsec; - } - return temp.tv_sec + (double)temp.tv_nsec/NANOSECONDS_PER_SEC; -} + timespec temp; + if ((end.tv_nsec - start.tv_nsec) < 0) + { + temp.tv_sec = end.tv_sec - start.tv_sec - 1; + temp.tv_nsec = 1000000000 + end.tv_nsec - start.tv_nsec; + } + else + { + temp.tv_sec = end.tv_sec - start.tv_sec; + temp.tv_nsec = end.tv_nsec - start.tv_nsec; + } + return temp.tv_sec + (double)temp.tv_nsec / NANOSECONDS_PER_SEC; +} void deleteObjects() { - //close connection + // close connection for (int i = 0; i < NUM_OF_PARTIES; i++) { if (i != partyNum) @@ -1524,18 +1613,17 @@ void deleteObjects() delete[] addrs; } - /************************ AlexNet on ImageNet ************************/ // NUM_LAYERS = 21; // WITH_NORMALIZATION = false; // CNNConfig* l0 = new CNNConfig(227,227,3,96,11,4,0,MINI_BATCH_SIZE); // MaxpoolConfig* l1 = new MaxpoolConfig(55,55,96,3,2,MINI_BATCH_SIZE); -// ReLUConfig* l2 = new ReLUConfig(27*27*96,MINI_BATCH_SIZE); +// ReLUConfig* l2 = new ReLUConfig(27*27*96,MINI_BATCH_SIZE); // BNConfig * l3 = new BNConfig(27*27*96,MINI_BATCH_SIZE); // CNNConfig* l4 = new CNNConfig(27,27,96,256,5,1,2,MINI_BATCH_SIZE); // MaxpoolConfig* l5 = new MaxpoolConfig(27,27,256,3,2,MINI_BATCH_SIZE); -// ReLUConfig* l6 = new ReLUConfig(13*13*256,MINI_BATCH_SIZE); +// ReLUConfig* l6 = new ReLUConfig(13*13*256,MINI_BATCH_SIZE); // BNConfig * l7 = new BNConfig(13*13*256,MINI_BATCH_SIZE); // CNNConfig* l8 = new CNNConfig(13,13,256,384,3,1,1,MINI_BATCH_SIZE); @@ -1574,12 +1662,11 @@ void deleteObjects() // config->addLayer(l19); // config->addLayer(l20); - /************************ VGG16 on ImageNet ************************/ // NUM_LAYERS = 37; // WITH_NORMALIZATION = false; // CNNConfig* l0 = new CNNConfig(224,224,3,64,3,1,1,MINI_BATCH_SIZE); -// ReLUConfig* l1 = new ReLUConfig(224*224*64,MINI_BATCH_SIZE); +// ReLUConfig* l1 = new ReLUConfig(224*224*64,MINI_BATCH_SIZE); // CNNConfig* l2 = new CNNConfig(224,224,64,64,3,1,1,MINI_BATCH_SIZE); // MaxpoolConfig* l3 = new MaxpoolConfig(224,224,64,2,2,MINI_BATCH_SIZE); // ReLUConfig* l4 = new ReLUConfig(112*112*64,MINI_BATCH_SIZE); diff --git a/src/tools.h b/src/tools.h index 80cdfcf..36ddb5e 100755 --- a/src/tools.h +++ b/src/tools.h @@ -22,6 +22,9 @@ #include "connect.h" #include "globals.h" +// custom heading +#include "randBit.h" + extern int partyNum; extern AESObject* aes_next; diff --git a/src/unitTests.cpp b/src/unitTests.cpp index d62833e..b5caad7 100755 --- a/src/unitTests.cpp +++ b/src/unitTests.cpp @@ -6,7 +6,11 @@ void runTest(string str, string whichTest, string &network) { if (str.compare("Debug") == 0) { - if (whichTest.compare("Mat-Mul") == 0) + if (whichTest.compare("RandBit") == 0) + { + network = "Debug RandBit"; + debugRandBit(); + } else if (whichTest.compare("Mat-Mul") == 0) { network = "Debug Mat-Mul"; debugMatMul(); diff --git a/tutorial/compare_with RingPPML.ipynb b/tutorial/compare_with RingPPML.ipynb new file mode 100644 index 0000000..e7c436a --- /dev/null +++ b/tutorial/compare_with RingPPML.ipynb @@ -0,0 +1,444 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import random" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "# Data from BitLT\n", + "bitLt_data = [\n", + " # {\"Ring size\": 1, \"Size\": 32768, \"Batch size\": 1, \"With Rand\": 0.058, \"Without Rand\": 0.007, \"Eval Time\": 0.028667},\n", + " # {\"Ring size\": 1, \"Size\": 32768, \"Batch size\": 10, \"With Rand\": 0.0530667, \"Without Rand\": 0.048667, \"Eval Time\": 0.0237333},\n", + " # {\"Ring size\": 1, \"Size\": 32768, \"Batch size\": 100, \"With Rand\": 5.73033, \"Without Rand\": 0.452333, \"Eval Time\": 2.641},\n", + " # {\"Ring size\": 1, \"Size\": 32768, \"Batch size\": 1000, \"With Rand\": 45.8927, \"Without Rand\": 5.216, \"Eval Time\": 19.6963},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1, \"With Rand\": 0.114667, \"Without Rand\": 0.013, \"Eval Time\": 2.23533},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 10, \"With Rand\": 0.764333, \"Without Rand\": 0.074667, \"Eval Time\": 6.69233},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 100, \"With Rand\": 7.51267, \"Without Rand\": 0.681333, \"Eval Time\": 79.6773},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1000, \"With Rand\": 40.541, \"Without Rand\": 8.46567, \"Eval Time\": 571.868},\n", + "]\n", + "\n", + "# Data from private compare\n", + "pc_data = [\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1, \"Without Rand\": 0.003, \"Eval Time\": 2.34533},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 10, \"Without Rand\": 0.023667, \"Eval Time\": 10.8127},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 100, \"Without Rand\": 2.004, \"Eval Time\": 100.561},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1000, \"Without Rand\": 13.048, \"Eval Time\": 1091.19},\n", + "]\n", + "\n", + "# Data from DReLU\n", + "DReLU_data = [\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1, \"Eval Time\": 2.253 },\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 10, \"Eval Time\": 10.5253 },\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 100, \"Eval Time\": 110.227},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1000, \"Eval Time\": 1477.15 },\n", + "]\n", + "\n", + "# Data from MSB\n", + "MSB_data = [\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1, \"Eval Time\": 1.17733},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 10, \"Eval Time\": 7.45767},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 100, \"Eval Time\": 97.4273},\n", + " {\"Ring size\": 32, \"Size\": 1024, \"Batch size\": 1000, \"Eval Time\": 796.407},\n", + "]\n" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1AAAAIyCAYAAADFUCvUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAADBHElEQVR4nOzdd3gU1dvG8e+mN0INHanSq6CISBMwVEXpWABFRKogiCgdAZEiIPUFBVGQLigdpUgTBEFREKQjvQdCSDbZef+YX1aWJJCEhEm5P9eVK7OzszvPzu7s7rPnnOfYDMMwEBERERERkQdyszoAERERERGR1EIJlIiIiIiISDwpgRIREREREYknJVAiIiIiIiLxpARKREREREQknpRAiYiIiIiIxJMSKBERERERkXhSAiUiIiIiIhJPSqBERERERETiSQmUhWw2G4MHD7Zk35s2bcJms7Fp0yZL9h8fVh6f2NSsWZPSpUtbHUaadeLECWw2G7Nnz7Y6FAA6d+5M3bp1E337wYMHY7PZkjCitOvpp5/m/ffftzqMGB7lc1igQAHatWv3SPYlKceuXbvw8vLi5MmTSX7f7dq1o0CBAkl+vwm1Zs0aAgICuHTpktWhJEhK+0xKbdq1a0dAQECy7sPK74npPoGaPXs2Npstzr9ffvnF6hAfypQpU1LMyf+gYx39lxLe8B+VESNGsGzZMqvDSDbRX0Cj/9zc3MiVKxeNGjWK17m1atWqON8cbTYbXbt2dVnXrl27eL3GHvRF9fjx48ycOZMPP/zQuS76w/Tux5IlSxbq16/Pjh07HvhYHqV7j0NgYCDlypVj7NixhIeHO7e79/nx8/OjZMmS9O/fn5CQEOd2d5+7W7dujbE/wzDIly8fNpuNRo0auVwX2/N0r759+zJ58mTOnz//kI/cGvE93lY6cOAAgwcP5sSJE1aHInf56KOPaN26Nfnz53euq1mzZpzvXX///beF0SZOvXr1KFKkCCNHjrQ6FKcffviBGjVqkD17dvz8/ChUqBAtWrRgzZo1VocGmJ83NWvWjPf2974HeXh4kC9fPlq1asWBAwcSFcPZs2cZPHgw+/btS9Tt0zoPqwNIKYYOHUrBggVjrC9SpIgF0SSdKVOmkC1bthhfGKtXr05YWBheXl6PLJbq1avz9ddfu6zr0KEDTz31FB07dnSui/7FIiwsDA+PtP0SHTFiBM2aNaNJkyZWh5Kspk6dSkBAAA6Hg9OnTzNjxgyqV6/Orl27KF++PAD58+cnLCwMT09P5+1WrVrF5MmT4/0L09tvv02dOnWcl48fP87AgQPp2LEj1apVc64vXLjwfe9nwoQJFCxYkFq1asW4rnXr1jRo0ICoqCgOHz7MlClTqFWrFr/++itlypRxbte/f38++OCDeMWdHLy9vZk5cyYA169fZ8mSJfTu3Ztff/2V+fPnu2wb/fzcunWLdevWMXz4cDZs2MC2bdtcWmB8fHyYN28ezz77rMvtN2/ezL///ou3t3eiYn3xxRcJDAxkypQpDB06NFH3YbWEHO/YHDp0CDe35PtN88CBAwwZMoSaNWumqx+pUrJ9+/bx448/sn379hjX5c2bN9aEI3fu3I8itCT39ttv07t3b4YMGUKGDBksjWXMmDH06dOHGjVq0K9fP/z8/Dhy5Ag//vgj8+fPp169ekDsn0nJKTIyks2bN1O7dm2X9Xfu3GHnzp3UqFHjvre/+z0oMjKSo0ePMm3aNNasWcOBAwcS/No5e/YsQ4YMoUCBAs7P6ZTGyu+JafvbaQLUr1+fSpUqWR3GI+Pm5oaPj88j3WehQoUoVKiQy7pOnTpRqFAhXn311RjbP+r4JPk0a9aMbNmyOS83adKE0qVLs2jRIucbs81me+jnvEqVKlSpUsV5effu3QwcOJAqVarE+hqLjd1uZ+7cuXTq1CnW65944gmX+6pWrRr169dn6tSpTJkyxbnew8PD0h8APDw8XOLs3LkzlStXZsGCBYwbN87lw/Tu56dTp040bdqUpUuX8ssvv7gczwYNGrBo0SImTpzo8tjmzZtHxYoVuXz5cqJidXNzo1mzZsyZM4chQ4akyq6PCTne0QzD4M6dO/j6+iY6+ZRH786dO3h5eT10wjtr1iwee+wxnn766RjXZcyYMd7vWalB06ZN6datG4sWLeKNN96wLI7IyEiGDRtG3bp1WbduXYzrL1686FxOis+khDh//jx9+vShRIkSvPvuu4A53KJz5840bNjwgQnUve9BYHaPbtSoEStXruStt95KrtAtY+X3xHTfhS8+7HY7WbJkoX379jGuCwkJwcfHh969ewMQERHBwIEDqVixIhkzZsTf359q1aqxcePGB+4nrv7KsfXDnzVrFs899xzZs2fH29ubkiVLMnXqVJdtChQowF9//cXmzZudzbrRTcJxjYFatGgRFStWxNfXl2zZsvHqq69y5syZGHEGBARw5swZmjRpQkBAAEFBQfTu3ZuoqKgHPs74urdva/RxOHz4MK+++ioZM2YkKCiIAQMGYBgGp0+fdv6SnTNnTsaOHRvjPsPDwxk0aBBFihTB29ubfPny8f777yeom82ePXt45pln8PX1pWDBgkybNi1R+7HZbISGhvLVV1+5dC37448/sNlsfP/99y77tNlsPPHEEy77qV+/PpUrV3ZZt3r1aqpVq4a/vz8ZMmSgYcOG/PXXXzFi/Pvvv2nWrBlZsmTBx8eHSpUquewT/uu6tW3bNnr16kVQUBD+/v689NJLD9WfPWfOnAAuX8Lv7W/erl07Jk+e7DxW0X/JbevWrVy+fNmlJet+olu2jh496rI+tvM2ujvbsmXLKF26NN7e3pQqVSrWbiObNm2iUqVK+Pj4ULhwYaZPn/5QY3Lc3Nyc5/+DunE999xzgNmCd7fWrVtz5coV1q9f71wXERHB4sWLadOmTaLiila3bl1OnjwZr+4iY8aM4ZlnniFr1qz4+vpSsWJFFi9eHGO7hBzvrVu38uSTT7oc74cR2/EuUKAAjRo1Yu3atVSqVAlfX1/nfu4eA7V7925sNhtfffVVjPtdu3YtNpuNFStWAHDy5Ek6d+5MsWLF8PX1JWvWrDRv3tzlOZ49ezbNmzcHoFatWs5z6e73//i+b8Tm+vXr9OzZkwIFCuDt7U3evHl5/fXXXRLqixcv8uabb5IjRw58fHwoV65cjMcX/R4wZswYJk+eTKFChfDz8+P555/n9OnTGIbBsGHDyJs3L76+vrz44otcvXrV5T6ij/G6desoX748Pj4+lCxZkqVLl7psd/XqVXr37k2ZMmUICAggMDCQ+vXr8/vvv7tsF/1ZOX/+fPr370+ePHnw8/NzdnHduXMn9erVI2PGjPj5+VGjRg22bdsWr+O2bNkynnvuuQSf08uXL6dhw4bkzp0bb29vChcuzLBhw+L1+etwOJgwYQJlypTBx8eHoKAg6tWrx+7du53bRCcZhQsXxtvbmwIFCvDhhx/G+JyMPtZbt27lqaeewsfHh0KFCjFnzpwY+82ePTtly5Zl+fLl941vzJgx2Gy2WMeE9evXDy8vL65duwbAP//8Q9OmTcmZMyc+Pj7kzZuXVq1acePGjTjv//Lly4SEhFC1atVYr8+ePbtz+d7PpOjXQnyGHiTmfMqbNy+7d++mTp06tGnThl27djFixAgWLlzI6NGj73vbuMT2WRuf1/6mTZt48sknAWjfvr3zcd49JGTnzp00aNCAzJkz4+/vT9myZZkwYUKMGBL7XXH37t0EBweTLVs25/ete5Pvu78n3tvN/t6/uz3MeRtNLVD/c+PGjRi/ntpsNrJmzYqnpycvvfQSS5cuZfr06S7d3pYtW0Z4eDitWrUCzIRq5syZtG7dmrfeeoubN2/yxRdfEBwc7NJd6WFNnTqVUqVK8cILL+Dh4cEPP/xA586dcTgcdOnSBYDx48fTrVs3AgIC+OijjwDIkSNHnPc5e/Zs2rdvz5NPPsnIkSO5cOECEyZMYNu2bezdu5dMmTI5t42KiiI4OJjKlSszZswYfvzxR8aOHUvhwoV55513kuQxxqVly5aUKFGCTz75hJUrV/Lxxx+TJUsWpk+fznPPPceoUaOYO3cuvXv35sknn6R69eqA+cHxwgsvsHXrVjp27EiJEiXYv38/n332GYcPH47XWKRr167RoEEDWrRoQevWrVm4cCHvvPMOXl5ezhM7vvv5+uuvY3RhLFy4MKVLlyZTpkz8/PPPvPDCCwBs2bIFNzc3fv/9d0JCQggMDMThcLB9+3aX7o9ff/01bdu2JTg4mFGjRnH79m2mTp3Ks88+y969e51v8n/99RdVq1YlT548fPDBB/j7+7Nw4UKaNGnCkiVLeOmll1wed7du3cicOTODBg3ixIkTjB8/nq5du7JgwYJ4PWfRX3IcDgdnzpxh2LBh+Pj40KJFizhv8/bbb3P27FnWr18fo+tnctq+fTs2m40KFSrEa/voL6qZM2eO1/Zbt25l6dKldO7cmQwZMjBx4kSaNm3KqVOnyJo1KwB79+6lXr165MqViyFDhhAVFcXQoUMJCgpK1GOKFp3kRe8nodsVKFCAKlWq8O2331K/fn3A/KJw48YNWrVqxcSJExMdW8WKFQHYtm3bA4/9hAkTeOGFF3jllVeIiIhg/vz5NG/enBUrVtCwYUOXbeNzvPfv38/zzz9PUFAQgwcPJjIykkGDBt33/TI+YjuOhw4donXr1rz99tu89dZbFCtWLMbtKlWqRKFChVi4cCFt27Z1uW7BggVkzpyZ4OBgAH799Ve2b99Oq1atyJs3LydOnGDq1KnUrFmTAwcO4OfnR/Xq1enevTsTJ07kww8/pESJEgDO//F934jNrVu3qFatGgcPHuSNN97giSee4PLly3z//ff8+++/ZMuWjbCwMGrWrMmRI0fo2rUrBQsWZNGiRbRr147r16/To0cPl/ucO3cuERERdOvWjatXr/Lpp5/SokULnnvuOTZt2kTfvn05cuQIn3/+Ob179+bLL790uf0///xDy5Yt6dSpE23btmXWrFk0b96cNWvWOAvDHDt2jGXLltG8eXMKFizIhQsXmD59OjVq1Ii1u9OwYcPw8vKid+/ehIeH4+XlxYYNG6hfvz4VK1Zk0KBBuLm5OX/c3LJlC0899VScx+3MmTOcOnUqxo9i0aKiomJ8J/Hx8SEgIIDZs2cTEBBAr169CAgIYMOGDQwcOJCQkJAHftF+8803mT17NvXr16dDhw5ERkayZcsWfvnlF2cvnA4dOvDVV1/RrFkz3nvvPXbu3MnIkSM5ePAg3333ncv9HTlyhGbNmvHmm2/Stm1bvvzyS9q1a0fFihUpVaqUy7YVK1Z84OdsixYteP/991m4cCF9+vRxuW7hwoU8//zzZM6cmYiICIKDgwkPD6dbt27kzJmTM2fOsGLFCq5fv07GjBljvf/s2bPj6+vLDz/8QLdu3ciSJct947lbiRIlYnwWXb9+nV69erkkXg9zPkWPr737ckIS7OjXTFRUFMeOHaNv375kzZrVZWxqfF77JUqUYOjQoTG6wD/zzDMArF+/nkaNGpErVy569OhBzpw5OXjwICtWrHA5nxP7XfHixYvO9+QPPviATJkyceLEiRg/hNwtKCgoxvNjt9vp2bOny/f2hzlvXRjp3KxZswwg1j9vb2/ndmvXrjUA44cffnC5fYMGDYxChQo5L0dGRhrh4eEu21y7ds3IkSOH8cYbb7isB4xBgwY5L7dt29bInz9/jBgHDRpk3PtU3b59O8Z2wcHBLrEYhmGUKlXKqFGjRoxtN27caADGxo0bDcMwjIiICCN79uxG6dKljbCwMOd2K1asMABj4MCBLnECxtChQ13us0KFCkbFihVj7Ot+/P39jbZt28Z63b3HJ/o4dOzY0bkuMjLSyJs3r2Gz2YxPPvnEuf7atWuGr6+vy31//fXXhpubm7FlyxaX/UybNs0AjG3btt031ho1ahiAMXbsWOe68PBwo3z58kb27NmNiIiIBO8nrsffsGFD46mnnnJefvnll42XX37ZcHd3N1avXm0YhmH89ttvBmAsX77cMAzDuHnzppEpUybjrbfecrmv8+fPGxkzZnRZX7t2baNMmTLGnTt3nOscDofxzDPPGI8//rhzXfT5UadOHcPhcDjX9+zZ03B3dzeuX79+32MW/Zzd+5cpUyZjzZo1LtseP37cAIxZs2Y513Xp0iXGaz8aYHTp0uW++//1119j3OeDvPrqq0bWrFljrI+Ob8iQIcalS5eM8+fPG1u2bDGefPJJAzAWLVrksn1s5y1geHl5GUeOHHGu+/333w3A+Pzzz53rGjdubPj5+Rlnzpxxrvvnn38MDw+POI/H3dq2bWv4+/sbly5dMi5dumQcOXLEGDFihGGz2YyyZcvGiPHQoUPGpUuXjOPHjxvTp083vL29jRw5chihoaGGYfz3Ovj111+NSZMmGRkyZHC+BzVv3tyoVauWYRiGkT9/fqNhw4YxHvODnqdoXl5exjvvvPPA7e59/4uIiDBKly5tPPfcczH2HZ/j3aRJE8PHx8c4efKkc92BAwcMd3f3JD3e+fPnN4AYr/3o6+5+L+jXr5/h6elpXL161bkuPDzcyJQpk8tnSWyfBTt27DAAY86cOc51ixYtcnnPj5aQ943YDBw40ACMpUuXxrgu+j1j/PjxBmB88803zusiIiKMKlWqGAEBAUZISIhhGP+dY0FBQS7vLf369TMAo1y5cobdbneub926teHl5eXyPhZ9jJcsWeJcd+PGDSNXrlxGhQoVnOvu3LljREVFucR7/Phxw9vb2+WzLfqzslChQi7H2uFwGI8//rgRHBzs8t54+/Zto2DBgkbdunXve9x+/PHHWL9TGMZ/nzX3/kW/PmJ7zt9++23Dz8/P5Vjc+51iw4YNBmB07949xu2jH8O+ffsMwOjQoYPL9b179zYAY8OGDc510cf6559/dq67ePGi4e3tbbz33nsx9jFixAgDMC5cuBDHUTFVqVIlxneJXbt2ubym9+7dG+v7bnxEv2b9/f2N+vXrG8OHDzf27NkTY7vYPpPu5nA4jEaNGhkBAQHGX3/9ZRjGw51P//77r1GpUiWjdevWxq5du4waNWoYGzduNIoXL2706dPnvo8p+nvZvX958uSJ8dji+9qP6/MzMjLSKFiwoJE/f37j2rVrMY7JvTEl5rvid9995/zMuZ97vyfeq3Pnzoa7u7vzdfuw5+3d1IXvfyZPnsz69etd/lavXu28/rnnniNbtmwuv7hfu3aN9evX07JlS+c6d3d3Z6brcDi4evUqkZGRVKpUid9++y3J4vX19XUuR7ee1ahRg2PHjt23+Touu3fv5uLFi3Tu3NmlT2nDhg0pXrw4K1eujHGbe8eIVKtWjWPHjiV43wnVoUMH57K7uzuVKlXCMAzefPNN5/pMmTJRrFgxl3gWLVpEiRIlKF68OJcvX3b+RXdXik83Sw8PD95++23nZS8vL95++20uXrzInj17kmw/1apV47fffiM0NBQwf0Vv0KAB5cuXZ8uWLYDZKmWz2ZwD+tevX8/169dp3bq1y37d3d2pXLmyc79Xr15lw4YNtGjRgps3bzq3u3LlCsHBwfzzzz8xum127NjR5VewatWqERUVFe/Su0uWLGH9+vWsW7eOWbNmUbRoUZo2bRrr4GmrXbly5b6tSYMGDSIoKIicOXM6f3kfO3YszZo1i9f916lTx6WIRdmyZQkMDHS+VqOiovjxxx9p0qSJy6/gRYoUcbb6xEdoaChBQUEEBQVRpEgRPvzwQ6pUqRLjF2SAYsWKERQURMGCBXn77bcpUqQIK1euxM/PL8a2LVq0ICwsjBUrVnDz5k1WrFjx0N33omXOnDle46jufv+7du0aN27ccJ4z94rP8V67di1NmjThsccec25XokQJZytPfMT3eBcsWDBe99uyZUvsdrvLL67r1q3j+vXrLp85dx8Lu93OlStXKFKkCJkyZYrXZ0583zfismTJEsqVKxej1RpwvmesWrWKnDlz0rp1a+d1np6edO/enVu3brF582aX2zVv3tylBSG6m/Krr77q0hWpcuXKRERExHi/yp07t0s8gYGBvP766+zdu9dZ6dHb29v5S39UVBRXrlwhICCAYsWKxXrc2rZt63Ks9+3bxz///EObNm24cuWK87iFhoZSu3Ztfv75ZxwOR5zH7cqVK0DcLdcFChSI8Z0kutT/3XFEv4dXq1aN27dv37dK35IlS7DZbAwaNCjGdXc/VwC9evVyuf69994DiPFdoGTJki4FeoKCgmJ89kaLfqwPOsdbtmzJnj17XLpFL1iwAG9vb1588UUA5+tj7dq13L59+773d68hQ4Ywb948KlSowNq1a/noo4+oWLEiTzzxBAcPHoz3/QwbNowVK1Ywe/ZsSpYsCTzc+ZQjRw5GjhzJvHnznL0Natasyd69e2O0rMfGx8fH+VpZu3Yt06dPJyAggAYNGnD48GHndgl97d9r7969HD9+nHfffdelZxIQa2tZYr4rRt/vihUrsNvtD4wpNnPmzGHKlCl8+umnzoJQD3ve3k1d+P7nqaeeum8RCQ8PD5o2bcq8efMIDw/H29ubpUuXYrfbXT7MAL766ivGjh3L33//7fLEx1blL7G2bdvGoEGD2LFjR4w3jxs3bsTZfB2X6C/CsXUnKV68eIzSxdF9p++WOXNmZ9/k5HT3lxww30h9fHxcihREr4/+kAKzW8fBgwfj7AZ19+DRuOTOnRt/f3+XdUWLFgXMrlxPP/10kuynWrVqREZGsmPHDvLly8fFixepVq0af/31l0sCVbJkSWcXhH/++Qf4b/zKvQIDAwGzy4VhGAwYMIABAwbEGWOePHmcl+895tEfhPF9vqtXr+7y/DRr1ozHH3+cbt26ORPPlMT8YSt2HTt2pHnz5ty5c4cNGzYwceLEBI39u/dYguu5c/HiRcLCwmKtAJqQqqA+Pj788MMPgPmBWbBgQfLmzRvrtkuWLCEwMBBPT0/y5s173yqFQUFB1KlTh3nz5nH79m2ioqLinTw+iGEY8equsmLFCj7++GP27dsXY1zhvR50vC9dukRYWBiPP/54jO2KFSvm/EL5IPE93vH9HChXrhzFixdnwYIFzh+HFixYQLZs2VzO8bCwMEaOHMmsWbM4c+aMy2s3Pj+mxfd9Iy5Hjx6ladOm993m5MmTPP744zGKLkR3Ibz3h5jY3uMB8uXLF+v6e9+HihQpEuO1cPf7dM6cOZ1jgaZMmcLx48ddzuHYurje+7xFH7d7u1je7caNGw/s2hvXe42/v3+c4zD/+usv+vfvz4YNG1ymG4jeZ1yOHj1K7ty579tt7eTJk7i5ucV4r8mZMyeZMmV64HMFcX8XiH6sDzrHmzdvTq9evViwYAEffvghhmGwaNEi6tev73w9FixYkF69ejFu3Djmzp1LtWrVeOGFF5zjox+kdevWtG7dmpCQEHbu3Mns2bOZN28ejRs35s8//3xgcYI1a9YwZMgQ+vXr5/L6f5jzycPDI9bn3MfH54EFJMD8Qfne2zdo0IDHH3+cfv36sWTJEoAEv/bvFZ3YxmdezMR+V6xRowZNmzZlyJAhfPbZZ9SsWZMmTZrQpk2beBXc2bdvH506daJ169YuPwYk1XkLSqASpFWrVkyfPp3Vq1fTpEkTFi5cSPHixSlXrpxzm2+++YZ27drRpEkT+vTpQ/bs2XF3d2fkyJExBpnfK643lXu/nB09epTatWtTvHhxxo0bR758+fDy8mLVqlV89tln8c6eH4a7u3uy7yMh+44rnrs/nBwOB2XKlGHcuHGxbnvvh3NiJcV+oosH/Pzzzzz22GNkz56dokWLUq1aNaZMmUJ4eDhbtmxx+ZU1+nn/+uuvnQNH7xb9y230dr17947zl/B7Pzzjc3wTIiAggMqVK7N8+XJCQ0NjJKVWypo1633f3B9//HHnh1SjRo1wd3fngw8+oFatWvGq5JnUx/J++4lvIYx7E9wHadOmDW+99Rbnz5+nfv36MX6FTKzr168/MI4tW7bwwgsvUL16daZMmUKuXLnw9PRk1qxZzJs3L8b2Ke1439168CAtW7Zk+PDhXL58mQwZMvD999/TunVrl1aYbt26MWvWLN59912qVKlCxowZsdlstGrVKl6fBfF933iU4nrOkvK5HDFiBAMGDOCNN95g2LBhZMmSBTc3N959991Yj9u9z1v0NqNHj45zbPP9JhGN/qKa0B8dr1+/To0aNQgMDGTo0KEULlwYHx8ffvvtN/r27Ztkn//xHXeTkOck+rE+6BzPnTs31apVY+HChXz44Yf88ssvnDp1ilGjRrlsN3bsWNq1a8fy5ctZt24d3bt3Z+TIkfzyyy9x/lh0r8DAQOrWrUvdunXx9PTkq6++emDJ8OPHj/PKK69Qt25dPv74Y5frkup8KlCgQIwiX4mRN29eihUrxs8//+xcl9DX/sNI7HdFm83G4sWL+eWXX/jhhx9Yu3Ytb7zxBmPHjuWXX36577l17do1mjZtStGiRZ1l3aM97Hl7NyVQCVC9enVy5crFggULePbZZ9mwYYOzOEO0xYsXU6hQIZYuXeryBhRbk/m9MmfOzPXr12Osv/cXnx9++IHw8HC+//57l19/Ymsaju+bYPQkfocOHYrxy8mhQ4dcJvlLrQoXLszvv/9O7dq1E13J7OzZszG+8Ec3jUcPDE3IfuK63svLi6eeeootW7bw2GOPObtIVKtWjfDwcObOncuFCxecBTKi9wvmINn7fZGLLiXv6ekZ7y/YySEyMhIwB6HHlUBZUc66ePHizJ07N94tuR999BEzZsygf//+STIJY/bs2fHx8eHIkSMxrottnRVeeukl3n77bX755Zd4FxJ5kDNnzhAREeFslYjLkiVL8PHxYe3atS6/RM6aNStR+w0KCsLX19f5y+TdDh06lKj7TCotW7ZkyJAhLFmyhBw5chASEuIsWBRt8eLFtG3b1qXq6J07d2J8lsR1LsX3fSMuhQsX5s8//7zvNvnz5+ePP/7A4XC4tEJFdzdL6s+X6Fb2ux/zve/TixcvplatWnzxxRcut41PEg//HbfAwMBEHbfixYsDMStdPsimTZu4cuUKS5cudXn/j8/9FC5cmLVr13L16tU4W6Hy58+Pw+Hgn3/+cTkXL1y4wPXr1x/quTp+/DjZsmWLVzGcli1b0rlzZw4dOsSCBQvw8/OjcePGMbYrU6YMZcqUoX///mzfvp2qVasybdq0GIlNfFSqVImvvvqKc+fOxblNWFgYL7/8MpkyZeLbb7+N0ar6sOdTcoiMjOTWrVvOy/F97T/oPePPP/9M9sf49NNP8/TTTzN8+HDmzZvHK6+8wvz5812GctzN4XDwyiuvcP36dX788ccY3dAf9ry9m8ZAJUD0XCU//PADX3/9NZGRkTG670Vn23f/+rJz50527NjxwPsvXLgwN27c4I8//nCuO3fuXIw+9LHt48aNG7F+gfD39481KbtXpUqVyJ49O9OmTXPpErN69WoOHjwYr/63KV2LFi04c+YMM2bMiHFdWFiYc7zR/URGRrqUN46IiGD69OkEBQU5q4glZD/3e36qVavGzp072bhxozOBypYtGyVKlHD+End33/Pg4GACAwMZMWJErH2Go8uOZ8+enZo1azJ9+vRYPygepjx5fF29epXt27eTM2dOl+pF94pOrOLzGk4qVapUwTCMeHctzJQpE2+//TZr165Nkhnbo1syli1bxtmzZ53rjxw54jIu00oBAQFMnTqVwYMHx/qlJjGij3d0lae4uLu7Y7PZXFrmT5w4Ea8qmnHdX3BwMMuWLePUqVPO9QcPHmTt2rWJus+kUqJECcqUKcOCBQtYsGABuXLlcvnSDGb89/7a//nnn8fouRDXuRTf9424NG3alN9//z3WsXXRcTVo0IDz58+7JNuRkZF8/vnnBAQExKt7UkKcPXvWJZ6QkBDmzJlD+fLlna0CsR23RYsWxRhPFZeKFStSuHBhxowZ4/LlNNqDjluePHnIly+fS/nw+Ijt8z8iIsJlDrq4NG3aFMMwGDJkSIzr7n6uwKzie7foHhUP811gz549LvPKPShWd3d3vv32WxYtWkSjRo1cfmgLCQlx/ggXrUyZMri5ud13WpLbt2/H+X0s+v01tqEM0Tp16sThw4f57rvvYu3m9bDnU1I7fPgwhw4dcukpFd/XflzvGU888QQFCxZk/PjxMa5Lqpb9a9euxbiv6Baj+z2/Q4YMYe3atXz77bexdpd+2PP2bmqB+p/Vq1fHOvjymWeecZn8tWXLlnz++ecMGjSIMmXKxPi1tFGjRixdupSXXnqJhg0bcvz4caZNm0bJkiVjfbLu1qpVK/r27ctLL71E9+7dnaUvixYt6jKw7/nnn8fLy4vGjRvz9ttvc+vWLWbMmEH27NljfCGuWLEiU6dO5eOPP6ZIkSJkz5491r65np6ejBo1ivbt21OjRg1at27tLGNeoEABevbsGa/jmJK99tprLFy4kE6dOrFx40aqVq1KVFQUf//9NwsXLnTOzXI/uXPnZtSoUZw4cYKiRYuyYMEC9u3bx//93/85ZytPyH4qVqzIjz/+6Jxos2DBgs4B09WqVWP48OGcPn3aJVGqXr0606dPp0CBAi7dFAIDA5k6dSqvvfYaTzzxBK1atSIoKIhTp06xcuVKqlatyqRJkwCzaMqzzz5LmTJleOuttyhUqBAXLlxgx44d/PvvvzHmQnlYixcvJiAgAMMwOHv2LF988QXXrl1j2rRp921lik5Ku3fvTnBwMO7u7i6/wO/evTvWXxpr1qzpLK6RUM8++yxZs2blxx9/jLMf+7169OjB+PHj+eSTT5g/f36i9nu3wYMHs27dOqpWrco777xDVFQUkyZNonTp0kmSpCWF+/Uhv1d8nqf169fz2GOPPbCEecOGDRk3bhz16tWjTZs2XLx4kcmTJ1OkSBGXH58SYsiQIaxZs4Zq1arRuXNn55f7UqVKJfo+k0rLli0ZOHAgPj4+vPnmmzF+8W7UqBFff/01GTNmpGTJkuzYsYMff/wxxliG8uXL4+7uzqhRo7hx4wbe3t7OuQTj+74Rmz59+rB48WKaN2/OG2+8QcWKFbl69Srff/8906ZNo1y5cnTs2JHp06fTrl079uzZQ4ECBVi8eDHbtm1j/PjxZMiQIUmPWdGiRXnzzTf59ddfyZEjB19++SUXLlxw+ZGxUaNGDB06lPbt2/PMM8+wf/9+5s6dG2Oy97i4ubkxc+ZM6tevT6lSpWjfvj158uThzJkzbNy4kcDAQOeYuLi8+OKLfPfdd/Ee+wfmd5LMmTPTtm1bunfvjs1m4+uvv47XF9datWrx2muvMXHiRP755x/q1auHw+Fgy5Yt1KpVi65du1KuXDnatm3L//3f/zm7C+7atYuvvvqKJk2aOAfjJ9TFixf5448/nNOsPEj27NmpVasW48aN4+bNmzF+rN6wYQNdu3alefPmFC1alMjISL7++mvc3d3vOybv9u3bPPPMMzz99NPUq1ePfPnycf36dZYtW8aWLVto0qRJnO9BK1euZM6cOTRt2pQ//vjD5b0hICCAJk2aJOhzOKlFRkbyzTffAGZLzIkTJ5g2bRoOh8OlF1R8X/uFCxcmU6ZMTJs2jQwZMuDv70/lypUpWLAgU6dOpXHjxpQvX5727duTK1cu/v77b/76668k+eHpq6++YsqUKbz00ksULlyYmzdvMmPGDAIDA51J/r3279/PsGHDqF69OhcvXnQei2ivvvpqkpy3TvGu15dG3a+MObGUb3Q4HEa+fPkMwPj4449j3J/D4TBGjBhh5M+f3/D29jYqVKhgrFixItYS5cRSfnHdunVG6dKlDS8vL6NYsWLGN998E2s55O+//94oW7as4ePjYxQoUMAYNWqU8eWXXxqAcfz4ced258+fNxo2bGhkyJDBAJwlze8tYx5twYIFRoUKFQxvb28jS5YsxiuvvGL8+++/LttEl+y9V2xxPkhiyphfunQpXvHUqFHDKFWqlMu6iIgIY9SoUUapUqUMb29vI3PmzEbFihWNIUOGGDdu3LhvrNH3t3v3bqNKlSqGj4+PkT9/fmPSpEkxto3vfv7++2+jevXqhq+vr0uZWsMwjJCQEMPd3d3IkCGDERkZ6Vz/zTffGIDx2muvxRrnxo0bjeDgYCNjxoyGj4+PUbhwYaNdu3bG7t27XbY7evSo8frrrxs5c+Y0PD09jTx58hiNGjUyFi9e7Nzm7vLV9+4jttfPvWIrY+7v729UqVLFWLhwocu2sZWMjYyMNLp162YEBQUZNpvN5fV1v/N22LBhhmEkroy5YRhG9+7djSJFisQa3+jRo2O9Tbt27Qx3d3dnyey4ypjHVtL73hLWhmEYP/30k1GhQgXDy8vLKFy4sDFz5kzjvffeM3x8fB4Yf1znxL3iOqfuFdfrILbHEVsZ8wc9T1FRUUauXLmM/v37PzBmwzCML774wnj88ccNb29vo3jx4sasWbMe+nhv3rzZqFixouHl5WUUKlTImDZtWrzf0+J7vGM7PveLyTDM8vXRx2vr1q0xrr927ZrRvn17I1u2bEZAQIARHBxs/P3337He34wZM4xChQo5y7Pfff7G930jNleuXDG6du1q5MmTx/Dy8jLy5s1rtG3b1rh8+bJzmwsXLjjj9PLyMsqUKRPjvIzrHIt+v7m3ZHVsr8voY7x27VqjbNmyztfIvbe9c+eO8d577xm5cuUyfH19japVqxo7duwwatSo4TL1R1z7jrZ3717j5ZdfNrJmzWp4e3sb+fPnN1q0aGH89NNPDzxu0VNR3DvlRWyfXXfbtm2b8fTTTxu+vr5G7ty5jffff9851crdz2ls3zsiIyON0aNHG8WLFze8vLyMoKAgo379+i6lru12uzFkyBCjYMGChqenp5EvXz6jX79+LiXSDSPu1/O9x9AwDGPq1KmGn5+fs2R9fMyYMcMAjAwZMrhMr2IYhnHs2DHjjTfeMAoXLmz4+PgYWbJkMWrVqmX8+OOP971Pu91uzJgxw2jSpInze5qfn59RoUIFY/To0S7T0Nz7mXS/74v3HueHOZ8SI7Yy5oGBgUbt2rVjHJP4vvYNwzCWL19ulCxZ0jmFxt3n7NatW426desaGTJkMPz9/Y2yZcu6TA/xMN8Vf/vtN6N169bGY489Znh7exvZs2c3GjVqFOP43f09Mfpcjevvbg9z3kaz/S8AERH5n2PHjlG8eHFWr15N7dq1rQ7HqUmTJvz111+xjtdJzZYtW0abNm04evQouXLlsjocScUKFChA6dKlWbFihdWhxEvt2rXJnTv3I50s3AoVKlSgZs2afPbZZ1aHIpIkNAZKROQehQoV4s033+STTz6xLIawsDCXy//88w+rVq2iZs2a1gSUjEaNGkXXrl2VPEm6M2LECBYsWBDvOfVSozVr1vDPP//Qr18/q0MRSTJqgRIRSYFy5cpFu3btKFSoECdPnmTq1KmEh4ezd+/eWOcsEpHU1wIlIqmTikiIiKRA9erV49tvv+X8+fN4e3tTpUoVRowYoeRJRETEYmqBEhERERERiSeNgRIREREREYknJVAiIiIiIiLxpARKREREREQknpRAiYiIiIiIxJMSKODnn3+mcePG5M6dG5vNxrJlyxJ8H4ZhMGbMGIoWLYq3tzd58uRh+PDhSR+siIiIiIhYRmXMgdDQUMqVK8cbb7zByy+/nKj76NGjB+vWrWPMmDGUKVOGq1evcvXq1SSOVERERERErKQy5vew2Wx89913NGnSxLkuPDycjz76iG+//Zbr169TunRpRo0aRc2aNQE4ePAgZcuW5c8//6RYsWLWBC4iIiIiIslOXfjioWvXruzYsYP58+fzxx9/0Lx5c+rVq8c///wDwA8//EChQoVYsWIFBQsWpECBAnTo0EEtUCIiIiIiaYwSqAc4deoUs2bNYtGiRVSrVo3ChQvTu3dvnn32WWbNmgXAsWPHOHnyJIsWLWLOnDnMnj2bPXv20KxZM4ujFxERERGRpKQxUA+wf/9+oqKiKFq0qMv68PBwsmbNCoDD4SA8PJw5c+Y4t/viiy+oWLEihw4dUrc+EREREZE0QgnUA9y6dQt3d3f27NmDu7u7y3UBAQEA5MqVCw8PD5ckq0SJEoDZgqUESkREREQkbVAC9QAVKlQgKiqKixcvUq1atVi3qVq1KpGRkRw9epTChQsDcPjwYQDy58//yGIVEREREZHkpSp8mK1MR44cAcyEady4cdSqVYssWbLw2GOP8eqrr7Jt2zbGjh1LhQoVuHTpEj/99BNly5alYcOGOBwOnnzySQICAhg/fjwOh4MuXboQGBjIunXrLH50IiIiIiKSVJRAAZs2baJWrVox1rdt25bZs2djt9v5+OOPmTNnDmfOnCFbtmw8/fTTDBkyhDJlygBw9uxZunXrxrp16/D396d+/fqMHTuWLFmyPOqHIyIiIiIiyUQJlIiIiIiISDypjLmIiIiIiEg8KYESERERERGJp3Rbhc/hcHD27FkyZMiAzWazOhwREREREbGIYRjcvHmT3Llz4+Z2/zamdJtAnT17lnz58lkdhoiIiIiIpBCnT58mb968990m3SZQGTJkAMyDFBgYaGksdruddevW8fzzz+Pp6WlpLCISN52rIimXPcrOrL2zAHi19Kts/GmjzlWRFC4lfa6GhISQL18+Z45wP+k2gYruthcYGJgiEig/Pz8CAwMtf/GISNx0roqkXKERofTZ0geADpU76FwVSQVS4udqfIb2qIiEiIiIiIhIPCmBEhERERERiSclUCIiIiIiIvGUbsdAxYdhGERGRhIVFZWs+7Hb7Xh4eHDnzp1k35ekXp6enri7u1sdhoiIiEi6pgQqDhEREZw7d47bt28n+74MwyBnzpycPn1ac1JJnGw2G3nz5iUgIMDqUERERETSLSVQsXA4HBw/fhx3d3dy586Nl5dXsiY2DoeDW7duERAQ8MCJuyR9MgyDS5cu8e+///L444+rJUpERETEIkqgYhEREYHD4SBfvnz4+fkl+/4cDgcRERH4+PgogZI4BQUFceLECex2uxIoEZF7eHt4s6L1CueyiEhyUQJ1H0pmJCVR904Rkbh5uHnQsGhDwBxbLCKSXJQhiIiIiIiIxJNaoERERCTVs0fZmbt/LgAtirewOBoRScvUApXMoqJg0yb49lvzv9VVyk+cOIHNZmPfvn3WBiIiIpKEIqIiaL+8Pe2XtyciKsLqcEQkDVMClYyWLoUCBaBWLWjTxvxfoIC5Prm0a9cOm83m/MuaNSv16tXjjz/+ACBfvnycO3eO0qVLA7Bp0yZsNhvXr1+PcT9NmjRxXo5OvO73N3v27OR7YCIiIiIiKYASqGSydCk0awb//uu6/swZc31yJlH16tXj3LlznDt3jp9++gkPDw8aNWoEgLu7Ozlz5sTDI2G9N6MTr+i/9957j1KlSrmsa9myZXI8HBERERGRFENjoOLJMCC+c+pGRUH37uZtYrsfmw169IA6dcDdHRwOCA01l+8t/OfnZ26fEN7e3uTMmROAnDlz8sEHH1CtWjUuXbpEaGgoBQsWZO/evWTKlIlatWoBkDlzZgDatm0ba0tSdOIVLSAgAA8PD5d1IiIiIiLxEhWFbfNm8vz8MzZ/f7OrViqZpkUJVDzdvg0BAUlzX4ZhtkxlzBi9xg3IFOu2t26Bv3/i93Xr1i2++eYbihQpQtasWQkNDXVely9fPpYsWULTpk05dOgQgYGB+Pr6Jn5nIiIiIiIPsnQp9OiBx7//Uglg3DjImxcmTICXX7Y6ugdSApUGrVixgoD/ZXuhoaHkypWLFStWxJjXyt3dnSxZsgCQPXt2MmXK9KhDFREREZH0JHqcy71dtaLHuSxenOKTKI2Biic/P7M1KD5/q1bF7z5XrTK3Dwlx8O+/1wkJccS4Lz+/hMdaq1Yt9u3bx759+9i1axfBwcHUr1+fkydPJvzORERERESSQlSUOY4lrnEuAO++a33Z6gdQC1Q82Wzx70r3/PNmK+SZM7G/Pmw28/rnn/9vDFRUlHn/946BSgx/f3+KFCnivDxz5kwyZszIjBkz6NChw8PvQEREJIXx9vBmYbOFzmURSYG2bIlZYe1uhgGnT5vb1az5yMJKKCVQycDd3ezC2ayZmSzdnURFF4QYP/7RjZOz2Wy4ubkRFhYW4zovLy8AolJ4pi8iInI/Hm4eNC/VHAC73W5xNCISq3PnknY7iyiBSiYvv2x24ezRwzXRzpvXTJ6Ss2tneHg458+fB+DatWtMmjSJW7du0bhx4xjb5s+fH5vNxooVK2jQoAG+vr7O8VM3btyIMeFu1qxZyZcvX/IFLyIiIiJpU65cSbudRZRAJaOXX4YXXzRbIc+dM18L1aolf8vTmjVryPW/F16GDBkoXrw4ixYtombNmpw4ccJl2zx58jBkyBA++OAD2rdvz+uvv+4sY75p0yYqVKjgsv2bb77JzJkzk/cBiIiIJFCkI5LvDn4HQKMijSyORkRiVa2a+YU4rham6HEu1ao92rgSyGYYsY3SSftCQkLImDEjN27cIDAw0OW6O3fucPz4cQoWLIiPj0+yx+JwOAgJCSEwMDBGpTyRaI/6dSkx2e12Vq1aRYMGDfD09LQ6HBG5S2hEKAEjzR4U13pfY/OPm3WuiqREzz4L27bFXB89zsWiKnz3yw3upW/rIiIiIiKS/NavN5Mnmw2yZ3e9Lm/eVFHCHJRAiYiIiIhIcrtzB7p0MZe7doWzZ4lcv57dvXoRuX49HD+eKpIn0BgoERERERFJbqNGwT//mGOghg0Dd3eMGjU4ExpKuRo1Hl156iSgFigREREREUk+//wDI0eay599BhkzWhvPQ1ICJSIiIiIiycMwzK574eFQty60aGF1RA9NCZSIiIiIiCSPhQvN4hHe3jB58n/V9lIxjYESERGRVM/L3YtZL85yLotICnDjBvTsaS736wePP25tPElECZSIiIikep7unrQr3w4w52wTkRRgwABz0twiRaBvX6ujSTLqwiciIiIiIknrt9/MLnsAU6aAj4+18SQhJVDJLSoKNm2Cb781/0dFWR1RrAoUKMD48eOtDkNERCRRIh2RrDy8kpWHVxLpiLQ6HJH0LSoKOnUChwNatTKLR6QhSqCS09KlUKAA1KoFbdqY/wsUMNcnk3bt2mGz2bDZbHh5eVGkSBGGDh1KZOT9P0x+/fVXOnbsmGRxnDhxApvNxr59+5Lk/s6fP0+3bt0oVKgQ3t7e5MuXj8aNG/PTTz8lyf2LiEjqFh4ZTqNvG9Ho20aER4ZbHY5I+jZ9Ovz6KwQGwrhxVkeT5DQGKrksXQrNmpmlG+925oy5fvHiZJttuV69esyaNYvw8HBWrVpFly5d8PT0pF+/fjG2jYiIwMvLi6CgoGSJJSmcOHGCqlWrkilTJkaPHk2ZMmWw2+2sXbuWLl268Pfff1sdYoJFH3cRERGRNOX8efjwQ3N5+HBz4tw0Ri1Q8WUYEBoav7+QEOjePWbyFH0/AD16mNs96L5iu48H8Pb2JmfOnOTPn5933nmHOnXq8P333wNmC1WTJk0YPnw4uXPnplixYoBrF742bdrQsmVLl/u02+1ky5aNOXPmALBmzRqeffZZMmXKRNasWWnUqBFHjx51bl+wYEEAKlSogM1mo2bNms7rZs6cSYkSJfDx8aF48eJMmTLlvo+nc+fO2Gw2du3aRdOmTSlatCilSpWiV69e/PLLL87tTp06xYsvvkhAQACBgYG0aNGCCxcuOK8fPHgw5cuX58svv+Sxxx4jICCAzp07ExUVxaeffkrOnDnJnj07w4cPd9m/zWZj6tSp1K9fH19fXwoVKsTixYtdtunbty9FixbFz8+PQoUKMWDAAJdBzNH7njlzJgULFsTnf/2Ar1+/TocOHQgKCiIwMJDnnnuO33///b7HQ0RERCTF6t3brL5XsSK8847V0SQLtUDF1+3bEBCQNPdlGPDvv85ZmN2ATHFte+sW+Ps/1O58fX25cuWK8/JPP/1EYGAg69evj3X7V155hebNm3Pr1i0C/veY165dy+3bt3nppZcACA0NpVevXpQtW5Zbt24xcOBAXnrpJfbt24ebmxu7du3iqaee4scff6RUqVLO1pa5c+cycOBAJk2aRIUKFdi7dy9vvfUW/v7+tG3bNkYsV69eZc2aNQwfPhz/WI5DpkyZAHA4HM7kafPmzURGRtKlSxdatmzJpk2bnNsfPXqU1atXs2bNGo4ePUqzZs04duwYRYsWZfPmzWzfvp033niDOnXqULlyZeftBgwYwCeffMKECRP4+uuvadWqFfv376dEiRIAZMiQgdmzZ5M7d27279/PW2+9RYYMGXj//fed93HkyBGWLFnC0qVLcXd3B6B58+b4+vqyevVqMmbMyPTp06lduzaHDx8mS5YsD3xuRURERFKMn36CuXPNuZ6mTYP/fd9Jc4x06saNGwZg3LhxI8Z1YWFhxoEDB4ywsLD/Vt66ZRhm6vNo/27dStDjatu2rfHiiy8ahmEYDofDWL9+veHt7W307t3beX2OHDmM8PBwl9vlz5/f+OyzzwzDMAy73W5ky5bNmDNnjvP61q1bGy1btoxzv5cuXTIAY//+/YZhGMbx48cNwNi7d6/LdoULFzbmzZvnsm7YsGFGlSpVYr3fnTt3GoCxdOnS+z7udevWGe7u7sapU6ec6/766y8DMHbt2mUYhmEMGjTI8PPzM0JCQpzbBAcHGwUKFDCioqKc64oVK2aMHDnSeRkwOnXq5LK/ypUrG++8806c8YwePdqoWLGi8/KgQYMMT09P4+LFi851W7ZsMQIDA407d+643LZw4cLG9OnTY9xnrK9LeaQiIiKMZcuWGREREVaHIiL3uBV+y2AwBoMxrt26pnNV5FG7c8cwihY1v7926RKvm6Skz9X75Qb3UgtUfPn5ma1B8fHzz9CgwYO3W7UKqlfH4XAQEhJCYGAgbm739Kr080twqCtWrCAgIAC73Y7D4aBNmzYMHjzYeX2ZMmXuO/7Gw8ODFi1aMHfuXF577TVCQ0NZvnw58+fPd27zzz//MHDgQHbu3Mnly5dxOByA2Y2udOnSsd5vaGgoR48e5c033+Stt95yro+MjCTj/1rj7mXEswvjwYMHyZcvH/ny5XOuK1myJJkyZeLgwYM8+eSTgNlVMUOGDM5tcuTIgbu7u8txz5EjBxcvXnS5/ypVqsS4fHeBjAULFjBx4kSOHj3KrVu3iIyMJDAw0OU2+fPndxlr9vvvv3Pr1i2yZs3qsl1YWJhLd0gRERGRFG/0aDh8GHLkgI8/tjqaZKUEKr5stvh3pXv+ecib1ywYEVsCYLOZ1z//vNm06XCY5R79/eHeBCoRatWqxdSpU/Hy8iJ37tx4eLg+zbF1hbvXK6+8Qo0aNbh48SLr16/H19eXevXqOa9v3Lgx+fPnZ8aMGeTOnRuHw0Hp0qWJiIiI8z5v/S8BnTFjhkv3OMDZpe1ejz/+ODabLckKRXh6erpcttlssa6LTgjjY8eOHbzyyisMGTKE4OBgMmbMyPz58xk7dqzLdvce91u3bpErVy6XLobRorsmioiIiKR4R4/+lzR99hmk8e8xSqCSg7s7TJhgVtuz2VyTKJvN/D9+fLL1C/X396dIkSIPdR/PPPMM+fLlY8GCBaxevZrmzZs7E40rV65w6NAhZsyYQbVq1QDYunWry+2jW7ii7pr3KkeOHOTOnZtjx47xyiuvxCuOLFmyEBwczOTJk+nevXuMJOT69etkypSJEiVKcPr0aU6fPu1shTpw4ADXr1+nZMmSiTsId/nll194/fXXXS5XqFABgO3bt5M/f34++ugj5/UnT5584H0+8cQTnD9/Hg8PDwoUKPDQMYqIpGde7l5Mqj/JuSwij4hhQNeuEB4OtWub8z6lcUqgksvLL5ulynv0MAtGRMub10yekqmEeVJq06YN06ZN4/Dhw2zcuNG5PnPmzGTNmpX/+7//I1euXJw6dYoPPvjA5bbZs2fH19eXNWvWkDdvXnx8fMiYMSNDhgyhe/fuZMyYkXr16hEeHs7u3bu5du0avXr1ijWOyZMnU7VqVZ566imGDh1K2bJliYyMZP369UydOpWDBw9Sp04dypQpwyuvvML48eOJjIykc+fO1KhRg0qVKj30sVi0aBGVKlXi2WefZe7cuezatYsvvvgCMFvJTp06xfz583nyySdZuXIl33333QPvs06dOlSpUoUmTZrw6aefUrRoUc6ePcvKlSt56aWXkiRuEZH0wtPdky5PdQFwqYIqIslsyRJYswa8vGDKlP8aC9IwlTFPTi+/DCdOwMaNMG+e+f/48VSRPIHZje/AgQPkyZOHqlWrOte7ubkxf/589uzZQ+nSpenZsyejR492ua2HhwcTJ05k+vTp5M6dmxdffBGADh06MHPmTGbNmkWZMmWoUaMGs2fPdpY9j02hQoX47bffqFWrFu+99x6lS5embt26/PTTT0ydOhUwu90tX76czJkzU716derUqUOhQoVYsGBBkhyLIUOGMH/+fMqWLcucOXP49ttvnS1bL7zwAj179qRr166UL1+e7du3M2DAgAfep81mY9WqVVSvXp327dtTtGhRWrVqxcmTJ8mRI0eSxC0iIiKSbEJCzMYCgA8+gKJFrY3nEbEZ8R2ln8aEhISQMWNGbty4EWOw/507dzh+/LjLfD3J6b5FJMRyNpuN7777jiZNmlgax6N+XUpMdrudVatW0aBBgxhj50TEWlGOKLac2gLA07meZu2atTpXRZJbz55mz6rCheHPPyGB309S0ufq/XKDe6kLn4iIiKR6dyLvUOurWgBc633N4mhE0oG9e2HiRHN58uQEJ0+pmZo7REREREQk/hwOeOcd83+LFhAcbHVEj5RaoEQeIJ32chURERGJ3YwZsHMnZMhgli1PZ9QCJSIiIiIi8XPhglkwAmDYMMid29p4LKAESkRERERE4qdPH7h+HSpUgC5drI7GEkqgRERERETkwTZuhK+/Nud6mjYNPNLnaKAUkUD9/PPPNG7cmNy5c2Oz2Vi2bNkDb7Np0yaeeOIJvL29KVKkCLNnz072OEVERERE0qWICOjc2Vzu1AmeesraeCyUIhKo0NBQypUrx+TJk+O1/fHjx2nYsCG1atVi3759vPvuu3To0IG1a9cmc6QiIiKSEnm6e/JpnU/5tM6neLpr7ieRJDdmDPz9N2TPDiNGWB2NpVJEu1v9+vWpX79+vLefNm0aBQsWZOzYsQCUKFGCrVu38tlnnxGczsooioiICHi5e9Gnah/AnJxTRJLQsWNmwQiAsWMhUyZLw7FaikigEmrHjh3UqVPHZV1wcDDvvvtunLcJDw8nPDzceTkkJAQw32TvfaO12+0YhoHD4cDhcCRd4HGILpMdvU+R2DgcDgzDwG634+7ubnU46VL0e4W+nImkbDpXRZKQYeDetStud+7gqFmTqBYtIInOrZR0riYkhlSZQJ0/f54cOXK4rMuRIwchISGEhYXh6+sb4zYjR45kyJAhMdavW7cOPz8/l3UeHh7kzJmTW7duERERkbTB38fNmzcf2b6S0pgxY1i3bh1//vknnp6enDx5MsY2p0+f5r333mPr1q34+/vTqlUrBg0ahMddgw+3bt3KRx99xN9//02ePHno3bs3bdq0cV7/xRdf8OWXX3L69GkAihcvTp8+fahbt65zm0aNGrFt2zaXfbdr147P/jdHwf79+xk/fjy//PILV69e5bHHHqN9+/Z06tQpSY9JcoiIiCAsLIyff/6ZyMhIq8NJ19avX291CCJyjygjimNhxwAo5FsId5u7zlWRJJBrxw6eWr0ah4cHG5s149bq1Um+j5Rwrt6+fTve26bKBCox+vXrR69evZyXQ0JCyJcvH88//zyBgYEu2965c4fTp08TEBCAj49PssdmGAY3b94kQ4YM2Gy2ZN9fUnNzc6NVq1b8+++/fPnllzGOZ1RUFG3atCFHjhxs3bqVc+fO0a5dOwICAhg+fDhgjmtr2bIlb7/9NvPmzWPDhg10796dggULOrtlFilShFGjRvH4449jGAZz5szhlVdeYc+ePZQqVQowk98OHTq4JMt+fn7OmA4dOkSePHn45ptvyJcvH9u3b6dTp074+/vTJYWX4rxz5w6+vr5Ur179kbwuJSa73c769eupW7cunp4aYyGSkoRGhNJ0TFMALva4yPbN23Wuijysmzfx6NoVAKN3b6p37Jikd5+SPleje6fFR6pMoHLmzMmFCxdc1l24cIHAwMBYW58AvL298fb2jrHe09MzxhMWFRWFzWbDzc0NNzfXOhuhEaFxxuXu5o6Ph0+8tnWzueHracbqcDgItYfibnd32Z+/l3+ct4/NpUuXKFOmDN27d+fDDz8EYPv27dSsWZPVq1dTu3btBN1ffA0dOhTAWQnx3mO2du1aDhw4wI8//uhsORw2bBh9+/ZlyJAheHl58X//938ULFiQcePGAVCqVCm2bdvGhAkTnOPjXnzxRZf7HTFiBNOmTWPXrl2UKVPGud7f35/ccUzq1qFDB5fLRYoUYefOnXz33Xd069YtkUfg0XBzc8Nms8X6mpVHS8+BSMrjafx3TkafnzpXRR7S8OHw779QqBDuAwfinkznU0o4VxOy/1SZQFWpUoVVq1a5rFu/fj1VqlRJ9n0HjAyI87oGjzdgZZuVzsvZx2Tntj325sAa+Wuwqd0m5+Vys8pxJeyKyzbGICNBsQUFBfHll1/SpEkTnn/+eYoVK8Zrr71G165d75s8lSpVKtZud9GqVavG6odort2xYwdlypRx6XYZHBzMO++8w19//UWFChUSPK4tKiqKRYsWERoaGuN5nzt3Lt988w05c+akcePGDBgwIEY3zbvduHGDLFmyJPrxiYiIiKQ5f/wBEyaYy5MmQRyNFOlRikigbt26xZEjR5yXjx8/zr59+8iSJQuPPfYY/fr148yZM8yZMweATp06MWnSJN5//33eeOMNNmzYwMKFC1m5cmVcu0g3GjRowFtvvcUrr7xCpUqV8Pf3Z+TIkfe9zapVq+47cC6uVr34imvMWvR199vm3nFt+/fvp0qVKty5c4eAgAC+++47SpYs6bxNmzZtyJ8/P7lz5+aPP/6gb9++HDp0iKVLl8Ya2/bt21mwYIFeOyIiIiLRHA5zrqeoKGjaFBJQLTs9SBEJ1O7du6lVq5bzcvRYpbZt2zJ79mzOnTvHqVOnnNcXLFiQlStX0rNnTyZMmEDevHmZOXPmIylhfqvfrTivc3dzrYx2sffFOLd1s7l2c/u9/e8EZgiM0f0tMcaMGUPp0qVZtGgRe/bsibXr4t3y58//0Pt8VIoVK8a+ffu4ceMGixcvpm3btmzevNmZRHW8q29umTJlyJUrF7Vr1+bo0aMULlzY5b7+/PNPXnzxRQYNGsTzzz//SB+HiIiISIr1xRewYwcEBMD48VZHk+KkiASqZs2azlLesYkeW3Pvbfbu3ZuMUcUuIeOSErStpz/+Xv5JkkAdPXqUs2fP4nA4OHHihMv4oNgkdxe+nDlzsmvXLpd10WPYcubM6fwfn3FtXl5eFClSBICKFSvy66+/MmHCBKZPnx7rvitXrgzAkSNHXBKoAwcOULt2bTp27Ej//v0T/dhERERE0pRLl6BvX3N56FDIm9faeFKgFJFASdKJiIjg1VdfpWXLlhQrVowOHTqwf/9+smfPHudtkrsLX5UqVRg+fDgXL150xrF+/XoCAwOdLUeJHdfmcDhc5ve61759+wDIlSuXc91ff/3Fc889R9u2bZ1VAEVEREQEeP99uHYNypWDFF5gyypKoNKYjz76iBs3bjBx4kQCAgJYtWoVb7zxBitWrIjzNg/bhe/UqVNcvXqVU6dOERUV5UxaihQpQkBAAM8//zwlS5bktdde49NPP+X8+fP079+fLl26OLsXxmdcW79+/ahfvz6PPfYYN2/eZN68eWzatIm1a9cCZsvbvHnzaNCgAVmzZuWPP/6gZ8+eVK9enbJlywJmt73nnnuO4OBgevXq5RyD5e7uTlBQ0EMdBxERsY6nuyeDagxyLotIIvz8M8yeDTYbTJsGHkoVYmWkUzdu3DAA48aNGzGuCwsLMw4cOGCEhYU9kliioqKMa9euGVFRUQ91Pxs3bjQ8PDyMLVu2ONcdP37cCAwMNKZMmfKwYcapbdu2BhDjb+PGjc5tTpw4YdSvX9/w9fU1smXLZrz33nuG3W6PEX/58uUNLy8vo1ChQsasWbNcrn/jjTeM/PnzG15eXkZQUJBRu3ZtY926dc7rT506ZVSvXt3IkiWL4e3tbRQpUsTo06ePy3M8aNCgWGPNnz9/chyaJPWoX5cSU0REhLFs2TIjIiLC6lBE5D50rookQni4YZQsaRhgGB07PpJdpqRz9X65wb1shnGfwUdpWEhICBkzZuTGjRuxTqR7/PhxChYs+EgmLHU4HISEhBAYmDRFJCRtetSvS4nJbrezatUqGjRoYPl8FSISN52rIonwySfQrx8EBcHff8MjmOIlJZ2r98sN7qV2OREREUn1HIaDg5cOAlAkUxGLoxFJZU6cMAtGAIwZ80iSp9RMCZSIiIikemH2MEpPLQ3Atd7XLI5GJBUxDLNYRFgY1KgBr71mdUQpnvqLiYiIiIikV8uXw4oV4OkJU6eaBSTkvpRAiYiIiIikR7duQffu5nLv3lCihLXxpBJKoO4jndbXkBRKr0cRERFJUkOHwunTUKAA9O9vdTSphhKoWERXAbl9+7bFkYj8JyIiAjDnrBIRERF5KPv3w7hx5vKkSeDnZ208qYiKSMTC3d2dTJkycfHiRQD8/PywJWN/UIfDQUREBHfu3FEZc4mVw+Hg0qVL+Pn54aFJ7URERORhOBzwzjsQFQUvvQQNG1odUaqib2JxyJkzJ4AziUpOhmEQFhaGr69vsiZqkrq5ubnx2GOP6TUiIiIiD2f2bNi2Dfz9YcIEq6NJdZRAxcFms5ErVy6yZ8+O3W5P1n3Z7XZ+/vlnqlevbvkkYpJyeXl5qYVSRCQOnu6e9K7S27ksInG4fBn69DGXhwyBfPmsjScVUgL1AO7u7sk+5sTd3Z3IyEh8fHyUQImIiCSCl7sXo58fDZDsP3yKpGp9+8LVq1CmzH8V+CRB9HO2iIiIiEh6sHUrfPmluTx1qjn3kySYWqBEREQk1XMYDk7dOAVALr9cFkcjkgLZ7WbhCIAOHaBqVWvjScWUQImIiEiqF2YPo+CEggBc633N4mhEUqDx4+HPPyFbNvjkE6ujSdXUhU9EREREJC07dQoGDzaXP/0Usma1NJzUTgmUiIiIiEha1r073L4N1apB27ZWR5PqKYESEREREUmrvv8eli8HDw+zcISmRHloOoIiIiIiImlRaCh062Yuv/celCplbTxphBIoEREREZG0aNgwc/xT/vwwYIDV0aQZSqBERERERNKav/6CsWPN5YkTwd/f2njSEJUxFxERkVTPw82DzpU6O5dF0jXDMOd8ioyEF1+EF16wOqI0Re8wIiIikup5e3gzueFkAOx2u8XRiFjsq69gyxbw8zNbnyRJqQufiIiIiEhaceUK9OljLg8aBI89Zm08aZBaoERERCTVMwyDy7cvA5DRM6PF0YhY6IMP4PJls+Jez55WR5MmKYESERGRVO+2/TbZx2QH4FrvaxZHI2KR7dth5kxzedo08PS0Np40Sl34RERERERSO7sdOnUyl9u3h2eftTaeNEwJlIiIiIhIajdxIuzfD1mywKefWh1NmqYESkREREQkNTt92iwYAWbylC2btfGkcUqgRERERERSs3ffhdBQeOYZs/ueJCslUCIiIiIiqdXKlbB0Kbi7m4Uj3PT1PrnpCIuIiIiIpEa3b0PXruZyz55Qpoy18aQTKmMuIiIiqZ6Hmwdty7V1LoukC8OHw4kTkC/ff2OgJNnpHUZERERSPW8Pb2Y3mQ2A3W63NhiRR+HgQRg92lyeOBECAqyNJx1RFz4RERERkdTEMKBzZ3Pup0aN4MUXrY4oXVELlIiIiKR6hmFw234bAE88LY5GJJl98w1s2gS+vvD552CzWR1RuqIESkRERFK92/bbBIw0uzBd633N4mhEktHVq/Dee+bywIFQoICl4aRH6sInIiIiIpJafPghXLoEJUtCr15WR5MuKYESEREREUkNfvkFpk83l6dOBS8va+NJp5RAiYiIiIikdJGR0KmTudy2LVSvbm086ZgSKBERERGRlG7SJPj9d8ic+b/y5WIJJVAiIiIiIinZv//CgAHm8qhREBRkbTzpnBIoEREREZGUrGdPuHULqlSBN9+0Opp0T2XMRUREJNVzd3OnWclmzmWRNGP1ali8GNzdzcIRbmr/sJoSKBEREUn1fDx8WNR8EQB2u93iaESSSFgYdO1qLvfoAeXKWRuPAOrCJyIiIiKSMo0YAceOQZ48MHiw1dHI/yiBEhERERFJaf7+2ywYATBhAmTIYG084qQESkRERFK90IhQbENs2IbYCI0ItTockYdjGNC5M9jt0KABvPyy1RHJXZRAiYiIiIikJPPmwcaN4OMDn38ONpvVEcldlECJiIiIiKQU165Br17mcv/+UKiQtfFIDEqgRERERERSio8+gosXoXhx6N3b6mgkFkqgRERERERSgl27YNo0c3nKFPD2tjYeiZUSKBERERERq0VGQqdOZgGJ116DWrWsjkjioARKRERERMRqU6bA3r2QKROMGWN1NHIfHlYHICIiIvKw3N3cafB4A+eySKpy9qxZMAJg5EjInt3aeOS+lECJiIhIqufj4cPKNisBsNvtFkcjkkA9e8LNm1C5MnTsaHU08gDqwiciIiIiYpV162DhQnBzg6lTzf+SoukZEhERERGxQlgYdO5sLnfrBhUqWBuPxIsSKBEREUn1QiNC8R/hj/8If0IjQq0ORyR+PvkEjh6F3Llh6FCro5F40hgoERERSRNu229bHYJI/B0+bCZQAOPHQ2CgpeFI/KkFSkRERETkUTIM6NIFIiIgOBiaNbM6IkkAJVAiIiIiIo/S/Pnw44/g7Q2TJ4PNZnVEkgBKoEREREREHpUbN6BXL3P5o4+gcGFr45EEUwIlIiIiIvKo9O8P589D0aLw/vtWRyOJkGISqMmTJ1OgQAF8fHyoXLkyu3btuu/248ePp1ixYvj6+pIvXz569uzJnTt3HlG0IiIiIiIJtHu32WUPYMoUswufpDopogrfggUL6NWrF9OmTaNy5cqMHz+e4OBgDh06RPbs2WNsP2/ePD744AO+/PJLnnnmGQ4fPky7du2w2WyMGzfOgkcgIiIiVnKzuVEjfw3nskiKExUFnTqZBSTatIHata2OSBIpRSRQ48aN46233qJ9+/YATJs2jZUrV/Lll1/ywQcfxNh++/btVK1alTZt2gBQoEABWrduzc6dOx9p3CIiIpIy+Hr6sqndJgDsdru1wYjEZto02LMHMmaEsWOtjkYeguUJVEREBHv27KFfv37OdW5ubtSpU4cdO3bEeptnnnmGb775hl27dvHUU09x7NgxVq1axWuvvRbnfsLDwwkPD3deDgkJAcw3WavfaKP3b3UcInJ/OldFUgedq5LinDuHx4cfYgOihg3DkTUr6PWZos7VhMRgeQJ1+fJloqKiyJEjh8v6HDly8Pfff8d6mzZt2nD58mWeffZZDMMgMjKSTp068eGHH8a5n5EjRzJkyJAY69etW4efn9/DPYgksn79eqtDEJF40LkqkjroXJWUouLYseQNCeFakSL8nCcPrFpldUgpSko4V2/fjv9E3JYnUImxadMmRowYwZQpU6hcuTJHjhyhR48eDBs2jAEDBsR6m379+tErumQkZgtUvnz5eP755wm0eOZnu93O+vXrqVu3Lp6enpbGIiJx07kqknKFRoTy+OTHATjQ8QDbN2/XuSopgu2nn/DYsgXDzY2Ab76hwRNPWB1SipGSPleje6fFh+UJVLZs2XB3d+fChQsu6y9cuEDOnDljvc2AAQN47bXX6NChAwBlypQhNDSUjh078tFHH+HmFnPwqLe3N96xVDrx9PS0/AmLlpJiEZG46VwVSXk8DU8uh102l/93fupcFcvduQPduwNg69IFz8qVLQ4oZUoJ52pC9m95mRovLy8qVqzITz/95FzncDj46aefqFKlSqy3uX37dowkyd3dHQDDMJIvWBERERGR+Pr0U/jnH8iZE4YNszoaSSKWt0AB9OrVi7Zt21KpUiWeeuopxo8fT2hoqLMq3+uvv06ePHkYOXIkAI0bN2bcuHFUqFDB2YVvwIABNG7c2JlIiYiIiIhY5sgRGDHCXB4/3qy+J2lCikigWrZsyaVLlxg4cCDnz5+nfPnyrFmzxllY4tSpUy4tTv3798dms9G/f3/OnDlDUFAQjRs3Zvjw4VY9BBERERERk2FAly4QHg5160KLFlZHJEkoRSRQAF27dqVr166xXrdp0yaXyx4eHgwaNIhBgwY9gshERERERBJg0SJYtw68vWHyZLDZrI5IkpDlY6BERERERNKMkBB4911zuV8/ePxxS8ORpJdiWqBEREREEsvN5kal3JWcyyKWGTAAzp2DIkWgb1+ro5FkoARKREREUj1fT19+fetXwJxbRsQSv/0GkyaZy1OmgI+PtfFIstBPNCIiIiIiDysqCjp1AocDWrUyi0dImqQESkRERETkYf3f/8Gvv0JgIIwbZ3U0koyUQImIiEiqd9t+mwLjC1BgfAFu229bHY6kNxcumAUjAD7+GHLlsjYeSVYaAyUiIiKpnmEYnLxx0rks8ki99x7cuAEVK0LnzlZHI8lMLVAiIiIiIom1YQPMnWvO9TRtGri7Wx2RJDMlUCIiIiIiiREe/l+LU+fOUKmStfHII6EESkREREQkMUaPhkOHIEcOc+yTpAtKoEREREREEurYMRg+3FweNw4yZbI0HHl0lECJiIiIiCSEYUCXLnDnDtSuDa1bWx2RPEKqwiciIiKpns1mo2RQSeeySLJasgTWrAEvL5gyxSwgIemGEigRERFJ9fw8/fir818A2O12i6ORNO3mTXj3XXO5b18oWtTScOTRUxc+EREREZH4GjQIzpyBwoX/mzxX0hUlUCIiIiIi8bFvH0yYYC5Pngy+vpaGI9ZQAiUiIiKp3m37bUpNKUWpKaW4bb9tdTiSFjkc0KmT+b95cwgOtjoisYjGQImIiEiqZxgGBy4dcC6LJLkZM2DnTsiQAT77zOpoxEJqgRIRERERuZ+LF+GDD8zlYcMgTx5r4xFLKYESEREREbmfPn3g+nUoX96c/0nSNSVQIiIiIiJx2bQJ5swx53qaNg08NAImvVMCJSIiIiISm4gI6NzZXH77bahc2dp4JEVQAiUiIiIiEpuxY+HgQcieHUaMsDoaSSHUBikiIiKpns1mI3/G/M5lkYd2/DgMHWoujx0LmTNbG4+kGEqgREREJNXz8/TjxLsnALDb7dYGI6mfYUC3bnDnDtSqBa+8YnVEkoKoC5+IiIiIyN2WLYOVK8HTE6ZMMQtIiPyPEigRERERkWi3bkH37uby++9D8eLWxiMpjhIoERERSfXC7GE8OeNJnpzxJGH2MKvDkdRs8GD4918oWBA++sjqaCQF0hgoERERSfUchoPdZ3c7l0US5Y8/YPx4c3nSJPD1tTQcSZnUAiUiIiIi4nBAp04QFQVNm0KDBlZHJCmUEigRERERkS+/hB07ICDgv1YokVgogRIRERGR9O3SJejb11weOhTy5rU2HknRlECJiIiISPr2/vtw9SqUK2fO/yRyH0qgRERERCT9+vlnmD3bXJ46FTxUY03uT68QERERSROy+WWzOgRJbSIi4J13zOWOHaFKFWvjkVRBCZSIiIikev5e/lzqcwkAu91ucTSSanz2GRw4AEFBMHKk1dFIKqEufCIiIiKS/pw4AUOGmMujR0OWLJaGI6mHEigRERERSX+6d4ewMKhRA15/3epoJBVRAiUiIiKpXpg9jJqza1Jzdk3C7GFWhyMp3fLl8MMPZsGIKVPAZrM6IklFNAZKREREUj2H4WDzyc3OZZE43br1X6ny3r2hZElr45FURy1QIiIiIpJ+DB0Kp09DgQIwYIDV0UgqlOgWqOPHj7NlyxZOnjzJ7du3CQoKokKFClSpUgUfH5+kjFFERERE5OH9+adZeQ/g88/Bz8/aeCRVSnACNXfuXCZMmMDu3bvJkSMHuXPnxtfXl6tXr3L06FF8fHx45ZVX6Nu3L/nz50+OmEVEREREEsbhMOd8ioyEl16CRo2sjkhSqQQlUBUqVMDLy4t27dqxZMkS8uXL53J9eHg4O3bsYP78+VSqVIkpU6bQvHnzJA1YRERERCTBZs+GrVvB3x8mTLA6GknFEpRAffLJJwQHB8d5vbe3NzVr1qRmzZoMHz6cEydOPGx8IiIiIiIP58oVeP99c3nwYLinEUAkIRKUQN0vebpX1qxZyZo1a4IDEhEREUkMP0+NZ5E49O1rJlFlykCPHlZHI6lcoqvw/fbbb+zfv995efny5TRp0oQPP/yQiIiIJAlOREREJD78vfwJ/TCU0A9D8ffytzocSUm2bYMvvjCXp04FT09r45FUL9EJ1Ntvv83hw4cBOHbsGK1atcLPz49FixbxfnQTqYiIiIiIVex26NTJXH7zTaha1dp4JE1IdAJ1+PBhypcvD8CiRYuoXr068+bNY/bs2SxZsiSp4hMRERERSZwJE8zS5VmzwqhRVkcjaUSiEyjDMHA4zJm+f/zxRxo0aABAvnz5uHz5ctJEJyIiIhIPdyLv0HBeQxrOa8idyDtWhyMpwalTMGiQuTx6tJlEiSSBRE+kW6lSJT7++GPq1KnD5s2bmTp1KmBOsJsjR44kC1BERETkQaIcUaz6Z5VzWYQePeD2bXj2WWjb1upoJA1JdAvU+PHj+e233+jatSsfffQRRYoUAWDx4sU888wzSRagiIiIiEiC/PADLFsGHh5m4Qi3RH/lFYkh0S1QZcuWdanCF2306NG4u7s/VFAiIiIiIokSGgrdupnLvXpB6dLWxiNpTqITqLvdunXLOR4qmqdKRIqIiIjIo/bxx3DyJDz2GAwcaHU0kgYluj3z+PHjNGzYEH9/fzJmzEjmzJnJnDkzmTJlInPmzEkZo4iIiIjIg/31F4wZYy5//jn4a04wSXqJboF69dVXMQyDL7/8khw5cmCz2ZIyLhERERGR+DMM6NwZIiPhhRfMP5FkkOgE6vfff2fPnj0UK1YsKeMREREREUm4OXPg55/Bzw8mTrQ6GknDEp1APfnkk5w+fVoJlIiIiFjO38sfY5ABgN1utzgaeeSuXIHevc3lQYMgf35r45E0LdEJ1MyZM+nUqRNnzpyhdOnSMYpGlC1b9qGDExERERF5oH794PJlKFUKeva0OhpJ4xKdQF26dImjR4/Svn175zqbzYZhGNhsNqKiNImdiIiIiCSz7dthxgxzeepUUCVoSWaJTqDeeOMNKlSowLfffqsiEiIiImKpO5F3eO271wD4stGXFkcjj0xkJLzzjrncvj1Uq2ZtPJIuJDqBOnnyJN9//z1FihRJynhEREREEizKEcXiA4sBmNFghsXRyCMzcSL88QdkyQKffmp1NJJOJHoeqOeee47ff/89KWMREREREYmf06f/myj3008hWzZr45F0I9EtUI0bN6Znz57s37+fMmXKxCgi8YJq74uIiIhIcnn3XQgNhWeeMbvviTwiiU6gOnXqBMDQoUNjXKciEiIiIiKSbFatgqVLwd3dLBzhluhOVSIJlugEyuFwJGUcIiIiIiIPdvs2dO1qLvfsCZo6Rx6xFJOuT548mQIFCuDj40PlypXZtWvXfbe/fv06Xbp0IVeuXHh7e1O0aFFWrVr1iKIVEREREUsMHw7Hj0O+fOakuSKPWIISqPnz58d729OnT7Nt27Z4bbtgwQJ69erFoEGD+O233yhXrhzBwcFcvHgx1u0jIiKoW7cuJ06cYPHixRw6dIgZM2aQJ0+eeMcnIiIiIqnMwYMwerS5PGECBARYG4+kSwlKoKZOnUqJEiX49NNPOXjwYIzrb9y4wapVq2jTpg1PPPEEV65cidf9jhs3jrfeeov27dtTsmRJpk2bhp+fH19+Gfs8Dl9++SVXr15l2bJlVK1alQIFClCjRg3KlSuXkIcjIiIiaYSfpx+3+t3iVr9b+Hn6WR2OJAfDgM6dwW6HRo2gSROrI5J0KkFjoDZv3sz333/P559/Tr9+/fD39ydHjhz4+Phw7do1zp8/T7Zs2WjXrh1//vknOXLkeOB9RkREsGfPHvr16+dc5+bmRp06ddixY0est/n++++pUqUKXbp0Yfny5QQFBdGmTRv69u2Lu7t7rLcJDw8nPDzceTkkJAQAu92O3W5PyGFIctH7tzoOEbk/nasiKZuXzQvQuZpW2b75Bo9NmzB8fYkcN86cRFdStZR0riYkhgQXkXjhhRd44YUXuHz5Mlu3buXkyZOEhYWRLVs2KlSoQIUKFXBLQCWUy5cvExUVFSPZypEjB3///Xestzl27BgbNmzglVdeYdWqVRw5coTOnTtjt9sZFEdf2JEjRzJkyJAY69etW4efX8r4pWr9+vVWhyAi8aBzVSR10LmadnjeukXtnj3xAA42bco/Bw7AgQNWhyVJJCWcq7dv3473tjbDMIxkjOWBzp49S548edi+fTtVqlRxrn///ffZvHkzO3fujHGbokWLcufOHY4fP+5scRo3bhyjR4/m3Llzse4nthaofPnycfnyZQIDA5P4USWM3W5n/fr11K1bN8Z8WiKScuhcFUm5wiPD6by6MwAT6kzg540/61xNQ9y6dMF9xgyM4sWJ3L0bvLysDkmSQEr6XA0JCSFbtmzcuHHjgblBosuYJ5Vs2bLh7u7OhQsXXNZfuHCBnDlzxnqbXLly4enp6dJdr0SJEpw/f56IiAi8YjmpvL298fb2jrHe09PT8icsWkqKRUTipnNVJOWJMCL4ev/XAEwMngjoXE0zdu6EmTMBsE2bhqe/v8UBSVJLCedqQvZveRlzLy8vKlasyE8//eRc53A4+Omnn1xapO5WtWpVjhw54jIX1eHDh8mVK1esyZOIiIiIpEKRkdCpk1lAom1bqFHD6ohErE+gAHr16sWMGTP46quvOHjwIO+88w6hoaG0b98egNdff92lyMQ777zD1atX6dGjB4cPH2blypWMGDGCLl26WPUQRERERCSpTZoE+/ZB5sz/lS8XsZjlXfgAWrZsyaVLlxg4cCDnz5+nfPnyrFmzxllY4tSpUy6FKfLly8fatWvp2bMnZcuWJU+ePPTo0YO+ffta9RBEREREJCmdOQMDBpjLn3wCQUHWxiPyPw+dQEVERHD8+HEKFy6Mh0fi765r16507do11us2bdoUY12VKlX45ZdfEr0/EREREUnBevaEW7fg6aehQwero5EkFhUFmzfb+PnnPPj726hVC+KYjSjFSXQXvtu3b/Pmm2/i5+dHqVKlOHXqFADdunXjk08+SbIARURERCSdWbMGFi0yv1FPmwYJmCJHUr6lS6FAAahb14Nx4ypRt64HBQqY61ODRL8a+/Xrx++//86mTZvw8fFxrq9Tpw4LFixIkuBEREREJJ0JC4Poce3du0O5ctbGI0lq6VJo1gz+/dd1/Zkz5vrUkEQlus/dsmXLWLBgAU8//TQ2m825vlSpUhw9ejRJghMRERGJDz9PPy72vuhcllRs5Eg4dgzy5IEhQ6yORpJQVBT06GEWVbyXYYDNBu++Cy++mLK78yW6BerSpUtkz549xvrQ0FCXhEpEREQkudlsNoL8gwjyD9L3kNTs0CGzYATAhAmQIYO18UiS2rIlZsvT3QwDTp82t0vJEp1AVapUiZUrVzovR79ZzZw5M875m0REREREYmUY0Lkz2O1Qvz68/LLVEUkSO3cuabezSqK78I0YMYL69etz4MABIiMjmTBhAgcOHGD79u1s3rw5KWMUERERua/wyHB6re0FwKjnRlkcjSTKt9/Chg3g42PO/6SWxDQnvpXoc+VK3jgeVqJboJ599ln27dtHZGQkZcqUYd26dWTPnp0dO3ZQsWLFpIxRRERE5L4iHZFM2T2FKbunEOmItDocSajr182y5QD9+0OhQpaGI0nv0iUYNuz+29hskC8fVKv2aGJKrIeaB6pw4cLMmDEjqWIRERERkfToo4/g4kUoVgx697Y6Gklie/dCkyZw6pTZwHjnjpks3V1MIrrBcfz4lF1AApJgIt2LFy9y8eJFHA6Hy/qyZcs+7F2LiIiISFr3668wdaq5PHUqeHtbG48kqW+/hTffNKvTP/44LF8OBw+a1fjuLiiRN6+ZPKWGoW+JTqD27NlD27ZtOXjwIMY9tQhtNhtRUVEPHZyIiIiIpGFRUdCpk9kU8eqrUKuW1RFJEomKgg8/hE8/NS/Xq2cmU5kyQYkSZqnyjRsjWb16H/Xrl6dWLY8U3/IULdEJ1BtvvEHRokX54osvyJEjh0qGioiIiEjCTJkCv/1mfqseM8bqaCSJXLsGbdrAmjXm5b59Yfhw16557u5Qo4ZBaOgZatQol2qSJ3iIBOrYsWMsWbKEIkWKJGU8IiIiIpIenD1rjn0Cc/LcHDmsjUeSxIEDZuvSkSPg6wtffgmtWlkdVdJKdBW+2rVr8/vvvydlLCIiIiKSXvTqBTdvwlNPQceOVkcjSWD5cqhc2UyeHnsMtm1Le8kTPEQL1MyZM2nbti1//vknpUuXxtPT0+X6F1544aGDExEREYkPX09fjvc47lyWFG7dOliwANzcYNo087+kWg4HfPwxDBpkXq5ZExYujP+8T6lNohOoHTt2sG3bNlavXh3jOhWREBERkUfJzeZGgUwFALDb7dYGI/d35w506WIud+sGFSpYG488lJs3oW1b+O4783K3bjB2LNzTtpKmJDrd79atG6+++irnzp3D4XC4/Cl5EhEREZFYffKJ2ccrd24YOtTqaOQhHDkCVaqYyZOXlzneaeLEtJ08wUO0QF25coWePXuSQwP+RERExGIRURF89JNZkGBw9cHWBiNx++cfs2AEmJP+BAZaGo4k3rp10LIlXL8OuXLB0qXw9NNWR/VoJLoF6uWXX2bjxo1JGYuIiIhIotij7IzZMYYxO8Zgj1IXvhTJMKBzZ4iIgOBgaNbM6ogkEQzDrDhfv76ZPD39NOzenX6SJ3iIFqiiRYvSr18/tm7dSpkyZWIUkejevftDByciIiIiacSCBfDjj+DtDZMmgeYQTXXCwuCtt2DuXPPyG2+YU3l5e1sb16P2UFX4AgIC2Lx5M5s3b3a5zmazKYESEREREdONG9Czp7n80UegeURTnVOn4KWXzHmP3d3NHphduqTPPDjRCdTx48eTMg4RERERSav694fz56FoUXj/faujkQTasgWaNoVLlyBbNli0yCxVnl6p6L6IiIiIJJ89e8x+XpA++3ulYoYBU6fCc8+ZyVP58uZ4p/ScPEECW6B69erFsGHD8Pf3p1evXvfddty4cQ8VmIiIiIikclFR0KmTOdNqmzZQu7bVEUk8hYebczrNmGFebtnSLFPu52dtXClBghKovXv3Oien27t3b5zb2dJjZ0gRERERcTVtmtlkkTGjObuqpArnz5td9rZvN8c4jRxp9rzUV3xTghKojRs3MmfOHFq2bKkS5iIiIpJi+Hr68uc7fzqXJQU4fx4+/NBcHj4ccua0Nh6Jl19/NYtFnDlj5r3z50O9elZHlbIkeAxU+/btuXHjRnLEIiIiIpIobjY3SmUvRanspXCzaYh3ivDeexASApUqmd34JMX76iuoVs1MnkqUMJMpJU8xJfgdxjCM5IhDRERERNKKH3+EefPAzc3sxufubnVEch+RkWaV+XbtzLFPL7wAv/wCjz9udWQpU6LKmGuMk4iIiKQkEVERjNgyAoA+T/exOJp0LjzcnCAIoHNnqFjR2njkvq5cgRYtYMMG8/LAgTBokJn7SuwSlUDVrl0bD4/73/S3335LVEAiIiIiCWWPsjNk8xAA3n3yXWuDSe8+/RQOHzbHPH38sdXRyH388Qe8+CKcOAH+/jBnDrz8stVRpXyJSqCCg4MJCAhI6lhEREREJDU7csQsGAHw2WdmFQJJkRYtMrvs3b4NhQrB8uVQurTVUaUOiUqg+vTpQ/bs2ZM6FhERERFJrQzD7LoXHg5165oTB0mK43DAgAEwwuzxSt26ZqW9LFmsjSs1SXACpfFPIiIiIhLDokWwbh14e8PkyZo0KAW6cQNeeQVWrjQvv/cefPIJPGBkjtwjwYdLVfhERERExEVICLz7rrn8wQcq35YCHTpkjnc6dAh8fGDmTDOZkoRLcAJ1/PhxgoKCkiMWEREREUmNBg6Ec+egSBEzgZIUZeVKaNPGzHPz5oVly1Qc8WEkuEBh/vz51Y1PREREREy//Qaff24uT5liNm9IimAY5linxo3N5OnZZ2H3biVPD0s9HkVERCTV8/HwYVeHXc5leUSiouCdd8zKBC1bmhUJJEW4dQvat4fFi83L77wD48eDl5elYaUJSqBEREQk1XN3c+fJPE8CYLfbLY4mHZkxA3btgsBAGDfO6mjkf44fhyZNzHmePD1h0iTo2NHqqNIOJVAiIiIiknAXLvw33unjjyF3bmvjEQA2bIAWLeDKFciRA5YsgapVrY4qbUlQAvXHH3/Ee9uyZcsmOBgRERGRxIiIimDCLxMA6Fyxs8XRpBO9e5t1sZ94AjrrmFvNMGDCBPNpiYqCJ5+EpUvNohGStBKUQJUvXx6bzRZnKfPo62w2G1FRUUkSoIiIiMiD2KPsvP/j+wC8Vf4ti6NJBzZuhG++Med6mjYN3N2tjihdu3MHOnWCr74yL7/+OkyfrnoeySVBCdTx48eTKw4RERERSQ3Cw82KBGD+f/JJa+NJ586cgZdegl9/NfPYMWOgRw/NY5ycEpRA5c+fP7niEBEREZHUYMwYczbWHDlg+HCro0nXtm+Hl182h6NlyQILF0Lt2lZHlfY9dBGJAwcOcOrUKSIiIlzWv/DCCw971yIiIiKSkhw7ZhaMALPqXqZMloaTns2YAV26gN0OZcqYk+MWKmR1VOlDohOoY8eO8dJLL7F//36XcVHRk+xqDJSIiIhIGmIY0LWrOeCmdm1o3drqiNKliAh4912YOtW83KwZzJoFAQGWhpWuuCX2hj169KBgwYJcvHgRPz8//vrrL37++WcqVarEpk2bkjBEEREREbHc0qWwerU5E+vkyRpkY4GLF6FOHTN5stnMxsCFC5U8PWqJboHasWMHGzZsIFu2bLi5ueHm5sazzz7LyJEj6d69O3v37k3KOEVERETEKjdvmpUJAPr2hWLFrI0nHdqzxywWcfq0OW/x3LnQqJHVUaVPiU6goqKiyJAhAwDZsmXj7NmzFCtWjPz583Po0KEkC1BERETkQXw8fNjYdqNzWZLYoEFmubdChaBfP6ujSXfmzoUOHczek0WLwvLlULy41VGlX4lOoEqXLs3vv/9OwYIFqVy5Mp9++ileXl783//9H4U0gk1EREQeIXc3d2oWqAmA3W63Npi0Zt8+mDjRXJ48GXx9LQ0nPYmKgg8+MAsfAjRoYCZTqt1hrUQnUP379yc0NBSAoUOH0qhRI6pVq0bWrFlZsGBBkgUoIiIiIhZxOMy5nqKioHlzqFfP6ojSjatXzTod69aZlz/8EIYO1ZzFKUGiE6jg4GDncpEiRfj777+5evUqmTNndlbiExEREXkU7FF2/m/P/wHQvmx7i6NJQ2bOhF9+MasUfPaZ1dGkG3/+CU2awNGj4OcHs2eb+aukDImuwvfNN984W6CiZcmSRcmTiIiIPHIRURF0Xd2Vrqu7EhEV8eAbyINdvGj2HwOz3FuePNbGk0589x08/bSZPBUoYE6Wq+QpZUl0AtWzZ09y5MhBmzZtWLVqleZ9EhEREUlL+vSBa9egfHlzxlZJVg4HDB4ML78MoaHw3HPw669QrpzVkcm9Ep1AnTt3jvnz52Oz2WjRogW5cuWiS5cubN++PSnjExEREZFHbfNmmDPHnGxo2jTwSPSoD4mHkBAzcRoyxLzcowesXQvZslkbl8Qu0QmUh4cHjRo1Yu7cuVy8eJHPPvuMEydOUKtWLQoXLpyUMYqIiIjIoxIRYRaOAHj7bahc2dp40rh//jG77C1fDt7eMGsWjB+vnDUlS5Knxs/Pj+DgYK5du8bJkyc5ePBgUtytiIiIiDxqY8fCwYOQPTuMGGF1NGnamjVmpb3r1yF3bnP801NPWR2VPEiiW6AAbt++zdy5c2nQoAF58uRh/PjxvPTSS/z1119JFZ+IiIiIPCrHj8OwYebymDGQObO18aRRhgGffgoNG5rJU5UqsHu3kqfUItEtUK1atWLFihX4+fnRokULBgwYQJUqVZIyNhERERF5VAwDunWDsDCoWRNefdXqiNKk27ehQwf49lvzcocOMGmS2X1PUodEJ1Du7u4sXLiQ4OBg3DWjl4iIiFjI28ObFa1XOJclEZYtg5UrwdMTpk41C0hIkjp5El56CfbuNcc4TZhgDjfToU5dEp1AzZ07NynjEBEREUk0DzcPGhZtCIDdbrc4mlTo1i3o3t1c7tMHihe3Np40aPNmaNYMLl+GoCBYvBiqV7c6KkmMBI+BatCgATdu3HBe/uSTT7h+/brz8pUrVyhZsmSSBCciIiIij8CQIfDvv1CwIHz0kdXRpCmGAZMnQ506ZvL0xBPmeCclT6lXghOotWvXEh4e7rw8YsQIrl696rwcGRnJoUOHkiY6ERERkXiwR9mZvW82s/fNxh6lFqgE+eMP+Owzc3nSJPDzszaeNCQ8HN56C7p2hchIs+Leli3w2GNWRyYPI8Fd+AzDuO9lERERkUctIiqC9svbA9Dk8SbWBpOaOBzmIJyoKGjaFBo0sDqiNOPcOfOQ7tgBbm4wahS8957GO6UFmqJLREREJL2aNQu2b4eAAHP2VkkSO3eaxSLOnYNMmWD+fAgOtjoqSSoJ7sJns9mw3ZM633tZRERERFK4y5fh/ffN5SFDIG9ea+NJI2bNMsc3nTsHJUvCr78qeUprEtWFr127dnj/r1j9nTt36NSpE/7+/gAu46NEREREJIV6/324ehXKlfuvAp8kmt0OvXvDxInm5SZNYM4cyJDB0rAkGSQ4gWrbtq3L5VdjmWTt9ddfT3xEIiIiIpK8tmwxm0rAnPPJQ6M6Hsbly9CiBWzcaF4ePBgGDDDHPknak+CzZVb0ySYiIiIiqY/dbhaOALNEXJUq1saTyu3bZ7Y2nTxpDiX7+mvzsqRdKSYvnjx5MgUKFMDHx4fKlSuza9eueN1u/vz52Gw2muiVKiIiIvJgn30Gf/0F2bLBJ59YHU2qtmABPPOMmTwVKQK//KLkKT1IEQnUggUL6NWrF4MGDeK3336jXLlyBAcHc/Hixfve7sSJE/Tu3Ztq1ao9okhFREQkJfL28GZhs4UsbLYQbw9vq8NJuU6eNAtGAIwZA1myWBtPKhUVBf36QatWEBZmFonYtQtKlbI6MnkUUkSH13HjxvHWW2/Rvr05f8O0adNYuXIlX375JR988EGst4mKiuKVV15hyJAhbNmyhevXr993H+Hh4S4FLkJCQgCw2+3Y7dZOuBe9f6vjEJH707kqkrI1KdoE0Ll6P+5du+J2+zaOatWIat3a7M4nCXL9Orz+ujtr1pjtEO+9F8XHHztwd9fhTKiUdK4mJAbLE6iIiAj27NlDv379nOvc3NyoU6cOO3bsiPN2Q4cOJXv27Lz55pts2bLlgfsZOXIkQ6J/cbnLunXr8EshM26vX7/e6hBEJB50roqkDjpXXeXcuZPKK1bgcHdnU4sW3Fy92uqQUp3TpwMYObIyZ88G4OUVRdeue6lW7Qxr11odWeqWEs7V27dvx3tbyxOoy5cvExUVRY4cOVzW58iRg7///jvW22zdupUvvviCffv2xXs//fr1o1evXs7LISEh5MuXj+eff57AwMBExZ5U7HY769evp27dunh6eloai4jETeeqSMoV6Yhk2aFlADQs1JCNP23UuXq30FA8/leq3OjVi2pvv21xQKnPihU2PvzQnZs3bTz2mMGiRQ4qVCgHlLM6tFQrJX2uRvdOiw/LE6iEunnzJq+99hozZswgW7Zs8b6dt7e3c+6qu3l6elr+hEVLSbGISNx0roqkPBEREbT5rg0A13pfA3Suuhg5Ek6dgvz5cR88GHcdl3hzOGDECBg4EAzDnCR30SIb2bPrGCaVlHCuJmT/lidQ2bJlw93dnQsXLrisv3DhAjlz5oyx/dGjRzlx4gSNGzd2rnM4HAB4eHhw6NAhChcunLxBi4iIiKQWf/4J48aZy5MmQQoZupAa3LoFbdvC0qXm5S5dzCKGyj/TN8ur8Hl5eVGxYkV++ukn5zqHw8FPP/1ElVjmJShevDj79+9n3759zr8XXniBWrVqsW/fPvLly/cowxcRERFJuRwOc86nyEizvnajRlZHlGocO2ZOkbV0qZkwzZhh5p9KnsTyFiiAXr160bZtWypVqsRTTz3F+PHjCQ0NdVble/3118mTJw8jR47Ex8eH0qVLu9w+U6ZMADHWi4iIiKRrX30FW7earU4TJlgdTarx44/QogVcuwY5c5pJlOYblmgpIoFq2bIlly5dYuDAgZw/f57y5cuzZs0aZ2GJU6dO4eZmeWOZiIiISOpx5Qr06WMuDxkCjz1mbTypgGGYXfT69DEb7556ykye8uSxOjJJSVJEAgXQtWtXunbtGut1mzZtuu9tZ8+enfQBiYiIiKRmffuaSVTp0tCjh9XRpHhhYdCxI3zzjXm5XTuYOhV8fCwNS1KgFJNAiYiIiEgS2bYNvvjCXJ42TQN3HuD0aXj5Zdi9G9zdzZob3bqBzWZ1ZJISKYESERGRVM/L3YtZL85yLqdrdrtZOALgzTehalVr40nhtm6Fpk3h4kXImhUWLYJatayOSlIyJVAiIiKS6nm6e9KufDvAnJwzXZswAfbvN7OBUaOsjiZFmz7dbGmy26FsWVi+HAoUsDoqSelUmUFEREQkrTh9GgYPNpdHjzaTKIkhIgI6dTL/7Haz4t727UqeJH7UAiUiIiKpXqQjkrVH1gLwXP7nLI7GQj16QGgoPPusOQOsxHDhAjRrZnbds9lgxAiz3obGO0l8KYESERGRVC88MpxG35qTxF7rfc3iaCyyYgV89x14eJjl4zQFTAy7d5vzCZ85Axkzwrx50KCB1VFJaqMzS0RERCS1u30boqeD6dXLLF0uLr7+2myYO3MGiheHnTuVPEniKIESERERSe2GDYOTJ83JcgcOtDqaFCUyEt57D15/HcLDoXFj+OUXKFbM6sgktVICJSIiIpKaHTgAY8aYyxMngr+/tfGkIFeuQP365rxOAP37w7JlZvc9kcTSGCgRERGR1MowzDmfIiPhhRfgxRetjijF2L/fPBzHj5s55VdfmfM9iTwstUCJiIiIpFZz5sDPP4Ofn9n6JAAsWQJVqpjJU8GCsGOHkidJOkqgRERERFKjq1ehd29zeeBAyJ/f2nhSAIcDBgwwy5SHhkLt2vDrr1CmjNWRSVqiLnwiIiKS6nm5ezGp/iTncrrQrx9cvgylSpmV99K5kBB49VX44Qfzcs+e8OmnZlV3kaSkl5SIiIikep7unnR5qgsAdrvd4mgegR074P/+z1yeOhU8Pa2Nx2KHD5vjnf7+G7y9zUPz+utWRyVplRIoERERkdQkMtIsHAHQrh1Uq2ZpOFZbtQratIEbNyBvXnMu4UqVrI5K0jKNgRIREZFUL8oRxaYTm9h0YhNRjiirw0len38Ov/8OWbKYfdTSKcOATz6BRo3M5KlqVXO8k5InSW5qgRIREZFU707kHWp9VQuAa72vWRxNMvr33/8myh01CoKCrI3HIqGh8OabsGCBebljRzOv9Eonw9/EWkqgRERERFKLd9+FW7fgmWfgjTesjsYSJ05AkyZmI5yHB0yaBG+/bXVUkp4ogRIRERFJDVatMic4cnc3C0e4pb+RGBs3QvPmcOUKZM9uHo5nn7U6Kklv0t+ZJyIiIpLa3L4NXbuay+++C2XLWhrOo2YYZhe9unXN5KliRdi9W8mTWEMJlIiIiEhKN2IEHD9ulpkbPNjqaB6pO3fM8U7du0NUlDnX05YtkC+f1ZFJeqUufCIiIiIp2d9//1dtb+JECAiwNp5H6OxZePll2LnT7LE4erQ5Qa7NZnVkkp4pgRIRERFJqQzDnPPJboeGDc3qCenEjh1m8nT+PGTObFbcq1vX6qhElECJiIhIGuDp7smndT51LqcZc+fCpk3g62sOAkonTS9ffAGdO0NEBJQuDcuWQeHCVkclYlICJSIiIqmel7sXfar2AcBut1scTRK5dg169TKXBwyAggWtjecRsNvNLnqTJ5uXX34ZvvoqXfValFRACZSIiIhISvThh3DpEpQoAe+9Z3U0ye7SJbNE+ebN5uVhw8xDkA6rtUsKpwRKREREUr0oRxS/nfsNgDLZylgcTRLYuROmTzeXp04FLy9r40lme/eaw7tOnYIMGeCbb+CFF6yOSiR2SqBEREQk1bsTeYenZj4FwLXe1yyO5iFFRkKnTmYBiddfhxo1rI4oWX37rVmmPCwMHn8cli83G91EUio1ioqIiIikJJMnw759Zum50aOtjibZREVB377Qpo2ZPNWvD7t2KXmSlE8JlIiIiEhKceaMWTAC4JNPIHt2a+NJJteuQaNG/01v9cEH8MMPkCmTpWGJxIu68ImIiIikFD17ws2b8PTT0KGD1dEkiwMH4MUX4cgRszr7rFnQsqXVUYnEnxIoERERkZRg7VpYtAjc3WHatDRZfm75cnj1Vbh1C/LnN+d3Kl/e6qhEEibtnZkiIiIiqU1YGHTpYi537w7lylkbTxJzOGDIELPS3q1bULMm/PqrkidJndQCJSIiImK1kSPh6FHIk8fMNNKQmzfNYoLLlpmXu3WDsWPB09PSsEQSTQmUiIiIpHqe7p4MqjHIuZyqHD4Mo0aZyxMmmBMhpRFHjpitTn/9ZU5lNW0atG9vdVQiD0cJlIiIiKR6Xu5eDK45GAC73W5tMAlhGNC5M0REmHW8X37Z6oiSzLp1ZnGI69chVy5YutSsjSGS2mkMlIiIiIhVvv0WfvoJfHxg0iSw2ayO6KEZBowZY+aD16+bSdPu3UqeJO1QC5SIiIikeg7DwcFLBwEokqmIxdHE0/Xr0KuXudy/PxQqZGk4SSEsDN56C+bONS+/8QZMmQLe3tbGJZKUlECJiIhIqhdmD6P01NIAXOt9zeJo4ql/f7hwAYoVg969rY7moZ06BS+9BL/9Bh4eMH682TsxDTSqibhQAiUiIiLyqP36q9k0A2miiWbLFmjaFC5dgmzZzOmsata0OiqR5KExUCIiIiKPUlQUvPOOOVjo1VfhueesjijRDAOmTjUfwqVL5rxOu3creZK0TQmUiIiIyKM0dSrs2QOZMpnVFlKp8HB4+22zm15kJLRqBdu2Qf78VkcmkrzUhU9ERETkUTl3Dj76yFweMQJy5LA2nkQ6dw6aNYPt280xTp98An36aLyTpA9KoEREREQelV69ICQEnnoKOna0OppE2bXLLBZx9ixkzAjz50O9elZHJfLoqAufiIiIyKOwfr2Zbbi5wbRp4O5udUQJ9tVXUL26mTyVKGHWwlDyJOmNWqBEREQk1fN096R3ld7O5RTnzh1zsBBA165QoYK18SRQZKRZaX3CBPPyCy/A119DYKC1cYlYQQmUiIiIpHpe7l6Mfn40AHa73eJoYjFqFBw5ArlywbBhVkeTIFeuQIsWsGGDeXngQBg0yGxIE0mPlECJiIiIJKd//jELRoA5u2wqarb54w948UU4cQL8/WHOHHj5ZaujErGWEigRERFJ9RyGg1M3TgGQyy+XxdHcxTCgSxeIiIDgYGje3OqI4m3RImjXDm7fhkKFYPlyKF3a6qhErKfGVxEREUn1wuxhFJxQkIITChJmD7M6nP8sXGgWj/D2hkmTUkWd76gos9J6ixZm8vT882axCCVPIia1QImIiIgkhxs34N13zeUPP4QiRSwN5//bu/f4nuv//+O3984b25w3hzllSyp8UKwitJqPQ858ln5U8ikdKCWpD4qKEEr6Kkl9ipKzjySHDGVphJzPjDFnZhs7vN+v3x8vezPe79nY9t7mfr1c3pf36/06Pl9vnu+9H+/n8/l45sT589CjB/z0k/n69ddh5Ejw0DdGETtVBxEREZH8MGQIJCRAWBgMGuTq0tzQrl3meKddu8DHB7780gymRCQrBVAiIiIieW3DBpg0yVz+7DOzC18htmiRGSwlJkJICMybBw0burpUIoWTxkCJiIiI5CWrFZ5/Hmw2iIqCRx5xdYmcMgwzQeDjj5vBU9OmsH69gieR7KgFSkRERCQvff65GYUEBsK4ca4ujVNJSfD00zB7tvm6b18zy7qXl0uLJVLoKYASERERySsJCWbCCID334fgYNeWx4kDB8zxTlu2gKen2duwTx9Xl0qkaFAAJSIiIkWeh5sHLzR6wb7sMq+9Zqaya9TI7MZXCK1YYaYoP3MGgoJgzhx48EFXl0qk6FAAJSIiIkWet4c3k9qYSRvS09NdU4gVK2DGDHBzg8mTwd3dNeVwwjDg44/N1ORWK9x3H8ydC1WquLpkIkWLkkiIiIiI3KrUVHjBbAHjhRcKXRaGS5fM8U6vvmoGTz17wurVCp5EboZaoERERKTIMwyDUymnAAj0DCz4AoweDbt3m2Oe3nuv4K+fjSNHoFMniI01G8U++gj69QOLxdUlEymaFECJiIhIkZeSnkKFsRUAOPv62YK9+N69ZsIIgPHjzex7hcTatWbwdPw4lCkDP/5YqLOqixQJ6sInIiIicrMMA156yezCFxEB3bu7ukR2U6ZA8+Zm8HTvvWYLlIInkVunAEpERETkZs2eDb/8Yk6eNGlSoegXl5ZmDsP6978hPR26doWYGKhZ09UlEyke1IVPRERE5GYkJsIrr5jLgwdDWJhLiwNw4gR06QJr1pix3HvvmUUrBHGdSLGhAEpERETkZgwdCkePQq1a8Oabri4NGzZAx45w+DAEBMD06dC2ratLJVL8qAufiIiISG5t3AgTJ5rLkyaBj49LizN9Ojz0kBk83XknrFun4EkkvxSaAGrSpElUr14dHx8fGjduzJ9//ul03ylTptC0aVNKly5N6dKliYiIyHZ/ERERkTxjtcLzz4PNZiaNeOwxlxZl4EB48klzrqc2bczgqXZtlxVJpNgrFAHUzJkzGTBgAMOGDeOvv/6iXr16REZGcuLECYf7R0dHExUVxcqVK4mJiSEkJITHHnuM+Pj4Ai65iIiIFAYebh70qteLXvV64eGWzyMUpkyBP/8Ef38YNy5/r5WNM2egdWsYO9Z8/dZbsGBBocqiLlIsFYoxUOPGjaNPnz48/fTTAEyePJmffvqJr776ijcd9CmePn16ltdffvklc+bMYcWKFfTs2dPhNVJTU0lNTbW/TkxMBCA9PZ309PS8upWbknl9V5dDRLKnuipSeLnhxpQ2U4B8rqvHj+MxeDAWwDp8OLby5c1UdwVs61bo2tWDffss+PkZfPmllS5dDGw2s2FMpCgoTH9Xc1MGlwdQaWlpbNiwgcGDB9vXubm5ERERQUxMTI7OkZKSQnp6OmXKlHG6z8iRI3n33XevW7906VL8/PxyX/B8sGzZMlcXQURyQHVVpGjIj7raYMIEQs6d41zNmqyqWhUWL87za9zIH39UZMKEBly6ZKFChWQGD/4TP79EVxRFJE8Uhr+rKSkpOd7X5QHUqVOnsFqtBAUFZVkfFBTEzp07c3SOQYMGUalSJSIiIpzuM3jwYAYMGGB/nZiYaO/6FxAQcHOFzyPp6eksW7aMRx99FE9PT5eWRUScU10VKbwMwyAl3fwC5Ikny5cvz/O6aomOxiM6GsNioeR339G6UaM8O3dO2GwwYoQbo0a5A9CihY3p070oV+6hAi2HSF4pTH9XM3un5YTLA6hbNWrUKH744Qeio6PxySYDjre3N97e3tet9/T0dPk/WKbCVBYRcU51VaTwSU5LpvTY0gCcff0skMd1NS0N+vUDwNK3Lx7h4Xlz3hxKTISePc0xTgD9+8PYsW54eBSK4ewit6Qw/F3NzfVdHkCVK1cOd3d3jh8/nmX98ePHCQ4OzvbYsWPHMmrUKJYvX07dunXzs5giIiJyOxs7FnbuhKAgeP/9Ar30nj3Qvj3s2AHe3vD559CrV4EWQUSu4vKfLby8vGjYsCErVqywr7PZbKxYsYLwbH7dGT16NCNGjGDJkiU0KuAmdBEREbmN7N8PI0aYyx99BKVKFdillyyB++83g6dKlWD1agVPIq7m8gAKYMCAAUyZMoVvvvmGHTt20LdvX5KTk+1Z+Xr27JklycSHH37IkCFD+Oqrr6hevToJCQkkJCSQlJTkqlsQERGR4sgw4OWXzUmWWraEJ54osMuOHm3O63TuHISHw/r1ZjAlIq7l8i58AN27d+fkyZMMHTqUhIQE6tevz5IlS+yJJeLi4nBzuxLr/d///R9paWl06dIly3mGDRvGO++8U5BFFxERkeJs3jwz056XF3z2GVgs+X7JlBR49ln4/nvzdZ8+MHGi2X1PRFyvUARQAC+99BIvvfSSw23R0dFZXh88eDD/CyQiIiK3twsX7IkjeOMNuPPOfL/koUPQoQNs2gQeHvDJJ/D88wUSt4lIDhWaAEpERESkUHnnHYiPh5o14a238v1yq1ZBly5w6hSULw+zZ0OzZvl+WRHJJQVQIiIiUuS5u7nTpU4X+/It27wZPv7YXJ40CXx9b/2cThiG2TvwlVcgIwMaNDB7Dlatmm+XFJFboABKREREijwfDx9mdZ0FmJNz3hKbDfr2BasVunaFVq3yoISOpabCCy/AV1+Zr594AqZMAT+/fLukiNwiBVAiIiIiV5s6FWJioGRJGD8+3y5z7Bh06gR//AFubvDhh/DaaxrvJFLYKYASERERyXTiBAwaZC6PGAGVK+fLZdatg44dzSCqVCn44QeIjMyXS4lIHisU80CJiIiI3IrktGQs71qwvGshOS355k/0xhtw9izUrw9OsgPfqmnTzOQQx47B3XdDbKyCJ5GiRAGUiIiICJhp8L75xuxDN3mymUc8D6Wnm1nRn3kG0tLMFqiYGKhVK08vIyL5TAGUiIiISFqamTgC4N//hsaN8/T0p06ZrUwTJ5qv333XTFPu75+nlxGRAqAxUCIiIiLjxsGOHeYETCNH5umpN20yJ8c9dMjMS/Hdd9C+fZ5eQkQKkFqgRERE5PZ28CAMH24uf/QRlC6dZ6eeORMeeMAMnmrVMpNHKHgSKdoUQImIiMjtyzDg5Zfh4kVo3hyefDJPTmu1wuDB8K9/maeOjIQ//4Q6dfLk9CLiQurCJyIiIrevBQtg0SLw9ITPPsuTSZjOnTMnxP35Z/P1G2/ABx+Au/stn1pECgEFUCIiIlLkubu50zq0tX05R5KSzLR4AAMHwl133XI5duwwu+jt2QO+vuacvFFRt3xaESlEFECJiIhIkefj4cNPT/wEQHp6es4OevddOHwYqleHt9++5TL873/QowdcuABVq8L8+fCPf9zyaUWkkNEYKBEREbn9bNkC48eby5MmgZ/fTZ/KZoP33jNbni5cMCfJXb9ewZNIcaUWKBEREbm92GzmnE9WK3TqBK1b3/SpkpKgVy+YO9d8/eKLZlzm6ZlHZRWRQkcBlIiIiBR5yWnJVBhbAYD4/vHZ7zxtGvz+O5QoARMm3PQ19+83W522bgUvLzMHRe/eN306ESkiFECJiIhIsZCSnnLjnU6dMtPigTn3U0jITV1r+XLo1g3OnoXgYLMFKjz8pk4lIkWMxkCJiIjI7WPQIDhzBurWvZKBLxcMA8aNM+d1OnsW7r/fHO+k4Enk9qEASkRERG4Pv/0GX31lLk+eDB6564hz8SL07AmvvWYOo3r6aVi1CipXzoeyikihpS58IiIiUvylp8Pzz5vLffrkusno8GHo2BE2bDAnxB0/Hl56KU/m3RWRIkYBlIiIiBR/48fDtm1QrhyMGpWrQ3/7DTp3hhMnoGxZmDULWrTIp3KKSKGnLnwiIiJSvB06ZE6aCzB2LJQpk+NDP/8cWrY0g6d69czxTgqeRG5vaoESERGRIs/N4sbD1R62L2fRvz+kpJgz3PbsmaPzpaWZOSY+/9x83b07TJ1qZj4XkdubAigREREp8nw9fYl+KhqA9PT0KxsWLoQFC8yEEZ99lqNBS8ePQ5cuZtc9iwU++MBM3qfxTiICCqBERESkuEpOhpdfNpdffx3uvvuGh6xfDx06QHw8BAbCjBnQunX+FlNEihYFUCIiIlJ8WK1YVq2i8urVuM2YAXFxUK0aDBlyw0O//dZM0JeaCrVrmw1XYWEFUGYRKVJynURiyZIlNGrUiLp169KkSRM2b94MQGxsLA8++CD16tWjfv36/Prrr/ZjOnbsSP369e0PNzc3Fi5cCMCLL76YZZuPjw+ffPIJAPPmzaNu3brUr1+fOnXq8Pbbb2MYhsNyNW/enBo1atjPM378ePs2wzB45513CAsL495776WFRn+KiIgUK8lpyZR/L5Dyb3uR2vpRGo0bh/uPP5obo6LAz8/psRkZMGCAOTwqNRXatYN16xQ8iYhjuWqBOnv2LD169GD16tXcfffdrFmzhh49erBlyxY6duzI119/TUREBLt37yYiIoJdu3bh6+vLvHnz7OdYv349rVq1olWrVgBMmjTJvi0hIYEaNWrQrVs3ACIiImjfvj1ubm6kpaXx0EMP0ahRIzp27OiwfOPHj6dDhw7Xrf/kk0/4+++/2bp1K15eXiQkJOTmtkVERKSwW7iAU9ZE8HWw7cMP4b77oFOn6zadPg3/+hcsX26+/s9/zIR9bspTLCJO5OrjYd++fZQtW5a7L/chbtq0KXFxccTGxnLy5EkiIiIACAsLo1SpUvz888/XnWPq1Kk8+eSTeHl5Xbftm2++ITIykuDgYAD8/f1xu/wJdunSJVJTU7HcxAjOMWPGMGrUKPs1M88vIiIixYDVCgMHZr/PK6+Y+11lyxYzrlq+3MyuN3s2jBih4ElEsperj4jQ0FBOnz7N2rVrAVi4cCEXLlzgyJEjVKxYkR8vN5XHxsaya9cuDh48mOX4ixcv8v3339O7d2+H5//qq6+u27Z27VruvfdeKlSoQMuWLWnfvr3T8r355pvce++9dO/enf379wOQmJjI8ePHWbBgAY0bN6Zx48bMnDkzN7ctIiIihUl6OuzZA4sXw4QJZstS/FHn+xsGHD4Ma9bYV82ZA+HhcOAA1KwJMTHmZLkiIjeSqy58gYGBzJ49m8GDB5OUlER4eDh16tTBw8ODBQsWMGjQIEaOHMndd9/NQw89hIdH1tPPnj3bPg7pWmvWrOHChQu0vibVzQMPPMCWLVs4efIknTt3Zs2aNTRr1uy647/99ltCQkIwDINJkybRtm1btm/fTkZGBhkZGVy8eJF169Zx8OBBHnjgAapUqQLArFmz8PPzo1OnTqxYsYLz589ToUIF7r//fhYtWgRAgwYNsNlsbNq0CYD27dvz22+/cfr0acqUKUOzZs2YP38+AHXr1sXT05MNGzYA0KZNG9avX8/x48cJCAjgscceY/bs2QDcfffdlCxZkj/++IPjx49z9uxZdu7cSXx8PCVKlKBt27b2YO/OO++kXLly/P777wD2rpJxcXF4e3vTqVMnZs6cic1m44477qBy5cqsXr0aMMeHxcXFsX//fjw8POjatStz5swhLS2NatWqcccdd9jHrD300EOcOHGC3bt3AxAVFcWCBQtISUmhSpUq1KlTh6VLlwIQHh7O+fPn2b59OwBdu3ZlyZIlXLhwgeDgYBo0aMDixYsBuO+++7h06RJbtmwBzHFx0dHRnD17lnLlyhEeHs7//vc/AP7xj38AsHHjRgDatWtHTEwMp06donTp0jRv3tzeLfTee+/Fx8eH2NhYAFq3bs1ff/1FQkIC/v7+tGrVilmzZgFQp04dAgMDiYmJAeCxxx5j+/btHDlyBD8/P9q3b8/3338PmK2oFSpU4LfffgOgZcuW7Nu3j0OHDuHl5UXnzp2ZNWsWGRkZ1KxZk6pVqxIdHQ1As2bNiI+PZ9++fbi5udG9e3fmzp1LamoqVatWJSwsjOWX+4o8+OCDnDp1il27dgHQvXt3Fi1aRHJyMpUrV+aee+7hl19+AaBx48YkJSWxbds2ALp06cLSpUtJTEwkKCiIRo0a8dNPPwHQsGFD0tPT+fvvvwHo0KEDq1ev5syZM5QtW5aHHnqIBQsWANjHJf71118AtG3blj///JMTJ04QGBjII488wty5cwG455578PPz488//wTgn//8J5s3b+bo0aOULFmS1q1b239IqV27NmXKlLH/4PLoo4+yc+dODh8+jK+vLx06dOCHH37AMAxCQ0MJDg5mzeUvNy1atODgwYMcOHAAT09PunTpwuzZs0lPT6dGjRpUr16dlStXAmZLeEJCAnv27MFisfCvf/2L+fPnc/HiRUJCQqhduzbLli0DzM+TM2fOsHPnTgC6devG4sWLSUpKolKlStSrV8/ecn7//feTkpLC1q1bAejUqRPLli1j//79REdH06RJkwL7jFi3bh0AkZGRbN26VZ8R+oy4PT4jHn+cH775BiMpidD0dIKPHWNNRgYkJdHi5585GBLCgRo18ExPp8vChfyvWxei8OQABzgRdJA/mprjnZuuXk1CxYrsCQ3Fsn493Zo1591357Nnz0Xq1w+hTJnadOmyjK1bISDg1j4jXPU9Qp8R+owoqp8RERERJCQkMGvWLPz9/V36PSIzr0NOWAxnWRlyIDU1leDgYGJjY6lVq1aWbXfddReffvopjzzyiH1d8+bNiYqK4rnnnrvuXE899RQVK1Zk5MiRTq83atQo4uPjmThx4g3L5uPjQ3x8PGXLlsXf35/NmzdTs2ZNwKygzZo1o1+/fpw/f56AgICc3nK+SE9PZ/HixbRu3RpPT0+XlkVEnFNdFcljhmEOQtq9+8pjz54rzxcvOj/W19fM8hAaCt7eJP84nZJvm5uS3ocS6dcfkrxoJVGfN+fy92wGDDCHR3koJ7GISxSmv6uJiYkEBgbmKDbI9UfGsWPHqFixIgAjRoygZcuW1KpVK8v6KVOmUKJECVq2bGk/bu/evaxfv96efe/aAs+ePdv+S0GmnTt3EhYWhpubGxcuXOCnn36ip4MZxDMyMjh9+jRBQUEAzJkzh6CgIMqWLQuYv34sWbKEF154gTNnzvDnn3/ywgsv5PbWRURE5GZcuJA1MLo6YDp3zvlxHh5m/7qwsCuP0FDzuVKlK4OVrFb4fSXgpBufxUJ6UBXuf60p23eBtzdMmQL/7//l9Y2KyO0g1wHU0KFDWbNmDRkZGYSHhzN16lQAvvjiC6ZPn45hGNx1113MmzcvS8KHr776is6dOzuM6H744QcaNmxIaGholvUzZ85k5syZeHp6YrVa6dKlC88++yxgZvMbOnQoixcvJjU1lTZt2pCamoqbmxvlypXLEqiNHDmSp59+ms8++wyAQYMG0bBhw9zeuoiIiDiTmgr79jkOkrLLfmuxQEiI4yCpevWcNQ+5u+M2ZiyNlj4BgNtVfWsMiwUMeCZxAtsT3KlSBebNg0aNbu12ReT2lasA6vTp08TGxuLj4wPAunXrKFeuHCdOnKB169YsXbqUpKQk4uLi2LNnDyEhIQCkpKRw4MABYmNjCQsL44MPPqBLly4AvP322yxcuBB3d3fq16/Pm2++yb/+9S8Ahg0bRvny5Zk4cSIeHh7MmzePN998Ex8fHxo1amTvF1uiRAn69+/P2LFjsV7OsFO6dGl7ucuWLcvChQuZNm0azzzzDJUqVbrFt01EROQ2ZLXCoUPXB0l79pjrbTbnx1aocH2AFBYGd9xhdse7Rb5dopj4pzdVx/XH13rEvj6eKvRjAvNSOvHgg2byiMsdVkREbkquAqiyZcvaB0ACjB07llWrVlG6dOls54EaO3Ys3t7e7N27lwMHDtC4cWNatGhB2bJlGThwIO+//z4A8fHx3HXXXURERFCuXDkWLFjA9OnT+eOPPwgMDOTkyZMO+0fu3LmTgQMHsnHjRipWrMh3331H37597QPhAA4ePMiUKVNo0qTJTb5VIiIitwHDgGPHrm9F2rPHbGFKS3N+bECA4yApNBQCA/O12HPnQpexnbAY7WnKGipyjGNUZI3RFBvuPPooLFoEDmZRERHJlVsaNjl16lRGjhzJ6dOnnc4DlZnVJbOrX40aNezZT5599llKlSplP19SUhKGYWC7/AvWmDFjGDZsGIGXP3TLly/vsBxbt26lbt269jFYrVu3pmfPnpw+fZqyZctis9l49tlnmThxIq+99tqt3LKIiEjxcObM9a1Imc9JSc6P8/Y2A6KrA6TMIKlCBbNLXgGzWqF/fzP2M3BnFc2v22fnTnB3L/CiiUgxdNMB1Nq1azl79ixt27bFw8PDPg9Ut27drpsHKi4ujmrVqtmPrV69OnFxcfbXn3zyCZMmTeLIkSN8+eWXVKhQAYDt27ezfv163n33XVJTU+nZsyf9+vW7riz16tXjr7/+Yvfu3YSFhfHdd99hGAaHDh2ibNmyjBs3jgcffFDjnkRE5PaSnAx7914fJO3ebWa/c8bd3Rx/5GhcUkiIS2eaTUyEuDizx2DmIzYWjhxPgVfqmDtN2g7pflmOy5wGqnnzgi+ziBQvNx1ATZ06lZ49e9rnesrJPFDO9OvXj379+rF582aefPJJHnvsMcqWLUtGRgYHDhxg9erVnD17locffpiaNWvStm3bLMeHhoYyefJkevbsSUZGBm3atKFUqVJ4eHiwdetW5syZY5/L4FqaB0rzN2j+Bs0DpXmg9BkBRfgz4uefISWFxn5+JB05wrZz5yApiS4LF7L0nntIDAwkKCGBRhs28FObNlC9Og1PnSI9KIi/GzeGEiXokJHB6pAQzvj5UbZiRR6KjGTB5bqQ5TPi1Kl8/YzYsWMn+/YdJj3dl8DADuze/QMpKQYJCaEcOBBMSMgaMjJg5coWVK9+kBo1DpCe7snKlV3o0m0hnqUe4AAHOBh0nBZN/wBg9eqmVKyYQGjoHtavt9C8ef59RmgeKH1GFMrPiEL8PeK2mgcqKSmJihUrEhsbS+3atR3uc/U8UHfffTdTp061jz/q1q0bjz32mD2j3tVatWpFnz596Ny5M/fccw+ffPKJPR36wIED8fb25r333su2fAkJCVSvXp0zZ87wzTffMHz4cLy9ve3bAgICePPNN3nttdc0D5SI5JjqqriMzWY2oThK3nDggNmHzZmyZR2PS6pVC0qUKLh7wBw+dfiw2Wp0dStS5vLhw2YyvxspUwaqVYOqVc3n9HT4vy+T4e2S5g7vJ0H69fe2cqVaoEQKk8L0dzVf54ECM714vXr1sgRP2c0D1bVrVyZPnkyTJk04cOAA0dHR9pTi27dvp04ds8l93759bNy40f76iSeeYMmSJbRs2ZKLFy8SHR3NG2+84bBMmde3Wq0MGjSIF198ET8/P/r27Uvfvn3t+zVv3pxXXnmFli1bajyUiIgUHoYBJ044Tt6wdy9cuuT82BIlnCdvKFOmwG7h/HnHgVHmIyHBvM3suLlB5cpXgqOrA6XM5ZIlsx5jtcKCn53OAoXFAlWqQNOmeXKbInKbu6kAaurUqfTp0yfLuuzmgRo4cCDPPPMMd9xxB+7u7nz66aeUK1cOgDfeeMPevObh4cGnn37KXXfdBcCAAQN47rnnqFOnDhaLhc6dO9O1a1cAJk+ezNGjRxk+fDgAzzzzDIcOHbLPCfXBBx/c3DsiIiKSn86fd5y8Yfduc4CPM56eZquRo+QNFSvme/IGmw2OH3ccHGUunz9/4/P4+l4JiK4OjDJfV65s3mpuuLvDmDHQY9v12zLflgkTlERCRPLGTXXhKw5y00yX3wpT86WIOKe6Kjl28eKVSWWvDZJOnHB+nMViJm+4thUpLMyMLnI4tvhmpKbeuHtddhnMM5Ut67jVKHO5XLn8ifWS05IpOfL6LnwhIWbw1KlT3l9TRG5NYfq7mu9d+ERERG57GRlw8OD1AdLu3Wa0kd3vk8HBjjPc1awJlyerz2vnzjnvXhcXZ079dCOZ3escBUbVqpnByrXd61xh4f8yiF66nn/+sz4tWnio5UlE8pQCKBEREWdsNjh61HGQtH+/GUQ5U6qU83FJ/v55XsyEhOy712XXOzCTr6/zcUfVqpnBUz42gt0Si8VCnfLmGOpmTcF2KZ6HH66n4ElE8lwh/RgUEREpIIZhzonkKHnDnj2QkuL8WF/fK8HRtUFSHvZVu3TJbNRy1oJ0+LCZie5GypVz3rWuWjWz+50L5sHNE36efmx7wRwElZ6TN0NE5CYpgBIRkdtDUpLz5A1nzzo/zsPD7FrnKHlD5cq3PKmsYdy4e11Cwo3P4+7uvHtd1armo4CzlouIFEsKoEREpPhITTW71jkKkm40yKdqVcfJG6pXz31auKvYbOalnXWtO3QILly48Xn8/LJPzlCpUuHtXiciUpzoo1ZERIoWq9WMPhwFSYcOmRGLM+XLO07ecMcdZoRyEy5dMovjLDg6ciTn3euuTel99XJR7l5XEFLSU7hvyn0ArH1qrYtLIyLFmQIoEREpfAzD7LfmKHnDvn3Z59P293eevKFUqVwX49y5rJPBXhsoHT9+4/O4u5sTuTprQapa9abjN7nMMAy2n9xuXxYRyS8KoERExHXOnnWcvGH3bnPMkjPe3uakso6CpKCgHDfVWK037l6XXTEyZXavc9aCVLGiuteJiBQX+jgXEZH8lZICe/c6DpJOnXJ+nJsb1KjhOHlDSAg5yU998eKNu9dll4k8U/ny2af3LlNG3etERG4XCqBEROTWpafDgQOOxyUdOZL9sZUrOw6SatYELy+nhxmG2YCVXfe6EyduXHQPD7N73bUpva/uXufrm8v3Q0REii0FUCIikjM2mxkMOQqSDhww+8M5U6aM4+QNtWpByZIOD7Fa4ejh7LvXJSffuNglSmSfnKFSpRw1ZomIiAAKoERE5GqGASdPOk7esHevmXLOGT8/58kbypa9bveUlBt3r8suJstUoYLjSWEzX5cure51IiKSdxRAiYjcjs6fNwMkRwkczp93fpynp5ny+9ogKSzMzJRwOVIxDDhz5nIwtDprYJQZKJ08eeNieniYw52czX0UEqLudWKyWCxUC6xmXxYRyS8KoEREiqtLl8yU346SN2SXe9tiMaMTR+OSqlUDDw8yMuDo0atajdZmbUGKi8tZ97qSJR0HRpmvK1ZU9zrJGT9PPw6+chCA9JxMvCUicpMUQImIFGUZGWbE4mhcUlyc2RTkTHDw9V3tLk8qm2Lzydqlbg3ETb/SghQfn7PudUFBzjPXVatmTsukxgIRESlKFECJiBR2hmE29zgKkvbvNzPgORMYeF0rkhEaxpmyoRw8E5A1SPrjynJ22cUzZXavc9aCFBICPj559zaIiIgUBgqgRERywmrFsmoVlVevxlKiBLRokfd9y06fdhwk7dljZlxwxsfnSutRaCjWO8I4WTqMg15h7D1XjkNxFjNQWguHvjeDpOxOl8nfP/u5j4KD1b1OCo+L6Rdp9nUzAFb0WOHi0ohIcaYASkTkRubOhf798ThyhEYA48aZEwd9/DF06pS7cyUlOU/ecOaM8+Pc3c15kcLCSK8eyqmyYRz2CWMPoey4UIWDcW5mkBSb8+51wcHOu9ZVrarudVK02Awb64+uty+LiOQXBVAiItmZOxe6dMEwDK6OJYz4eCxdusDs2dcHUWlpZtc6R8kbjh7N9nJGSAjp1UM5VyGMYyXDOOAZxra0UDaeq8GBI54c+gNO/3TjYnt6Ou9eV7WquteJiIjcLAVQIiLOWK3Qv/91wROAxTAwsGB5/nlzwqK9e68ESQcPmpPOOjtt2fIkVQzlZOkw4rzD2E0Ym5JD+eNULXYf8ePi4RsXLSAg++QMwcHg5nZLdy8iIiIOKIByMWualb8nrubCr1v5e29J6r/cHHcvDSqQ24TNZmaRs1qzfy6IfRytuzybq7NebBYuTzrbv/912zJ8S3K2XBhHS4axzz2MramhxJ4PY+3JUM6cLg2ns39rgoMdTwp7dfc6ERERKXgKoFzojzfmUnVcfxpaj9AQYDEcHVSFuAEf02R0LsdVSOFksxVsEODqgCO3+2SXYrsI2RPYiLXeLfgrKYyNKWaL0vGLQXDYcejl5XWle52jFqSQEPD2LuCbEBERkRwpNAHUpEmTGDNmDAkJCdSrV4+JEydy//33O91/1qxZDBkyhIMHDxIaGsqHH35I69atC7DEt+aPN+Zy/5guQNYvkMHWeILHdOEPZheNIOrqAKGIfYk3MjIgw4qRnmFfJiMDw37cta8zsFx9HuuV81kuv7ZcXmd/LiYBgitkWDyw4oHV4o7V4oEV89mGe5ZtGVzedvk54/K+GXiQYVy1fPnZariTftVzhnF52+XndOPKtkrE8/+YfsOy9jk/hlU0t78ODIS62XSvCwpS9zoREZGiqlAEUDNnzmTAgAFMnjyZxo0bM2HCBCIjI9m1axcVKlS4bv+1a9cSFRXFyJEjadu2LTNmzKBDhw789ddf3HPPPS64g9yxplmpOq4/YHDtdyg3DAyg9tjexB4+aHYRysgAmxVLRsblL+nWrF/UbeZz5nqL7fLy1c82K26Z2zKXDQfPtozLDytuhrmvu83cZn82Lu9jWHGj6AYIlmueC1o6HvYv/46es9tW3PcxcLvy24IL/4u5YaU5q6hMvMP/6zYsHKEKNXs25fWuV4KkwEAXFFZEKOdXztVFEJHbgMUwXP8TeePGjbnvvvv49NNPAbDZbISEhPDyyy/z5ptvXrd/9+7dSU5OZtGiRfZ1TZo0oX79+kyePDlH10xMTCQwMJDz588TEBCQNzeSQ5smRFP/1RYFek1XuFGAUFi/vDvbZsUDw+1ya4jFXLa5eWCzmM+ZrzOfrRYPcM+6LvO1xd0Ni8VshXD2yO/tBXGNol7GtWvhmw5zmU0XgCxBlO1y6N2F2fRb2YnmzV1Ry0TEkfT0dBYvXkzr1q3x9PR0dXFExInCVFdzExu4vAUqLS2NDRs2MHjwYPs6Nzc3IiIiiImJcXhMTEwMAwYMyLIuMjKS+fPnO71Oamoqqamp9teJiYmA+Q+Xnp5+C3eQexd2H8nRfht9wzld6g5smV/ML3dlsrmby1c/c82Xd8PdXDbc3S8/e1z/7O4B7m72dXhc+YKfuWy4e2DxyPqM++V9Pa4sWzwyn83zuXm4Of1ymtsvtO5u4OlwH8PpufL6S7XFkpfz4VgvP6Swi4yEFyt3pGv8LCbwCiFcqbtHqMKrjCe2SkeaNEmngD9GRCQbmX/XC/rvu4jkTmGqq7kpg8sDqFOnTmG1WgkKCsqyPigoiJ07dzo8JiEhweH+CQkJTq8zcuRI3n333evWL126FD8/v5so+c1LTD1L0xzstzUqioDHqzrd7nb5+dZz9mV+oU+75TPdCpst28zPIi7x5JMV+fDDTsynPU35jYoc4xgVWcND2HBnUI9YfvnlmKuLKSIOLFu2zNVFEJEcKAx1NSUlJcf7ujyAKiiDBw/O0mqVmJhISEgIjz32WIF34bNGWDn69WiCbc7HVRxzr0LXT/oopbmIi7VuDQ0aWBkwwJ1V8c3t66tUMfjoIysdO/4D+IfLyicipovpF2k3sx0AczvN5bfo33j00Udd3i1IRJxLT09n2bJlhaKuZvZOywmXB1DlypXD3d2d48ePZ1l//PhxgoODHR4THBycq/0BvL298XaQF9jT07PA/8E8PT2Je+1jgsd0wYbF4biKwwMmULmET4GWS0Qc69YNOneGlSsz+PnnTfzzn/Vp0cIDd3eXf4SKyGVpRhqr41YD4O5h/vjoir/xIpJ7haGu5ub6bjfeJX95eXnRsGFDVqxYYV9ns9lYsWIF4eHhDo8JDw/Psj+YTX/O9i+MmozuxJ8DZ5PgXjnL+mPuVfhzYBFJYS5yG3F3h4cfNmjWLJ6HHzZwV+OwiIjIbalQ/Hw6YMAAevXqRaNGjbj//vuZMGECycnJPP300wD07NmTypUrM3LkSAD69+/Pww8/zEcffUSbNm344YcfWL9+PV988YUrbyPXmozuhPW99myYGM2OX//grpZNqP9ycyqr256IiIiISKFUKAKo7t27c/LkSYYOHUpCQgL169dnyZIl9kQRcXFxuLldaSx74IEHmDFjBv/5z3946623CA0NZf78+UViDqhruXu5U7dfM47USqJu62a4eyp4EhEREREprApFAAXw0ksv8dJLLzncFh0dfd26rl270rVr13wulYiIiIiIyBUuHwMlIiIiIiJSVBSaFigRERGRW+HnWbDzOorI7UkBlIiIiBR5JbxKkPxWMmDOLSMikl/UhU9ERERERCSHFECJiIiIiIjkkAIoERERKfIuZVyizYw2tJnRhksZl1xdHBEpxjQGSkRERIo8q83K4j2L7csiIvlFLVAiIiIiIiI5pABKREREREQkhxRAiYiIiIiI5JACKBERERERkRxSACUiIiIiIpJDt20WPsMwAEhMTHRxScwZ01NSUkhMTMTT09PVxRERJ1RXRQqv5LRkuJy9PDExUXVVpAgoTH9XM2OCzBghOxYjJ3sVQ0eOHCEkJMTVxRARERERkULi8OHDVKlSJdt9btsAymazcfToUfz9/bFYLC4tS2JiIiEhIRw+fJiAgACXlkVEnFNdFSkaVFdFiobCVFcNw+DChQtUqlQJN7fsRzndtl343NzcbhhdFrSAgACX/+cRkRtTXRUpGlRXRYqGwlJXAwMDc7SfkkiIiIiIiIjkkAIoERERERGRHFIAVQh4e3szbNgwvL29XV0UEcmG6qpI0aC6KlI0FNW6etsmkRAREREREckttUCJiIiIiIjkkAIoERERERGRHFIAJSIiIiIikkMKoERERERERHJIAVQeWb16Ne3ataNSpUpYLBbmz5+fZbthGAwdOpSKFSvi6+tLREQEe/bssW8/ePAgvXv3pkaNGvj6+nLHHXcwbNgw0tLSHF5v7969+Pv7U6pUqXy8K5GirzDVzVmzZlG7dm18fHy49957Wbx4cV7eqkiRUZTq5Y3KIlKcFLe6eebMGXr06EFAQAClSpWid+/eJCUl3dybcxUFUHkkOTmZevXqMWnSJIfbR48ezSeffMLkyZNZt24dJUqUIDIykkuXLgGwc+dObDYbn3/+Odu2bWP8+PFMnjyZt95667pzpaenExUVRdOmTfP1nkSKg8JSN9euXUtUVBS9e/dm48aNdOjQgQ4dOrB169a8vWGRIqAo1csblUWkOCludbNHjx5s27aNZcuWsWjRIlavXs2///3vW32bwJA8Bxjz5s2zv7bZbEZwcLAxZswY+7pz584Z3t7exvfff+/0PKNHjzZq1Khx3fo33njDePLJJ41p06YZgYGBeVl0kWLNlXWzW7duRps2bbKsa9y4sfHcc8/d3M2IFBOFuV7ebFlEioOiXje3b99uAEZsbKx9n59//tmwWCxGfHz8jd+AbKgFqgAcOHCAhIQEIiIi7OsCAwNp3LgxMTExTo87f/48ZcqUybLu119/ZdasWU5/GRCRnCvIuhkTE5PlOgCRkZHZXkfkdlSY6uXNlkWkOCpqdTMmJoZSpUrRqFEj+z4RERG4ubmxbt26HN61YwqgCkBCQgIAQUFBWdYHBQXZt11r7969TJw4keeee86+7vTp0zz11FN8/fXXBAQE5F+BRW4TBVk3ExIScnUdkdtVYaqXN1MWkeKqqNXNhIQEKlSokGW7h4cHZcqUueX6qwCqEIqPj6dVq1Z07dqVPn362Nf36dOHJ554gmbNmrmwdCK3L9VNkcJH9VKkcCrOdVMBVAEIDg4G4Pjx41nWHz9+3L4t09GjR2nRogUPPPAAX3zxRZZtv/76K2PHjsXDwwMPDw969+7N+fPn8fDw4KuvvsrfmxAphgqybgYHB+foOiK3u8JUL3NTFpHirqjVzeDgYE6cOJFle0ZGBmfOnLnl+qsAqgDUqFGD4OBgVqxYYV+XmJjIunXrCA8Pt6+Lj4+nefPmNGzYkGnTpuHmlvWfJyYmhk2bNtkfw4cPx9/fn02bNtGxY8cCux+R4qIg62Z4eHiW6wAsW7Ysy3VEpHDVy5yWReR2UNTqZnh4OOfOnWPDhg32fX799VdsNhuNGze+tTfjllJQiN2FCxeMjRs3Ghs3bjQAY9y4ccbGjRuNQ4cOGYZhGKNGjTJKlSplLFiwwPj777+N9u3bGzVq1DAuXrxoGIZhHDlyxKhVq5bxyCOPGEeOHDGOHTtmfzijLHwiN1ZY6ubvv/9ueHh4GGPHjjV27NhhDBs2zPD09DS2bNmSb/cuUlgVpXp5o7KIFCfFrW62atXK+Mc//mGsW7fO+O2334zQ0FAjKirqlt8nBVB5ZOXKlQZw3aNXr16GYZjpFocMGWIEBQUZ3t7exiOPPGLs2rXLfvy0adMcHp9djKsASuTGClPd/PHHH42wsDDDy8vLuPvuu42ffvopr29XpEgoSvXyRmURKU6KW908ffq0ERUVZZQsWdIICAgwnn76aePChQs3/wZdZjEMw7i1NiwREREREZHbg8ZAiYiIiIiI5JACKBERERERkRxSACUiIiIiIpJDCqBERERERERySAGUiIiIiIhIDimAEhERERERySEFUCIiIiIiIjmkAEpERERERCSHFECJiMgts1gszJ8/39XFcLmDBw9isVjYtGlTnp2zevXqTJgwIc/OJyIit0YBlIiIZOvkyZP07duXqlWr4u3tTXBwMJGRkfz+++/2fY4dO8Y///nPfC9L8+bNb7jP119/jcVisT9KlixJw4YNmTt3bq6uFR0djcVi4dy5czdX2DwSGxvLv//9b5eWQURErvBwdQFERKRw69y5M2lpaXzzzTfUrFmT48ePs2LFCk6fPm3fJzg4ON+uv2jRIipVqkSDBg3s63744QcaNGhAWFiYw2MCAgLYtWsXABcuXGDatGl069aNbdu2ceedd+ZbWfND+fLlXV0EERG5ilqgRETEqXPnzrFmzRo+/PBDWrRoQbVq1bj//vsZPHgwjz/+uH2/q7vwvfPOO1lagDIfX3/9NQA2m42RI0dSo0YNfH19qVevHrNnz3Zahpo1azJ48GCGDRvGuXPn6NatGytXrqRcuXJOj7FYLAQHBxMcHExoaCjvvfcebm5u/P333/Z9vv32Wxo1aoS/vz/BwcE88cQTnDhxAjC74rVo0QKA0qVLY7FYeOqpp+zlHz16NLVq1cLb25uqVavy/vvvZ7n+/v37adGiBX5+ftSrV4+YmBinZTUMg3feecfewlepUiX69etn3351F75rW9cyH++88459/y+//JK77roLHx8fateuzWeffeb02iIiknsKoERExKmSJUtSsmRJ5s+fT2pqao6Oef311zl27Jj9MXbsWPz8/GjUqBEAI0eO5L///S+TJ09m27ZtvPrqqzz55JOsWrXK4fnq1KnDL7/8wp49e9i8eTMRERF8/vnnlClTJkflsVqtfPPNNwBZWrHS09MZMWIEmzdvZv78+Rw8eNAeJIWEhDBnzhwAdu3axbFjx/j4448BGDx4MKNGjWLIkCFs376dGTNmEBQUlOWab7/9Nq+//jqbNm0iLCyMqKgoMjIyHJZvzpw5jB8/ns8//5w9e/Ywf/587r33Xof7du/ePct7+/333+Ph4cGDDz4IwPTp0xk6dCjvv/8+O3bs4IMPPmDIkCH2+xcRkTxgiIiIZGP27NlG6dKlDR8fH+OBBx4wBg8ebGzevDnLPoAxb968646NiYkxfHx8jJkzZxqGYRiXLl0y/Pz8jLVr12bZr3fv3kZUVJTD6+/cudNo1aqVMWTIEKNevXpG165djb59+xpnzpxxuP+0adMMwChRooRRokQJw83NzfD29jamTZuW7X3GxsYagHHhwgXDMAxj5cqVBmCcPXvWvk9iYqLh7e1tTJkyxeE5Dhw4YADGl19+aV+3bds2AzB27Njh8JiPPvrICAsLM9LS0hxur1atmjF+/Pjr1u/du9coU6aMMXr0aPu6O+64w5gxY0aW/UaMGGGEh4c7PLeIiOSeWqBERCRbnTt35ujRoyxcuJBWrVoRHR1NgwYN7F3ynImLi6NDhw68/vrrdOvWDYC9e/eSkpLCo48+am/dKlmyJP/973/Zt2+fw/Ps3r2b999/n+HDh1OqVCl+/PFHmjZtysmTJ51e29/fn02bNrFp0yY2btzIBx98wPPPP8///vc/+z4bNmygXbt2VK1aFX9/fx5++GF7uZ3ZsWMHqampPPLII9nee926de3LFStWBLB3D7xW165duXjxIjVr1qRPnz7MmzfPaWtVpvPnz9O2bVvatGnDwIEDAUhOTmbfvn307t07y3v73nvvOX1vRUQk95REQkREbsjHx4dHH32URx99lCFDhvDss88ybNgwe5e3ayUnJ/P4448THh7O8OHD7euTkpIA+Omnn6hcuXKWY7y9vR2eq127dteti4qKyra8bm5u1KpVy/66bt26LF26lA8//JB27dqRnJxMZGQkkZGRTJ8+nfLlyxMXF0dkZCRpaWlOz+vr65vtdTN5enraly0WC2COnXIkJCSEXbt2sXz5cpYtW8YLL7zAmDFjWLVqVZbzZLJarXTv3p2AgAC++OIL+/rM93bKlCk0btw4yzHu7u45KreIiNyYAigREcm1OnXqOJ33yTAMnnzySWw2G99++609gMg8ztvbm7i4OHuLT25ER0ffZInNIOLixYsA7Ny5k9OnTzNq1ChCQkIAWL9+fZb9vby8ADNgyRQaGoqvry8rVqzg2WefvemyXMvX15d27drRrl07XnzxRWrXrs2WLVuyjNnK9Oqrr7JlyxbWr1+Pj4+PfX1QUBCVKlVi//799OjRI8/KJiIiWSmAEhERp06fPk3Xrl155plnqFu3Lv7+/qxfv57Ro0fTvn17h8e88847LF++nKVLl5KUlGRvGQkMDMTf35/XX3+dV199FZvNxkMPPcT58+f5/fffCQgIoFevXnlSbsMwSEhIAODixYssW7aMX375haFDhwJQtWpVvLy8mDhxIs8//zxbt25lxIgRWc5RrVo1LBYLixYtonXr1vj6+lKyZEkGDRrEG2+8gZeXFw8++CAnT55k27Zt9O7d+6bK+vXXX2O1WmncuDF+fn589913+Pr6Uq1atev2nTZtGp999hnz5s3DYrHY7zGzu967775Lv379CAwMpFWrVqSmprJ+/XrOnj3LgAEDbqp8IiJyDVcPwhIRkcLr0qVLxptvvmk0aNDACAwMNPz8/Iw777zT+M9//mOkpKTY9+OqJBIPP/ywAVz3yEziYLPZjAkTJhh33nmn4enpaZQvX96IjIw0Vq1alSdlzkwikfnw9vY2wsLCjPfff9/IyMiw7zdjxgyjevXqhre3txEeHm4sXLjQAIyNGzfa9xk+fLgRHBxsWCwWo1evXoZhGIbVajXee+89o1q1aoanp6dRtWpV44MPPjAM40oSiavPcfbsWQMwVq5c6bC88+bNMxo3bmwEBAQYJUqUMJo0aWIsX77cvv3qJBK9evVy+N4OGzbMvv/06dON+vXrG15eXkbp0qWNZs2aGXPnzr2l91RERK6wGIZhFHjUJiIiIiIiUgQpC5+IiIiIiEgOKYASERERERHJIQVQIiIiIiIiOaQASkREREREJIcUQImIiIiIiOSQAigREREREZEcUgAlIiIiIiKSQwqgREREREREckgBlIiIiIiISA4pgBIREREREckhBVAiIiIiIiI59P8BhDKCMNuk5JcAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "## Evaluation Time between BitLT (Ring PPML) and Private compare (Falcon) vs Size * Batch size\n", + "# Data extraction\n", + "bitLt_x = [entry[\"Size\"] * entry[\"Batch size\"] for entry in bitLt_data]\n", + "bitLt_y = [entry[\"Eval Time\"] * 1000 for entry in bitLt_data] # Convert to milliseconds\n", + "\n", + "pc_x = [entry[\"Size\"] * entry[\"Batch size\"] for entry in pc_data]\n", + "pc_y = [entry[\"Eval Time\"] * 1000 for entry in pc_data] # Convert to milliseconds\n", + "\n", + "# Calculate log2 of the x values\n", + "bitLt_x_log2 = [np.log2(x) for x in bitLt_x]\n", + "pc_x_log2 = [np.log2(x) for x in pc_x]\n", + "\n", + "# Value for the vertical line\n", + "vertical_line_x = 28 * 28 * 128\n", + "vertical_line_log2 = np.log2(vertical_line_x)\n", + "\n", + "# Function to interpolate the y-value at the vertical line\n", + "def interpolate_y(x_vals, y_vals, target_x):\n", + " return np.interp(target_x, x_vals, y_vals)\n", + "\n", + "# Calculate the intersection points\n", + "bitLt_y_intersection = interpolate_y(bitLt_x_log2, bitLt_y, vertical_line_log2)\n", + "pc_y_intersection = interpolate_y(pc_x_log2, pc_y, vertical_line_log2)\n", + "\n", + "# Plotting\n", + "plt.figure(figsize=(10, 6))\n", + "plt.plot(bitLt_x_log2, bitLt_y, marker='o', linestyle='-', color='b', label='BitLT')\n", + "plt.plot(pc_x_log2, pc_y, marker='o', linestyle='-', color='r', label='Private Compare')\n", + "\n", + "# Add vertical line\n", + "plt.axvline(vertical_line_log2, color='g', linestyle='--', label=f'x = {vertical_line_x}')\n", + "\n", + "# Add horizontal lines at the intersection points\n", + "plt.axhline(bitLt_y_intersection, color='gray', linestyle='--', linewidth=0.5)\n", + "plt.axhline(pc_y_intersection, color='gray', linestyle='--', linewidth=0.5)\n", + "\n", + "# Annotate the intersection points on the left side\n", + "plt.text(plt.xlim()[0], bitLt_y_intersection, f'{bitLt_y_intersection:.2f}', color='black', fontsize=8, verticalalignment='center', horizontalalignment='right')\n", + "plt.text(plt.xlim()[0], pc_y_intersection, f'{pc_y_intersection:.2f}', color='black', fontsize=8, verticalalignment='center', horizontalalignment='right')\n", + "\n", + "plt.xlabel('Size * Batch size')\n", + "plt.ylabel('Eval Time (ms)')\n", + "plt.title('Evaluation Time between BitLT (Ring PPML) and Private compare (Falcon) vs Size * Batch size')\n", + "plt.legend()\n", + "plt.grid(True)\n", + "\n", + "# Set the x-ticks to the specific log2 values\n", + "all_x_log2 = sorted(set(bitLt_x_log2 + pc_x_log2))\n", + "plt.xticks(all_x_log2, [f\"{2**x:.0f}\" for x in all_x_log2])\n", + "\n", + "plt.show()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAIjCAYAAADvBuGTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAADCH0lEQVR4nOzdd3hTZfsH8G+SpuluKZ3Q0s0uQ5YgG6EvMh1slCEqL1sEEVSWDEFBUBmKAg5AhoCiTJEN/lSWBcroooyWtlC626zn90dtXkI3JJw0/X6ui4vm5DnnuTOeJPc5z7mPTAghQERERERERI9FLnUARERERERE1oDJFRERERERkQkwuSIiIiIiIjIBJldEREREREQmwOSKiIiIiIjIBJhcERERERERmQCTKyIiIiIiIhNgckVERERERGQCTK6IiIiIiIhMgMkVmY1MJsPs2bMl6fvw4cOQyWQ4fPiwJP2Xh5TPjyWrDK+dJevYsSMaNmxotu3Hx8dDJpNh/fr1ZuvjYevXr4dMJsPff//9xPoky6bX69GwYUPMnz/faPlff/2FNm3awNHRETKZDOfOnSv3NgvfZ/Hx8aYNtoLu3r0LR0dH7N69W9I4itOxY0d07NhR6jBMavHixahbty70ev0jrW8p7xtLotFo4O/vj5UrV0odiiSYXFm5wkFf0r8//vhD6hAfy8qVK5/oj7zSlPVcF/4LDAyUOlRJ6PV6fPvtt2jVqhXc3d3h7OyM2rVr45VXXrGY9+Hhw4cxfPjwcrfv2LGj0Wtra2uLoKAgvP7667hx48YjxXDp0iXMnj2bX9Rmtnv3bqveuVGYBBf+UyqV8PDwQJs2bTBjxgwkJCSYpd/Zs2dDJpMhNTXVLNsvtGnTJty4cQPjxo0zLNNoNOjXrx/u3buHTz75BN999x0CAgLMGoc5VK9eHaNGjcL7778vdSiPLDAwEDKZDM8++2yx969Zs8bw3nx4p8nx48fRvXt31KxZE3Z2dqhVqxZ69eqFjRs3GrV7+LvV0dER9evXx7x585CTk1OuODMyMrBo0SJMmzYNcvn/fhI/vG0XFxd06NABv/76awWfCdN5+PvG3d0dLVq0wNq1a40Sw+HDhxeJvXHjxliyZAny8/MN7QrHqlwuL/b7KiMjA/b29pDJZEbjrPCz5eOPPy4xVqVSicmTJ2P+/PnIy8sz0TNQedhIHQA9GXPnzkVQUFCR5aGhoRJEYzorV66Eh4dHkR/E7du3R25uLmxtbZ9YLO3bt8d3331ntGzUqFFo2bIlXn/9dcMyJycnAEBubi5sbKrOEJwwYQJWrFiBPn36YMiQIbCxscGVK1ewZ88eBAcH4+mnnwbw5F+79PR0REVFGfovdP/+fVy5cgWtWrUqdX0/Pz8sXLgQAKBWq3Hp0iWsXr0a+/btQ1RUFBwcHCoUz6VLlzBnzhx07NjRIhPxgIAA5ObmQqlUSh3KY9m9ezdWrFhh1QkWAAwaNAjPPfcc9Ho90tLS8Ndff2HZsmVYvnw5vv76awwcOFDqEB/JRx99hIEDB8LV1dWwLCYmBtevX8eaNWswatQoCaN7fKNHj8ann36K33//HZ07d5Y6nEdiZ2eHQ4cOISkpCT4+Pkb3bdiwAXZ2dkV+eG/duhUDBgxAkyZNMHHiRFSrVg1xcXE4evQo1qxZg8GDBxu179q1K1555RUAQFZWFo4dO4b3338f58+fx9atW8uMce3atdBqtRg0aFCR+wq3LYTA9evXsWrVKvTq1Qt79uxBRESEod3LL7+MgQMHQqVSlfu5eVQPft+kpKTg22+/xauvvoqrV6/iww8/NLRTqVT46quvABR8l/3444+YMmUK/vrrL/zwww9G21SpVNi0aRPefvtto+Xbt29/rFhHjBiBd955Bxs3bsTIkSMfa1uVjiCrtm7dOgFA/PXXX0+8bwBi1qxZZu2jQYMGokOHDmbt43E4OjqKYcOGSR2G5JKSkoRMJhOvvfZakfv0er24c+eOBFEVOH/+vKhdu7aYMGGC2LVrlxg2bJjYtm2bCAoKEp988kmp63bo0EE0aNCgyPLPP/9cABD79++vcDxbt24VAMShQ4cqvG5pMVVm5vgcGzt2rLDmr8C4uDgBQHz00UdF7ouPjxe1a9cWtra24ty5c6VuJysrq0L9zpo1SwAQKSkpFVqvIs6cOSMAiN9++81o+ZEjRwQAsXXr1kfabuH7LC4uzgRRPr6GDRuKl19+WeowjHTo0KFc37kBAQGiS5cuwsXFRSxbtszovhs3bgi5XC5efPHFIuO6fv36okGDBiI/P7/INh/+ngAgxo4dW6TdSy+9JORyucjNzS0zzkaNGomhQ4cWWV7cti9duiQAiO7du5e5XXMo7rM9Oztb+Pn5CUdHR6FWq4UQQgwbNkw4OjoatdPpdKJ58+YCgLh165YQ4n9j9YUXXhBNmjQp0l/Xrl0Nr9GDz0Vpny0P69mzp2jXrl2FH2tlx2mBBI1GA3d3d4wYMaLIfRkZGbCzs8OUKVMAFOyZnzlzJpo1awZXV1c4OjqiXbt2OHToUJn9DB8+vNg98YWHph+0bt06dO7cGV5eXlCpVKhfvz5WrVpl1CYwMBAXL17EkSNHDIe/C+eCl3TeztatW9GsWTPY29vDw8MDQ4cOxa1bt4rE6eTkhFu3bqFv375wcnKCp6cnpkyZAp1OV+bjLK+Hz7kqfB6uXr2KoUOHwtXVFZ6ennj//fchhMCNGzfQp08fuLi4wMfHB0uWLCmyzfz8fMyaNQuhoaFQqVTw9/fH22+/bTQVoDjjxo2Dk5NTsVMpBg0aBB8fH8Nj//vvvxEREQEPDw/Y29sjKCiozL1ScXFxEELgmWeeKfZ58PLyMtx++LUrbbrlw3P/v//+e8Pr6+7ujoEDB5Y5Pa9Ro0aIjIyEv78/3njjDWzbtg1btmzBoUOHMGnSpFLXLUnhXtoHj0xev34dY8aMQZ06dWBvb4/q1aujX79+RtP/1q9fj379+gEAOnXqZHicD76P9+zZgw4dOsDZ2RkuLi5o0aJFkekyQMERsE6dOsHBwQE1a9bE4sWLyxX7gQMH0LZtW7i5ucHJyQl16tTBjBkzDPc/fM5V4etVnimwe/bsQbt27eDo6AhnZ2f06NEDFy9eLFdcAJCTk4M33ngD1atXh4uLC1555RWkpaUVaVdWP8OHD8eKFSsAGE//AYCnnnoKL7zwgtH2wsPDIZPJ8M8//xiWbd68GTKZDFFRUYZlt27dwsiRI+Ht7Q2VSoUGDRpg7dq1ReIr7zgtnI6zc+dONGzY0LDNvXv3lvs5K05AQADWr18PtVpt9L4oHGtHjhzBmDFj4OXlBT8/P8P9j/v6FSrpvJ2SviMetnPnTtja2qJ9+/ZG63bo0AEA0K9fP6PPh3/++QfDhw9HcHAw7Ozs4OPjg5EjR+Lu3bvlirc8Y84c3y1du3bFrl27IIQoNb579+5hypQpCA8Ph5OTE1xcXNC9e3ecP3/eqF3hWN2yZQvmz58PPz8/2NnZoUuXLoiOji6y3S+//BIhISGwt7dHy5YtcezYsXI9X4Xs7OzwwgsvFHmuNm3ahGrVqhkd/SkUExODFi1aFDtz4cHvidL4+PhAJpOVOTMkLi4O//zzT4lTFx9Wr149eHh4ICYmxmh5cedcBQYGomfPnjh+/DhatmwJOzs7BAcH49tvvy2y3X/++QcdOnSAvb09/Pz8MG/ePKxbt65c53E5ODjg6aefRnZ2NlJSUkpsJ5fLDePh4W0OHjwY586dw+XLlw3LkpKS8Pvvvxc5UlhRXbt2xfHjx3Hv3r3H2k5lU3XmJFVx6enpRebAy2QyVK9eHUqlEs8//zy2b9+OL774wuhDbefOncjPzzdMHcnIyMBXX32FQYMG4bXXXkNmZia+/vprRERE4M8//0STJk1MEu+qVavQoEED9O7dGzY2Nti1axfGjBkDvV6PsWPHAgCWLVuG8ePHw8nJCe+++y4AwNvbu8Rtrl+/HiNGjECLFi2wcOFC3LlzB8uXL8eJEydw9uxZuLm5GdrqdDpERESgVatW+Pjjj/Hbb79hyZIlCAkJwX//+1+TPMaSDBgwAPXq1cOHH36IX3/9FfPmzYO7uzu++OILdO7cGYsWLcKGDRswZcoUtGjRwvADQ6/Xo3fv3jh+/Dhef/111KtXD5GRkfjkk09w9epV7Ny5s9Q+V6xYgV9//dXw4x4o+DG7a9cuDB8+HAqFAsnJyejWrRs8PT3xzjvvwM3NDfHx8WVOHyg872Hr1q3o169fhabKFTfd8vr163jvvfeMvmznz5+P999/H/3798eoUaOQkpKCzz77DO3bty/y+j6scN554Q/sB39sl0Wn0xnGlkajQVRUlOGH84PJ5F9//YWTJ09i4MCB8PPzQ3x8PFatWoWOHTvi0qVLcHBwQPv27TFhwgR8+umnmDFjBurVqwcAhv/Xr1+PkSNHokGDBpg+fTrc3Nxw9uxZ7N271+hLMC0tDf/5z3/wwgsvoH///ti2bRumTZuG8PBwdO/evcTHcvHiRfTs2RONGjXC3LlzoVKpEB0djRMnTpS4Tr169Yq8Pvfv38fkyZONXp/vvvsOw4YNQ0REBBYtWoScnBysWrUKbdu2xdmzZ8v1o3rcuHFwc3PD7NmzceXKFaxatQrXr183/Ggsbz9vvPEGbt++jQMHDhSJvV27dti0aZPh9r1793Dx4kXI5XIcO3YMjRo1AgAcO3YMnp6ehtfmzp07ePrppw0JkaenJ/bs2YNXX30VGRkZhkS9ouP0+PHj2L59O8aMGQNnZ2d8+umnePHFF5GQkIDq1auX+ZyVpHXr1ggJCcGBAweK3DdmzBh4enpi5syZyM7OLvfz+qScPHkSDRs2NJqa+sYbb6BmzZpYsGABJkyYgBYtWhi+Dw4cOIDY2FiMGDECPj4+uHjxIr788ktcvHgRf/zxR6ljvTxjzlzfLc2aNcMnn3yCixcvllqkJjY2Fjt37kS/fv0QFBSEO3fu4IsvvkCHDh1w6dIl1KhRw6j9hx9+CLlcjilTpiA9PR2LFy/GkCFD8H//93+GNl9//TXeeOMNtGnTBpMmTUJsbCx69+4Nd3d3+Pv7l/0i/Wvw4MHo1q0bYmJiEBISAgDYuHEjXnrppWKnFgcEBODgwYO4efOmUWJfkry8PMPnb3Z2Nk6cOIFvvvkGgwcPLjO5OnnyJICCHSrlkZ6ejrS0NMPjKEt0dDReeuklvPrqqxg2bBjWrl2L4cOHo1mzZmjQoAGAgh0yhTvSpk+fDkdHR3z11VcVmmIYGxsLhUJR6nccAENS+PDnRvv27eHn54eNGzdi7ty5AAp2Hjk5OaFHjx7ljqM4zZo1gxACJ0+eRM+ePR9rW5WKtAfOyNwKpzkU90+lUhna7du3TwAQu3btMlr/ueeeE8HBwYbbWq22yOH6tLQ04e3tLUaOHGm0HA9NCxw2bJgICAgoEmPhoekH5eTkFGkXERFhFIsQJU8LPHTokNHUKrVaLby8vETDhg2Npgr88ssvAoCYOXOmUZwAxNy5c4222bRpU9GsWbMifZWmtGmBDz8/hc/D66+/blim1WqFn5+fkMlk4sMPPzQsT0tLE/b29kbb/u6774RcLhfHjh0z6mf16tUCgDhx4kSJcer1elGzZk3x4osvGi3fsmWLACCOHj0qhBBix44djzw965VXXhEARLVq1cTzzz8vPv74YxEVFVWk3cOv3cNyc3NFs2bNRI0aNURiYqIQomCak0KhEPPnzzdqGxkZKWxsbIosf9A///wj6tatK8aPH2+YFrh161YRFBRUZDrLwzp06FDs2KpXr56IjY01alvce/rUqVMCgPj2228Ny0qaFnj//n3h7OwsWrVqVWS6i16vLxLTg9vMz88XPj4+RV7fh33yySdlTukqnBKybt26Yu/X6/WiZ8+ewsnJSVy8eFEIIURmZqZwc3MrMi00KSlJuLq6Fjtd9EGFn2PNmjUzTH0RQojFixcLAOKnn36qcD8lTQssfP4vXbokhBDi559/FiqVSvTu3VsMGDDA0K5Ro0bi+eefN9x+9dVXha+vr0hNTTXa3sCBA4Wrq6vh9a/IOAUgbG1tRXR0tGHZ+fPnBQDx2WeflfqclWfqTp8+fQQAkZ6eLoT43/Pctm1bodVqDe0q8ryWZ1pgSVPLSvqOeJifn1+x7+XCz46HpwUWN/Y2bdpk9NkmRNFpgeUZc+b8bjl58qQAIDZv3lzSUyGEECIvL0/odDqjZXFxcUKlUhn1Vfj81KtXz+h7fPny5QKAiIyMNHpMTZo0MWr35ZdfCgDlnhbYo0cPodVqhY+Pj/jggw+EEP+bWnfkyJFip/t+/fXXhvd9p06dxPvvvy+OHTtW5PEJIUr8bdO3b1+Rl5dXZozvvfeeACAyMzOL3farr74qUlJSRHJysvj777/Ff/7zn2LHVHHTSQMCAoq8v5KTk4VKpRJvvfWWYdn48eOFTCYTZ8+eNSy7e/eucHd3L7LNDh06iLp164qUlBSRkpIioqKixIQJEwQA0atXL0O7wmmBhe2io6PFggULhEwmE40aNTK0e3CsTpkyRYSGhhrua9GihRgxYoThuXjUaYG3b98WAMSiRYvKbGtNOC2wilixYgUOHDhg9G/Pnj2G+zt37gwPDw9s3rzZsCwtLQ0HDhzAgAEDDMsUCoXhyJZer8e9e/eg1WrRvHlznDlzxmTx2tvbG/4uPOrWoUMHxMbGIj09vcLb+/vvv5GcnIwxY8bAzs7OsLxHjx6oW7dusRWARo8ebXS7Xbt2iI2NrXDfFfXgidgKhQLNmzeHEAKvvvqqYbmbmxvq1KljFM/WrVtRr1491K1bF6mpqYZ/hSdDlzZ1UyaToV+/fti9ezeysrIMyzdv3oyaNWuibdu2hn4B4JdffoFGo6nQ41q3bh0+//xzBAUFYceOHZgyZQrq1auHLl26FJk+U5oxY8YgMjISP/74o2H63fbt26HX69G/f3+jx+7j44OwsLBSH3utWrWwbt06fPrpp4ZiIy+99BLOnDlTpMhFcQIDA43G1LJly5Ceno7u3bsbTdN48D2t0Whw9+5dhIaGws3NrVxj58CBA8jMzMQ777xj9B4GUGTPu5OTE4YOHWq4bWtri5YtW5b5/i18fX/66adHLkv8wQcf4JdffsH69etRv359Q+z379/HoEGDjF4fhUKBVq1alWtaMQC8/vrrRnu7//vf/8LGxsZQstoU/bRr1w4AcPToUQAFR6hatGiBrl27GqZF3b9/HxcuXDC0FULgxx9/RK9evSCEMOo7IiIC6enphte4ouP02WefNdpT3qhRI7i4uJjks6jw/Z6ZmWm0/LXXXoNCoTDcNtXrZyp3795FtWrVyt3+wbFXeKSjcGyXNvbKM+bM+d1S+BjLqryoUqkMle50Oh3u3r1rmNJb3OMbMWKE0QyVwvdxYQyFj2n06NFG7YYPH25UQKQ8FAoF+vfvbzgavGHDBvj7+xv6fNjIkSOxd+9edOzYEcePH8cHH3yAdu3aISwszHCk6UF9+vQxfP7+9NNPmD59uuGooihjOuXdu3dhY2NjGAcP+/rrr+Hp6QkvLy80b94cBw8exNtvv43JkyeX67HXr1/f6HF6enoW+d7eu3cvWrdubTTrx93dHUOGDCl2m5cvX4anp6fhqPlnn32GHj16FJl+nJ2dbWgXGhqKGTNmoHXr1tixY0ex2x08eDCio6Px119/Gf5/3CmBQPnfw9aG0wLLcPToUXz00Uc4ffo0EhMTsWPHDvTt27dC2xBCYMmSJfjyyy9x/fp1eHh4YMyYMYapbE9Cy5Yt0bx58xLvt7GxwYsvvoiNGzciPz8fKpUK27dvh0ajMUquAOCbb77BkiVLcPnyZaMf2MVVI3xUJ06cwKxZs3Dq1Kki5wGlp6dX+AP++vXrAIA6deoUua9u3bo4fvy40TI7Ozt4enoaLatWrVqx53eYWq1atYxuu7q6ws7ODh4eHkWWP3jOwLVr1xAVFVUk7kLJycml9jtgwAAsW7YMP//8MwYPHoysrCzs3r0bb7zxhuGHRIcOHfDiiy9izpw5+OSTT9CxY0f07dsXgwcPLnMag1wux9ixYzF27FjcvXsXJ06cwOrVq7Fnzx4MHDiwXPP5v/jiC6xbtw5ffPGFUeJz7do1CCEQFhZW7HqlVbZzdXUtNolyc3Mrs1IgADg6OhrN2f/Pf/6Dtm3bonnz5vjwww8N58bl5uZi4cKFWLduHW7dumX0xV+eHQaFUzrKcw0rPz+/IglXtWrVjM4ZKs6AAQPw1VdfYdSoUXjnnXfQpUsXvPDCC3jppZeMyhSXZO/evZgzZw6mT5+OF1980bD82rVrAFBi1TMXF5cytw2gyOvr5OQEX19fwzkEpujH29sbYWFhOHbsGN544w0cO3YMnTp1Qvv27TF+/HjExsYiKioKer3e8MMpJSUF9+/fx5dffokvv/yy2O0Wjr+KjtOHPw8A030WFe5IcXZ2Nlr+8Ge5qV4/Uyrrh/OD7t27hzlz5uCHH34o8vyWNvbKM+bM+d1S+BjLmqKs1+uxfPlyrFy5EnFxcUbnbxU3dfTh91ThD+DCGAof08PjTalUIjg4uNRYijN48GB8+umnOH/+PDZu3IiBAweW+pgiIiIQERGBnJwcnD59Gps3b8bq1avRs2dPXL582Wi6sZ+fn9Hnb+/evVG9enVMmTIFv/zyC3r16lXheAv16dMH48aNg1qtxl9//YUFCxYgJyenXJ+FQPnG7vXr19G6desi7Uqq5BwYGGgoY29nZ4ewsLBiz0Wzs7PDrl27ABQk30FBQaVOs2zatCnq1q2LjRs3ws3NDT4+PiapUlne97C1YXJVhuzsbDRu3BgjR44scpJzeU2cOBH79+/Hxx9/jPDwcNy7d88iT+4bOHAgvvjiC+zZswd9+/bFli1bULduXTRu3NjQ5vvvv8fw4cPRt29fTJ06FV5eXlAoFFi4cGGRkzwfVtLgevhE3piYGHTp0gV169bF0qVL4e/vD1tbW+zevRuffPLJI+9Rr4gH99o+acX1XVI8D/7A0Ov1CA8Px9KlS4ttW9Y8+aeffhqBgYHYsmULBg8ejF27diE3N9couZbJZNi2bRv++OMP7Nq1C/v27cPIkSOxZMkS/PHHHyXuAXxY9erV0bt3b/Tu3RsdO3bEkSNHcP369VKvSfPnn39i4sSJGDVqlFFp+8LHLpPJsGfPnmKfq/LGVdKJ9hVVWPCl8OgHAIwfPx7r1q3DpEmT0Lp1a7i6ukImk2HgwIEmf0+X5/1SHHt7exw9ehSHDh3Cr7/+ir1792Lz5s3o3Lkz9u/fX+q4iIuLw5AhQ9C1a1fMmzfP6L7Cx/fdd98VKckMwGSXJDBVP23btsXBgweRm5uL06dPY+bMmWjYsCHc3Nxw7NgxREVFwcnJCU2bNjXqd+jQoRg2bFix2yw8V6ui4/RRX8vyuHDhAry8vIokRw8e6QFM//rJZLJi4y9vwaDq1atXKLns378/Tp48ialTp6JJkyZwcnKCXq/Hf/7znyfyffKginy3FD7Gh3esPWzBggV4//33MXLkSHzwwQdwd3eHXC7HpEmTin185nxPFadVq1YICQnBpEmTEBcXV+4jIg4ODmjXrh3atWsHDw8PzJkzB3v27ClxjBXq0qULgIKd46UlV9WrV4dWq0VmZmaRHQyAceL23HPPwcPDA+PGjUOnTp3K9XvQHM/zwzvzSuu7vIU6Cg0ePBirVq2Cs7MzBgwYUO4ksjTlfQ9bGyZXZejevXupJ4Dn5+fj3XffxaZNm3D//n00bNgQixYtMvxAi4qKwqpVq3DhwgXDni1THuExpfbt28PX1xebN29G27Zt8fvvvxc5urZt2zYEBwdj+/btRsnSrFmzytx+tWrVcP/+/SLLC/eSFdq1axfy8/Px888/G+35KW7qSXn3hhT+aL9y5UqRvTFXrlyplBeafFhISAjOnz+PLl26PPJeov79+2P58uXIyMjA5s2bERgYWOxRnaeffhpPP/005s+fj40bN2LIkCH44YcfHunaMs2bN8eRI0eQmJhY4uuQkpKCl156CU2aNDFUeXtQSEgIhBAICgpC7dq1KxyDOeh0OqMpltu2bcOwYcOMqjzm5eUVGRMlvXaFU8MuXLhg1uvTyeVydOnSBV26dMHSpUuxYMECvPvuuzh06FCJX9a5ubl44YUX4Obmhk2bNhX5Ui6M3cvLq8Jf+A+6du0aOnXqZLidlZWFxMREPPfccxXup7Qx0q5dO6xbtw4//PADdDod2rRpA7lcjrZt2xqSqzZt2hh+PHl6esLZ2Rk6na7Mfk0xTk3h1KlTiImJMZo+WhJTvX6FqlWrVuw0uIe/C0pSt25dxMXFlattWloaDh48iDlz5mDmzJmG5YVH40pTnjFnzu+WwsdYWDSlJNu2bUOnTp3w9ddfGy2/f//+I/2oLYz52rVrRo9Jo9EgLi7OaIdreQ0aNAjz5s1DvXr1HqnwVeHMm8TExDLbarVaADD6/C1O3bp1ARQ8z4U7P0rzxhtv4JNPPsF7772H559/3iTjNyAgoNhKjcUtM7fBgwdj5syZSExMLFLo51GV9z1sbXjO1WMaN24cTp06hR9++AH//PMP+vXrh//85z+GD+5du3YhODgYv/zyC4KCghAYGIhRo0ZZ5JEruVyOl156Cbt27cJ3330HrVZbZEpg4Y+JB/e8/N///R9OnTpV5vZDQkKQnp5uNDWpcKplWX2kp6dj3bp1Rbbp6OhYbML2sObNm8PLywurV682Kne8Z88eREVFPXZFHEvQv39/3Lp1C2vWrClyX25urqHqV2kGDBiA/Px8fPPNN9i7dy/69+9vdH9aWlqRvW6FX5SllXtPSkrCpUuXiixXq9U4ePAg5HJ5iT9edDodBg4cCLVajR9//LHYEr0vvPACFAoF5syZUyQ+IUS5Sy6byqFDh5CVlWX0I0ShUBSJ7bPPPiuyt97R0REAiryvu3XrBmdnZyxcuLDIhTdNtce5uM+l8ry+o0ePxtWrV7Fjx45iz4WJiIiAi4sLFixYUOy5eqWVEH7Ql19+abT+qlWroNVqDTvAKtJPSc8z8L9zUBYtWoRGjRoZpiG3a9cOBw8exN9//210LoVCocCLL76IH3/8ERcuXCi1X1OM08d1/fp1DB8+HLa2tpg6dWqZ7U31+hUKCQnB5cuXjdY7f/58qVUpH9S6dWtcuHChzEtMAMV/nwAF1WbLUp4xZ87vltOnT8PV1dVQWa4kxX22bN26tULnsj6oefPm8PT0xOrVq6FWqw3L169fX67v2+KMGjUKs2bNKvYSIg86ePBgscsLz6ssbvrlwwqnw5WVBBZOx/v777/L3CZQcIT2rbfeQlRUFH766adyrVOWiIgInDp1CufOnTMsu3fvHjZs2GCS7VdESEgIli1bhoULF6Jly5Ym2ebp06chk8mKnfpozXjk6jEkJCRg3bp1SEhIMJQ6nTJlCvbu3Yt169ZhwYIFiI2NxfXr17F161Z8++230Ol0ePPNN/HSSy/h999/f2Kx7tmzx+gaBoXatGljNId6wIAB+OyzzzBr1iyEh4cX2dvQs2dPbN++Hc8//zx69OiBuLg4rF69GvXr1y9zL9HAgQMxbdo0PP/885gwYYKhlG/t2rWNTrrt1q0bbG1t0atXL7zxxhvIysrCmjVr4OXlVWSvVbNmzbBq1SrMmzcPoaGh8PLyKnaesFKpxKJFizBixAh06NABgwYNMpTLDQwMxJtvvlmu59GSvfzyy9iyZQtGjx6NQ4cO4ZlnnoFOp8Ply5exZcsW7Nu3r9Tz7oCCkrShoaF49913kZ+fX+z5ditXrsTzzz+PkJAQZGZmYs2aNXBxcTEcPSjOzZs30bJlS3Tu3BldunSBj48PkpOTsWnTJpw/fx6TJk0qcQ/r6tWr8fvvvxse14O8vb3RtWtXhISEYN68eZg+fTri4+PRt29fODs7Iy4uDjt27MDrr79uuFabqaWnp+P7778HULDHtLBEuL29Pd555x1Du549e+K7776Dq6sr6tevj1OnTuG3334rck5EkyZNoFAosGjRIqSnp0OlUhmu+fbJJ59g1KhRaNGiBQYPHoxq1arh/PnzyMnJwTfffPPYj2Xu3Lk4evQoevTogYCAACQnJ2PlypXw8/MzFDV52K+//opvv/0WL774Iv755x+jnSdOTk7o27cvXFxcsGrVKrz88st46qmnMHDgQHh6eiIhIQG//vornnnmGXz++edlxqdWq9GlSxf0798fV65cwcqVK9G2bVv07t0bACrUT7NmzQAAEyZMQEREBBQKheGSE6GhofDx8cGVK1cwfvx4Q//t27fHtGnTAKDICfkffvghDh06hFatWuG1115D/fr1ce/ePZw5cwa//fabIXE1xTitiDNnzuD777+HXq/H/fv38ddff+HHH3+ETCbDd999V6499o/y+i1durTIJRfkcjlmzJiBkSNHYunSpYiIiMCrr76K5ORkrF69Gg0aNEBGRkaZ8fTp0wcffPABjhw5gm7dupUZe/v27bF48WJoNBrUrFkT+/fvL9eRLxcXlzLHnDm/Ww4cOIBevXqVeYSkZ8+emDt3LkaMGIE2bdogMjISGzZseKTzo4CC78t58+bhjTfeQOfOnTFgwADExcVh3bp1j7zNgIAAo+s6lqRPnz4ICgpCr169EBISguzsbPz222/YtWsXWrRoUWSa39WrVw2fvzk5Ofjjjz/wzTffIDQ0FC+//HKpfQUHB6Nhw4b47bffyrxWY6Hhw4dj5syZWLRoUYXPvy/O22+/je+//x5du3bF+PHjDaXYa9WqhXv37j3xo9sTJ04sd9uDBw8W2ekAAH379jWcp3jgwAE888wzj3XZiErpCVUltAoAxI4dOwy3C0utOjo6Gv2zsbER/fv3F0II8dprrwkA4sqVK4b1Tp8+LQCIy5cvmz3m0kqxo5hyynq9Xvj7+wsAYt68eUW2p9frxYIFC0RAQIBQqVSiadOm4pdffim2hC4eKjUuhBD79+8XDRs2FLa2tqJOnTri+++/L7YU+88//ywaNWok7OzsRGBgoFi0aJFYu3ZtkdKkSUlJokePHsLZ2dmoRGxJ5bw3b94smjZtKlQqlXB3dxdDhgwRN2/eNGpT3NXNhSi+ZHxZHqUU+8MljEuKp7irtavVarFo0SLRoEEDoVKpRLVq1USzZs3EnDlzDOWWy/Luu+8KAEZlWQudOXNGDBo0SNSqVUuoVCrh5eUlevbsKf7+++9St5mRkSGWL18uIiIihJ+fn1AqlcLZ2Vm0bt1arFmzxqiU+MOvXeHzUty/h0sC//jjj6Jt27aGsVi3bl0xduxYo/FnSg+XYpfJZMLd3V307t1bnD592qhtWlqaGDFihPDw8BBOTk4iIiJCXL58WQQEBBR5j6xZs0YEBwcLhUJR5H38888/izZt2gh7e3vh4uIiWrZsKTZt2mQU08PvCyHKV+b64MGDok+fPqJGjRrC1tZW1KhRQwwaNEhcvXrV0ObhUuylfcY83N+hQ4dERESEcHV1FXZ2diIkJEQMHz68zPdPYR9HjhwRr7/+uqhWrZpwcnISQ4YMEXfv3i3Svjz9aLVaMX78eOHp6SlkMlmRsd2vX78iZbDVarVwcHAQtra2RUpzCyHEnTt3xNixY4W/v79QKpXCx8dHdOnSRXz55ZdG7co7TvFQCeRCxb1nHlb4OhX+s7GxEe7u7qJVq1Zi+vTp4vr160XWKa409oPK87yWNl4VCoWh3ffffy+Cg4OFra2taNKkidi3b1+5S7ELUVAK/9VXXy0SH4opxX7z5k3x/PPPCzc3N+Hq6ir69etnKBH94GdwcSW1hSh7zAlh+u+WqKgoAUD89ttvZT4XeXl54q233hK+vr7C3t5ePPPMM+LUqVNFSt6X9PyUdHmFlStXiqCgIKFSqUTz5s3F0aNHSyyj/7DCUuylKe79tmnTJjFw4EAREhIi7O3thZ2dnahfv7549913RUZGhtH6xb2//Pz8xOuvvy7u3LlTZoxCCLF06VLh5ORUpFx/SWNPCCFmz55t9LlcUin24h5/cc/f2bNnRbt27YRKpRJ+fn5i4cKF4tNPPxUARFJSktG6xX22P6yk99nDynPZBCFKLsVe0r/vvvtOCFFwKQNbW1vx1VdflRmLtZEJYaYzGK2QTCYzqha4efNmDBkyBBcvXixy4qKTkxN8fHwwa9asIlMpcnNz4eDggP3796Nr165P8iEQERFVet999x3Gjh2LhISEMi+eWhlNmjQJR48eNUyrIvNIT09HcHAwFi9ebHS5E6lNmjQJX3zxBbKysiQtsPU4li1bhsWLFyMmJqZIkRxrx3OuHkPTpk2h0+mQnJyM0NBQo3+FFZWeeeYZaLVao0p6V69eBQCrKKJARET0pA0ZMgS1atUqtsBNZXf37l189dVXmDdvHhMrM3N1dcXbb7+Njz766IlXjiyUm5trdPvu3bv47rvv0LZt20qbWGk0GixduhTvvfdelUusAIBHrsqQlZVlqNrStGlTLF26FJ06dYK7uztq1aqFoUOH4sSJE1iyZAmaNm2KlJQUHDx4EI0aNUKPHj2g1+vRokULODk5YdmyZdDr9Rg7dixcXFywf/9+iR8dEREREUmlSZMm6NixI+rVq4c7d+7g66+/xu3bt3Hw4EG0b99e6vDoETC5KsPhw4eNSv8WGjZsGNavXw+NRoN58+bh22+/xa1bt+Dh4YGnn34ac+bMQXh4OADg9u3bGD9+PPbv3w9HR0d0794dS5Ysgbu7+5N+OERERERkIWbMmIFt27bh5s2bkMlkeOqppzBr1iyTXPaApMHkioiIiIiIyAR4zhUREREREZEJMLkiIiIiIiIyAV5EuBh6vR63b9+Gs7MzK/UQEREREVVhQghkZmaiRo0akMtLPzbF5KoYt2/fhr+/v9RhEBERERGRhbhx4wb8/PxKbcPkqhjOzs4ACp5AFxcXSWPRaDTYv38/unXrBqVSKWksRFQyjlUiy6XRabDu7DoAwNCGQ3Ho4CGOVSILZ0nfqxkZGfD39zfkCKVhclWMwqmALi4uFpFcOTg4wMXFRfI3FhGVjGOVyHJlq7Mx9dhUAMCoVqM4VokqAUv8Xi3P6UIsaEFERERERGQCTK6IiIiIiIhMgMkVERERERGRCfCcq0ckhIBWq4VOpzNrPxqNBjY2NsjLyzN7X1QxSqUSCoVC6jCIiIiIyEIwuXoEarUaiYmJyMnJMXtfQgj4+Pjgxo0bvOaWhZHJZPDz84OTk5PUoRARERGRBWByVUF6vR5xcXFQKBSoUaMGbG1tzZr06PV6ZGVlwcnJqcyLltGTI4RASkoKbt68ibCwMB7BIiIiIiImVxWlVquh1+vh7+8PBwcHs/en1+uhVqthZ2fH5MrCeHp6Ij4+HhqNhskVEZEFU9mo8MugXwx/ExGZC5OrR8REhzhNk4iocrCR26BH7R4ACs5lJiIyF2YIREREREREJsAjV0RERGTVNDoNNkRuAAD0r9tf4miIyJoxuZKQ0AncP3Yf6kQ1bH1t4dbODTIFp5oRERGZklqnxoifRgAA+ob1lTYYIrJqnBYokZTtKfgj8A+c73QeUYOjcL7TefwR+AdStqeYrc/hw4dDJpNBJpNBqVTC29sbXbt2xdq1a6HX6w3tAgMDDe0cHBwQHh6Or776qkJ9rV+/Hm5ubsXeN3v2bDRp0qTI8vj4eMhkMpw7d65CfRERERERWQImVxJI2Z6Ciy9dRP7NfKPl+bfycfGli2ZNsP7zn/8gMTER8fHx2LNnDzp16oSJEyeiZ8+e0Gq1hnZz585FYmIiLly4gKFDh+K1117Dnj17zBYXEREREVFlx+TKBIQQ0GXryvVPm6HFtQnXAFHchgr+i54YDW2GtsxtCVHcRkqnUqng4+ODmjVr4qmnnsKMGTPw008/Yc+ePVi/fr2hnbOzM3x8fBAcHIxp06bB3d0dBw4cMNx///59jBo1Cp6ennBxcUHnzp1x/vz5CsdDRERERPQgoRNIP5IO5VEl0o+kQ+gq/ptXKjznygT0OXocczpmmo0JIP9mPo67Hi+zabusdlA4Pv71lTp37ozGjRtj+/btGDVqlNF9er0eO3bsQFpaGmxtbQ3L+/XrB3t7e+zZsweurq744osv0KVLF1y9ehXu7u6PHRMRERERVT0p21MQPTEa+Tfz4QAHXFh6ASo/FUKXh8LzBU+pwysTj1wRAKBu3bqIj4833J42bRqcnJygUqnw0ksvoVq1aobE6/jx4/jzzz+xdetWNG/eHGFhYfj444/h5uaGbdu2SfQIiIiIiKgyk/LUGVPhkSsTkDvI0S6rXbna3j96H5HPRZbZLnx3ONzau0Gv1yMjIwMuLi5FLlwsdzBdbiyEMLoo7tSpUzF8+HAkJiZi6tSpGDNmDEJDQwEA58+fR1ZWFqpXr260jdzcXMTExJgsJiIiIiKqGoROIHpidMmnzsiA6EnR8OjjYdHVtZlcmYBMJiv39Dz3bu5Q+amQfyu/+DePDFD5qeDezR0yhQwyvQwKnQIKR0WR5MqUoqKiEBQUZLjt4eGB0NBQhIaGYuvWrQgPD0fz5s1Rv359ZGVlwdfXF4cPHy6ynZIqBD7IxcUF6enpRZbfv38fAODq6vqoD4OIiKgIlY0KW17aYvibiCzP/WP3ixyxMiKA/Bv5uH/sPqp1rPbkAqsgJldPmEwhQ+jyUFx86SIgg3GC9W8SHros9Ilm5L///jsiIyPx5ptvFnu/v78/BgwYgOnTp+Onn37CU089haSkJNjY2CAwMLDC/dWpUwc3b97EnTt34O3tbVh+5swZ2NnZoVatWo/6UIiIiIqwkdugX4N+AACNRiNxNERUHHWi2qTtpMLkSgKeL3iiwbYGhpP1Cqn8VAhdZt6T9fLz85GUlASdToc7d+5g7969WLhwIXr27IlXXnmlxPUmTpyIhg0b4u+//8azzz6L1q1bo2/fvli8eDFq166N27dv49dff8Xzzz+P5s2bAwB0Ol2Ra1apVCpERESgTp06GDRoEObNmwcfHx+cOXMG7733HiZOnAiF4vGLdBARERFR5WHra1t2owq0kwqTK4l4vuAJjz4euH/sPtSJatj62sKtnZvZj1jt3bsXvr6+sLGxQbVq1dC4cWN8+umnGDZsWKnTDuvXr49u3bph5syZ2L17N3bv3o13330XI0aMQEpKCnx8fNC+fXujI1FZWVlo2rSp0XZCQkIQHR2N/fv3Y8aMGRg0aBBSUlIQFBSEiRMnYvLkyWZ77EREVDVp9VrsiNoBAOgZ2lPiaIioOG7t3GBbwxbq2yUcmfr31Bm3dm5PNK6KkolHuViSlcvIyICrqyvS09Ph4uJidF9eXh7i4uIQFBQEOzs7s8dSWkELktaTfi+QZdNoNNi9ezeee+45KJVKqcMhogdkq7PhtNAJAJA2JQ1HfjvCsUpkgc49ew73D94vese/xx4abGsgSTn20nKDh/HXOhERERERSSr9j3RDYqX0NN7xofJTSZZYVRSnBRIRERERkWSETuDa2GsAAO9h3qj7dV3cPXQXf+75Ey27t0T1TtUtuvz6g5hcERERERGRZG5/cRtZZ7KgcFUgZHEIZAoZXDu4QpOtgWsH10qTWAGcFkhERERERBJRJ6sR924cACB4fjBsvSy7GmBZmFwREREREZEkYqfFQntfC6emTqgxuobU4Tw2JldERERERPTEpZ9IR9L6JABA2MqwSjX9ryQ854qIiIismq3CFuv6rDP8TUTS02v1uDrmKgDAd5QvXJ92lTgi02ByRURERFZNqVBieJPhAAquSUdE0ru94jay/8mGjbsNghYGSR2OyXBaIBERERERPTH5ifmIe//fIhYLg2HrYT1HlHnkSgIJeXlILWXPmYdSiVp2dk8wIiIiIuul1WuxL3ofAKBzQGeJoyGimKkx0GXq4NzSGb6jfKUOx6R45OoJS8jLQ50//0Sz06dL/Ffnzz+RkJdn8r6HDx8OmUyG0aNHF7lv7NixkMlkGD58OAAgJSUF//3vf1GrVi2oVCr4+PggIiICJ06cMKwTGBgImUwGmUwGhUKBGjVq4NVXX0VaWprJYyciInpU+dp89NzUEz039US+Nl/qcIiqtLTDaUjekAzIgLAVYZDJK38RiwcxuXrCUjUa5On1pbbJ0+tLPbL1OPz9/fHDDz8gNzf3f/3l5WHjxo2oVauWYdmLL76Is2fP4ptvvsHVq1fx888/o2PHjrh7967R9ubOnYvExEQkJCRgw4YNOHr0KCZMmGCW2ImIiIio8tJr9Lg29hoAoMboGnBp7iJxRKbHaYEmIIRAThkJU6HcCrTL1umg//d/hU4HuRBGbRzkcshkFcv2n3rqKcTExGD79u0YMmQIAGD79u2oVasWgoIKTia8f/8+jh07hsOHD6NDhw4AgICAALRs2bLI9pydneHj4wMAqFmzJoYNG4ZNmzZVKCYiIiIisn43l99EzqUcKD2UCJpnPUUsHsTkygRy9Ho4HTtm0m22PXu2zDZZ7drBUaGo8LZHjhyJdevWGZKrtWvXYsSIETh8+DAAwMnJCU5OTti5cyeefvppqFSqcm331q1b2LVrF1q1alXhmIiIiIjIeuXdzEP87HgAQPDiYCjdldIGZCYWPy3w6NGj6NWrF2rUqAGZTIadO3eWe90TJ07AxsYGTZo0MVt8ldHQoUNx/PhxXL9+HdevX8eJEycwdOhQw/02NjZYv349vvnmG7i5ueGZZ57BjBkz8M8//xTZ1rRp0+Dk5AR7e3v4+flBJpNh6dKlT/LhEBEREZGFi3krBvpsPVxau8BnmI/U4ZiNxR+5ys7ORuPGjTFy5Ei88MIL5V7v/v37eOWVV9ClSxfcuXPHjBEWTM/LateuXG3PZWWV66jU8aZN0cTJCXq9HhkZGXBxcYFcbpwLO8gfLTf29PREjx49sH79eggh0KNHD3h4eBi1efHFF9GjRw8cO3YMf/zxB/bs2YPFixfjq6++MhS9AICpU6di+PDhEELgxo0bmDFjBnr06IGjR49C8QhH1YiIiIjIutz77R5StqQAciBspfUVsXiQxSdX3bt3R/fu3Su83ujRozF48GAoFIoKHe16FDKZrNzT8+zLmRDZy+VwVCigl8mgUyjgqFAUSa4ex8iRIzFu3DgAwIoVK4ptY2dnh65du6Jr1654//33MWrUKMyaNcsoufLw8EBoaCgAICwsDMuWLUPr1q1x6NAhPPvssyaLl4iIiIgqH33+/4pY1BxXE85NnCWOyLwsPrl6FOvWrUNsbCy+//57zJs3r8z2+fn5yM//X2nWjIwMAAVXcX/4Su4ajQZCCOj1eujLWZziQeVdp3D74t8iFoV9Pg4hhGE73bp1g1qthkwmQ9euXQ19ldZPvXr1sHPnTqP7H25fWGAjOzv7seO1dIXPmUaj4VE6MnxWPPyZQUTSk+llWN5tueFvgGOV6Em5+dFN5F7NhdJbCb/3/co99izpe7UiMVhdcnXt2jW88847OHbsGGxsyvfwFi5ciDlz5hRZvn//fjg4OBgts7GxgY+PD7KysqBWqyscn0qthkomQ/5Dlf+M2shkUOXnI+OBNpmZmRXu62EajQZardaQPJ46dQpAQSIEAFqtFhqNBvHx8Rg+fDiGDBmCBg0awNnZGWfPnsXixYvRvXt3w/p6vR6pqam4du0ahBC4desWZs2aBQ8PD4SHhxvaWSu1Wo3c3FwcPXoUWq1W6nDIQhw4cEDqEIioGAEIAAAc/v0wAI5VoidBliKD8wfOkEGG9EHp2H9if4W3YQljNScnp9xtrSq50ul0GDx4MObMmYPatWuXe73p06dj8uTJhtsZGRnw9/dHt27d4OJiXH8/Ly8PN27cgJOTE+zs7CocYwMAl1u0KPU6Vh5KJWr9u20hBDIzM+Hs7FzhsusPUyqVsLGxMTymhx+bjY0NlEolfH190aZNG3z55ZeIiYmBRqOBv78/XnvtNUyfPh329vYAALlcjgULFmDBggUACs7lat68Ofbt24fAwMDHirUyyMvLg729Pdq3b/9I7wWyLhqNBgcOHEDXrl2hVFpnBSQia8CxSvTkRPWLwj31Pbi0c0GbRW0q9FvWksZqRQ4YWFVylZmZib///htnz541nE9UOHXLxsYG+/fvR+fOnYusp1Kpii03rlQqi7yYOp0OMpkMcrn8kc+BCnRwQGA52xZOrSvs83F88803pd7/008/Gf7+8MMP8eGHH5baPj4+/rHiqezk/15nrLj3CVVdfD8QWR6dXodjCQWXTHna92kAHKtE5nZ3z13c++keoABqr6wNW1vbR9qOJYzVivRvVcmVi4sLIiMjjZatXLkSv//+O7Zt22a4SC4RERFVHXnaPHT6phMAIG1KmsTREFk/XZ4O18YXFLHwm+gHp4ZOEkf05Fh8cpWVlYXo6GjD7bi4OJw7dw7u7u6oVasWpk+fjlu3buHbb7+FXC5Hw4YNjdb38vKCnZ1dkeVERERERGR6NxbfQF5MHmxr2CJwdqDU4TxRFp9c/f333+jUqZPhduG5UcOGDcP69euRmJiIhIQEqcIjIiIiIqJ/5cbmImFhwW/z0KWhsHG2+HTDpCz+0Xbs2NFQjrw469evL3X92bNnY/bs2aYNioiIiIiIioieGA19nh5und3g2d9T6nCeONNdlZaIiIiIiKqs1F2puPvLXciUMoStCHvsSteVEZMrIiIiIiJ6LLocHaInFNRJ8JvsB8e6jhJHJA0mV0RERERE9FgSPkxAXnweVP4qBL4fKHU4krH4c66IiIiIHodSocTiZxcb/iYi08q5loOERf8WsfgkFApHhcQRSYfJFREREVk1W4Utpj4zFQCg0WgkjobIugghcG38NQi1QLWIavB4wUPqkCTFaYFERERERPRIUnekIm1fGmS2MoR9VjWLWDyIyRWZ1fz589GmTRs4ODjAzc2t2DYJCQno0aMHHBwc4OXlhalTp0Kr1Rq1OXz4MJ566imoVCqEhoYWKcG/atUqNGrUCC4uLnBxcUHr1q2xZ88eozYdO3aETCYz+jd69GjD/efPn8egQYPg7+8Pe3t71KtXD8uXLzfJ80BERNLR6XX469Zf+OvWX9DpdVKHQ2Q1dNk6RE8qKGJR6+1acAhzkDgi6XFaIJmVWq1Gv3790Lp1a3z99ddF7tfpdOjRowd8fHxw8uRJJCYm4pVXXoFSqcSCBQsAAHFxcejRowdGjx6NDRs24ODBgxg1ahR8fX0REREBAPDz88OHH36IsLAwCCHwzTffoE+fPjh79iwaNGhg6O+1117D3LlzDbcdHP73IXD69Gl4eXnh+++/h7+/P06ePInXX38dCoUC48aNM9dTREREZpanzUPLr1oCANKmpEkcDZH1uD7vOvJv5MMu0A61pteSOhyLwOTKhLLV2SXep5ArYGdjV662cpkc9kr7/7XVZEOhVkAu/9+BRkfbipW3TElJQXh4OCZMmIAZM2YAAE6ePImOHTtiz5496NKlS4W2V15z5swBUPLFnvfv349Lly7ht99+g7e3N5o0aYIPPvgA06ZNw+zZs2Fra4vVq1cjKCgIS5YsAQDUq1cPx48fxyeffGJIrnr16mW03fnz52PVqlX4448/jJIrBwcH+Pj4FBvLyJEjjW4HBwfj1KlT2L59O5MrIiIiogdkX87GjSU3AAChy0OhcKi6RSwexOTKhJwWOpV433Nhz+HXwb8abnt97IUcTU6xbTsEdMDh4YcNtxuva4y7uXeN2ohZokKxeXp6Yu3atejbty+6deuGOnXq4OWXX8a4ceNKTawaNGiA69evl3h/u3btiky/q4hTp04hPDwc3t7ehmURERH473//i4sXL6Jp06Y4deoUnn32WaP1IiIiMGnSpGK3qdPpsHXrVmRnZ6N169ZG923YsAHff/89fHx80KtXL7z//vtGR68elp6eDnd390d+fERERETWRgiBa+OuQWgEqvesDo/eVbuIxYOYXFUhzz33HF577TUMGTIEzZs3h6OjIxYuXFjqOrt37y61spK9vX2J95VHUlKSUWIFwHA7KSmp1DYZGRnIzc01xBAZGYnWrVsjLy8PTk5O2LFjB+rXr29YZ/DgwQgICECNGjXwzz//YNq0abhy5Qq2b99ebGwnT57E5s2b8euvvxZ7PxEREVFVlLIlBfcP3ofcTo7Q5aFSh2NRmFyZUNb0rBLvU8iND5UmT0kusa1cZlxn5PyI83BxdjGaFvioPv74YzRs2BBbt27F6dOnoVKpSm0fEBDw2H0+KXXq1MG5c+eQnp6Obdu2YdiwYThy5IghwXr99dcNbcPDw+Hr64suXbogJiYGISEhRtu6cOEC+vTpg1mzZqFbt25P9HEQERERWSptphbRk/8tYjG9FuyDH29Hu7VhcmVCFTkPqkJtlY5wtHU0SXIVExOD27dvQ6/XIz4+HuHh4aW2N/e0QB8fH/z5559Gy+7cuWO4r/D/wmUPtnFxcTE6cmZra4vQ0IK9J82aNcNff/2F5cuX44svvii271atWgEAoqOjjZKrS5cuoUuXLnj99dfx3nvvPfJjIyIiIrI28XPiob6thl2IHfzf9pc6HIvD5KoKUavVGDp0KAYMGIA6depg1KhRiIyMhJeXV4nrmHtaYOvWrTF//nwkJycb4jhw4ABcXFwMR5xat26N3bt3G6134MCBIudTPUyv1yM/P7/E+8+dOwcA8PX1NSy7ePEiOnfujGHDhmH+/PmP8pCIiIiIrFLWhSzcXHYTABD2WRgUdixi8TAmV1XIu+++i/T0dHz66adwcnLC7t27MXLkSPzyyy8lrvO40wITEhJw7949JCQkQKfTGRKa0NBQODk5oVu3bqhfvz5efvllLF68GElJSXjvvfcwduxYw5TF0aNH4/PPP8fbb7+NkSNH4vfff8eWLVuMzoWaPn06unfvjlq1aiEzMxMbN27E4cOHsW/fPgAFR+w2btyI5557DtWrV8c///yDN998E+3bt0ejRo0AFEwF7Ny5MyIiIjB58mTDOV8KhQKenp6P9TwQEZF0lAolZnWYZfibiCpOCIFrY68BOsDjeQ9U715d6pAsk6Ai0tPTBQCRnp5e5L7c3Fxx6dIlkZub+0Ri0el0Ii0tTeh0usfazqFDh4SNjY04duyYYVlcXJxwcXERK1eufNwwSzRs2DABoMi/Q4cOGdrEx8eL7t27C3t7e+Hh4SHeeustodFoisTfpEkTYWtrK4KDg8W6deuM7h85cqQICAgQtra2wtPTU3Tp0kXs37/fcH9CQoJo3769cHd3FyqVSoSGhoqpU6cavcazZs0qNtaAgIBiH9uTfi+QZVOr1WLnzp1CrVZLHQoRlYJjlejRJH6XKA7hkDhif0Tkxpv/t48ljdXScoOHyYQQFavpXQVkZGTA1dUV6enpcHFxMbovLy8PcXFxCAoKgp2dXQlbMB29Xo+MjAy4uJimoAWZzpN+L5Bl02g02L17N5577jkoldwzTmSpOFaJKk6brsX/1fk/aO5oELQgCAHTzV/wzJLGamm5wcM4LZCIiIisml7oEZUSBQAIdWPZaKKKipsZB80dDexr28N/MotYlIbJFREREVm1XE0uGq5qCABIm5ImcTRElUvW+Szc+vwWACDs8zDIVZxJVRo+O0REREREVITQC1wdcxXQA579POHe1V3qkCwekysiIiIiIioi6ZskZJzMgNxRjpClIWWvQEyuHhXrgBDfA0RERGStNGkaxL4dCwAInB0IOz8W7yoPJlcVVFitJCcnR+JISGpqtRpAwXWwiIiIiKxJ3Ltx0KRq4FDfAX4T/aQOp9JgQYsKUigUcHNzQ3JyMgDAwcEBMpnMbP3p9Xqo1Wrk5eWxFLsF0ev1SElJgYODA2xsOIyIiIjIemSezsTt1bcBAGErwiBX8jdoefFX4SPw8fEBAEOCZU5CCOTm5sLe3t6sSRxVnFwuR61atfi6EBERkdUwFLEQgNdgL1TrWE3qkCoVJlePQCaTwdfXF15eXtBoNGbtS6PR4OjRo2jfvr3kF1AjY7a2tjyaSERUCSgVSkxpPcXwNxGVLPHrRGT+mQmFswIhH7OIRUUxuXoMCoXC7OfbKBQKaLVa2NnZMbkiIiJ6BLYKW3zU7SMAMPtOUaLKTJ2qRuw7/xaxmBsIla9K4ogqH+52JyIiIiIixM2Ig/aeFo7hjqg5rqbU4VRKPHJFREREVk0v9EhITwAA+Dr4ShwNkWXK+L8MJH6VCAAIWxkGuQ2PwTwKJldERERk1XI1uQhaHgQASJuSJnE0RJZH6P5XxMJ7mDfc2rpJHVKlxZSUiIiIiKgKu/3FbWSdyYLCVYGQxSxi8TiYXBERERERVVHqZDXi3o0DAATPD4atl63EEVVuTK6IiIiIiKqo2Gmx0N7XwqmpE2qMriF1OJUekysiIiIioioo/UQ6ktYnASgoYiFTyCSOqPJjckVEREREVMXotfqCIhYAfEf5wvVpV4kjsg5MroiIiIiIqpjbK24j+59s2LjbIGhhkNThWA2WYiciIiKrZiO3wZjmYwx/E1V1+Yn5iHv/3yIWC4Nh68EiFqbCTxgiIiKyaiobFVb0WAEA0Gg0EkdDJL2YqTHQZerg3NIZvqN4YW1T4rRAIiIiIqIqIu1wGpI3JAMyIGxFGGRyFrEwJR65IiIiIqsmhEBqTioAwFXJk/ap6tJr9Lg29hoAoMboGnBp7iJxRNaHyRURERFZtRxNDrw+9gIApE1JkzgaIuncXH4TOZdyoPRQImgei1iYA6cFEhERERFZubybeYifHQ8ACF4cDKW7UtqArBSTKyIiIiIiKxfzVgz02Xq4tHaBzzAfqcOxWkyuiIiIiIis2L3f7iFlSwogB8JWsoiFOTG5IiIiIiKyUvp8Pa6NKyhiUXNcTTg3cZY4IuvG5IqIiIiIyErdWHoDuVdyofRWImgui1iYm8UnV0ePHkWvXr1Qo0YNyGQy7Ny5s9T227dvR9euXeHp6QkXFxe0bt0a+/btezLBEhERERFZiLyEPFz/4DoAIOTjENi4slC4uVl8cpWdnY3GjRtjxYoV5Wp/9OhRdO3aFbt378bp06fRqVMn9OrVC2fPnjVzpERERGSJbOQ2GNZ4GIY1HgYbOX9cUtURPSka+lw9XNu7wnuIt9ThVAkW/wnTvXt3dO/evdztly1bZnR7wYIF+Omnn7Br1y40bdrUxNERERGRpVPZqLC+73oAgEajkTYYoifk7p67SN2RCiiAsBVhkMlYxOJJsPjk6nHp9XpkZmbC3d29xDb5+fnIz8833M7IyABQ8AEs9YdwYf9Sx0FEpeNYJaocOFapKtDn6XFtfEERixrja0BVR1Xp3vOWNFYrEoPVJ1cff/wxsrKy0L9//xLbLFy4EHPmzCmyfP/+/XBwcDBneOV24MABqUMgonLgWCWyPEII5OsLdqKq5CrIZDKOVbJqqs0q2MXYQe+uR1SrKETtjpI6pEdmCWM1Jyen3G1lQghhxlhMSiaTYceOHejbt2+52m/cuBGvvfYafvrpJzz77LMltivuyJW/vz9SU1Ph4uLyuGE/Fo1GgwMHDqBr165QKnklbSJLxbFKZLmy1dmo9nE1AEDyxGScPHKSY5WsVl5cHs42Pgt9nh61v6sNzwGeUof0SCzpezUjIwMeHh5IT08vMzew2iNXP/zwA0aNGoWtW7eWmlgBgEqlgkqlKrJcqVRK/mIWsqRYiKhkHKtElkcp/jcmC8cnxypZq8tvXYY+Tw+3zm7wHeJb6c+1soSxWpH+Lb5a4KPYtGkTRowYgU2bNqFHjx5Sh0NEREREZHapu1Jx95e7kCllLGIhEYs/cpWVlYXo6GjD7bi4OJw7dw7u7u6oVasWpk+fjlu3buHbb78FUDAVcNiwYVi+fDlatWqFpKQkAIC9vT1cXV0leQxEREREROaky9EhekLBb2a/yX5wrOsocURVk8Ufufr777/RtGlTQxn1yZMno2nTppg5cyYAIDExEQkJCYb2X375JbRaLcaOHQtfX1/Dv4kTJ0oSPxERERGRuSV8mIC8+Dyo/FUIfD9Q6nCqLIs/ctWxY0eUVnNj/fr1RrcPHz5s3oCIiIiIiCxIzrUcJCwqONgQ+kkoFI4KiSOquiz+yBURERERERVPCIFr469BqAWqRVSDxwseUodUpVn8kSsiIiKix6GQK/BS/ZcMfxNZk9QdqUjblwaZrQxhn7GIhdSYXBEREZFVs7Oxw9Z+WwEUXDuHyFrosnWInlRQxKLW27XgEOYgcUTEaYFERERERJXQ9XnXkX8jH3aBdqg1vZbU4RCYXBERERERVTrZl7NxY8kNAEDo8lAoHDjl1RIwuSIiIiKrlq3OhmyODLI5MmSrs6UOh+ixCSFwbdw1CI1A9Z7V4dGbRSwsBZMrIiIiIqJKJGVLCu4fvA+5nRyhy0OlDocewOSKiIiIiKiS0GZqET353yIW02vBPthe4ojoQUyuiIiIiIgqifg58VDfVsMuxA7+b/tLHQ49hMkVEREREVElkHUhCzeX3QQAhH0WBoUdi1hYGiZXREREREQWTgiBa2OvATrAo68HqnevLnVIVAwmV0REREREFu7OhjtIP5oOub0coctYxMJS2UgdABEREZE5KeQKPBf2nOFvospGm65FzJQYAEDA+wGwC7CTOCIqCZMrIiIismp2Nnb4dfCvAACNRiNxNEQVFzczDpo7GtjXtof/ZBaxsGScFkhEREREZKGyzmfh1ue3AABhn4dBruLPd0vGV4eIiIiIyAIJvcDVMVcBPeDZzxPuXd2lDonKwOSKiIiIrFq2OhuOCxzhuMAR2epsqcMhKrekb5OQcTIDckc5QpaGSB0OlQPPuSIiIiKrl6PJkToEogrRpGkQ+3YsACBwViDs/FjEojLgkSsiIiIiIgsT924cNCkaONR3gN8kP6nDoXJickVEREREZEEyT2fi9urbAICwFWGQK/mTvbLgK0VEREREZCEMRSwE4DXYC9U6VpM6JKoAJldERERERBYi8etEZP6ZCYWzAiEfs4hFZcPkioiIiIjIAqhT1Yh9598iFnMDofJVSRwRVRSrBRIREZFVk8vk6BDQwfA3kaWKmxEH7T0tHMMdUXNcTanDoUfA5IqIiIismr3SHoeHHwYAaDQaaYMhKkHG/2Ug8atEAEDYyjDIbbgjoDLiq0ZEREREJCGh+18RC+9h3nBr6yZ1SPSImFwREREREUno9he3kXUmCwpXBUIWs4hFZcbkioiIiKxatjobnh95wvMjT2Srs6UOh8iIOlmNuHfjAADB84Nh62UrcUT0OHjOFREREVm91JxUqUMgKlbstFho72vh1NQJNUbXkDocekw8ckVEREREJIH0E+lIWp8EoKCIhUwhkzgielxMroiIiIiInjC9Vl9QxAKA7yhfuD7tKnFEZApMroiIiIiInrDbK24j+59s2LjbIGhhkNThkIkwuSIiIiIieoLyE/MRN/PfIhYLg2HrwSIW1oLJFRERERHRExQzNQa6DB2cWzjD91VfqcMhE2K1QCIiIrJqcpkczWs0N/xNJKW0w2lI3pAMyFjEwhoxuSIiIiKrZq+0x1+v/QUA0Gg0EkdDVZleo8e1sdcAADVG14BLcxeJIyJT4+4bIiIiIqIn4Obym8i5lAOlhxJB81jEwhoxuSIiIiIiMrO8m3mInx0PAAheHAylu1LagMgsmFwRERGRVcvR5CBwWSAClwUiR5MjdThURcW8FQN9th4urV3gM8xH6nDITHjOFREREVk1IQSup183/E30pN377R5StqQA8n+LWMhZxMJa8cgVEREREZGZ6PP1uDauoIhFzbE14dzEWeKIyJyYXBERERERmcmNT24g90oulN5KBH3AIhbWjskVEREREZEZ5CXk4foHBVNSQz4OgY0rz8ixdkyuiIiIiIjMIHpSNPQ5eri2d4X3EG+pw6EngMkVEREREZGJ3d1zF6k7UgEFELYiDDIZi1hUBTw2SURERFZNJpOhvmd9w99E5qbL0+Ha+IIiFn4T/eDU0EniiOhJsfgjV0ePHkWvXr1Qo0YNyGQy7Ny5s8x1Dh8+jKeeegoqlQqhoaFYv3692eMkIiIiy+SgdMDFMRdxccxFOCgdpA6HqoAbH91AXkwebGvYInB2oNTh0BNk8clVdnY2GjdujBUrVpSrfVxcHHr06IFOnTrh3LlzmDRpEkaNGoV9+/aZOVIiIiIiqupy43KRsCABABCyJAQ2zpwoVpVY/KvdvXt3dO/evdztV69ejaCgICxZsgQAUK9ePRw/fhyffPIJIiIiil0nPz8f+fn5htsZGRkAAI1GA41G8xjRP77C/qWOg4hKx7FKVDlwrJK5XR1/Ffo8PVw7uaLaC9X4XntEljRWKxKDxSdXFXXq1Ck8++yzRssiIiIwadKkEtdZuHAh5syZU2T5/v374eBgGdMHDhw4IHUIRFQOHKtElidfn48pV6cAAD6u/TFUchXHKpmFzZ82cPzVEcJG4OaLN5GwJ0HqkCo9SxirOTk55W5rdclVUlISvL2NS116e3sjIyMDubm5sLe3L7LO9OnTMXnyZMPtjIwM+Pv7o1u3bnBxcTF7zKXRaDQ4cOAAunbtCqVSKWksRFQyjlUiy5WtzsaNf24AADp16oSTR05yrJLJ6XJ0ODvpLPKRD79Jfgh8PVDqkCo1S/peLZzVVh5Wl1w9CpVKBZVKVWS5UqmU/MUsZEmxEFHJOFaJLI9S/G9MFo5PjlUytZtLbiI/Ph8qfxWCZwdDoVRIHZJVsISxWpH+Lb6gRUX5+Pjgzp07Rsvu3LkDFxeXYo9aERERERE9jpxrOUhYVDAFMPSTUCgcmVhVVVaXXLVu3RoHDx40WnbgwAG0bt1aooiIiIiIyFoJIXBt/DUItUC1iGrweMFD6pBIQhafXGVlZeHcuXM4d+4cgIJS6+fOnUNCQsHegenTp+OVV14xtB89ejRiY2Px9ttv4/Lly1i5ciW2bNmCN998U4rwiYiIiMiKpe5IRdq+NMhsZQj7LIwXqq7iLD65+vvvv9G0aVM0bdoUADB58mQ0bdoUM2fOBAAkJiYaEi0ACAoKwq+//ooDBw6gcePGWLJkCb766qsSy7ATERERET0KXbYO0ZOiAQC13q4FhzDLqDJN0rH4ghYdO3aEEKLE+9evX1/sOmfPnjVjVERERFRZyGQyBLgGGP4mMpXr864j/0Y+VAEq1JpeS+pwyAJYfHJFRERE9DgclA6InxQPwDIuSErWIftyNm4sKSjxH/ZpGBQOLGJBlWBaIBERERGRJRFC4Nq4axAageo9q8OjN4tYUAEmV0REREREFZCyJQX3D96H3E6O0OWhUodDFoTJFREREVm1XE0uWqxpgRZrWiBXkyt1OFTJaTO1iJ78bxGL6bVgH8zrqNL/8JwrIiIismp6ocfft/82/E30OOLnxEN9Ww27EDv4v+0vdThkYXjkioiIiIioHLIuZOHmspsAgLDPwqCwYxELMsbkioiIiIioDEIIXBt7DdABHn09UL17dalDIgvE5IqIiIiIqAzJG5ORfjQdcns5QpexiAUVj8kVEREREVEptOlaRL9VUMQi4P0A2AXYSRwRWSomV0REREREpYibGQfNHQ3sa9vDfzKLWFDJWC2QiIiIrJ6HAy/ySo8m63wWbn1+CwAQ9nkY5Coem6CSMbkiIiIiq+Zo64iUqSkAAI1GI3E0VJkIvcDVMVcBPeDZzxPuXd2lDoksHFNvIiIiIqJiJH2bhIyTGZA7yhGyNETqcKgSYHJFRERERPQQTZoGsW/HAgACZwXCzo9FLKhsTK6IiIjIquVqctFxfUd0XN8RuZpcqcOhSiLuvThoUjRwqO8Av0l+UodDlQTPuSIiIiKrphd6HLl+xPA3UVkyT2fi9qrbAICwFWGQK3k8gsqH7xQiIiIion8ZilgIwGuwF6p1rCZ1SFSJmOXIVVxcHI4dO4br168jJycHnp6eaNq0KVq3bg07O85XJSIiIiLLlPh1IjL/zITCWYGQj1nEgirGpMnVhg0bsHz5cvz999/w9vZGjRo1YG9vj3v37iEmJgZ2dnYYMmQIpk2bhoCAAFN2TURERET0WNSpasS+828Ri7mBUPmqJI6IKhuTJVdNmzaFra0thg8fjh9//BH+/sZXr87Pz8epU6fwww8/oHnz5li5ciX69etnqu6JiIiIiB5L3Iw4aO9p4RjuiJrjakodDlVCJkuuPvzwQ0RERJR4v0qlQseOHdGxY0fMnz8f8fHxpuqaiIiIiOixZPxfBhK/SgQAhK0Mg9yGpQmo4kyWXJWWWD2sevXqqF69uqm6JiIiIiqVg9JB6hDIggnd/4pYeA/zhltbN6lDokrKLCn5mTNnEBkZabj9008/oW/fvpgxYwbUarU5uiQiIiIqlqOtI7JnZCN7RjYcbR2lDocs0O0vbiPrTBYUrgqELGIRC3p0Zkmu3njjDVy9ehUAEBsbi4EDB8LBwQFbt27F22+/bY4uiYiIiIgqTJ2sRty7cQCA4PnBsPW2lTgiqszMklxdvXoVTZo0AQBs3boV7du3x8aNG7F+/Xr8+OOP5uiSiIiIiKjCYqfFQntfC6emTqgxuobU4VAlZ5bkSggBvb7gCui//fYbnnvuOQCAv78/UlNTzdElERERUbHytHnosbEHemzsgTxtntThkAVJP5GOpPVJAAqKWMgUMokjosrOLBcRbt68OebNm4dnn30WR44cwapVqwAUXFzY29vbHF0SERERFUun12H3td2Gv4kAQK/VFxSxAOA7yheuT7tKHBFZA7McuVq2bBnOnDmDcePG4d1330VoaCgAYNu2bWjTpo05uiQiIiIiKrfbK24j+59s2LjbIGhhkNThkJUwy5GrRo0aGVULLPTRRx9BoVCYo0siIiIionLJT8xH3Mx/i1gsDIatB4tYkGmYJbl6UFZWluH8q0JKpdLc3RIRERERFStmagx0GTo4t3CG76u+UodDVsQs0wLj4uLQo0cPODo6wtXVFdWqVUO1atXg5uaGatWqmaNLIiIiIqIy3T9yH8kbkgEZi1iQ6ZnlyNXQoUMhhMDatWvh7e0NmYxvWiIiIiKSll6jx9WxBUUsaoyuAZfmLhJHRNbGLMnV+fPncfr0adSpU8ccmyciIiIiqrCby28i52IOlB5KBM1jEQsyPbMkVy1atMCNGzeYXBEREZHkHG0dIWYJAIBGo5E4GpJK3s08xM+OBwAELw6G0p01AMj0zJJcffXVVxg9ejRu3bqFhg0bFilg0ahRI3N0S0RERERUrJi3YqDP1sOltQt8hvlIHQ5ZKbMkVykpKYiJicGIESMMy2QyGYQQkMlk0Ol4AT8iIiIiejLu/XYPKVtSAPm/RSzkrAdA5mGW5GrkyJFo2rQpNm3axIIWREREJKk8bR5e3vEyAGBtz7USR0NPmj5fj2vjrgEAao6tCecmzhJHRNbMLMnV9evX8fPPPyM0NNQcmyciIiIqN51eh22XtgEA1jy3RuJo6Em78ckN5F7JhdJbicC5gVKHQ1bOLNe56ty5M86fP2+OTRMRERERlUteQh6uf3AdABDyUQiUbixiQeZlliNXvXr1wptvvonIyEiEh4cXKWjRu3dvc3RLRERERGQQ/WY09Dl6uLZ3hfdQb6nDoSrALMnV6NGjAQBz584tch8LWhARERGRud3dexep21MBBRC2Iow1AOiJMEtypdfrzbFZIiIiIqIy6fJ0hiIWfhP94NTQSeKIqKowyzlXRERERERSufHRDeTF5MG2hi0CZwdKHQ5VISZLrn744Ydyt71x4wZOnDhhqq6JiIiIiAAAuXG5SFiQAAAIWRICG2ezTNQiKpbJkqtVq1ahXr16WLx4MaKioorcn56ejt27d2Pw4MF46qmncPfuXVN1TURERFQiB6UDsqZnIWt6FhyUDlKHQ2YWPTEa+jw93Dq7wWuAl9ThUBVjslT+yJEj+Pnnn/HZZ59h+vTpcHR0hLe3N+zs7JCWloakpCR4eHhg+PDhuHDhAry9WbGFiIiIzE8mk8HR1hEAoNFoJI6GzCl1Vyru7roLmVKGsM9ZxKKyScjLQ+q/Y1Sr1SJGLsfZrCzY2BSkLB5KJWrZ2UkZYplMepy0d+/e6N27N1JTU3H8+HFcv34dubm58PDwQNOmTdG0aVPI5RU/WLZixQp89NFHSEpKQuPGjfHZZ5+hZcuWJbZftmwZVq1ahYSEBHh4eOCll17CwoULYWfhLwYRERERPRpdrg7RE6IBAH6T/eBYz1HiiKgiEvLyUOfPP5H3YGE8Z2fggWvn2snluNKypUUnWGaZhOrh4YG+ffuaZFubN2/G5MmTsXr1arRq1QrLli1DREQErly5Ai+vood6N27ciHfeeQdr165FmzZtcPXqVQwfPhwymQxLly41SUxERERUeeRr8/HGL28AAD6P+FziaMhcEhYmIC8+Dyp/FQLfD5Q6HKqgVI3GOLEqRp5ej1SNpuolV6a0dOlSvPbaaxgxYgQAYPXq1fj111+xdu1avPPOO0Xanzx5Es888wwGDx4MAAgMDMSgQYPwf//3fyX2kZ+fj/z8fMPtjIwMAAVTB6SePlDYv9RxEFHpOFaJLFeuOhffnP8GAPBRx48AcKxam9xruUhYVFDEIvCjQOht9dBreGmgykSr1Za73ZMevxXpz6KTK7VajdOnT2P69OmGZXK5HM8++yxOnTpV7Dpt2rTB999/jz///BMtW7ZEbGwsdu/ejZdffrnEfhYuXIg5c+YUWb5//344OFjGia8HDhyQOgQiKgeOVSLLk6fLM/z9+++/w05hx7FqTQTgMNcBSrUSmqYa/Kn6E9gtdVBUUTFyecE0wDIcP34ciU/4mro5OTnlbmvRyVVqaip0Ol2R4hfe3t64fPlysesMHjwYqampaNu2LYQQ0Gq1GD16NGbMmFFiP9OnT8fkyZMNtzMyMuDv749u3brBxcXFNA/mEWk0Ghw4cABdu3aFUqmUNBYiKhnHKpHlylZnA5EFf3fu3Bknj5zkWLUid3fcxeWzlyGzlaHV961gH2YvdUj0CM5mZRmdX1WStm3boqnTk70odOGstvKw6OTqURw+fBgLFizAypUr0apVK0RHR2PixIn44IMP8P777xe7jkqlgkqlKrJcqVRazAevJcVCRCXjWCWyPErxvzFZOD45Vq2DLluHuClxAIBab9eCS31pd4rToyusCFiedk967FakP7MmV2q1GnFxcQgJCSn3E/YgDw8PKBQK3Llzx2j5nTt34OPjU+w677//Pl5++WWMGjUKABAeHo7s7Gy8/vrrePfddx+pWiERERERWZ7r864j/0Y+VAEq1JpeS+pw6DGcycyUOgSTMEumkZOTg1dffRUODg5o0KABEhIKTjAcP348Pvzww3Jvx9bWFs2aNcPBgwcNy/R6PQ4ePIjWrVuX2PfDCZRCoQAACCEq+lCIiIiIyAJlX87GjSU3AABhn4ZB4aCQOCJ6VN8mJeG/V69KHYZJmCW5mj59Os6fP4/Dhw8bXVvq2WefxebNmyu0rcmTJ2PNmjX45ptvEBUVhf/+97/Izs42VA985ZVXjApe9OrVC6tWrcIPP/yAuLg4HDhwAO+//z569eplSLKIiIiIqPISQuDauGsQGoHqPavDo7eH1CHRI9ALgfdiYzHs8mVoUXZiYieXw8PCp/OaZVrgzp07sXnzZjz99NNGV8Zu0KABYmJiKrStAQMGICUlBTNnzkRSUhKaNGmCvXv3GopcJCQkGB2peu+99yCTyfDee+/h1q1b8PT0RK9evTB//nzTPDgiIiKqVByUDkiekmz4myq/lK0puH/wPuR2coQuD5U6HHoEOTodhl2+jG0pKQCA6bVq4XVfX9z7tyS7VqvF8ePH0bZtW8PpRR5KpUVf4wowU3KVkpJS7AV+s7OzjZKt8ho3bhzGjRtX7H2HDx82um1jY4NZs2Zh1qxZFe6HiIiIrI9MJoOnoycAXt/KGmgztYh+MxoAUGt6LdgHszpgZZOUn4/eFy7gr8xMKGUyrKlTB8P+racQ+G8bjUaDRL0eTZ2cKlXxGbNMC2zevDl+/fVXw+3ChOqrr74q8VwpIiIiIqKyxM+Jh/q2GnYhdvB/21/qcKiCzmdloeWZM/grMxPuNjb4rXFjQ2JlDcxy5GrBggXo3r07Ll26BK1Wi+XLl+PSpUs4efIkjhw5Yo4uiYiIiIqVr83H5H0F17Nc1HmRxNHQ48i6kIWby24CAMI+C4PCjufTVya/pKZiUFQUsnQ61La3x6/h4Qh1sK6pumY5ctW2bVucO3cOWq0W4eHh2L9/P7y8vHDq1Ck0a9bMHF0SERERFUur12Ll3yux8u+V0Oq1UodDj0gIgWtjrwE6wKOvB6p3ry51SFROQggsu3EDfS5cQJZOh85ubvjjqaesLrECzHidq5CQEKxZs8ZcmyciIiKiKiR5YzLSj6ZDbi9H6DIWsagsNHo9JkRHY/Xt2wCA13x9sSIsDEorvfasWS8inJycjOTkZOj1eqPljRo1Mme3RERERGRFtOlaRL9VUMQi4L0A2AVYdsU4KnBfo0H/S5dwIC0NMgAfhYRgsp/fIxW4qyzMklydPn0aw4YNQ1RUVJEL98pkMuh0OnN0S0RERERWKG5WHDR3NLCvbQ//t1jEojKIzc1Fz8hIROXkwEEux8b69dHHw/qvR2aW5GrkyJGoXbs2vv76a3h7e1t1dkpERERE5pN1Pgu3PrsFAAj7PAxylXVOJ7MmJ9LT0ffCBaRqNKhpa4td4eFo6uwsdVhPhFmSq9jYWPz4448IDeV8WCIiIiJ6NEIvcHXMVUAPePbzhHtXd6lDojJsuHMHIy9fhloINHNyws/h4aihUkkd1hNjltS/S5cuOH/+vDk2TURERERVRNK3Scg4mQG5oxwhS0OkDodKoRcCM+PiMDQqCmoh8LyHB440bVqlEivATEeuvvrqKwwbNgwXLlxAw4YNi1xVuXfv3uboloiIiKgIe6U94ibGGf6mykGTpkHs27EAgMBZgbDzYxELS5Wr02H45cvYkpICAJjm748FwcGQV8FTg8ySXJ06dQonTpzAnj17itzHghZERET0JMllcgS6BQIANBqNtMFQucW9FwdNigYO9R3gN8lP6nCoBHfUavSJjMT/ZWbCRibDF7VrY6Svr9RhScYs0wLHjx+PoUOHIjExEXq93ugfEysiIiIiKk3m6UzcXlVwXaSwFWGQK1nEwhJFZmWh5enT+L/MTFSzscGBRo2qdGIFmOnI1d27d/Hmm2/C29vbHJsnIiIiKje1To13D74LAJjdfra0wVCZDEUsBOA12AvVOlaTOiQqxu67dzHw0iVk6nQIs7fHL+HhqO3gIHVYkjPLboAXXngBhw4dMsemiYiIiCpEo9Pg41Mf4+NTH0Oj47RAS5f4dSIy/8yEwlmBkI9ZxMISfXbzJnpFRiJTp0MHV1f88dRTTKz+ZZYjV7Vr18b06dNx/PhxhIeHFyloMWHCBHN0S0RERESVmOauBrHv/FvEYm4gVL5Vq9KcpdPq9ZgUHY0VtwumbI708cGq2rVhK+e0zUJmqxbo5OSEI0eO4MiRI0b3yWQyJldEREREVETs9Fho72nhGO6ImuNqSh0OPSBdq8WAixexLy0NMgAfBgdjqr8/ZFWwImBpzJJcxcXFmWOzRERERGSlMv4vA4lfJQIAwlaGQW7DoyGWIi43Fz0jI3EpJwf2cjk21KuH5z09pQ7LIpkluSIiIiIiKi+h+18RC+9h3nBr6yZ1SPSvU+np6HPhAlI0GtSwtcXP4eFo5uwsdVgWy2TJ1eTJk/HBBx/A0dERkydPLrXt0qVLTdUtEREREVVyt7+4jawzWVC4KhCyiEUsLMWmO3cw4vJl5AuBpk5O2BUejpoqngdXGpMlV2fPnjVcmO/s2bMltuO8TCIiIiIqpE5WI+7dglNKgucHw9bbVuKISAiBudevY3Z8PACgT/Xq+L5ePTjZcNJbWUz2DB06dAjffvstBgwYwDLsREREZDHslfa48N8Lhr/JssROi4X2vhZOTZ1QY3QNqcOp8vJ0Ooy8cgWbkpMBAFP8/fFhcDAUPEBSLiY9U3DEiBFIT0835SaJiIiIHotcJkcDrwZo4NUAchmLJFiS9BPpSFqfBKCgiIVMwR/wUkpWq9H5/HlsSk6GjUyGNbVr46OQECZWFWDSY3tCCFNujoiIiIislF6rLyhiAcB3lC9cn3aVOKKq7WJ2NnpGRiI+Lw9uNjb4sUEDdK5WTeqwKh2TT5zkOVVERERkSdQ6NRYcWwAAmPr0VImjoUK3V9xG9j/ZsHG3QdDCIKnDqdL23buH/hcvIkOnQ4idHX5t1Ah1HBykDqtSMnly1aVLF9iUcbLbmTNnTN0tERERUbE0Og3mHJkDAJjUYpK0wRAAID8xH3Ez/y1isTAYth4sYiGVFbduYcK1a9ADaO/qiu0NG6K6Uil1WJWWyZOriIgIODk5mXqzRERERGQlYqbGQJehg3MLZ/i+6it1OFWSVq/H5JgYfHbrFgBgmLc3vqhTByo5z0t8HCZPrqZOnQovLy9Tb5aIiIiIrMD9I/eRvCEZkLGIhVQytFoMvHQJe+7dAwAsDArCtFq1eHqPCZg0ueILQkREREQl0Wv0uDq2oIhFjTdqwKW5i8QRVT3X8/LQMzISF7KzYS+X47t69fCip6fUYVkNVgskIiIioifi1qe3kHMxB0oPJYLms4jFk/ZHejr6XLiAZI0GPra2+LlhQ7RwYYJrSiZNruLi4uDJzJeIiIiIHpJ/Kx/xs+MBAMGLg6F0Z9GEJ2lzcjKGRUUhXwg0dnTErvBw+NvZSR2W1TFpchUQEGDKzRERERGRlYieHA1dlg4urV3gM8xH6nCqDCEE5l2/jpnx8QCAntWrY1O9enAqo7o3PRo+q0RERGTV7Gzs8OeoPw1/05N377d7SNmSAsj/LWIh53n6T0KeTodRV65gQ3IyAOBNPz98FBICBeskmA2TKyIiIrJqCrkCLWq2AABoNBqJo6l69Pl6XBt3DQBQc2xNODdxljiiqiFFrcbzFy7gREYGFABW1K6NN2rUkDosq8fkioiIiIjM5sYnN5B7JRdKbyUC5wZKHU6VcCk7Gz0jIxGXlwdXhQLbGjTAs+7uUodVJZgsufrnn3/K3bZRo0am6paIiIioVGqdGsv/WA4AGNNsjMTRVC15CXm4/sF1AEDIRyFQurGIhbkduHcP/S5eRLpOh2A7O/wSHo56jo5Sh1VlmCy5atKkCWQyWYnl2Avvk8lk0Ol0puqWiIiIqFQanQZv//Y2AOC1Jq9JHE3VEv1mNPQ5eri2d4X3UG+pw7F6q2/dwrhr16AD0NbVFTsaNICHra3UYVUpJkuu4uLiTLUpIiIiIqrk7u69i9TtqYACCFsRBhmLKJiNTghMiYnBsps3AQAve3tjTZ06UMnlEkdW9ZgsuWIZdiIiIiICAF2ezlDEwm+iH5waOkkckfXK1GoxOCoKv9y9CwCYFxSEGbVqMZmViFkLWly6dAkJCQlQq9VGy3v37m3ObomIiIhIQjc+uoG8mDzY1rBF4OxAqcOxWgl5eegVGYl/srNhJ5fjm7p10d/LS+qwqjSzJFexsbF4/vnnERkZaXQeVmEGzXOuiIiIiKxTblwuEhYkAABCloTAxpnFqc3hz4wM9I6MxB2NBt5KJX4KD0crFxepw6ryzDIRc+LEiQgKCkJycjIcHBxw8eJFHD16FM2bN8fhw4fN0SURERERWYDoidHQ5+nh1tkNXgN4FMUctiUno8O5c7ij0SDc0RH/16wZEysLYZZdCadOncLvv/8ODw8PyOVyyOVytG3bFgsXLsSECRNw9uxZc3RLRERERBJK3ZWKu7vuQqaUIexzFrEwNSEEFiYk4N1/C8n1cHfHpvr14WzDo4OWwiyvhE6ng7NzwdW3PTw8cPv2bdSpUwcBAQG4cuWKObokIiIiKpadjR0ODTtk+JvMQ5erQ/SEaACA32Q/ONbjtZVMKV+vx+tXruDbO3cAABNr1sSS0FAomMBaFLMkVw0bNsT58+cRFBSEVq1aYfHixbC1tcWXX36J4OBgc3RJREREVCyFXIGOgR0BABqNRtpgrFjCwgTkxedB5adCwHusIm1KqWo1Xrh4EcfS06EA8FlYGP5bs6bUYVExzJJcvffee8jOzgYAzJ07Fz179kS7du1QvXp1bN682RxdEhEREZFEcqJzkLCooIhF6LJQ2DhxmpqpXM7ORs/ISMTk5cFFocDWBg3Qzd1d6rCoBGYpaBEREYEXXngBABAaGorLly8jNTUVycnJ6Ny5c4W3t2LFCgQGBsLOzg6tWrXCn3/+WWr7+/fvY+zYsfD19YVKpULt2rWxe/fuR3osREREVLlpdBqs+HMFVvy5Ahodj1yZmhAC0eOjIdQC1SKqweMFD6lDshoH09LQ+uxZxOTlIcjODqeeeoqJlYUzS3L1/fffG45cFXJ3d3+kkxo3b96MyZMnY9asWThz5gwaN26MiIgIJCcnF9terVaja9euiI+Px7Zt23DlyhWsWbMGNXnolIiIqEpS69QYt2ccxu0ZB7VOXfYKVCGpO1Jxb+89yGxlCPuMRSxM5cvbtxFx/jzua7Vo4+KC/3vqKdR35Hlsls4sydWbb74Jb29vDB48GLt3736s61otXboUr732GkaMGIH69etj9erVcHBwwNq1a4ttv3btWty7dw87d+7EM888g8DAQHTo0AGNGzd+5BiIiIiIqChdtg7RkwqKWNR6uxYcwhwkjqjy0wmBt6Kj8cbVq9ABGOLlhYONG8PT1lbq0KgczDIhNjExEXv37sWmTZvQv39/ODg4oF+/fhgyZAjatGlT7u2o1WqcPn0a06dPNyyTy+V49tlncerUqWLX+fnnn9G6dWuMHTsWP/30Ezw9PTF48GBMmzYNCoWi2HXy8/ORn59vuJ2RkQGg4KRXqU98Lexf6jiIqHQcq0SW68FxybFqWvFz4pF/Ix+qABV8p/jyeX1MWTodXrl6Fb/cuwcAmFWrFmb4+UGm10Oj10sc3ZNlSWO1IjGYJbmysbFBz5490bNnT+Tk5GDHjh3YuHEjOnXqBD8/P8TExJRrO6mpqdDpdPD29jZa7u3tjcuXLxe7TmxsLH7//XcMGTIEu3fvRnR0NMaMGQONRoNZs2YVu87ChQsxZ86cIsv3798PBwfL2ANz4MABqUMgonLgWCWyPHm6PMPfv//+O+wUdhyrJiC/KYfTJ06QQYZ7Q+5h3+F9UodUqaXKZJjv6Ig4hQJKITAhNxdNIyOxJzJS6tAkZQljNScnp9xtzV7KxcHBAREREUhLS8P169cRFRVl1v70ej28vLzw5ZdfQqFQoFmzZrh16xY++uijEpOr6dOnY/LkyYbbGRkZ8Pf3R7du3eAi8dWuNRoNDhw4gK5du0KpVEoaCxGVjGOVyHJlq7OBf3+fdu7cGSePnORYfUxCCFzsfhHp2nRUe64a2sxqw3OtHsPpzEz8NyoKiRoNvJRK/FivHlr9e83YqsqSvlcLZ7WVh9mSq8IjVhs2bMDBgwfh7++PQYMGYdu2beXehoeHBxQKBe78e7G0Qnfu3IGPj0+x6/j6+kKpVBpNAaxXrx6SkpKgVqthW8x8VZVKBZVKVWS5UqmU/MUsZEmxEFHJOFaJLI9S/G9MFo5PjtXHk7wlGem/p0NuJ0ftz2oX+/uKymd7SgqGRkUhV69HQ0dH7GrYEIH29lKHZTEsYaxWpH+zFLQYOHAgvLy88OabbyI4OBiHDx9GdHQ0PvjgA9StW7fc27G1tUWzZs1w8OBBwzK9Xo+DBw+idevWxa7zzDPPIDo6GvoH5qVevXoVvr6+HPhEREREj0mbqUX0m/8WsZheC/bBTAQehRACH16/jhcvXkSuXo//uLvjRNOmTKwqObMcuVIoFNiyZQsiIiJKLCJRXpMnT8awYcPQvHlztGzZEsuWLUN2djZGjBgBAHjllVdQs2ZNLFy4EADw3//+F59//jkmTpyI8ePH49q1a1iwYAEmTJjw2I+LiIiIKh+VjQq/DPrF8Dc9nutzr0N9Ww27EDv4v+0vdTiVklqvx+irV7EuKQkAMK5mTXwSEgIbuVmOe9ATZJbkasOGDSbb1oABA5CSkoKZM2ciKSkJTZo0wd69ew1FLhISEiB/4I3o7++Pffv24c0330SjRo1Qs2ZNTJw4EdOmTTNZTERERFR52Mht0KN2DwCWUXmsMsu+mI2by24CAMI+C4PC7vF2oldFdzUavHDhAo6mp0MOYHloKMb5+UkdFpmISZOr5557Dps2bYKrqysA4MMPP8To0aPh5uYGALh79y7atWuHS5cuVWi748aNw7hx44q97/Dhw0WWtW7dGn/88UeF+iAiIiKikgkhcHXMVQitgEdfD1TvXl3qkCqdqzk56BEZiejcXDgrFNhSvz7+U53PozUx6bHHffv2GV0vasGCBbj3b51+ANBqtbhy5YopuyQiIiIqlUanwfpz67H+3HpodDxy9aiSNyYj/Wg65PZyhC4LlTqcSudQWhqePnMG0bm5CFCpcLJpUyZWVsikR66EEKXeJiIiInrS1Do1RvxUcK5237C+0gZTSWnTtYh+q6CIRcB7AbALsJM4osrl68REjL56FVoh8LSLC3Y2bAhvFlqzSma/zhURERERVW5xs+KguaOBfW17+L/FIhblpRcC78TG4qMbNwAAA728sLZOHdg/ZsE3slwmTa5kMlmRC8jxgnJERERElVfW+Szc+uwWACDs8zDIVaxoVx7ZOh2GRkVhZ2oqAGBWQABmBQbyt7GVM/m0wOHDhxsuyJuXl4fRo0fD0dERAIzOxyIiIiIiyyb0AlfHXgX0gGc/T7h3dZc6pErhVn4+ekVG4mxWFmxlMqyrWxeD/610TdbNpMnVsGHDjG4PHTq0SJtXXnnFlF0SERERkZkkfZuEjBMZkDvKEbI0ROpwKoUzmZnoFRmJ22o1PJVK7GzYEG3+raRN1s+kydW6detMuTkiIiIikogmTYPYt2MBAIGzAmHnxyIWZdmZkoIhUVHI0etR38EBv4SHI8jeXuqw6AliQQsiIiIiKiLuvThoUjRwqO8Av0m8yG1phBD4+MYNTIuNhQDQrVo1bGnQAK42/Kld1fAVJyIiIqumslFhy0tbDH9T2TJPZ+L2qtsAgLAVYZArWcSiJGq9HmOuXsXXSUkAgDE1amB5aChs5HzOqiImV0RERGTVbOQ26NegHwBAo+FFhMsi9AJXx1wFBOA12AvVOlaTOiSLdU+jwUsXL+LQ/fuQA/gkNBTja9ZkRcAqjMkVERERERkkfp2IzD8zoXBWIORjFrEoybWcHPSMjMTV3Fw4KRT4oX599KheXeqwSGJMroiIiMiqafVa7IjaAQDoGdpT4mgsm+auBrHv/FvEYm4gVL6cRlmcI/fv44ULF3BPq0UtlQq/hIcj3MlJ6rDIAjC5IiIiIquWr81H/239AQBpU9IkjsayxU6PhfaeFo7hjqg5rqbU4VikdYmJeOPqVWiEQEtnZ/zUsCF8VExCqQCTKyIiIiJCxv9lIPGrRABA2MowyG1YkOFBeiEwIzYWi27cAAD09/TE+rp1Ya9QSBwZWRImV0RERERVnND9r4iF9zBvuLV1kzoki5Kj0+HlqChsT00FALwXEIA5gYGQs3AFPYTJFREREVEVd/uL28g6kwWFqwIhi1jE4kG38/PROzISp7OyYCuT4as6dfCyj4/UYZGFYnJFREREVIWpk9WIezcOABA8Pxi23rYSR2Q5zmVmoteFC7iZn4/qNjbY2bAh2rq5SR0WWTAmV0RERERVWOy0WGjva+HU1Ak1RteQOhyL8XNqKgZfuoRsvR51HRzwS3g4QuztpQ6LLByTKyIiIqIqKv1EOpLWJwEoKGIhU/AcIiEElt68iakxMRAAnq1WDVvr14ebUil1aFQJMLkiIiIiq2arsMW6PusMf1MBvVZfUMQCgO8oX7g+7SpxRNLT6PUYd+0avkwsqJr4hq8vPgsLg1LOyolUPkyuiIiIyKopFUoMbzIcAKDRaKQNxoLcXnkb2f9kw8bdBkELg6QOR3JpGg36XbyIg/fvQwZgSUgIJvn5QcaKgFQBTK6IiIiIqpj8xHzEvf9vEYuFwbD1qNpH9GJyc9Hjn39wJTcXjnI5NtWvj14eHlKHRZUQkysiIiKyalq9Fvui9wEAOgd0ljgayxAzNQa6DB2cWzjD91VfqcOR1LH79/H8hQu4q9XCT6XCL+HhaOzkJHVYVEkxuSIiIiKrlq/NR89NPQEAaVPSJI5GeveP3EfyhmRAxiIW3yYlYdSVK9AIgebOzvi5YUP4qlRSh0WVGJMrIiIioipCr9Hj6tiCIhY13qgBl+YuEkckDb0QeD8uDgsSEgAAL3p44Nt69eCgUEgcGVV2TK6IiIiIqohbn95CzsUcKD2UCJpfNYtY5Oh0GHb5MralpAAAZtSqhQ+CgiBn4QoyASZXRERERFVA/q18xM+OBwAELwqG0r3qXbcpKT8fvS9cwF+ZmVDKZFhTpw6G+fhIHRZZESZXRERERFVA9FvR0GXp4NLaBT7Dq15CcT4rC70iI3EjPx/uNjbY0bAh2ru5SR0WWRkmV0RERERWLu1gGlI2pwDyf4tYyKvWFLhfUlMxKCoKWTod6tjb45fwcIQ6OEgdFlkhJldEREREVkyv/l8Ri5pja8K5ibPEET05Qggsv3kTb8XEQA+gs5sbtjVogGrKqjclkp4MJldERERk1WwVtvi8++eGv6uaG0tvIPdKLpTeSgTODZQ6nCdGo9djQnQ0Vt++DQB4zdcXK8LCoJTLJY6MrBmTKyIiIrJqSoUSY1uOBQBoNBqJo3my8hLycP2D6wCAkI9CoHSrGkds7ms06H/pEg6kpUEG4KOQEEz284OMFQHJzJhcEREREVmp6Dejoc/Rw7WdK7yHeksdzhMRm5uLnpGRiMrJgYNcjo3166OPh4fUYVEVweSKiIiIrJpOr8OxhGMAgKd9n5Y4mifn7t67SN2eCiiAsBVhVeKozYn0dPS9cAGpGg1q2tpiV3g4mjpXnXPMSHpMroiIiMiq5Wnz0OmbTgCAtClpEkfzZOjydLg27hoAwG+iH5zCnSSOyPw23LmDkZcvQy0Emjk54efwcNRQqaQOi6oYJldEREREVubGRzeQF5MH2xq2CJwdKHU4ZqUXArPj4/HB9YJzy5738MB39erBUaGQODKqiphcEREREVmR3LhcJCxIAACELAmBjbP1/tzL1ekw/PJlbElJAQBM8/fHguBgyKvAFEiyTNY72oiIiIiqoOiJ0dDn6eHW2Q1eA7ykDsds7qjV6BMZif/LzISNTIYva9fGCF9fqcOiKo7JFREREZGVSN2Viru77kKmlCHsc+stYhGZlYWekZFIyM9HNRsbbG/QAB2rVZM6LCImV0RERETWQJerQ/SEaACA32Q/ONZzlDgi89h99y4GXLqELJ0OYfb2+CU8HLUdHKQOiwgAkysiIiIiq5CwMAF58XlQ+akQ8F6A1OGYxWc3b2JSdDT0ADq6ueHHBg3grqwaF0amyoHJFREREVk1pUKJxc8uNvxtjXKic5CwqKCIReiyUNg4WddPPK1ej0nR0Vhx+zYAYKSPD1bVrg1buVziyIiMWdfIIyIiInqIrcIWU5+ZCgDQaDQSR2N6QghEj4+GUAtUi6gGjxc8pA7JpNK1Wgy4eBH70tIgA/BhcDCm+vtb7flkVLkxuSIiIiKqxFJ3puLe3nuQ2coQ9pl1FbGIy81Fz8hIXMrJgYNcju/r1cPznp5Sh0VUIiZXREREZNV0eh3OJJ4BAIR7hEscjWnpsnWInlhQxKLW27XgEGY9hR1Opaejz4ULSNFoUMPWFrvCw/GUs7PUYRGViskVERERWbU8bR5aftUSAJA2JU3iaEzr+rzryL+RD1WACrWm15I6HJPZdOcORly+jHwh0NTJCbvCw1FTpZI6LKIyVYqzAFesWIHAwEDY2dmhVatW+PPPP8u13g8//ACZTIa+ffuaN0AiIiKiJyz7cjZuLLkBAAj7NAwKB4XEET0+IQTmxMdjcFQU8oVAn+rVcbRJEyZWVGlYfHK1efNmTJ48GbNmzcKZM2fQuHFjREREIDk5udT14uPjMWXKFLRr1+4JRUpERET0ZAghcG3cNQiNgHsPd1TvVV3qkB5bnk6HIVFRmB0fDwCY6u+PHxs2hJMNJ1pR5WHxydXSpUvx2muvYcSIEahfvz5Wr14NBwcHrF27tsR1dDodhgwZgjlz5iA4OPgJRktERERkfilbU3D/4H3IVDKEfVr5i1gkq9XofP48NiUnw0Ymw5ratbE4JASKSv64qOqx6F0BarUap0+fxvTp0w3L5HI5nn32WZw6darE9ebOnQsvLy+8+uqrOHbsWJn95OfnIz8/33A7IyMDQEG5VqlLthb2L3UcRFQ6jlUiy/XguLSGsarN1CL6zYIiFn5v+8HG36ZSP56LOTl4/tIlxOfnw02hwOa6ddHJza1SPyZ6fJY0VisSg0UnV6mpqdDpdPD29jZa7u3tjcuXLxe7zvHjx/H111/j3Llz5e5n4cKFmDNnTpHl+/fvh4ODZVTdOXDggNQhEFE5cKwSWZ48XZ7h799//x12CrtKPVbt1ttBdVsFnY8OF8Iv4MLuC1KH9MjO2tjgIwcH5Mhk8NHp8H5mJnJPnsRuqQMji2EJYzUnJ6fcbS06uaqozMxMvPzyy1izZg08PMp/Ab3p06dj8uTJhtsZGRnw9/dHt27d4OLiYo5Qy02j0eDAgQPo2rUrlErrvKo8kTXgWCWyXNnqbCCy4O/OnTvj5JGTlXas5lzMwblfzkFAIPzLcFT7TzWpQ3pkqxIT8UFsLPQA2rm4YEvduqheCV8TMg9L+l4tnNVWHhadXHl4eEChUODOnTtGy+/cuQMfH58i7WNiYhAfH49evXoZlun1egCAjY0Nrly5gpCQkCLrqVQqqIqpQqNUKiV/MQtZUixEVDKOVSLL4yB3wKwOswr+tiuYkVIZx6oQArGTYiG0Ah59PeDVy0vqkB6JVq/H5JgYfHbrFgBguI8PvqhdG7Zyiy8FQBKwhLFakf4tOrmytbVFs2bNcPDgQUM5db1ej4MHD2LcuHFF2tetWxeRkZFGy9577z1kZmZi+fLl8Pf3fxJhExERkQWxVdhidsfZACzj/I1HlbwxGelH0iG3lyN0WajU4TySDK0WAy9dwp579wAAC4OCMK1WrUpfkIOokEUnVwAwefJkDBs2DM2bN0fLli2xbNkyZGdnY8SIEQCAV155BTVr1sTChQthZ2eHhg0bGq3v5uYGAEWWExEREVUW2nQtot8qKGIR8F4A7ALsJI6o4q7n5aFnZCQuZGfDXi7Hd/Xq4UVPT6nDIjIpi0+uBgwYgJSUFMycORNJSUlo0qQJ9u7dayhykZCQADkPIxMREVEJ9EKPqJQoAECoW+U84hM3Kw6aOxrY17aH/1uVbybOH+np6HPhApI1GvjY2mJXw4ZoLvF57UTmYPHJFQCMGzeu2GmAAHD48OFS112/fr3pAyIiIqJKI1eTi4arCmawpE1Jkziaiss6n4VbnxWcnxT2eRjkqsq1U3lzcjKGRUUhXwg0dnTErvBw+NtVviNvROVRKZKr/2/vvuOjKtP+j39mkplJD4RIQkmhBJAuIIhiYQVBEWVVZFEfEVnLuq6rPDb2UXBtWFBwXV0su2DDAioiukpRVJCfioKGTigJkF5IJZPJzP37IzAQSAiBIZmE7/v1mldOzrnPOdcM3JO55tz3dUREREROR8Zj2PrnreCBM8aeQdTwqMYO6bgZY3g8NZWpu3YBcHmrVrx75pmEBerjpzRf+t8tIiIi4qcy38ykaFUR1lArnZ4/uuKxvyp3u/njli28k50NwOT27XmmUycCVLhCmjklVyIiIiJ+yFXgYsf9OwBInJZIUPumMZQup6KC369fz6qiIgKAl7t04da2bRs7LJEGoeRKRERExA/tfGgnrhwXId1DaH93+8YO57hsLC3l8uRkdpaXExkQwIIePRgW1XSGMoqcLCVXIiIiIn6m+Odi0v+VDkDSS0lYbf5fxGJpfj5jN2yg0O2mY1AQi3v14szQ0MYOS6RBKbkSERER8SPGY9h6x1Yw0Pq61rS8qGVjh1Sn2Xv3cue2bbiBIZGRfNyjB9F2e2OHJdLglFyJiIhIs2YLsHHv4Hu9y/4u498ZFP9YTEB4AJ1m+HcRC7cx3Lt9O7P27AHgf2JieK1rVxy6B6mcppRciYiISLNmD7Dz7CXPAuByuRo5mmNz5bnY8eCBIhaPJuJo42jkiGpXXFnJdZs2sTgvD4DHO3Tgb/HxWFQRUE5jSq5ERERE/MSOKTuozK8ktFco7e5s19jh1CqtvJzRycn8VlpKkNXKG926cW3r1o0dlkijU3IlIiIizZrHeEgrTAOgTUibRo6mdkU/FJHxegYASS8nYQ30z6F1PxYVcUVyMlkuFzE2G4t69WJgRERjhyXiF5RciYiISLO237WfDi90AKDg3oJGjqZmxn2oiEXMhBhaDGnR2CHVaEF2Nv+zeTPlHg+9QkNZ3KsX8UFN4/5bIg1ByZWIiIhII0t/NZ2SX0oIiAyg09P+V8TCGMP0tDT+b+dOAEZFRfFu9+6EB+qjpMjh1CNEREREGlFFdgU7/1aVtHR8oiP2GP8qYe70eLh1yxbezMoC4O727ZnRqRMBKlwhchQlVyIiIiKNaMcDO6jcV0nYWWG0vb1tY4dTTW5FBVdt2MB3hYUEAP9MSuL2dv5baEOksSm5EhEREWkkhasKyZybCVQVsbAE+M/VoM2lpVyenMz28nIiAgKY36MHl0RFNXZYIn5NyZWIiIhII/BUeqqKWACxk2KJPCeykSM6ZHlBAdds2MC+yko6BAWxuFcvuoeGNnZYIn5PyZWIiIhII0h/OZ3S30oJjAqk41MdGzscr1fT07lj61bcwLkRESzs2ZMz7P41D0zEXym5EhERkWYt0BrIHQPu8C77A2eGk50PHyhiMb0j9ujGT17cxnD/9u08v2cPANe3bs3rXbsSFBDQyJGJNB3+8Q4jIiIicoo4Ah28NOolAFwuVyNHU2XH/TtwF7kJPzucNpMa/8bGJZWVXL9pE4vy8gB4NDGRhxISsKgioEi9KLkSERERaUD7vtlH1ttZYPGPIhZ7yssZvX4960pKcFgszO3WjT/ExDRqTCJNlZIrERERadaMMeSW5QIQaWvcohEel4etf64qYtH2trZEDIho1HjWFBVxxfr1ZFRU0Npm45OePTkn0n8Ka4g0NUquREREpFkrc5XRekZrAAruLWjUWPb+Yy9lG8qwRdvo8ESHRo3lo5wcbti0if0eDz1DQ/m0Z08Sg4MbNSaRpk7JlYiIiEgDcO51suuRXQB0fLojtihbo8RhjOHptDSm7KwqqDEyKor3u3cnIlAfC0VOlnqRiIiISANI+d8U3CVuIgZHEHtTbKPEUOHxcPvWrczJrLpx8Z3t2jGzUycCrdZGiUekuVFyJSIiInKKFSwvIOf9HLAeKGJhbfgiFnkuF1etX8+3hYVYgRc6d+bO9u0bPA6R5kzJlYiIiMgp5Kk4VMSi3Z/bEd43vMFj2FpWxqjkZFL27yc8IIAPundnZKtWDR6HSHOn5EpERETkFNr9/G72b9mPLcZG4qOJDX7+rwsKuHrDBgoqK0lwOFjcqxc9w8IaPA6R04GSKxEREZFTpDytnNTHUgHo9GwnbC0atojFvzMyuH3rViqN4ZyICBb27EmM3d6gMYicTpRciYiISLMWaA1kQp8J3uWGlHJPCp4yD5HnRxJzQ8PdmNdjDA/u2MGzu3cDML51a/7TtStBAQENFoPI6UjJlYiIiDRrjkAHc8fMBcDlcjXYefO+yCP3o1wIgKSXkrBYGqaIRanbzQ2bNrEwt+rGydMSEpiWmNhg5xc5nSm5EhEREfExd7mbbXduA6D9X9sT1qth5jjtdToZnZzM2pIS7BYLc7p147qYhrtiJnK6U3IlIiIizZoxhjJXGQA2GmbO0+5nd1O+vRx7GzuJ0xIb5Jy/FBczOjmZ9IoKzrDZWNizJ+dGRjbIuUWkipIrERERadbKXGWETa+6clRwb8EpP9/+nftJezINgE7PdyIw4tR/3FqYk8P1mzZR5vHQPSSExb160SE4+JSfV0SqU3IlIiIi4kMpf03BU+6hxe9a0Hpc61N6LmMMM3bv5oEdOzDAJS1b8kGPHkQG6iOeSGNQzxMRERHxkdxPc8n7NA+LzULSP09tEYsKj4c7tm7l35mZANzRti0vdO5MoNV6ys4pIsem5EpERETEB9z73aT8NQWA9pPbE3pm6Ck7V77LxTUbNvD1vn1YgVmdO3Nnu3aqCCjSyJRciYiIiPhA2vQ0yneW42jvIOGhhFN2nm1lZVyenMzW/fsJCwjg/e7duaxVq1N2PhE5fkquRERERE5SWUoZaU9XFbHoPKszgWGn5iPWN/v2cdX69eRXVhLvcLC4Vy96hTVMmXcRqZuSKxEREZGTYIwh5S8pmApDy0taEn1V9Ck5z5yMDG7buhWXMQwKD2dhz57EOhyn5FwicmKUXImIiEizFmAN4Jru13iXfS13YS75X+RjsZ+aIhYeY/jbjh08vXs3ANeecQZzu3UjOMD3z0VETo6SKxEREWnWggKDmD92PgAul8unx3aXHipiEX9/PCFJIT49fpnbzf9s2sRHubkAPJSQwN8TE7GqcIWIX1JyJSIiInKCUp9IxbnbiSPBQfyUeJ8eO93p5IrkZH4uKcFusfB61678T2ysT88hIr6l5EpERETkBJRuLmX3jKqhekn/SCIgxHfD9NYWFzM6OZm9FRVE22x83KMHQ1q08NnxReTUUHIlIiIizVppRSlh06sq6hXcW+CTYxpj2HbnNozLEDUqilajfVcKfVFuLtdt3Eipx0O3kBA+69WLjsHBPju+iJw6Sq5ERERE6ilnfg77lu/D4rCQ9A/fFLEwxvD8nj3ct307BhjWsiXzu3enhc128gGLSINQciUiIiJSD5XFlaTcU1XEImFKAsEdT/6qksvj4c5t23g1IwOA29q04cWkJGxW60kfW0QaTpPosS+99BKJiYkEBQUxaNAgfvzxx1rbvvbaa5x//vm0bNmSli1bMmzYsGO2FxEREamP1EdTqUivIKhTEHEPxJ308QpcLi797TdezcjAAszs1Il/demixEqkCfL7Xvv+++8zefJkpk2bxi+//EKfPn0YMWIE2dnZNbZfsWIF48eP5+uvv2b16tXExcVxySWXsHfv3gaOXERERJqb0g2l7Jm1B4CkF5MICDq5Ihbb9+9n8C+/sHzfPkKtVhb17MndcXE+v1eWiDQMv0+unn/+eW655RYmTpxI9+7dmT17NiEhIfznP/+psf0777zDHXfcQd++fenWrRuvv/46Ho+H5cuXN3DkIiIi0pwYY9j6562YSkP0mGhaXXpyRSy+27ePQT//zJb9+2nvcLCqXz8uj472UbQi0hj8es5VRUUFP//8M1OmTPGus1qtDBs2jNWrVx/XMcrKynC5XERFRdXaxul04nQ6vb8XFRUBVTca9PXNBuvr4PkbOw4ROTb1VRH/dXi/PJm+mj0vm8JvCrEGW0l4NuGk+vtb2dncnpKCyxj6h4Xx0Zln0sZu13uIyAH+9He1PjH4dXKVm5uL2+0mJiam2vqYmBg2b958XMd44IEHaNu2LcOGDau1zfTp0/n73/9+1PolS5YQEuLbO62fqKVLlzZ2CCJyHNRXRfxPhaeC/hH9AVjx9QrsVnv9+2ophN8TjhUrZVeV8dWGr2BD/WPxAPMcDhYEBQFwrsvFX/fuZe3evayt/+FEmj1/+LtaVlZ23G39Ork6WU899RTvvfceK1asIOjAm1hNpkyZwuTJk72/FxUVeedqRURENESotXK5XCxdupThw4djUylWEb+lviri38YwBjjxvrrjf3eQUZBBUFIQg2cPxuqo/8yKMrebm7dt46O8PAAeaN+ev8fHY9X8KpGj+NPf1YOj2o6HXydX0dHRBAQEkJWVVW19VlYWsbGxx9x3xowZPPXUUyxbtozevXsfs63D4cDhcBy13mazNfo/5kH+FIuI1E59VaRpqE9fLfm1hIyXqkqkd3mpC46woz8z1CXT6eSKDRv4qbgYm8XCa127MqGOzzIi4h9/V+tzfr8uaGG32+nfv3+1YhQHi1MMHjy41v2eeeYZHnvsMb744gsGDBjQEKGKiIhIM2Q8VUUs8MAZY88ganjtc7hr82tJCQN/+YWfiouJCgxkWZ8+SqxEmim/vnIFMHnyZCZMmMCAAQMYOHAgs2bNorS0lIkTJwJw44030q5dO6ZPnw7A008/zdSpU5k3bx6JiYlkZmYCEBYWRlhYWKM9DxEREWkcpRWltJ7RGoC9f63frVky38ykaFUR1lArnZ7vVO9zL87NZfymTZS43XQNDmZxr1509pP53CLie36fXI0bN46cnBymTp1KZmYmffv25YsvvvAWuUhLS8N62E32/vWvf1FRUcE111xT7TjTpk3jkUceacjQRURExE+UuY5/QvpBrgIXO+7fAUDitESC2tc+f/tIxhhe2LOH/92+HQ/wuxYtWNCjBy01bFikWfP75Argzjvv5M4776xx24oVK6r9vmvXrlMfkIiIiDR7Ox/aiSvHRUj3ENrf3f6493N5PNyVksLs9HQAbmnThpeSkrBZ/Xo2hoj4QJNIrkREREQaUvHPxaT/qyo5SnopCavt+BKjfS4X127cyNKCAizAs506Mbl9eyyqCChyWlByJSIiInIYbxELA62va03Li1oe13479u/n8uRkNpWVEWq1Mq97d66Ijj7F0YqIP1FyJSIiInKYjH9nUPxDMQHhAXSacXxFLFYVFjJm/XpyXS7a2e182qsXZ4WHn+JIRcTfKLkSEREROcCV52LHgweKWDyaiKNN3fe0ejszk0lbtlBhDP3DwljUqxdta7h/pog0f0quREREpFmzWqxcmHChd/lYdkzZQWV+JaG9Qml3Z7tjtvUYw7Rdu3g8NRWAq6KjefPMMwkNCPBN4CLS5Ci5EhERkWYt2BbMiptWAOByuWptV/RDERmvZwCQ9HIS1sDaE7H9bjc3bd7MBzk5ADwQF8eTHTtiVeEKkdOakisRERE57Rm3YesdVUUsYibE0GJIi1rbZlVUcGVyMj8UFxNosfBqly5MbNOm4YIVEb+l5EpEREROe+mvplPySwkBkQF0err2IhbJJSVcnpxMmtNJVGAgH/bowUUtj6+aoIg0f0quREREpFkrrSgl8YVEALbdse2o7RXZFez8204AOj7REXuMvcbjfJ6Xx7iNGylxu0kKDuazXr1ICgk5ZXGLSNOj5EpERESavdyy3Fq37XhwB5X7Kgk7K4y2t7etsc2Le/Zwd0oKHuCiFi34sEcPomy2UxStiDRVSq5ERETktFW4qpDMOZlAVRELS0D1ghSVHg93p6TwUno6ADfHxvKvLl2wW49ddVBETk9KrkREROS05Kn0VBWxAGInxRJ5TmS17YWVlYzbsIEvCwqwAE937Mi9cXFYVBFQRGqh5EpEREROS+kvp1P6WymBLQPp+FTHatt27t/P5cnJbCwrI8Rq5Z0zz2TMGWc0UqQi0lQouRIREZHTTkVGBTsfPlDEYnpH7NGHilisLizkyvXryXG5aGu382mvXvQLD2+sUEWkCVFyJSIiIqeNwpWF2L61sfXFrbiL3ISfHU6bPx66R9W8rCxu3rwZpzGcFRbGp7160c7haMSIRaQpqddszLvuuovExEQsFgvr1q3zrt+2bRvnnnsuXbp04eyzz2bDhg3ebV988QUDBgygd+/enHPOOfz666/ebRMnTqR379707duXs88+m+XLl3u3XXTRRXTo0IG+ffvSt29fZs6cWW2/Ll260KdPH8477zx++umnGuNNT09nxIgRdO3ald69e3P11VeTc+BO6uXl5YwZM8Z7nOHDh5OSklKfl0NERESaAKvFSt/gvnTL6cbmyzcT8nwIhcsLAYi+JhpLgAVjDI/s3Mn1mzbhNIYrW7Xi2759lViJSL3U68rVNddcw/3338+QIUOqrb/tttu49dZbuemmm1iwYAE33XQTP/30EwUFBVx//fV8++239OjRg++++47rr7+e9evXAzBz5kxatGgBwNq1a7n44ovJzc3FeqACz8yZMxkzZsxRcfz+97/ntddeIzAwkMWLFzN27Fh27dp1VLuAgAAefvhhb7z33Xcf9913H3PnzgXg1ltv5dJLL8VisfDPf/6TP/7xj6xYsaI+L4mIiIj4uZJPS5j54EwwR2/b+eBOrJ2DeKBbHu9mZwNwX1wcT3XsiFWFK0Sknup15eqCCy6gffv21dZlZ2ezZs0abrjhBgCuvvpqdu/eTUpKCtu3b6dVq1b06NEDgPPPP5+0tDR++eUXAG9iBVBYWHjccVxxxRUEBlblheeccw579+6lsrLyqHYxMTHVEsFBgwZ5k7CgoCAuu+wyb8Wfc845p8YETURERJou4zasfHQbWzvD1qSjH2v6wbDCTbybnU2gxcJrXbrwTKdOSqxE5ISc9Jyr3bt306ZNGwIDA/niiy946KGHKCoq4oorruDVV18lLy+PVatWsXTpUl577TWKi4u58cYbvVevHnjgAV599VWKiopISEjg4osv5uuvvwZgwoQJOJ1OgoODiYuLY+bMmVx88cVAVaLkdDrJzMwkODgYm83Gr7/+Su/evY+K8a233mLGjBmkpKTQrl070tLSiI+PB6qGOi5atIjU1FTGjx9/si+HiIiINDBjDK4cF+Wp5ZSnluNMdXqXt2eU8IenK6ioY3RfOFYW9u7F71q2bJigRaRZ8llBi8OHAE6cOJHrrruO22+/nQULFjBhwgRyc3MZP3483377Lffdd593v7Zt2zJ06FBuueUWpk6dyhtvvAFUJUTh4eFERkby0ksv8fzzzzN27FjvsMEffviBt99+m8cee4zHH3+cF154ocbEavPmzdx3330MHz6cjh078vvf/54//elPfPbZZ0DVUMfg4GBmzZrFX/7yl2r7zp8/n5CQEK666iqWL19OYWEhrVu3ZuDAgSxevBiAfv364fF4vHPQrrzySlauXEleXh5RUVFccMEFLFy4EIDevXtjs9n4+eefARg1ahRr1qwhKyuLiIgILrnkEhYsWABAjx49CAsL4//9v/9HVlYWBQUFbN68mb179xIaGsrll1/O+++/D0DXrl2Jjo5m1apVAAwbNoytW7eSlpaGw+Hgqquu4v3338fj8dCpUyfatWvHt99+C1TNbUtLS2PHjh0EBgYyduxYPvzwQyoqKkhISKBTp0589dVXAAwZMoTs7Gy2bq26J8j48eP55JNPKCsro3379nTv3p0lS5YAMHjwYAoLC9m4cSMAY8eO5YsvvqC4uJjY2Fj69evH559/DsDZZ59NeXk5ycnJQNWwzxUrVlBQUEB0dDSDBw/m008/BeCss84CqoaRAowePZrVq1eTm5tLy5Ytueiii/j4448B6NWrF0FBQd45eZdddhm//PILmZmZhIeHM3LkSObPnw9A9+7diYyMZPXq1QBccsklbNy4kT179hASEsKVV17Ju+++C0CXLl1o3bo1K1euBOB3v/sd27dvJzU1FbvdztVXX838+fOprKykY8eOxMfHe4ebXnDBBezdu5ft27djtVoZN24cH330EU6nk/j4eLp06cKyZcsAOO+888jNzWXLli0AjBs3jsWLF1NaWkq7du3o2bMnX375JVD1ZUNJSYl3vuM111zDkiVLKCoqIiYmhgEDBnj/z/fv3x+Xy8Vvv/0GwJgxY/j222/Jz8+nVatWDBkyhE8++QSAvn37YrVavVebL7/8cn788Ueys7OJjIzk4osv5qOPPgKgZ8+ehISE8OOPPwJw6aWX8uuvv5Kenk5YWBiXXXYZH3zwAQDdunUjKiqK77//HoDhw4ezefNmdu/eTXBwMGPGjOG9997DGENSUhKxsbF89913AAwdOpRdu3axc+dObDYb11xzDQsWLMDlctGhQwcSExO9X9Ccf/75ZGZmsm3bNiwWC3/4wx9YuHAh+/fvJy4ujm7durF06VIAzj33XPLz89m8eTMA1157LZ9//jklJSW0bduWPn368N///heAgQMHUlZW5v2S6KqrrmLp0qXs2LGDFStWcM455zTYe8QPP/wAwIgRI1i/fr3eI/Qe0SzfIy4eejEb12wkbVcaNqeNwfsHs6xkGe79bqK2RxG0KYjUQakAJH6dyL7EfezrsA+r3Urgvu5ckZKMLf8Xdlr2sqvXbQzdnQHAt/HxtCkpISk/n76uYH530QWn7D2isT5H6D1C7xFN9T1i2LBhZGZmMn/+fMLDwxv1c8ThNSPqYjHG1DAC+dgSExNZuHAhffv2JTs7m86dO7NkyRJuvPFGtmzZQps2bVi5ciX9+vVjxYoVXHHFFXz11VckJCQQGxvLTz/9ROfOnQFo3749X331FV26dKFbt26888479O/f/6hz2u12QkNDycvLw2q18v777/PQQw+xfPlybrvtNkaMGMHdd9991H4LFixg8uTJ9OzZk4ULF1JSUkJ0dDQ5OTm0atWKGTNm8N5775Gdnc2iRYvo27cvRUVFREZGUlhYSERERH1fHp9yuVx8/vnnXHbZZdhstkaNRURqp74qcuLc+9040w5dbTry6pNzrxPcdRzEAvY2doISgghKCMKR4CAoIYh1rlIu77EdVl5W1W7I5xAQfNTuX5PERRe18/2TE5ET4k9/V+uTG5z0lavWrVvTr18/fvnlF/Ly8njyySdp3749GzdupLi4mJUrV5KVlcUnn3zCrFmzsFgs/PzzzyQkJLB+/XrvtjfeeIOUlBTWrVtHnz59yMvLIyYmhgcffJC5c+fidrv56KOPsFqtfPDBBzz00EMsW7YMi8XCN998w1tvvVVjfIsWLSIzM5PPP/8cu93O22+/jTGG1NRU3njjDd59912WLVvm/SZDREREfMcYQ+W+yqMSpsN/d+W46jyOxWbBEec4Knny/t7egdVxaCp5pcdDRkUF2XkW2Fx3nOFn6T5WInLy6pVc3XbbbXz22WdkZmYyYsQIwsPDSUlJ4ZVXXuGmm24iODiYp556iri4OJYsWUL37t356KOPqKysZPr06YwaNYr77ruPkSNHkpCQwF133UVlZSXPPfccnTp14s033+Tee++lR48e3HnnnTidTqxWKz169ODaa6/l/vvvZ9WqVVx//fXExsZy5ZVXkpWVhd1u9xammD17Nunp6Tz66KOsWrWKt956i7Zt2zJo0CCMMcTHx9OiRQvy8vL43//9Xzp27MjQoUNJT0/nhhtu8F7CFxERkboZj6Eis6J6wpRWPXlyF9d12QkCwgKOTpgO+90ea8dirfpb7/R4SHc6SXE62eN0ssdZzJ60g8tVj8yKCjz1eB4Hjy0icjLqlVy98sorNa7v2rWrd4zpQU6nk9jYWD755BPOOuss1qxZQ8eOHYGqMaDr16/nxx9/JDw8nO+//9677eOPP2b9+vWsWbPmqPPMnDmT5ORkXK6qb7iMMXTq1Im5c+fSqlUrAG6//XZv+/POO48jRz1mZmaSmJh41LbExETefvvtam0150pjpTVWWnOuNOdK7xFwer9HXHHZFaz4cgV5WXmEV4ZzZtGZfFf2HZ79HmKTY6nMriSjT9UcpqTFSewduJey1mU4PA46bO3A5qur+lSb1DaERoSS1j0Na7CV86LOY2fITvLIIyI6glFjRjF//nzcxhDdqRP7Q0NJWbOG/Zs85Nh6U/79LgJycigMCOC9Ll34w/r1WIBtUVFkhIVxQVoaDiAtMZHEffu4cN8+Kq1WvuvThyEbkrExnp3sZFdJKUN3bweqz7n6cc8e+t1wg+Zc6T1C7xH1fI/QnKvqTmjOVW0yMjJo06bqLucPPfQQmzZt4sMPP+TWW2+lb9++3HHHHeTn53PWWWcxf/58Bg4cWOu2s846i9TUVO/crB9//JGRI0eyfft2Wh6o5LN8+XImTpzIrl27vPfGOlZcbrebm2++mejoaJ577rlqbQ6fR6Y5VyJSX+qr0lRVllTWOlyvPK2civSKGu8PVY0VHO0cNQ/XS3AQFB+ENdhKkdtd7epSTY99NdxapSZBVivtHY5jPs6w2VhXUkL/H1fWOefq5/796ReuoYEi/sKf/q426Jyrw02dOpXvvvuOyspKBg8ezL///W8Apk+fzsSJE3n55ZeBqvLrAwcOPOa2srIyJkyYQGFhIYGBgYSGhrJgwQJvYgXw73//m4kTJx6VWB0+NBDg5ptvJjU1FafTyahRo3jyySe9bWsa6ngwuxYREWnKjDG4cl21DtcrTy2nMr/uZMYaZMURX/t8J1sbGwVUJU47vIlSOXuchewpd7J3fQV7nE5K3HUPDwQICwggrpaEqd2Bn1GBgd4pASIi/sKnydVrr71W4/pWrVqxaNGiem0LCQnxXp6uzbx582pcf/jQQMB7mb4mNQ11LCoqOuZ5RURE/IFxG5zpztqLRaSV4ymre+ZRYItA7xWmI5MnW7yDghaGvRUVhyVOTvY686uWM5zs2eXEeZwDYaICA2tNmA4+IgJ99/Ek2mbDYQ3AGZJwYM3RCVmQ1Uq0rjiLiA/4NLkSERER33GXVy9RfvhwPWeqE+ceJ6ay7qTGHmuvcbheQLydgrZWMm1udjqd7D28QIQzlz1OJ+lbKqg8zsQpxmY7ZtLUzuEgJCDgZF+WeokPCmLr4AvIHVA1H6eyspKVK1cyZMgQAg8kcdE2G/FBQQ0al4g0T0quREREGolrn6vW4XrlqeW4so6jRHngoRLl3gQqPgiTYCO/nZWcVrDLuA6b17SfPc59VRX18isw+XXHaQXaOhy0s9trnd/U1uHAfoz5z40pPijImzy5XC4yPB7OCgtr9HkcItL8KLkSERE5BYwxVGRV1F4sIrUcd1Hdc5CsIdaj5jq5E+3kx1nJaQ1ZYR72uioOS57y2ON0kutyQRZVj2OwWSxHXWE68hFjsxHop4mTiIg/UXIlIiJyAjwuD849zhqH6x1cNs66h9MFtgqsljw5O9rIjw8gt42F7BaGDFvlYcP1CtnjzKbQ7YZKIP3Yxw4+jop60TYb1mZeGKLMVcbZr50NwPc3fd/I0YhIc6bkSkREpAbuUnetw/WcqU6c6U7qvEutpapEuSPegSPBQVlnO/kdrOS2rRqulxXmJv3AkL29zlL2OPMp9Rw4aNmBRy0iAgKOmTS1czhoqYp6QNVVxI05G73LIiKnipIrERE57RhjqMyvrHW4XnlqOZV5dZcot9gtBMUHEZhop7SrnfxOgeS1s5LT2pAdYci0V3qH7O11FlevqFdy4FGDVjVU1Ds8aWrn44p6IiLiG3pnFhGRZse4Dc4MZ83D9Q48PKV1lygPiAggsIODou528pMCyI8PIK+1hawWHrJCPKRbqq46pTsLOWr2VA139bAAMTUUhTi8UEQ7h4PgBq6oJyIivqHkSkREmhyP00P57lru7ZRaXlWi3FX38C9Pu0CKejnY1yWQvMQA8mItZLcyZIV6yAisZK+7gqyKMsyR4/PcQHH1VQFUVdSrrQx5e4eDNna731bUExGRk6fkSkRE/E5lUWWtw/WcqU4qMivqPEZZGBT2tFHYzU5+hwDy2lnIjobscA8ZdjfpVJBXWUlVZYgjuA48DrBbLLUmTN6KenY7AZrfJCJyWlNyJSIiDcoYgyvbVetwPWeqk8p9tc93MkBJGOTGWSjsbie/UwB57a3ktoasSENWkJt0i4si4+aoLOmgww4fcpwV9VQYQkRE6qLkSkREfMpT6aFib0WtxSKcaU485TXPd/JYoDAScpIgv4OVgiQbefFWcmMhp6UhM8RDZkAlZXioSrOcNQdxYERgZB0V9do7HESqol6zZ7FYSIhM8C6LiJwqSq5ERKRe3GXuWq84laeV49zr5OjqDuC2Qn4U5HSAnDOgoFMA+YkB5LWxeMuSZ9rcVFgOzpXyUGvyBETbbMcuRW63E6aKegKE2ELYdfcuAFyuGq5kioj4iP7qiIiIlzGGyoLK2pOn1HJcOUd/OHUFQl6rqqQppxPkxkB+x4CqsuTRkB3hIdvhxl3tooGbmrIwCxBbQ0W9w+c8tbPbCVJFPRER8TNKrkRETiPGY6jIrDhmsQh3SfWEp9wBudFViVNunwM/21rIi7OSGwPZkYbc4JqG+R2dPAVaLLStqRT5ERX1bKqoJyIiTZCSKxGRZsRT4cG5u+bheuWp5Th3OzEVh0qUl4YcSpxy4iCnX9VyXjsLubEWsqMMhUE1lTQ3HJk4OQ5U1DtWVb3WqqgnjWC/az8XzL0AgOXXL2/kaESkOVNyJSJykozbUPhNIbZvbRSGFtJqaCssAacmgagsqax1uF55ajkVGRVgqlKf4vADSdPBxwUHrjqdAbltLOS0MpQG1fqs8FaFAEKtVuKCgqrd7PbIRytV1BM/5TEe1qSv8S6LiJwqSq5ERE5QWnk5277MZvezu6nIdgEh7Pl0PfbWNuLuiyNpRGvig2rNXo5ijMGV66p1uF55ajmVBZV4LLCvRfXEKbcH5Fx0KHnKaQ1O+zHP5l1qERhYZ0W9iIAAJU4iIiJ1UHLlxxry23ARqZ+08nK6rP4BZ6SBx4/c6gJ24Fi9k62DB3kTLE+lh4r0imolyY8cuucq91RV1Dv8ilMbyOl92LynaKi0HV+cZ9RVUc/hIFSFIURERHxCyZUf8vW34SLieznOCpyWmuYiHeK0GH58YAt56zyUp5ZTnOUkr8URV5yiIWcw5FxR9Xt+FHiOI9exAG2OMUSvncNBW1XUExERaVBNIrl66aWXePbZZ8nMzKRPnz68+OKLDBw4sNb28+fP5+GHH2bXrl0kJSXx9NNPc9lllzVgxCfuRL4NFzHGgKfqaqdxH7bsMeCuqhBn3EcsH2hz2ux3sG1N+x1xbI/bg8eAxxjcxmA84DYGjzHe9VvauuGpuv9tHutQwP4eVUlUQdTx/XsGWiy1zm06WCgiVhX1RERE/I7fJ1fvv/8+kydPZvbs2QwaNIhZs2YxYsQItmzZQuvWrY9q//333zN+/HimT5/O5Zdfzrx58xgzZgy//PILPXv2bIRnUD/H+214jrOiUZMr74f5E/kAXEObE/kA7Mv9PG4PHo/B4wG3x1QtG4PbfXC5ar0x4PEc/MBtqtqaA/uZA2050MZjquqpHfbTQ1VbjzG4qYrDzWH7HfzwfnDZ+/Ow9d7lqu0Hyw54rGAsdf/0VZvj+cnxxGXz0XHq8fyP2c7HF3p+61v99yCrtcab3R5ZUc+q+U0iIiJNjsUYc+xP8o1s0KBBnH322fzzn/8EwOPxEBcXx1/+8hcefPDBo9qPGzeO0tJSFi9e7F13zjnn0LdvX2bPnn1c5ywqKiIyMpLCwkIiIiJ880SO04oVexnKtjrb3b7MTju3DQy4TW0fxA98qD9s2duOgx/OD7S1VCVMB7cZy6EP7wfbVq070Jbj+2Duqw/Dvvjwf8xtugAgp8hkWvO7XjHexCkqMFCFIUQaWGlFKYkvJAKw7Y5tfLPsGy677DJstuOcvCgiDc7lcvH555/7RV+tT27g11euKioq+Pnnn5kyZYp3ndVqZdiwYaxevbrGfVavXs3kyZOrrRsxYgQLFy6s9TxOpxOn0+n9vaioCKj6R3W5XCfxDOrPmemE2LrbzR5WAVSc8nikZhYDVsBqwIoFC1VzYKrWWbB6f69ath62bLEcvt6C1XL4z0PLloPrLAfaWo7xe7XlQz8DrIe2WQ7f73h+HrZc076WI9rU53g1HdcXxzsyPouPj3f4unXFJZy3PrnO/yvX9oqlX0S49/fKyso69xER37Jb7KTfnQ7g/bve0H/fRaR+/Kmv1icGv06ucnNzcbvdxMTEVFsfExPD5s2ba9wnMzOzxvaZmZm1nmf69On8/e9/P2r9kiVLCAkJOYHIT9zODBvE1n3ObukeQqwGq+HAh0hz2Ad6qn3Y92631LDeAlaMd/nQB1KwWMyhtpbD2lqOaFPt9wPtDhzICliMqSWmQ8tWY+rcTm3bDq6r5TzV1h12nlpfK1PXa1n1kIZ18Cqqv9hutUJ4eJ3tVn2/ikyPP0UuIgBLly5t7BBE5Dj4Q18tKys77rZ+nVw1lClTplS72lVUVERcXByXXHJJgw8L/KWoGJJ/q7Pd3BF9q30bLiINa21JCfz6a53thgwZwllhYQ0QkYgcD5fLxdKlSxk+fHijDzUSkdr5U189OKrtePh1chUdHU1AQABZWVnV1mdlZREbW/PYudjY2Hq1B3A4HDgcjqPW22y2Bv/HtNmP73w2e8PHJiKHxAYHE2S1Un6Mq1JBViuxwcHqqyKNbL9rP5e+cykAi65dBDTO33gRqT9/6Kv1Ob9fJ1d2u53+/fuzfPlyxowZA1QVtFi+fDl33nlnjfsMHjyY5cuXc/fdd3vXLV26lMGDBzdAxCJyuogPCmLLwIHkHhiHXVlZycqVKxkyZAiBgVVvrdE2m26ZIOIHPMbDN6nfeJdFRE4Vv06uACZPnsyECRMYMGAAAwcOZNasWZSWljJx4kQAbrzxRtq1a8f06dMB+Otf/8qFF17Ic889x6hRo3jvvfdYs2YNr776amM+jeMWbbMd17fh0fq2TaTRxQcFeZMnl8tFhsfDWWFhjf4Nm4iIiDQOv0+uxo0bR05ODlOnTiUzM5O+ffvyxRdfeItWpKWlYT3sRprnnnsu8+bN46GHHuJvf/sbSUlJLFy4sEnc4wr0bbiIiIiISFPl98kVwJ133lnrMMAVK1YctW7s2LGMHTv2FEd16ujbcBERERGRpke3ThUREREREfEBJVciIiIiIiI+0CSGBYqIiIicjBBbSGOHICKnASVXIiIi0qyF2kMp/VspUDWXWUTkVNGwQBERERERER9QciUiIiIiIuIDSq5ERESkWSuvLGfUvFGMmjeK8sryxg5HRJoxzbkSERGRZs3tcfP5ts+9yyIip4quXImIiIiIiPiAkisREREREREfUHIlIiIiIiLiA0quREREREREfEDJlYiIiIiIiA+oWmANjDEAFBUVNXIkVXeSLysro6ioCJvN1tjhiEgt1FdF/FdpRSkcqMBeVFSkvirSBPjT39WDOcHBHOFYLOZ4Wp1m9uzZQ1xcXGOHISIiIiIifmL37t20b9/+mG2UXNXA4/GQnp5OeHg4FoulUWMpKioiLi6O3bt3ExER0aixiEjt1FdFmgb1VZGmwZ/6qjGG4uJi2rZti9V67FlVGhZYA6vVWmdW2tAiIiIa/T+WiNRNfVWkaVBfFWka/KWvRkZGHlc7FbQQERERERHxASVXIiIiIiIiPqDkys85HA6mTZuGw+Fo7FBE5BjUV0WaBvVVkaahqfZVFbQQERERERHxAV25EhERERER8QElVyIiIiIiIj6g5EpERERERMQHlFyJiIiIiIj4gJKrBvDtt98yevRo2rZti8ViYeHChdW2G2OYOnUqbdq0ITg4mGHDhrFt2zbv9l27djFp0iQ6dOhAcHAwnTp1Ytq0aVRUVNR4vpSUFMLDw2nRosUpfFYiTZ8/9c358+fTrVs3goKC6NWrF59//rkvn6pIk9GU+mVdsYg0J82tb+bn53P99dcTERFBixYtmDRpEiUlJSf24hxGyVUDKC0tpU+fPrz00ks1bn/mmWf4xz/+wezZs/nhhx8IDQ1lxIgRlJeXA7B582Y8Hg+vvPIKGzZsYObMmcyePZu//e1vRx3L5XIxfvx4zj///FP6nESaA3/pm99//z3jx49n0qRJrF27ljFjxjBmzBjWr1/v2ycs0gQ0pX5ZVywizUlz65vXX389GzZsYOnSpSxevJhvv/2WW2+99WRfJjDSoADz8ccfe3/3eDwmNjbWPPvss951+/btMw6Hw7z77ru1HueZZ54xHTp0OGr9/fffb2644QYzZ84cExkZ6cvQRZq1xuyb1157rRk1alS1dYMGDTK33XbbiT0ZkWbCn/vlicYi0hw09b65ceNGA5iffvrJ2+a///2vsVgsZu/evXW/AMegK1eNbOfOnWRmZjJs2DDvusjISAYNGsTq1atr3a+wsJCoqKhq67766ivmz59f6zcKInL8GrJvrl69utp5AEaMGHHM84icjvypX55oLCLNUVPrm6tXr6ZFixYMGDDA22bYsGFYrVZ++OGH43zWNVNy1cgyMzMBiImJqbY+JibGu+1IKSkpvPjii9x2223edXl5edx0003MnTuXiIiIUxewyGmiIftmZmZmvc4jcrryp355IrGINFdNrW9mZmbSunXratsDAwOJioo66f6r5KqJ2bt3LyNHjmTs2LHccsst3vW33HIL1113HRdccEEjRidy+lLfFPE/6pci/qk5900lV40sNjYWgKysrGrrs7KyvNsOSk9PZ+jQoZx77rm8+uqr1bZ99dVXzJgxg8DAQAIDA5k0aRKFhYUEBgbyn//859Q+CZFmqCH7Zmxs7HGdR+R050/9sj6xiDR3Ta1vxsbGkp2dXW17ZWUl+fn5J91/lVw1sg4dOhAbG8vy5cu964qKivjhhx8YPHiwd93evXu56KKL6N+/P3PmzMFqrf5Pt3r1atatW+d9PProo4SHh7Nu3Tp+//vfN9jzEWkuGrJvDh48uNp5AJYuXVrtPCLiX/3yeGMROR00tb45ePBg9u3bx88//+xt89VXX+HxeBg0aNDJvRgnVQ5DjktxcbFZu3atWbt2rQHM888/b9auXWtSU1ONMcY89dRTpkWLFuaTTz4xv/32m7nyyitNhw4dzP79+40xxuzZs8d07tzZXHzxxWbPnj0mIyPD+6iNqgWK1M1f+uaqVatMYGCgmTFjhtm0aZOZNm2asdlsJjk5+ZQ9dxF/1ZT6ZV2xiDQnza1vjhw50px11lnmhx9+MCtXrjRJSUlm/PjxJ/06KblqAF9//bUBjnpMmDDBGFNVMvLhhx82MTExxuFwmIsvvths2bLFu/+cOXNq3P9YubGSK5G6+VPf/OCDD0yXLl2M3W43PXr0MJ999pmvn65Ik9CU+mVdsYg0J82tb+bl5Znx48ebsLAwExERYSZOnGiKi4tP/AU6wGKMMSd37UtEREREREQ050pERERERMQHlFyJiIiIiIj4gJIrERERERERH1ByJSIiIiIi4gNKrkRERERERHxAyZWIiIiIiIgPKLkSERERERHxASVXIiIiIiIiPqDkSkRETimLxcLChQsbO4xGt2vXLiwWC+vWrfPZMRMTE5k1a5bPjiciIidHyZWIiJywnJwc/vSnPxEfH4/D4SA2NpYRI0awatUqb5uMjAwuvfTSUx7LRRddVGebuXPnYrFYvI+wsDD69+/PRx99VK9zrVixAovFwr59+04sWB/56aefuPXWWxs1BhEROSSwsQMQEZGm6+qrr6aiooI33niDjh07kpWVxfLly8nLy/O2iY2NPWXnX7x4MW3btqVfv37ede+99x79+vWjS5cuNe4TERHBli1bACguLmbOnDlce+21bNiwga5du56yWE+FM844o7FDEBGRw+jKlYiInJB9+/bx3Xff8fTTTzN06FASEhIYOHAgU6ZM4YorrvC2O3xY4COPPFLtytHBx9y5cwHweDxMnz6dDh06EBwcTJ8+fViwYEGtMXTs2JEpU6Ywbdo09u3bx7XXXsvXX39NdHR0rftYLBZiY2OJjY0lKSmJxx9/HKvVym+//eZt89ZbbzFgwADCw8OJjY3luuuuIzs7G6ga3jd06FAAWrZsicVi4aabbvLG/8wzz9C5c2ccDgfx8fE88cQT1c6/Y8cOhg4dSkhICH369GH16tW1xmqM4ZFHHvFeGWzbti133XWXd/vhwwKPvCp38PHII49427/++uuceeaZBAUF0a1bN15++eVazy0iIvWn5EpERE5IWFgYYWFhLFy4EKfTeVz73HvvvWRkZHgfM2bMICQkhAEDBgAwffp03nzzTWbPns2GDRu45557uOGGG/jmm29qPF737t358ssv2bZtG7/++ivDhg3jlVdeISoq6rjicbvdvPHGGwDVrn65XC4ee+wxfv31VxYuXMiuXbu8CVRcXBwffvghAFu2bCEjI4MXXngBgClTpvDUU0/x8MMPs3HjRubNm0dMTEy1c/7f//0f9957L+vWraNLly6MHz+eysrKGuP78MMPmTlzJq+88grbtm1j4cKF9OrVq8a248aNq/bavvvuuwQGBnLeeecB8M477zB16lSeeOIJNm3axJNPPsnDDz/sff4iIuIDRkRE5AQtWLDAtGzZ0gQFBZlzzz3XTJkyxfz666/V2gDm448/Pmrf1atXm6CgIPP+++8bY4wpLy83ISEh5vvvv6/WbtKkSWb8+PE1nn/z5s1m5MiR5uGHHzZ9+vQxY8eONX/6059Mfn5+je3nzJljABMaGmpCQ0ON1Wo1DofDzJkz55jP86effjKAKS4uNsYY8/XXXxvAFBQUeNsUFRUZh8NhXnvttRqPsXPnTgOY119/3btuw4YNBjCbNm2qcZ/nnnvOdOnSxVRUVNS4PSEhwcycOfOo9SkpKSYqKso888wz3nWdOnUy8+bNq9buscceM4MHD67x2CIiUn+6ciUiIifs6quvJj09nUWLFjFy5EhWrFhBv379vMP8apOWlsaYMWO49957ufbaawFISUmhrKyM4cOHe6+KhYWF8eabb7J9+/Yaj7N161aeeOIJHn30UVq0aMEHH3zA+eefT05OTq3nDg8PZ926daxbt461a9fy5JNPcvvtt/Ppp5962/z888+MHj2a+Ph4wsPDufDCC71x12bTpk04nU4uvvjiYz733r17e5fbtGkD4B1yeKSxY8eyf/9+OnbsyC233MLHH39c61WugwoLC7n88ssZNWoU9913HwClpaVs376dSZMmVXttH3/88VpfWxERqT8VtBARkZMSFBTE8OHDGT58OA8//DB//OMfmTZtmncY3ZFKS0u54oorGDx4MI8++qh3fUlJCQCfffYZ7dq1q7aPw+Go8VijR48+at348eOPGa/VaqVz587e33v37s2SJUt4+umnGT16NKWlpYwYMYIRI0bwzjvvcMYZZ5CWlsaIESOoqKio9bjBwcHHPO9BNpvNu2yxWICquVo1iYuLY8uWLSxbtoylS5dyxx138Oyzz/LNN99UO85BbrebcePGERERwauvvupdf/C1fe211xg0aFC1fQICAo4rbhERqZuSKxER8anu3bvXel8rYww33HADHo+Ht956y5tcHNzP4XCQlpbmvVJUHytWrDjBiKsSjP379wOwefNm8vLyeOqpp4iLiwNgzZo11drb7XagKpk5KCkpieDgYJYvX84f//jHE47lSMHBwYwePZrRo0fz5z//mW7dupGcnFxtjthB99xzD8nJyaxZs4agoCDv+piYGNq2bcuOHTu4/vrrfRabiIhUp+RKREROSF5eHmPHjuXmm2+md+/ehIeHs2bNGp555hmuvPLKGvd55JFHWLZsGUuWLKGkpMR7RSUyMpLw8HDuvfde7rnnHjweD0OGDKGwsJBVq1YRERHBhAkTfBK3MYbMzEwA9u/fz9KlS/nyyy+ZOnUqAPHx8djtdl588UVuv/121q9fz2OPPVbtGAkJCVgsFhYvXsxll11GcHAwYWFhPPDAA9x///3Y7XbOO+88cnJy2LBhA5MmTTqhWOfOnYvb7WbQoEGEhITw9ttvExwcTEJCwlFt58yZw8svv8zHH3+MxWLxPseDQwD//ve/c9dddxEZGcnIkSNxOp2sWbOGgoICJk+efELxiYjIERp70peIiDRN5eXl5sEHHzT9+vUzkZGRJiQkxHTt2tU89NBDpqyszNuOwwpaXHjhhQY46nGwoITH4zGzZs0yXbt2NTabzZxxxhlmxIgR5ptvvvFJzAcLWhx8OBwO06VLF/PEE0+YyspKb7t58+aZxMRE43A4zODBg82iRYsMYNauXett8+ijj5rY2FhjsVjMhAkTjDHGuN1u8/jjj5uEhARjs9lMfHy8efLJJ40xhwpaHH6MgoICA5ivv/66xng//vhjM2jQIBMREWFCQ0PNOeecY5YtW+bdfnhBiwkTJtT42k6bNs3b/p133jF9+/Y1drvdtGzZ0lxwwQXmo48+OqnXVEREDrEYY0yDZ3QiIiIiIiLNjKoFioiIiIiI+ICSKxERERERER9QciUiIiIiIuIDSq5ERERERER8QMmViIiIiIiIDyi5EhERERER8QElVyIiIiIiIj6g5EpERERERMQHlFyJiIiIiIj4gJIrERERERERH1ByJSIiIiIi4gP/H5VnvgVA0gfkAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "\n", + "# Extract x and y values and calculate log2\n", + "def extract_values(data):\n", + " x = [entry[\"Size\"] * entry[\"Batch size\"] for entry in data]\n", + " y = [entry[\"Eval Time\"] * 1000 for entry in data] # Convert to milliseconds\n", + " x_log2 = [np.log2(x_val) for x_val in x]\n", + " return x, y, x_log2\n", + "\n", + "DReLU_x, DReLU_y, DReLU_x_log2 = extract_values(DReLU_data)\n", + "MSB_x, MSB_y, MSB_x_log2 = extract_values(MSB_data)\n", + "\n", + "# Value for the vertical line\n", + "vertical_line_x = 28 * 28 * 128\n", + "vertical_line_log2 = np.log2(vertical_line_x)\n", + "\n", + "# Plotting\n", + "plt.figure(figsize=(10, 6))\n", + "plt.plot(DReLU_x_log2, DReLU_y, marker='o', linestyle='-', color='m', label='DReLU')\n", + "plt.plot(MSB_x_log2, MSB_y, marker='s', linestyle='-', color='c', label='MSB')\n", + "\n", + "# Add vertical line\n", + "plt.axvline(vertical_line_log2, color='g', linestyle='--', label=f'x = {vertical_line_x}')\n", + "\n", + "# Add horizontal lines at the intersection points\n", + "def interpolate_y(x_vals, y_vals, target_x):\n", + " return np.interp(target_x, x_vals, y_vals)\n", + "\n", + "DReLU_y_intersection = interpolate_y(DReLU_x_log2, DReLU_y, vertical_line_log2)\n", + "MSB_y_intersection = interpolate_y(MSB_x_log2, MSB_y, vertical_line_log2)\n", + "\n", + "plt.axhline(DReLU_y_intersection, color='gray', linestyle='--', linewidth=0.5)\n", + "plt.axhline(MSB_y_intersection, color='gray', linestyle='--', linewidth=0.5)\n", + "\n", + "# Annotate the intersection points on the left side\n", + "plt.text(plt.xlim()[0], DReLU_y_intersection, f'{DReLU_y_intersection:.2f}', color='black', fontsize=8, verticalalignment='center', horizontalalignment='right')\n", + "plt.text(plt.xlim()[0], MSB_y_intersection, f'{MSB_y_intersection:.2f}', color='black', fontsize=8, verticalalignment='center', horizontalalignment='right')\n", + "\n", + "plt.xlabel('Size * Batch size')\n", + "plt.ylabel('Eval Time (ms)')\n", + "plt.title('Evaluation Time vs Size * Batch size between DreLu (falcon) and MSB (RingPPML)')\n", + "plt.legend()\n", + "plt.grid(True)\n", + "\n", + "# Set the x-ticks to the specific log2 values\n", + "all_x_log2 = sorted(set(DReLU_x_log2 + MSB_x_log2))\n", + "plt.xticks(all_x_log2, [f\"{2**x:.0f}\" for x in all_x_log2])\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "metadata": {}, + "outputs": [], + "source": [ + "def float_to_fixedpoint(a, Float_precision=FLOAT_PRECISION):\n", + " return int(a * (1 << Float_precision))\n", + "\n", + "def fixedpoint_to_float(value, Float_precision=FLOAT_PRECISION):\n", + " return value / (1 << Float_precision)\n", + "\n", + "def getShare_int(ls, _randrange=ring):\n", + " size = len(ls)\n", + " np.random.seed(42)\n", + " share0 = np.random.randint(_randrange, size=size).tolist()\n", + " np.random.seed(43)\n", + " share1 = np.random.randint(_randrange, size=size).tolist()\n", + " share2 = [(ls[i] - share0[i] - share1[i]) & ((1 << ring_size) - 1) for i in range(size)]\n", + " return [share0, share1, share2]\n", + "\n", + "def open(share):\n", + " size = len(share[0])\n", + " return [(share[0][i] + share[1][i] + share[2][i]) & ((1 << ring_size) - 1) for i in range(size)]\n", + "\n", + "def getShare_float(ls, Float_precision=FLOAT_PRECISION):\n", + " ls_fixed = [float_to_fixedpoint(x, Float_precision) for x in ls]\n", + " print(\"float to fixed point: \", ls_fixed)\n", + " size = len(ls_fixed)\n", + " np.random.seed(42)\n", + " share0 = [float_to_fixedpoint(np.random.rand(), Float_precision) for _ in range(size)]\n", + " np.random.seed(43)\n", + " share1 = [float_to_fixedpoint(np.random.rand(), Float_precision) for _ in range(size)]\n", + " share2 = [(ls_fixed[i] - share0[i] - share1[i]) & ((1 << ring_size) - 1) for i in range(size)]\n", + " return [share0, share1, share2]\n", + "\n", + "def open_float(share, Float_precision=FLOAT_PRECISION):\n", + " size = len(share[0])\n", + " result_fixed = [(share[0][i] + share[1][i] + share[2][i]) & ((1 << ring_size) - 1) for i in range(size)]\n", + " print(\"fixed input opened: \", result_fixed)\n", + " return [fixedpoint_to_float(x, Float_precision) for x in result_fixed]\n", + "\n", + "def list_delta(list_val_a:list, list_val_b:list):\n", + "\n", + " sz = len(list_val_a)\n", + " if(sz != len(list_val_b)):\n", + " return False\n", + " \n", + " return [abs(list_val_a[i] - list_val_b[i]) for i in range(sz)]\n", + "\n", + "def delta(list_val_a:list, list_val_b:list):\n", + " _list_delta = list_delta(list_val_a, list_val_b)\n", + " sz = len(_list_delta)\n", + " sum = 0\n", + " for item in _list_delta:\n", + " sum += item\n", + " return sum/sz\n", + "\n", + "def ss_mult(ss_list_a:list[list], ss_list_b:list[list], size=32):\n", + "\n", + " # define mult calculate by each party\n", + " mult0 = []\n", + " mult1 = []\n", + " mult2 = []\n", + " # mult = []\n", + "\n", + " sz = len(ss_list_a[0])\n", + " if sz != len(ss_list_b[0]): \n", + " return (False)\n", + " for i in range(sz):\n", + " mult0.append((ss_list_a[0][i]* ss_list_b[0][i] \\\n", + " + ss_list_a[0][i]* ss_list_b[1][i] \\\n", + " + ss_list_a[0][i] * ss_list_b[2][i]) & (size-1))\n", + " \n", + " mult1.append((ss_list_a[1][i]* ss_list_b[0][i] \\\n", + " + ss_list_a[1][i]* ss_list_b[1][i] \\\n", + " + ss_list_a[1][i] * ss_list_b[2][i]) & (size-1))\n", + " \n", + " mult2.append((ss_list_a[2][i]* ss_list_b[0][i] \\\n", + " + ss_list_a[2][i]* ss_list_b[1][i] \\\n", + " + ss_list_a[2][i] * ss_list_b[2][i]) & (size-1))\n", + "\n", + " # mult.append(mult0[i] + mult1[i] + mult2[i])\n", + "\n", + " return [mult0, mult1, mult2]\n", + " # return (True, mult)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Mô phỏng quá trình nhân với kỹ thuật Secret Sharing với kiểu dữ liệu int, và float" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[8, 7, 6, 6, 23]\n", + "[[102, 179, 92, 14, 106], [68, 64, 255, 49, 21], [94, 20, 171, 199, 152]]\n", + "[8, 7, 6, 6, 23]\n", + "[[304, 997, 552, 596, 902], [544, 448, 1018, 550, 739], [240, 140, 514, 938, 424]]\n", + "[64, 49, 36, 36, 17]\n" + ] + } + ], + "source": [ + "## Multiplication visualization with int value\n", + "\n", + "# ring setting\n", + "ring_size = 8\n", + "ring = 1 << ring_size\n", + "# Setting integer input\n", + "RANDOM_BITS = 5\n", + "RAN_RANGE = 2**RANDOM_BITS\n", + "input_size = 5\n", + "\n", + "# Generate a list of random integers\n", + "random_list = [random.randrange(RAN_RANGE) for _ in range(input_size)]\n", + "\n", + "print(random_list)\n", + "ss_random_list = getShare_int(random_list)\n", + "print(ss_random_list)\n", + "open_random_list = open(ss_random_list)\n", + "print(open_random_list)\n", + "\n", + "ss_mult_list = ss_mult(ss_random_list, ss_random_list, ring)\n", + "print(ss_mult_list)\n", + "print(open(ss_mult_list))\n", + "#\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "| Input | party 0 | party 1 | party 2 |\n", + "|:-------:|:-------:|:-------:| :-------:|\n", + "| 8
7
6
6
23 | 102
179
92
14
106 | 68
64
255
49
21 | 94
20
171
199
152 |\n", + "\n", + "*Cơ chế phép nhân*\n", + "\n", + "```\n", + "c = a x b \n", + " = (a0 + a1 + a2) x (b0 + b1 + b2)\n", + " = a0 x ((b0 + b1 + b2)) + a1 x ((b0 + b1 + b2)) + a2 x ((b0 + b1 + b2))\n", + " = mult0 + mult1 + mult2\n", + "```\n", + "\n", + "*Xét ví dụ bên trên*\n", + "\n", + "- Lấy chính vector khởi tạo nhân với nó ta được vector mới có giá trị ouput cuối cùng ko đúng với dự kiến: `23*23 = 529 >< 17`.\n", + "- Nguyên nhân sai lệch là do vành 8 bits nên giá trị thu được là `529` sẽ bị chuyển về `17`: `17 = 529 % (2**8)`\n", + "- Khi đó ta cần tính toán trong việc chuyển một số qua tính toán trên fixed point và xác định được số lượng fixed point tối đa\n" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[255.7661313696008, 190.88843954566104, 135.14617148837283, 253.27622063027235, 82.1768290184628]\n", + "float to fixed point: [65476, 48867, 34597, 64838, 21037]\n", + "[[95, 243, 187, 153, 39], [29, 155, 34, 61, 83], [65352, 48469, 34376, 64624, 20915]]\n", + "fixed input opened: [65476, 48867, 34597, 64838, 21037]\n", + "[255.765625, 190.88671875, 135.14453125, 253.2734375, 82.17578125]\n", + "--------------------------\n", + "mult list fixed float: [[6220220, 11874681, 6469639, 9920214, 820443], [1898804, 7574385, 1176298, 3955118, 1746071], [4278987552, 2368534623, 1189306472, 4190090912, 439988855]]\n", + "open [4287106576, 2387983689, 1196952409, 4203966244, 442555369]\n", + "fixed input opened: [4287106576, 2387983689, 1196952409, 4203966244, 442555369]\n", + "[65416.054931640625, 36437.7393951416, 18264.044326782227, 64147.434143066406, 6752.859024047852]\n", + "65416.054931640625\n" + ] + } + ], + "source": [ + "## Multiplication visualization with fixedpoint value\n", + "\n", + "# ring setting\n", + "ring_size = 32\n", + "ring = 1 << ring_size\n", + "# Setting integer input\n", + "RANDOM_BITS = 8\n", + "RAN_RANGE = 2**RANDOM_BITS\n", + "FLOAT_PRECISION_MAX = (ring_size>>1)- RANDOM_BITS\n", + "FLOAT_PRECISION = FLOAT_PRECISION_MAX\n", + "\n", + "input_size = 5\n", + "\n", + "# Generate a list of random floating-point numbers\n", + "random_floats = [random.uniform(0, RAN_RANGE) for _ in range(input_size)]\n", + "\n", + "print(random_floats)\n", + "\n", + "## Get float share\n", + "ss_random_list_float = getShare_float(random_floats, Float_precision=FLOAT_PRECISION)\n", + "print(ss_random_list_float)\n", + "\n", + "## check float share\n", + "open_random_list_float = open_float(ss_random_list_float, Float_precision=FLOAT_PRECISION)\n", + "print(open_random_list_float)\n", + "\n", + "# ## get delta between floating and fixed\n", + "# list_delta_float = list_delta(random_floats, open_random_list_float)\n", + "# print(list_delta_float)\n", + "# print(delta(random_floats, open_random_list_float))\n", + "\n", + "## mult\n", + "print(\"--------------------------\")\n", + "ss_mult_list_float = ss_mult(ss_random_list_float, ss_random_list_float, ring)\n", + "print(\"mult list fixed float: \", ss_mult_list_float)\n", + "print (\"open \", open(ss_mult_list_float))\n", + "print(open_float(ss_mult_list_float, FLOAT_PRECISION<<1))\n", + "print(open_random_list_float[0]**2)\n", + "\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/tutorial/images/LeNet.png b/tutorial/images/LeNet.png new file mode 100644 index 0000000..b574d2a Binary files /dev/null and b/tutorial/images/LeNet.png differ diff --git a/tutorial/images/MinioNN.png b/tutorial/images/MinioNN.png new file mode 100644 index 0000000..a377cd8 Binary files /dev/null and b/tutorial/images/MinioNN.png differ diff --git a/tutorial/images/Screenshot from 2024-07-28 23-50-55.png b/tutorial/images/Screenshot from 2024-07-28 23-50-55.png new file mode 100644 index 0000000..fc93b9a Binary files /dev/null and b/tutorial/images/Screenshot from 2024-07-28 23-50-55.png differ diff --git a/tutorial/images/maxpolling.png b/tutorial/images/maxpolling.png new file mode 100644 index 0000000..56f0650 Binary files /dev/null and b/tutorial/images/maxpolling.png differ diff --git a/tutorial/images/training_batchsize.png b/tutorial/images/training_batchsize.png new file mode 100644 index 0000000..330ceab Binary files /dev/null and b/tutorial/images/training_batchsize.png differ diff --git a/tutorial/tutorial.md b/tutorial/tutorial.md new file mode 100644 index 0000000..057fa6e --- /dev/null +++ b/tutorial/tutorial.md @@ -0,0 +1,120 @@ +# Hướng dẫn traning model với falcon + +## Mục lục +- [Cơ chế hoạt động của ppml](#cơ-chế-hoạt-động-của-ppml) +- [Giai đoạn chuẩn bị](#chuẩn-bị) + - [Cài đặt môi trường tương ứng](#môi-trường-chạy) + - [Dataset - MNIST](#chuẩn-bị-dữ-liệu---mnist) + - [Thiết lập thông số](#điều-chỉnh-thông-số-của-mô-hình-ml) +- [Huấn luyện mô hình](#training) +- [Kết quả](#kết-quả) +- [Nhận xét - đánh giá](#nhận-xét---đánh-giá) +- [Phụ Lục](#phụ-lục) + +## Cơ chế hoạt động của ppml +- Sử dụng fixed point value để lưu trữ các giá trị tính toán - `FLOAT_PRECISION` trong file `src/globals.h` +- Sau khi chuyển giá trị dưới kiểu `int`, `float` sang dạng `fixed point value`, các protocol bên trong ppml sẽ coi các giá trị đó đơn thuần là 1 chuốt các bit và áp dụng các phép toán `+`, `&`, `^` để ra chuỗi bit tương ứng, khi cần sẽ reconstruct lại giá trị `int`, `float` như ban đầu để đạt được kết quả cùng tính toán nhằm đảo bảo tính bảo mật xuyên xuốt quá trình thực thi + +## Chuẩn bị + +### Môi trường chạy +- Yêu cầu cài đặt các hàm thư viện sau: + - [g++]() + - [make]() + - [libssl-dev]() + +### Chuẩn bị dữ liệu - MNIST +- Sử dụng script tại `MNIST/run.sh` để tạo các file data cho từng bên tham gia với nguồn (.csv file tương ứng) +- Chỉnh sửa đường dẫn để load data tại hàm `loadData` trong file `src/secondary.cpp` + +### Điều chỉnh thông số của mô hình machine learning +- Điều chỉnh số lượng image trainnig và testing tại hàm `loadData` trong file `src/secondary.cpp` +- Điều chỉnh fixed point và số lượng epoch: `src/globals.h` + +### Tự điều chỉnh các layer bên trong một mô hình +- Có thể thêm mới, sửa các layer có sẵn tại hàm `selectNetwork` tròn file `src/secondary.cpp` +- Xác định yêu cầu chuẩn hóa dữ liệu, với biến `WITH_NORMALIZATION` thiết lập trong hàm `selectNetwork`, tuy nhiên `WITH_NORMALIZATION` trong toàn bộ project chỉ thiết lập cho hàm tính toán accuracy. + +## Training model +- build toàn bộ project với lệnh `make all -j$(nproc)` + +- Training local + - `Cách thức train`: sử dụng 3 terminal riêng biệt và chạy các lệnh tương ứng trên từng terminal + - `Lựa chọn mô hình`: Có thể thay đổi mô hình từ `MiniONN` sang `SecureML`, `Sarda`, `LeNet` cũng như các mô hình có thể tự thêm vào. + ``` + ./Falcon.out 0 files/IP_localhost files/keyA files/keyAB files/keyAC MiniONN MNIST Semi-honest + ./Falcon.out 1 files/IP_localhost files/keyA files/keyAB files/keyAC MiniONN MNIST Semi-honest + ./Falcon.out 2 files/IP_localhost files/keyA files/keyAB files/keyAC MiniONN MNIST Semi-honest + ``` + - Hình ảnh mô phỏng một quá trình train với `2^11` ảnh và `24` epoch +![Alt text](./images/Screenshot%20from%202024-07-28%2023-50-55.png) + - `Accuracy`: Khi thực hiện không tiệm cận được như accuracy được nêu bên trong [paper gốc]() + - `Thời gian`: Thời gian training cho một batchsize gồm `128 images`, một image tương đương một ma trận `28x28` +![Alt text](./images/training_batchsize.png) + +- Training global (Chưa thực hiện) + +## Kết quả +- Thời gian thực thi cho từng batch size (`128 images`/`1 batch`,`28x28`/`1 image`) trong quá trình trainnig bao gồm `forward` và `backward` ~ `2.7555 second` + +## Nhận xét - đánh giá +- Dưới dữ liệu đầu vào là giá trị `int` của bộ ảnh MNIST khi chưa chuẩn hóa cho ra accuracy không thể áp dụng vào thực tế . +- Có thể xem xét, áp dụng phương pháp chuẩn hóa dữ liệu cho bộ ảnh đầu vào, dựa vào gợi ý tại `files\preload\input_0` (format input của mô hình với preload khác so với input từ tập ảnh MNIST được tạo ra tại `MNIST/run.sh`) +- Thời gian chạy 50 batch với một số models + +| ML model | Times - fixed point value - semi-honet - Falcon | communications - sending and receving | cpu - colab +| :---: | :---: | :---: | :---: | +| [MinioNN](./images/MinioNN.png) | ~146 s | 3744.5 MB | ---- | +| [LeNet](./images/LeNet.png) | ~570 s | 5577.61 MB | [~540 ms](https://github.com/trhieung/CSC14120/blob/main/report_run.ipynb) | + +# Phụ Lục + +## Các protocol thiết lập bên trong Framework +Nhằm hỗ trợ cho training và inferencing và tuân thủ mộ mô hình ppml, falcon cài đặt một số giao thức an toàn (thông tin cụ thể được nêu trong luận văn + [paper gốc]()) +- Cách kiểm tra riêng lẻ từng protocol: chọn protocol tương ứng được liệt kê trong hàm `runTest` tại file `src/unitTests.cpp`, uncomment dòng debug trong main và đưa tên protocol tương ứng vào hàm. + - `Mat-Mul` + - `DotProd` + - `PrivateCompare` + - `Wrap` + - `ReLUPrime` + - `ReLU`: + - `Division` + - `BN` + - `SS Bits` + - `SelectShares` + - `Maxpool` + +- `Ví dụ `với `Maxpooling`, xác định vị trí có giá trị lớn nhất trong một hàng, đồng thời chỉ ra giá trị đó +![Alt text](./images/maxpolling.png) + + +
+ +
+ +| col 0 | col 1 | col 2 | +|:-------:|:-------:|:-------:| +| 1
3
1
5
6 | 2
1
5
1
3 | 3
2
3
6
9 | +
+
+ +*giá trị lớn nhất trong hàng* + + 3 + 3 + 5 + 6 + 9 +
+
+ +*Ma trận biểu thị ví trí lớn nhất* + + 0 0 1 + 1 0 0 + 0 1 0 + 0 0 1 + 0 0 1 +
+