From 7fb8130bc6566acdbd8bbd3f5091718f0eeacfbe Mon Sep 17 00:00:00 2001 From: ByronHsu Date: Tue, 13 Aug 2024 01:02:10 +0000 Subject: [PATCH] Polish readme init readme wi[ test looks good emoji modify add wip wip --- Readme.md | 163 +++++++++++++++++++++++++------------ docs/images/e2e-memory.png | Bin 0 -> 17055 bytes docs/images/e2e-tps.png | Bin 0 -> 16070 bytes 3 files changed, 109 insertions(+), 54 deletions(-) create mode 100644 docs/images/e2e-memory.png create mode 100644 docs/images/e2e-tps.png diff --git a/Readme.md b/Readme.md index 0e358dd8a..1ba4210a0 100644 --- a/Readme.md +++ b/Readme.md @@ -1,71 +1,84 @@ # Liger Kernel -Liger Kernel is the collection of Triton-native kernels for LLM Training. It is designed to be performant, correct, and light-weight. +**Liger Kernel** is a collection of Triton-native kernels designed specifically for LLM training. It aims to be **performant**, **correct**, and **lightweight**. We welcome contributions from the community to help us enhance and grow this project. -## Overview +### ✨ Key Features +- **🚀 Performant:** All kernels are written in OpenAI Triton with optimized tuning, increasing multi-GPU training throughput by 20% and reducing memory usage by 60%. +- **✅ Correct:** Each kernel undergoes rigorous unit and convergence testing to ensure accuracy. +- **🌱 Lightweight:** The kernels have minimal dependencies, requiring only Torch and Triton—no extra libraries needed! +### 🎯 Target Audiences -| Speed Up | Memory Reduction | -|--------------------------|-------------------------| -| ![Speed up](docs/images/speedup.png) | ![Memory](docs/images/memory.png) | +- **Researchers**: Looking to compose models using efficient and reliable kernels for frontier experiments. +- **ML Practitioners**: Focused on maximizing GPU training efficiency with optimal, high-performance kernels. +- **Curious Novices**: Eager to learn how to write reliable Triton kernels to enhance training efficiency. + +## 🌟 Overview -> **Note:** -> -> 1. Benchmark conditions: LLaMA 3-8B, Batch Size = 4, Sequence Length = 2048, Data Type = bf16, Full Pass (Forward + Backward). -> -> 2. **Fused Linear Cross Entropy Loss** trades time for memory by not materializing full logits, and it is recommended to use it when memory is the bottleneck. +### Supercharge Your Model with Liger Kernel +Gain +20% throughput and -60% memory usage. Achieve longer context lengths and larger batch sizes. -| Patch existing HF model | Compose your own model | +| ⚡ Speed Up | 💾 Memory Reduction | |--------------------------|-------------------------| -| ![Patch](docs/images/patch.gif) | ![Compose](docs/images/compose.gif) | +| ![Speed up](docs/images/e2e-tps.png) | ![Memory](docs/images/e2e-memory.png) | +> **Note:** +> 1. Benchmark conditions: LLaMA 3-8B, Batch Size = 8, Data Type = bf16, Optimizer = AdamW, Gradient Checkpointing = True, Distributed Strategy = FSDP1 on 8 A100s. +> 2. HuggingFace models start to OOM at 4K context length, whereas Liger Kernel scales up to 16K. +> 3. **Fused Linear Cross Entropy Loss** is enabled to significantly reduce memory usage. +### ✨ Utilize Individual Kernels or Enhance Existing Models +| 🛠️ Patch Existing HF Model | 🧩 Compose Your Own Model | +|--------------------------|-------------------------| +| ![Patch](docs/images/patch.gif) | ![Compose](docs/images/compose.gif) | -## Features +## 🚀 Features -- Forward + Backward -- Hugging Face model compatible. Easily patch model to speed up with 1 line -- Robust unit and convergence tests for kernels -- Compatible with multi GPUs (PyTorch FSDP) -- Compatible with `torch.compile` +- +20% throughput and -60% memory usage for multi-GPU training. +- Unlock large vocabulary sizes, long contexts, or multi-head training. +- Minimal dependencies—only `torch` and `triton` are required. +- Hugging Face model compatible—speed up your models with just one line of code. +- Forward and backward passes implemented. +- 0% loss in correctness—kernels are validated through robust unit and convergence tests. +- Compatible with multi-GPU setups (PyTorch FSDP and DeepSpeed). +- Seamless integration with Torch Compile. +## 🔧 Installation -## Installation +### Dependencies +- `torch >= 2.1.2` +- `triton >= 2.3.0` +- `transformers >= 4.40.1` -- dependencies - - torch >= `2.1.2` - - triton >= `2.3.0` - - transformers >= `4.40.1` +To install the stable version: ```bash $ pip install liger-kernel ``` -## Usage +To install the nightly version: + +```bash +$ pip install liger-kernel-nightly +``` -1. Patch existing Hugging Face models +## 🚀 Getting Started +### 1. 🛠️ Patch Existing Hugging Face Models ```python from liger_kernel.transformers import apply_liger_kernel_to_llama from transformers import Trainer +# By adding this line, it automatically monkey patches the model with the optimized kernels apply_liger_kernel_to_llama() model = transformers.AutoModelForCausalLM.from_pretrained("") ``` -| **Model** | **API** | **Supported Operations** | -|-------------|--------------------------------------------------------------|-------------------------------------------------------------------------| -| LLaMA (2 & 3) | `liger_kernel.transformers.apply_liger_kernel_to_llama` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy | -| Mistral | `liger_kernel.transformers.apply_liger_kernel_to_mistral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy | -| Mixtral | `liger_kernel.transformers.apply_liger_kernel_to_mixtral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy | - -2. Compose your own model - -For example, use `LigerFusedLinearCrossEntropyLoss` with `torch.nn.Linear` model +### 2. 🧩 Compose Your Own Model ```python from liger_kernel.transformers import LigerFusedLinearCrossEntropyLoss @@ -73,6 +86,8 @@ import torch.nn as nn import torch model = nn.Linear(128, 256).to("cuda") + +# LigerFusedLinearCrossEntropyLoss fuses linear and cross entropy layer together and performs chunk-by-chunk computation to reduce memory loss_fn = LigerFusedLinearCrossEntropyLoss() input = torch.randn(4, 128, requires_grad=True, device="cuda") @@ -82,36 +97,76 @@ loss = loss_fn(model.weight, input, target) loss.backward() ``` -| **Kernels** | **API** | **Description** | **Benchmark (A100) ** | -|----------------------------|-------------------------------------------------------------|-----------------|--------------------------------------------------------| -| RMSNorm | `liger_kernel.transformers.LigerRMSNorm` | TBA | [time](./benchmark/rms_norm_speed/) / [memory](./benchmark/rms_norm_memory/) | -| RoPE | `liger_kernel.transformers.liger_rotary_pos_emb` | TBA | [time](./benchmark/rope_speed/) / [memory](./benchmark/rope_memory/) | -| SwiGLU | `liger_kernel.transformers.LigerSwiGLUMLP` | TBA | [time](./benchmark/swiglu_speed/) / [memory](./benchmark/swiglu_memory/) | -| CrossEntropy | `liger_kernel.transformers.LigerCrossEntropyLoss` | This liger Cross Entropy loss computes both loss and the gradient in the forward path with inplace replacement of input to reduce the peak memory (avoid the materialization of both input logits and gradient) thus reducing the peak memory. We only consider hard label + mean reduction for now. Please refer to https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html for the math. | [time](./benchmark/cross_entropy_speed/) / [memory](./benchmark/cross_entropy_memory/) | -| FusedLinearCrossEntropy | `liger_kernel.transformers.LigerFusedLinearCrossEntropyLoss`| This Liger Cross Entropy loss further improves upon the basic Liger Cross Entropy kernel by reducing peak memory usage through fusion of the model's final output head layer with the CE loss, and chunking the input for block-wise loss and gradient calculation. The same strategy of computing both loss and gradient in the forward path with inplace replacement of input is used here. | [time](./benchmark/fused_linear_cross_entropy_speed/) / [memory](./benchmark/fused_linear_cross_entropy_memory/) | +## ⚙️ Note on ML Compiler + +### 1. ⚡ Torch Compile + +Since Liger Kernel is 100% Triton-based, it works seamlessly with Torch Compile. In the following example, Liger Kernel can further optimize on top of Torch Compile, reducing the memory by more than half. -## Structure +| Configuration | ⚡ Throughput (tokens/sec) | 💾 Memory Reserved (MB) | +|--------------------------------|----------------------------|-------------------------| +| Torch Compile | 3780 | 66358 | +| Torch Compile + Liger Kernel | 3702 | 31000 | + +> **Note:** +> 1. **Fused Linear Cross Entropy Loss** is enabled. +> 2. Benchmark conditions: LLaMA 3-8B, Batch Size = 8, Seq Len = 4096, Data Type = bf16, Optimizer = AdamW, Gradient Checkpointing = True, Distributed Strategy = FSDP1 on 8 A100s. +> 3. Tested on torch `2.5.0.dev20240731+cu118` + +### 2. 🌩️ Lightning Thunder + +*WIP* + +## 📂 Structure + +### Source Code + +- `ops/`: Core Triton operations. +- `transformers/`: PyTorch `nn.Module` implementations built on Triton operations, compliant with the `transformers` API. + +### Tests + +- `transformers/`: Correctness tests for the Triton-based layers. +- `convergence/`: Patches Hugging Face models with all kernels, runs multiple iterations, and compares weights, logits, and loss layer by layer. + +### Benchmark + +- `benchmark/`: Execution time and memory benchmarks compared to Hugging Face layers. + +## 🔧 APIs + +### Patching + +| **Model** | **API** | **Supported Operations** | +|-------------|--------------------------------------------------------------|-------------------------------------------------------------------------| +| LLaMA (2 & 3) | `liger_kernel.transformers.apply_liger_kernel_to_llama` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy | +| Mistral | `liger_kernel.transformers.apply_liger_kernel_to_mistral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss | +| Mixtral | `liger_kernel.transformers.apply_liger_kernel_to_mixtral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss | -1. Source code -- `ops/`: Core Triton operations implementation -- `transformers/`: PyTorch `nn.Module` on top of Triton operations complying with `transformers` API +### 🧩 Kernels -2. Tests +| **Kernel** | **API** | **Description** | +|---------------------------|-------------------------------------------------------------|-----------------| +| RMSNorm | `liger_kernel.transformers.LigerRMSNorm` | [RMSNorm Paper](https://arxiv.org/pdf/1910.07467) | +| RoPE | `liger_kernel.transformers.liger_rotary_pos_emb` | [RoPE Paper](https://arxiv.org/pdf/2104.09864) | +| SwiGLU | `liger_kernel.transformers.LigerSwiGLUMLP` | [SwiGLU Paper](https://arxiv.org/pdf/2002.05202) | +| CrossEntropy | `liger_kernel.transformers.LigerCrossEntropyLoss` | [PyTorch CrossEntropyLoss Documentation](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) | +| FusedLinearCrossEntropy | `liger_kernel.transformers.LigerFusedLinearCrossEntropyLoss`| Inspired by [Efficient Cross Entropy](https://github.com/mgmalek/efficient_cross_entropy), with additional optimizations | -- `transformers/`: Correctness tests for the triton-based layers -- `convergence/`: Patch Hugging Face models with all kernels, run X iterations, and compare the weights layer by layer, logits, and loss. +## 🛣️ Roadmap -3. Benchmark +WIP -- `benchmark/`: Execution time and memory benchmark versus Hugging Face layers. +## 🤝 Contributing -## Roadmap +WIP -## Contributing +## 📜 License -## Acknowledgements +WIP +## Citation -## License \ No newline at end of file +WIP \ No newline at end of file diff --git a/docs/images/e2e-memory.png b/docs/images/e2e-memory.png new file mode 100644 index 0000000000000000000000000000000000000000..ab2f9176055e353199e0bc0ac73e891c8acfe804 GIT binary patch literal 17055 zcmeHv2UJu`w%Q^UqB$~Ee>8fSh$-pdpp=Wx`}&BUH@~1ICzc!nD09CpG(~Bq^|3# zs58quxmqxb@ZRFRbzPd2nVDJA)!b5C^Nzyt?ckl%b!&HbXK_A0FE1}%FF{@>S1Ue# zF)=Z|TLOFn0zBXf9ycFHcQbDuM>m$g8aZj_j)mJ}R~u({8z)C*e7k0koIKp6u3yJ@ z^!J~?#_4Wj`LCWF-Hsm%JRl$b8$N#CTYP`F4Q`dhe=4r#YGVO<#@^+`l8Mk+*@muELU_P1P@6Z1A{Y{x*YC(?gf8Rl@Yk2CfTbYAp zI*EvN`JZTBJ-Hj)D*XTHt?Eo}K7lUK-6o0z>Y5Q_KfP?#Wc&4>O`aQV>;?^ixCHd2 zozt+0ql2EKo7L7Z1y4f1wL19r(m=~wiPhT@%Q@+#WqLcrNk-C#(8J#m4voQtha;4V zSIwTBCbzZR?Uq;@?ci&$=Y|z-HSjhZY*_ARcr*{FK^=-}H^NjBggBXBHV%w8~9rr=OkiS||yLgEYe5@p&HiapmR9q2_J%OPYm zqd`YILE{O+cDZA|E5ikq{(G+b+eJYRMfF%)ZFuHIt_HK_(+0KPB~l0L&6sQ`j?*fp z+-Pr1Z9j?*Lgjqeb94}q?Ap$$a__wWvee==k6!6F%kCr_r}{n1D7W16J5xaoOa!8B zrhj^mm$}ZO+kH#YHI2XyS?%H3BegSX;8bVfw$NQs6o9sK90>~|hu@9CEv82th*rcI z`?fI#Y-!Mo*z10Wr_phZ*7TsbQske1p+mh7?h?@a}5O8Wx_rWA9K+mHoLsf_hfuOhD+QObD96Mi{fc%i z(TiZyMCydxQ7m>%<7(9$tMze8N>^)dlcvoj~ zZDsl$jhEpybiAn3N2$$D!9$0KLu#7?3U9C4C^7|JJYsOaBZ9Vy#31L>6(dBt zxHai5dp@E~SjfrA4`041ELiu*Sy4?d?tvcFLO*2%8;q=smOM^GEp*4^@om+m9#mIX zI}08Jst0YM5CezXCPx+bjlJ)t-^YHWz3rU3jUE?G8Zxiq$x7Nz&v+{kg9$3TOnr{t!I9OlUXg)q2the zXZ&L+->r2xk5TpD6mMAzqhBY*gi!!hE7G-tM`fNVV7=)~U?b}pqauXtT~Cu%6}Y5u z%FTVu#;3cl58Q&5AEuBl$J)uVRbv$2TzSVWF{+bqqb_4D+np%lPZyK&05lfUJu z>&{I^|MjN8_3XyosFZ`*kLO}9$Pk1?lSJQa?B{qfIU2N6GqqPf%~b6xuj8C4bC>YF zg0s-_#{SgNWKjB$Aw{?dr_|x@%wZm8fd2Jlj!;6TXv<|2fBr9!r8O$7np2q9q)#@q z&6tG+Tduv2LCgEy;j4c45>+RZ`;H-px}Op9PU#Z6>Xzd4*6Az_$qFOcbh2Vn&VeC~ zs3h&R$vVINM5<2C#{PQ`Z#TYAd{58MJ!`m351}@ycK^LF$a!xdF_$Rp(R&3CcQPf} zsY-|O2Q2q|hkO^hlU3aOT{GpDQEy?v8+B^|JrrERarZ(70SH(nlx3jB<3g<=W~A*# z!I^yu5JjvO3f6$t4$4qjNifN&6|8mYIJCmUnEW0{h?j0;kXD zRTrchaPo4ds%u83F?EvqyN}DSDEV%>FNJXz-ctD@g&BQ}(|}0vPVde{?35YM2_vf9 zvT@6>hK(oONUlUlAJ;tiMdY5o?~sEmE>_y2o$3f<;N2iMmi|-rK0ImA5&27NeDH@d zU!4#~WuyR7*5u&VD6K6^kS5cIT~omkV{T!6=(x z-%H$WDx%BO{F$$o%ekbz!V_+6lOngAkBd>2+SY>Rz%Gbdc3ufe45Yk`+Zw>3mtXVw zPm>*bhUZkzI%?#0y}0W7a%x_lLTa$#Zb87;kvs{u8on(KQpo$FI!r>{(c!+M=#jYk z&&+q_!$j<&wjX*})WZ15xrV-mE+^ZakM;B-tj&8&cDI-oxUC$$koV!HOR=vM#N2D- z8n&0ozp(~%8ropDmK+~O!MqIkYc*K-t8tl+k@O{@&kSvb^Ysqzeb^A{$*SOdV1n8;+8Hxj8Dm^L+84B&fV;(6-?D6&_Y386?zR9%0+xLlZTv? zmPdv)`Lm`4klRIT`~ei!j2VpNE(0o}d+Pi;y~8$d{bEe6D1WyFCikGkF`!VIRJxbTGZYB;H>8;2CH9ANOq@JCycF8SyV`! zK+B?2^75UfYK$ZqiSl_P(mn>(`a`~%H6ehAf&>&Lwq zD=DNkl39kdK2P^El2Dv zq)g35f=$xLh0N_Q;OMIBtEH=J1E?6 z#z4CB6NX|{-I)eYvEMQko(xhXJj^*9S#WX=X%e7N%`ndd_0cEjf@W#fI7O_G{ zuI-xsV%p!U#uz6j2XLdGBXaR-zP~JzK09%O-Xm$B{3w)+cRj_hKs0lG@y@Q!TW&=s z;_7Fal&4q&F81LtYZ%TgZKiQpKaWUP`q^x`v6;66^zRgSglw>%Uh{5@MM3X@*5jt)CgtnKTU=oF}qB<+00LIY!)j-84w(-o6c^%5CcYk9kG)V@~PPpR@W% zDv&DgJyQ-JBA!Ll7AhB7?34uVyy{#>H_7$Pwf7gmwv$7RAdIVz530ITdPFXtvo=?} zPpG36Ho(3t+3v=4y0?Ho{PyQA(pwg((LHFXAjLH(5~J0awW6yAg|+W^ixzc2#1ES& zkEm_OG&Qh|A@efJm*SKhZ*bQ(Bqql7+%w$09ZLxeRi85FrNPsLo-|QSr0c*$mUBOv zW@pgaVT6ae3)X+Y9$>+a`#JJ$(eOUpvs-fW!BLfp@0V-^3Q}5!R^EqhB|^4o2C;M{ zE^q{?P0uU#kBe~m2#a3W?PpLMy?gP~0#d z_Zpa*GR!$vO@02{rZ74D+Q?(N=%TeA!w6gTz^|R-0EfPbh zs?llVx(O+tM4gbN?(m>N3c6?gFJI3mExxFcK8p!=@@FCWD%8Nm#Yxg%AXnG6_)y*9 zer}v&5$7|-5>lIpw@28KhSf9Twh;=8R9pTl22-R8N?L)}ENqX4-{*N@7GvH5w_fd# zUA;UIL3-Xj@1dFyALjmQ;bf`DrDp9c#&OCR($8*MVtcboSjn9UH}um>s_WzV9!S!G zq)I~yT8DzpbskwBMIu&{P`8a_#-;czydT4sxmb#6onK6_cYeiu|p#)1pN@DOkJJSVreu z3jK0Vdf}9fU!#gFeSkKnAN|z3+}fhSYk?23hi<4-vgi&GD;QC}RF>I4 zZ+z+Jr@PzFUp;5UdEcY&Wx9gEpWs6u!u9C9j{a{OuWCq^N+Jyb$354eF!s?8EuoI= z%Td{@MK{c$Q9KdXA0h8XjSEkk^n7pM(Xq;>PQ8U6vK;Z1rrgN4sdP)1dTV*Qss;?X za1IZ;9JtsxbNWx&ZGRqjWgnUbQH*QZi?!Gq<0K}qrW_RnZpBJG-FpCG!CRTsc?f7_ zk2fpiIMi3SB_zt%eC;L;zQkXV;{)C&tUaQ8c`PJk(l9r2Roq^{a~wQOCtPN3{6F15 zPSGAszYOu%^+<$8yu;e8wwzHjb6K^g54uA&^U`QCzd_yPK3_!nnzTnH%!pBq;B9ceWp4mZkIEuj^ zgU>I4TQ9IS@_glB*wEyQbaH0R@!w&%?QE(#5?i@2QS~z1C9Y_rv>_}<=hur_dc)7x z_g3jGXk#ubK6Z-$au;z-z9ounwe8(V_OZ;$=9Wnkhy5_ z!?^9@WnJ4#5Ex(G)_WsAbOoy;XSO^+2nV4;@hA6I`P2tQ2o~b$ZI6{<#MBXPYFtB+ z!2*u-OoWEb=7QdMD{L2{pP$iXsdW|QExfxQ@Xe$&yh#k7 zUvLuP**Kz zhm2!fkRBt5%k}HeTC=7=NP=l)i5PuDE0|wc=(3@EZBD`HjKmYsyzgy7HE*o2G?&s99N%(Si#PV#ON}_lC(gYNcTIQNafzT(VG>3aB zM+iQ*rcj7y8*AJsHO!C_iTtsMd3{`6t|={ zoCo9c+egE38XRyZE#oiec(@~%kTvk2U(Uu_wBZ%@2PQ2~qw?gv6JxzrCzc}KR~@jt zz5SKRLj%{mRF;Cvm)e_+3M?-6+i|3q@-4>!DTaJab8usnTa?;;V&ZeM0TPOesdKx{ z*DjD-$Hg)Mop$4J8Y{N}p+W@WcMH>61oqh`8K>yJufJXjIt6n|V_eSle0fkfeZvZxX|QqG&3u@J?ufQXez=<|cA`T%r#g!Bd+ zEQ_Ddf-r*r5o_+j%jN`g+zj+E9X(dj!KRvIU1r&3`#$@pk}Qbs5oOsyx~4p%{zhF+f>C_w}N%Z-At-TDz$_aAP**=wP8~))Gw>Gm;VcGUj%>Zn^cl zWcU92hpl03123W+rEFGvYkfBD2Xi!JV=+PKiV=Gd%zCykc($LD$r(0eYNl~>j8h)v=~s@ z5(f<~wF)08G_}c|VK4`w?b(g=jnHI$y_e~4@T8}HAGtP$CkH`XxBS~dfH&9>kb}Xs z1RW?wW}*{x3ou-b{jRL+$o{Uw_Clv<3qv%6fceX0pZR!Cghkw5c0}I9m-aRhGlr0I zl3Q(3JW?tWuP@$v^xjW{y3Hk$EyC_&6kF|-$B2;%<|VL;$KRB$THK*BDb2sZR&`l_ zPru}5IT95ilK}ZJ`*D6U8(ly;xV=}3c)nnO?3qeiSwFzo%^3Sehd44hPSzrVerW9y zMrru$eH3{suptHG#Lx#gOZd$o4-U1uWpTe428)C{G`B7<$fWAv{TQI73@09lpXm2t zwGMyRr|W;q&wIoq#qCx-@qljq@mdN1Pi)H{+t2_o#>>#rYQY3~{+J|u=!n0>RtvmW zkchQBRP3> zveF(Qka){1)J990^OaEW@@CnQ$7d10Sqn7`%f;|R%(mk#5VU5tJx{v(}KyMZuj z?tH(U)k9k#h1)OnXYzKuZzUtOB0OH13mj=OC>Grw$;~$)@N=mQ*!Q%Xs;|SNU{em8yYDD}n_7cCpT|%e5VBA7Yp4B0P>xP>ul?t&CsuUe_1WCOC`=!HOe3H#sS+y zojDrm0+VRyk@gC&PfuO@Rrg+5S|>-Hd4N#eH+72}EiC@)iHn+0roHOQd(6#bQZGUt zHQd#G6`rAbGAbEPur-uEcU2uv8Z$M>+6>V_{lT#B^U6~Cb%kOm-wSz zX-A=tX<`B5!iq_sT}fy_^)TY@B|4PgnMymC6x%LS;yiu7x4tZYOz=WkqcBA*?-$J> z>ei*CiVt8bmcVTAya;@Kmn2`$B;j42^H94>_xfIxK3k)bQmpIo#$~R+j|S~a_x5-+ z_s?E1Wo1>aIv#$cJ^<@MPbqT6&_^S?$CETak9yWghC zdyE4aI>7|o6-H{!C;f4P_A<;lFCQlm4PXdzz)uuaAfGnVPv#$zf!` z`J_K3&>p8=``zQ=43b_j&8%FpxqaN<4OY-z^v${Jf9*{fTQK>Wz4l@HMzqriU|9C_DPqa4sX7ZJP<=&05BRPl&Z1lq?B^JPg!LXaI3ScLtYKNE=tiJnpK4wGE1*AKN8-k8>6R!9L3j=Wnq)hAZ zoD`p;^ziVIz3XA19yN95{ME&V^K?QSV65vAm!HyHHuOZ~!?GK@tL8q#rSB@^!HY}v zXWB+*Q||YaS0z)|C4nl92BAeUa7PFfqRio7SISKVEazjuBTH2hMG&3wf)2B-@0u!N z?ij1zCOgJ}V41U!M@Lj?^Li@r0^cbL-`@U3GH}C-yoyt5A5|ve^6SpgrKj@G&z0=2 z*7Tg0Kox)6I0)UN2q6hcb{kimWsB2>dMdg;dnf=VPd$Sm<@(Kwye(!WXLQB7o7jlC z03zi*s(BuEEm(#YfY~qk2po->zjpeKV~CMS6$hY^TQ-#=d<-gD#Q>BbUgxz>LhNNK z5Mp-+rVk&FSiK3L9`%Hn7WuKMT9KE$EYQaVBsy4Z5)FQ7sM(#no@A004thge#o5n>{9Er9QVSW*=j&?K{vtPb z+iLgV=13(l1BV^Hf)g_kAh}=~Lzw*V#MwB2MIQ1z>K8v=WR54`2HIXR)?+&w{J*h* z8#Ja;8?7ulFxW`OeX`5Eu}f{_jLabCVB(2ra^B@}7k;^J^$O&1iy0(5bwR-B0R9bsewc0A@E@UP=>vzL z)vASdcT3gRGMS|3xRt-}8h2W%c|p%zxz&i>SxX7m9P|ZV5QNXR>`BgY>a+pxZ$-83 zu2l2noLxP2=KPk)8FD%*R2&%4ZVQ>*Yck#%`{BHYrVA4rg9a&|SOE^*`o;bK*QENgmGNYA$|^et1ot6R)iBDQhpc zta5bth;b6daNWYRZ%kZI0!}jXeRQVrmb^cTm7yuDXe|2aF;*1B z)N=dZrjMOKVikbIm~qj;b0?6f#f(`n)&4BsmY1w2(c;M?K4m{HktycY%xve?O0F|v z=Wo;9qW9MW-76tr(Y07UFe)-uul}jzutg+xlw@Jcm;D#-V^#r;&n=@1I+TOvU6abT z3TEH@)o-*B0A7@l4CG{Dh5^Le7AV?K92@Zes~0TW708yK0dvWr^)sHe29li5q92J$PxQfCt<2DD* z@%?tD`?2ZWO9O^LDv``~;^|63>up#~@#Fc3FXuS?U@H~o{%z44nWwy zs%E#H^p_9XE3J<{iAOkrt5q_~9xy-YPY1N8XQFfxk5J(QfZ_ogytBu1{u|t&y?nmR zli`r!1Ax&bp%WqEJwBiqj_W@e&NDDvPLz8Bc=A6Dw2=<78%ToNWuhAel|2!{6!vAp z^a2_+-(yFz?UIdz(wd~*OlwBlGnnxs3YiB|v9PEa84>HHFY`_P5SVQ=kyvunpN$fE z@+>kLjS1VCv{j_APgfVtrKDJZTbap0&o+w|V<$R3EA>mMMxd2u+k?_007&c-Kv8t~ z{p-8z6Po`nGL6q5W73&$t$>l+WhZ+kBm*`r>z1XK;uyuIQaY3e8~<&Sn=kWrC!&E&og=9@2s@2V+W<)zr2hkqGjkMJ>iMTm{Nby^( z5=gERP%}6TyrM&MrSOTpC-#~_#@5P~>{6FoGCvY`%e0YTWvU^F_d_V0^`a?{N&Q4& z&8j%QE~GC*?%#v=A0K%5JGA##Xa{di?{=~qnbh@q-YxVn%HM_rF8^yBXowb}`h^u5*&D90_VP}y)yMkGrC}h-$l2@-Zx0hBT*q#;JSGG?A z@GyL~X_xm?=<)tlcK~KRH2iq*B*N!128jHt<|XL~OR4}0gSJ70hRJa!Io<#f!q%>H zos46M4NzeRDTRRJJ^G)6_}@AcsrA3??m;MJQmY}JYeM5CefBFaEaQ=uID(;jC`S+6=~!?}8c4ZaOBH<qRM*VuED~h#plI`5mTFh2wH~e@8M`H9BUAA6F<-LRf8(BQf z6(SRU);eRkZlTjS-U5TcStjblqqRpyTamP7b#;{ahclh$B}VLfKS;^y@VB5R9#z`E zMJU5WW2Xv}5&W=(ZE61=4us!k*T-bb%4zhEfb-Zay$H2oonY-Z$wFab=Qn?>aHJf) z{+(&E@vdMu;Kcg1IoA6OEQ=-`F<72Sw2afY<;Ef(!X6DvkIC{L3X@HRsQCn*x*k8b zsX2&2T_=_SIokRt!tD-Ps9=!nr^rpw@r`=DI6}*sO23u04-T6OMbQl%WQVr=)OIkG zOZTm=`EKUr3f(Dik4KtG(jA2H+2VGEo>!DP8X-pGov~0v1d2Q$vv|ahm~g^aqJr>AG3rH5 z8@~e9PVO?rrMPxf+{GcKhSfiXJ-ft}fu@xhR=&vJ3gN!KO#>lT-fjiw?aoy$kNGqW znwCAObqG-OOIJ$loD52H0fQtPseOGpXUQ&IbVqpsrYVydoYxWgqv%IL#dc7M=Xb`- z!^(L+qp6v5j{;sx@fuQKS@ShwEfk8|zjoiJ1Z507ljfMNHDQtB7oxA~_Cvbp$GgZp ztNku@#Z=agdrPP_!T4J|Xa{xQ9I`Q zDK%(WHbm#>5bLsr&I^%bD3~Vh+$98fC~a(}z<%bPieEHl zuTsUxiIynW1M{s5V>EvxIK!m4qVF?h^6GXz z-*p&CDo%LteT>@AkmeCrSJ^q&LYfr=l#5ml8XRw@EfA;7& zfA~D&bk*?dZ63@Qlz1;6;ySAldm|U8D7!o}gvi%BdtS~)efddT*RT%-Z5QK@{wGkV zZtu&!&kZ}0xdHSk_q97+5E;pJ8}1V7n}s@3eOIhGfA$wN>#KZ#S@!hv#US`tbE!e0 z0&$nkg?r(_^G4gWtKrwle~0E>?wtHGLZ7^h*!`)dVm>l}G-+t@+rUPJr|W>>(|K-( zruaHdYJoY&u$8`e_@XepX>gSxd_jFO@KUF^pR8d48!$<$W0MRc$h*^HJ71(`aK);2nHQ5|Qm^#xO}<^RDi?v;;4m@OltEJV zuBBxZ#6G#!Ae4@EfFo3)Xj$ULPHuq(=mYwXKD5rdZ!_NTT_33v)0yPxFGWnIbnHEz ziIZPk?5gTNb&nz$CB?Eb(<01KG&*BgwW>|14O{ckB`rQ!AoN+O_!w5fjrn5VBa~o8 zKda9-3?H2-U6E-pe;sXKw|px98QJAASp5dIA3lq}-b?|fcdG4sD>nb(J)!G$;tQ?U zQZ+K_CmrNejMr8Ipvyfov)IQy*z^u8&s654w|P5^UByScKYR>9O`#6MzIW5O*ksC2 zS~Skg#z{Js(LGmD{@mfC0o;U--E_qjiHe#JGq-wNfNaj@w^2dd? zwCdS__jZe91O6u_KS)>9D6hm}|l#XAtCr zzIZPYmv_FvUimJx{LuxNCpUEZm$-`S9G*v7c78BB;gM_)*4fJ510B+pc#FQ4?c+*b zq!!BMSyH;r4$pAm=#1?U|Eec7eyg~Qq->C0To2sP_WFxkt_Ik)DIaSB*p7o2Cjn`+ zaw?c~9)n8VfPfhp!x!fn?7?A*xzgF%V`2i1*Qjs16@z-!$KWJKIR-sG4l0xXWYvJ; zk%Zg_g*p9OcaL*0gv6zgdK}s*vHsURZ6;F&mv59JR^34i83Xm4ji<@!2J`RXuA%;+@SrvaAF3hNr<)5NW~+Ll zN?OK#eSiH#621<~M|6#hk`3mB<(S(AbD0d^Rpk${pQJ~#)H(p4{D$tf<0h(-OvjoH zCaDT1GezI1~4V0x=Dnu|c%vM3xRHUq`cvD8C zh%`QG;C?E#TA(4Ycu=Nu)aC7jiAR)KS*hNhCWcEC6_CtsyjgxB*dr5}lwnfNHj+*N}O+IvndmJ#Y zN9ttHhw;Q{7iVNTIT^W$;RLD^bwdu9Stli?~$EVC6+@euTv8 zmrkam-Ih1lRd9NvRjhp!HEZ8mK1(}90q~&1JIJF13s%f%+?)8q@ov>@Aa|c{p(`@5cAlmc0~VTNB|9Zh zd8S8e`>4>H?ZU_v>f)yFl;*$12InIS!3fmTg z|8|hur#kkVN8&;DW}jz}<7MEJ9#v3Wra2Fa2W(3bIGs=7RqYO?g4`;^uK1;l-Uj*o znFzxwG2N>9HeoR_?k1_KFeXag@lbd44XNXyyG#eVW#V>aJD# z7@DCotM*m(4Tg+G;Y3yT9(jCV0yin@C>}tfH3De2%eXn1=g5e4g47oZY|k|L-N;$f`h%TiP}BenczNRnX4`;R8!~CEoL5V*>4{83D=zC z66JXMOy&=8h7#32vtNOk1PBY-4sskdpYP9l6QE60|>4C z+(4F`tk0e!(TZslt^Z9fv6K;o0=Qjv_ce_k$UvaM>8(KRijel}!7{!e$nGB)DW?j_ zrSS$(;{d4JaHjI*aj&TWbb?)B2_BHF2e8{KXl|3eFudT_eS7BIc(Iv$WOgbF)L|Ec zbCEruDrZWMpG8|^F0a1=$xe z1<<`}uQ0Lzz0C+IWxW&5%yB(n*ahYBws@gwzUIT}>*{_)fcur<2{+zzkAmX7Sy5+j zPy`UZN=W!nfjGRduMlv>QDZ-}#W>t)N3p>bqNx$0p|yu-u_!td^9an2CJU%39&{Hb zzYCNSUkr-Y-?kX&kC@~&{}$sQ+;*x)o!<`T2|Gj;3z9URS3t$FbG|}C*nN$;%MTe_ z@Ok^c^MSSsKRX3`%JH?fpj^_HuBm_H1OMYt4F23M1{5Idh775QBF)nk!bvxT*p<_b z5_SUtdpSK$ruUMoA<}gesA>Jzz-qk>%!(n=s%spAnV@^z=s;x!^m>J96H(CD8oLif z+4!TxqTxcgjktz#pea`0tU1`|Ku^D5s#Np;b?LA~R^crKIoWa8@2D}i!0}uKoP%{G zoHxxpAWGdM$C8-xvbjz~B1GM~efDdXU-5IW zQ_5AqoI3t!R1MrZvXX%eq zN(3R5K*6l&iby}!Cka90*mdZ$p% HH1xj!L3tgk literal 0 HcmV?d00001 diff --git a/docs/images/e2e-tps.png b/docs/images/e2e-tps.png new file mode 100644 index 0000000000000000000000000000000000000000..624ba96d956a92b4612e07edb00a3891328d7c78 GIT binary patch literal 16070 zcmeHuXIK>7vMwMfq98#)kR(yEA4$nUBqzyn29z8n49EZjh>C*bFp`ld5+zDTQAv_> zHW7!MVH~(?5Zrrz`}{cP$9Y`f!U$m?Ym95p)ak#)|LL59?a-1{RA>c1g zupJKm@n;+ycJLbq=S({8KhCD(ou0)rOh0q_8P^>Po1-A^71JRm>z4F6wzH~9a^2A4`;kBVu!*;#=*W93WzC2>6RUyhyJM}i-F z^M6w2_mdtU1&=C4B*Fh@+N6kRAm7e`iELDsm(_iUyF7V5_VQqL)4GfKQwx~2p(V}4 zgWGIexWuFc6nB)rlS7%$e~x+f<0(9x<0-3R+H6$aM;nIMn(h+L^K_=+=+J|V=qK+! z5)eH4%7mA}gKKGEPPc>D9()j*m{U_zi~Wc4kGO9zTv!8JUDfZOo~Y5&i7JVXB$dIz z#XtTbM`25CtbF+~4jv&>0QQIZ1yXZY=HH*efotIFlZz^@Cuak2aH;10j3q6_!PRd~ zzZ`se1YAn;|LLW=PmQMd5$mA^A?VekLzlnM^KN-L_bRVNDYc?k8~KCIUFu3)o%EWH zyCnTf>vII7%W9IU(d^exV+DrG09!U>Ol^@Npp+ zP_!|mJy#3+#sWVO-FT#DxY`rpIhW$qPN69g1*^_CD2_iuHy*{fcS{bB-n2E+z-6l} zdANSB_Ol|RpGU7l5oYasD1XS#h4a|gp1wsONB@3<_6b)uS(x@xSGN?J2W7uHS!MYvo5Z0-HI4rPuMR!zz6SW|z@c3%;FBRcjg z=bbn*U{8tx{-5YW_IT#AWr=e%q%m82LSlK@a8ZwizHIHgcdseZ!*sQsU8n5*3yjJX z%Kep&O)8t0bdgY>@3Oe(Ql#|ZdPLvA0PEo3pn%7`mgrovd-C+pPNkP!aS0+$i3KKA z8sVa|SN7X95%(B9hl|Vv?ti|RAnsMM`FVdzP4%_5SA< zQQ`FBVvix)y=uGgvgEoc&X4zH6^kK$+j@D$#mK38A90f`Zf+fDuFUYGvx40cn=d4% zQ3E=faW4MoanF@hNuQ;fFA0K*Qb)hsMd#ClmZZc{1!WTGr8()NYCoFg7)`0SlTRvvNNppB5NcYX(3Q5MZlN9SV7qAE)42f6k)I{33)rW^Vy z91J+P`!RxrwN!O}XwUr)=pi$s?^>YH&6}!6hnSgPo848=lrtt5jFAY;!6>^Jb?9B5 zDv}N`*)n$Lmm(xWOjlbNeAK9TjV*`ef@p~Ggl^DE!ab4_7pN-`qKmCTh-(gg{I$Uh zaHXx#Vb+wU0^H$^nT{EtK=(n1${8|V_t8?T93^z9NziLu&VG@)wGS7%u0KKSH(++g zOWkHvs&~d5%8G2>)MObKBR>hf?ET8x9~)X+O_xWp9wD)zlq)T~*DJH186oBN=B@Xp z^*(y~$l1g;&5p6QAG15{a<02Z<4nWNsfW!ZpF2f2hD=tP;$%r8yg#sy%_N} zWBB56FJLR(+#i_$Dnal9-D{jIq*9>=(q`LbNl0#Cv(uaHZ(bR-xsRqYO zr;q%ZT0l0mNE)-CnXMSXuyKGY?tmU082CJH@%5%1Dm1AY%e*D&vm=zh)T9`c-<2ZX zaS5?@M)dnLmL+r4LT(xrCZmsxnEq2M`-0L-1etxp{g&`J+%spNX@nqlMs0McV(hM# zFEJjj1-t!PZxP=fz;PHVFfy9Flfz_X!JOVUUCn&R*J;HD41J=sx;30GOXo^w{({#B;gN|xU%t9dhjT8YN} z*`$n_m+taGg6cm$hm|Q32$S~p{>7_P1%r}r6O{-|N+;DcZgz=#J{YX`@v8nAtttP)MDPYel}2aX7K(=CjL%rsiZmU)vNa_Chls)zCw04vSgo}g}_I( zryVvH%P#P}YnsYYMe@0Qrm-0rU$9M9w)VKISXC||iF|sAk&jkl>fw(tYr)Aus1dG81=SzrpSR_I}NstB1!SO3Mu|isrpQ zV|BiHu11QZ5+*}U0~osP2A9u=$|Aa356qJ!Hlf zw8pS5sf478OGFouS3&zlwdx9=DCm&b)2?SP8w>r^-)BuiS6R zcZq#k^5w!I1maZe38J?3IG{=Vb`&!2Hg)3iT3Uq{khxPcw$XVuZZ~rG7Z&Z>TetKS9n0$%XJ8V= zZrVmK;XeEJGMRqDutnX`5{n?osAuzH%* zWW%_6)gT#}Uz!7ae+Ic6MOm0DwX2#ImDPrsf2@YGQ~35J`K<>`(0$~~1k>(LUVHoJ zVY5^|)MIC-(C{xWJs06At62-#NJuI1RxjPvo6hCa4r{OcllaFy(92chfuU2?xtMIL z#I}vu&xFLz+uHeEjRMQUEp3+mlOjnqsO-DyaA*6-T-?~h=gjKXBuN7y?a5?&?C zT{bkwKtu>%Wrh3|b-vUi*;SAJyLc(4%NlUxOEeiVEzkV&o>)ePbfU-i8k%o6(Do&n z5k!X8lTNeSR;#eQzNMrUy~d?&8k3bQ=*QU(!(kg^&6MtJuf9pxuo-Yu^*?8k9(@f` zoTeP`+B2Gy9C}8hd+dxR{QMA)-a8v#gD!IwhgT!l;z$mn;BWDG!xU@7{wtVh~J&sK@ zBz5rI8+MLmYIXP=*Kd^NLC}5=UzI=C%=HuHP9a*%IAe92JXDI2bN^aqjq||kyF0&F zFdM^P7eV|x#Fc!Vc@=5qnc$by*p<@Tp&)j!`GfIjWJ&}nDj+7U9FgyiC{I1uJL9Ji zEpBDp?T@bVFI!dV7fI>}S#gU-!Up8SqWS$6J+ZIx=KjM2NDxiE?(5CAFY{psi7Cv4lYU_Io7y>G8Rv%0Z+lJj=pqiu`uoooUKCmX@Wk!v0G-v#VcSXonk({pqhx(e1@y_@K|H;Lna#N zL{8}P=K@m#!V))1OMmRuSP@Lh=>iLq;CvHr`2{Ig4ITe-aA$|Di3nG6 z6Qz!c<>)~@4gK00WclYSZZWHJ-mBMuq6j=w=B@r~{5yCFXAOwr7Am*5w^NJqhJ2jq z%&kUd9K744ao@!4nh6jWl`|(eqihzC)C%|O+xDSv*}U~{e;DZW})*2FvWfRO1cM-yRu8i_aI zWo0Js$4wZ-=4|H%X4N06A+>LbO+_R_EZ;03=lE_T0rr-qk@9^<=Q&~_;CJAm^LQ`r zn`5lC#mYpr+u{CPN3S9yFG&94ZPNUPMt}5cO2@e%Hj~`GKj&zFK_i&FZBpeN$6fSJ z?kox2bz)-TQV@$diesrJx)AY*-vG&kHUfi!iknQpQH27YONQ0xmD-tj6KB^U zhsd?EG>qD8XA%cUJX3#lAN`7mF?e$U5x+2IVqm}lvLLng9L*FlJ>XI9RD;`Qm%1H& zdgj=n4r9pT=CJzI3Jl34o!Nqohg7D?#C-P!*XoPQ4&2=nam+SH_Pc~FhZTjYvm4RQ z_v3|IFo@R3Ya7VMUw7bJ^Vhib3rZAUeW^rvuLyV;ok;*0*4R1Ex&63a3cE`qrqU-yro(1x zEO`(i=)4&1=~HE}@p(l;TD5Ve{>!6{;cWP%tz`kj$QVSmRkVoX2$Qg|@JYU0#p^zlv!kg^x zR3K^UG?3+UPS~$(*)`GF<@1%Gtywr%9#HOiCZ9o+MI{ zr?xs(KNHH|P%>O-Rx9$OPUPxA-8%QQO_hidHo$2H_^m;xx_w%K`e zJor-nL`l1{$TvwmS2FWxA5(au=UMXB7S>%E%JmUtjtGw@oU;_=p4FN_Ti z>-dBvAvWDnC($8Y4Frew^80pFCo=yx3Se@<9m2XxN=izn5h@--rZeYAViyZdzs%}y0^t}%En31#R)VekYju=kZF*q5GI!H%*)H` z`5>EyUd$tMdAv$|9KX|@$Q<_+HavSKV>PnfQ*9sypNU&sT(krwGHpFQHgT_&SWwFl zPa*)rN2*G!J86yT;jj(+sXzFk0S|Mq9n;{4K5&k)v!2HtLKQbULurL5d&JLoT78i6 zI~bvuK%r2}!-p7jQYWizs!me_!ISeBII120AtizEG2*#a5a(tl<&F1od?>4CY+5`u zIFP4fHBw?xGo~TC`Q@vp)tege{7}9#XD>$1rTS=S z-1>&^vzhKaYxpnMI0R6}DQI6hMJDNyc(0($JeD(}r{>5+dI!Lr>pMg+xfA#k4Wc90 z!b;}G3Eug?GJp`{LKa_sHcpW#0&kUflEtg8?)H$2`p@<2=+KqHLOJI>&x&NYB^twR zbgb9)X8@8ASj;jJKATuq`Syr=dq_qjLizW(I^fB^b&B3oBN%ew{L&hiRCT4rgE7qt z3#p0#FnZz8-S3QlkEdy7?24C^x?;Kv-V1X|!o+57#ka4-r)w^<7HGHfH1<>n!g4aECi~|1S5IFplNWf86U3ZbtT)P`WR{!z*t4J5CcO3IUY6f&n{N ztG{#k`sbkMp{`$;$=&Cf0sf{upW1w4m@@kS9LKp#_X(WnaUBbeqrQsroG2b!48HI0 z@bcb1F|YrIfJ?Nz(edk26|3LtOei%l9Pj_^r|c4&B;5^qQ!f4&hP>iPvp1eU{z38TN=I- zN+n93c=N$mnY^N+_-nVHwt|wsTi4%w74R}_NK*>WUO;qcMd95tnz&i4O^301F_0oAtim8G&T@5Uk zcvOgY7#pt#?@C+{k4L8r+W<-pwwgj& z$)p*?L3@v3dk_n`G7!@7`HM31G;sWLXN%_g~wvFP^aY9jAuDu_}J%>dutyKkhE zs}ToU6*I|y3#L713KSW<<%?=p#1^LNYbEx+2JJ-HiN5$U-@Cpbd#2nLo?-a_y&_4~+7o9b zXO4;g@WQk@U-!lRnuxS+E4%{VMQxGORfP3&@sk>vC##4l4FKQ$fo#VU(|VQ-Rvk}m zio!`qm5~AvX(LZ{^u!`aRe>in^b`+MIB^-i*aTb$S$Zlj9>{yUkh@eezdn~8uoG3xbpP(!+&$NZ3s#wq^zACXWkQal^yac>fOmp*qdy0Mf^lV;U zUetLCb|oDho&Um5yH%+wz6}>&xn(o>MlV0BvT_J9ovX=cROx8+_tC8m0o(JL5$##Z zG08cV<>g9NR@s<_+rcEIAe~O$OvW?(10|T5duTziNC4aWl)QuzpWYAGft#77ER9vD zzIpS;a;__7NB&_9jZpWJ3uL&!hz0b>MO;D;;o4F5H*C6+{S+AtV`-P95qC*ey|oTL z#IWUX@0-f!{#dwh@WB^K3C(~uBKoRarnR`DkvI-B?Qw3yW?pTQyA%C6(QqRzj(5Yu z!{fym^sd)N(5xuIl8M&Kk_XWpBpZm`7fe1g4G~k>cDd3w=OBgwi{lR`TqS((ADiY0nZwqz=!|a zD|KsQ+W_qVB_y9&Rz}8$!@ryc6>q5}iKuV6ebv#FWTW8F5Hcbc`giwgTH|5Yo>!o>7u*J%EEPY5$0h|9UtnCZ7{IQP zDq#c&JN&4j!kFGn#g14`on)s+V9o!{MLPU!jdBCsCvOp#e}kcJl^6bAZ2$ntdvmtU z3TZYwDnUQ^rm(PL^I&U%1prL<|8g|oo#Z+itEz_T>g$i~|EU3-{d*0-p!4*3nYLy*$I|fFCfQ{*Wl_V7a|G*p?rpS;q|O2>_dPz{`40mb;8e6&V$x_2AV z{}5QpLSL@t&nrFFj_KhiI~cSFy|BKKk?R1J3xGB+jfmsRGVi~8yURjNHCuhER{x3~ zn@Ruf+q`S#Un;SZ*nS6GhWY z484CXN_vX+?f~$_C#f!enlqULbViUlD|3qGr~x>7wvFdN=GzPI-2&Akhs`)$G?eL1X_>;D0^?f$i^vVm+AyWN`6!R;WfR z)Gg6-%+KqY2hTnCC_#*Qtw}%Z!TeGUgF0)qq@zD+-YTOyw|%h&YF%+J9}mzPDX%0q zU(<@mJNUHriGw+RihSQUiqhO^g4%rWIi1I2rRi_XlHt^OOCy#0qbpS?)u=(2&;qoL zgJ|3!Ge(1Y0W*l1afaM;+RSN1qD+QijJ8gKa#=dvnDN+g?aJ^R%;-vRW7H<1bErQy zdKs}56cxI~jjNwk8{B?P3lC3ByTUWPH8_l!8*bE= ztfSW$S^Y?iXVo*FG(Lf&ouQ5KDr(Ug{isxsN<|l@unG7?w+>Mjj?=-OD^312bJd){ zRX2eN{cReq;f;}g#)0|DW+Z|FrP>^BJ3_oW4#WsTVo7_v4 zo(O;1-260T+jlX_s@n)U?^9r&AFP!2s*sYnCe6;;Cq>Sl;DL!jNVy~F!kDIIw+0H+ zDDP>QX^`01LUh=g0$fi_!W^QPO zLL8(-q414-_+DV^bAzhzD|P`DpNJ4t4EHb-9Hq0xL!P(g?F&?&($}0F?^f@BtBI;D z?6^}|=uBK9)H7yHuX9FHy`~|V;;?t#?BE(LhV-rRd~pDlcEa?sX-A#;>YPd73t{_# zNut#7x}HP_n4Ck2Uhfo94hgkryYVnu$U!&H$g&w-IDU4r+pMEVKRri!yr3Ei_f@CW zvTGWI$)Ge*V;dNMti4pTtxp$|4G7L7&=C^{G|$j;P0M7jtQ1O^2WWH&0;4Hx-?Eqa zS65by!G>6O##k;n%7sxb4H#2&3Rn0J!9_KLikZ-` zQOyCQH=juVN^bE{?C~O#mNDLt^fj?z6`c=>51cVxGNr?-9VJzTB_ii*pxO&!8CFmR zqidyYv~7Jd`Rz3hcZX5QC8e6O^g$~~d|Ds>AIEbt`wk6R2xg76t*d||u}%&D^8D*e z{h^e=#H~>Sv5?s0t@oMY1=~9;SK&rp|;BwPS#9I&XFGK}W73{r@4q@hnw#4r| z{OGi?^Uap@d2;$8{;~ZOd1o`ekk5jgFJT^}*F76ZPSF$WwP<>sK6Y#?u+^kQty25 zv%yEAhGZzxH$gJW=jZG4Awh+AvvBnvY&W#FbQ@;sOqwwM%A*+FURY60txT;u)JMJ6 z<40*UJL|9M*mWiml}F={dbwiv|dJubt+D^>wDDSaf=(L7dryN!+RcuzZsf%O^FzQ*iq=2_73q zPbWNb{w=kMdqKh7teI{Tf4;fwA;$hljiF?2Kb(bi;vvD(9myiu%||OX?ZU;hTClRz z;B4G@j3YbBFZuW6kNB2N!C#xNkh?YUeF>Z?G-#mjADrp|r1}od?Z*L%uwIJ$X{J-`89!CcEgWCLlq# z-pEE`)-?Tg@Yk~#wSD4cHST;+x$Dvb_MZbiP)?JXvK3iL zVA|c4TAt>C(S#*f9xrI)YwPH+P_U~-=CC^y5ggNjut4G2YZ+yH!+ib1=rLo|O$+Yr zg?`0&UK4BnP3aRHAj1@b$3|veah&_2=)y_p#aefk$1`n}Q%;k4&_icsuijl8RMjtr z@apO5mELb5Sr{$Lb)2jjEW5x6Sg^f}$CNQF4v!3LiZg(49#AN7J;yt3C$prf5*#vvYQi&(q0mFR^H5R#J{TRUuG6 zF#5{-#Q(Bm^B(?^=^qcAj(uPNr8}1YXr*B*Q|7r^pppK&fb9gp zRo{xV{V6|&?Dv4IH+?zkNm>3!hvzI&jLi(U9;E9U7<5ElyKMq7wI*@->a;@h|V>m7RdN}}%eg7kkO|pf^obVZ3ex1gKck>yc zxO%s;@lQ~}>4RSCPbpzAJRCrh(B*R3mvuYw0vrh^UK)nK@gNARo5qS3$c*TIL? zviKVI^^Wz;ZSgcvy8ijG`I2<<9@s!Y?llQB_yEU`4w@TIvt}voPHFog9Cd4LoHkC0 zXlrFsd>wF9ywA!3g1O^8&`|WOFKq08dFycRhvaxVl-&%}9Ab7C^NYuzvyecOBx3I$ zoRN-+Mf}>{UP~jrhjPO~j^mZQ78r#EZwY;d>yhmP#r!p1t1Ay3w0tI9TSN&^)S=t9 zSLP%yB%I$%X+|7RKDxCnP1p-EfyDR>-vez?&xUO;T8g0=nQqTQ!?(Hk4pICm{C(fbrhj4{VDKLUP z??yn-Igh%OFpe?DbasC9S*=GNr2o8iZ0W8m)pP8rh=Ya4!bcpg-=0uc?tht|v-$q< z`FTtjgr97&GHVMjA-NfH)U~($1pOL2j|?4!c#wvLt~p+lN>)IJfijz?jt9QHzq_Ue;81B!JZL6%(odR}@yIr)b`qKG#KnVv zHTP}Kubne@gv9`G%!f^@&u}#n%~jcED@g@o4&)TG2TT<;R{(?e3}4v1uY$T*d-ix_ z-vvuI5{})dcrekkuOVr|F)b(rE0NJ)*;7(SU^g8*&uY>;8X#r3cBP>x_rdvC} zA*L0&zFzE)>2MAq$u#l0Txl0ym1a*!2o9@Kn zLs#hBOwz)B0ZZjdbs7S*uv;DK!}xBz`#x=fVD>=F_PBhz^wYvBQ7GU`x&fZ!c!cYN zszEWbE%tM0#bJ~HXZ`UizT)u15wiVk$R$(&kYCWQxLD2+#x`tA*y4gTSi>bN8WbpJ z^|y$!DyU*O-GaICrq<85EesS6TkTy}1+8Diw8+^9rq7iCTkspw9)3($6)?SD5V7kG zSSg=s@R&irrizj5{?0Z^v#~p6G|fP@=@D~WWPkzEsJp(@g>Bej9zLh5qZ8B5jEcB# zBcw8;SENID!3qDD8s^}GnNsO4*g+I~1+a4bf*v(2`e=KCg@N!}2Mixc*Pl^}W61vP zx9or%>_L(fA+*09;h%g9?06xEpBncbZyg~3z{z?wJ)t@jG4>qFZ;-04T z|JjIkTCM-@?$`T9$YWNc`}qY83-HQO43h9xEKC@$bh?nc=9jIjtNYlP;R$xOLHD>r zX$^GHqR97pBM_{|burMcrQ$Qa4#3C8X4;X2HY`g31b7)P5V&;(3KVeyGl;$<-P9~< zIIv$BtH`JGU3>4=LhscI=#R?-cGwnb{i>g)y@SKdhYS3%VBd|HH*IVRpvSn)zY1Vw*cb$SxY|`LRsKeLz^$Cp{1q$8FWeNc(+;Cfg{zg_F*>= zDx0W z>8-$jjr&{LDc;4IW`G3LS7atWTJ4@Ic`!q%v`?UbnBiDO{+i0W3?_%v1ecHOb0x~| z7G5vaJZQbT51WI=2)6%RYS~7f+Xpr{l?QPUCvUaj8`Q}9`PG*dhYJxXBWBKVM)-kT z8Ukuv>x(YNp)zfR#{@TkQrbXUTwbxn!yA<9G%$caZMwhKd}+C9@esX=Bx6yyY)^rA20E}U)ns5uP1MEGw)@#8dXfYaw3vh4#eZ`xb+|!v?^s#2? zg#R+=HxWcD9PdjMR|_OtolxEc&=_oYFIAlKGROgP*$#kkO29Dv+JGg@0-S6W)sMeP;{Puz5qNaY>r47g*=OVK*j?eucQoXS I