Skip to content

Project 3 Part 2

Jump to bottom
CleverParty edited this page May 8, 2020 · 29 revisions

We posit that a microservice architecture should be integrated with a strong service mesh service to fully take advantages that each service mesh offers. As we started implementing different service mesh architectures we have found that each of the technologies have a multitude of unique configuration requirements to begin with the configuration process. When trying to utilise the linkerd’s in-built graphing feature grafana , it was configured with a DNS reroute disabled which prevented the tunnelling of the hosted localhost port.

Weather App (Live)

Methodology :

The setup of istio as a service mesh was a tumultuous affair but it provides unique advantages to the overall architecture which are not explicitly observable through low throughput numbers. Unique features of istio: The Traffic management of istio’s architecture is heavily dependant on its unique approach to Virtual Services and their configuration. Distinct sidecar proxies that communicate through mLTS.

Deviations from original proposal :

We had planned to implement linkerd service mesh but we faced unique hurdles during setup itself.

Problem Statement Development :

Through the software development cycle , we played with a variety of development patterns that we had to keep improving as the mode of interaction developed. Using kiali and jaeger : Visualisations

Testing and Conclusions:

We ran jmeter test with various different replication specs of our services. We found that given our current implementation, scaling up the services only resulted in more error rates and slower throughput. We further investigated this issue and found that the major bottleneck of our application to be the Kafka service pipeline. Since we can't scale it normally like other services. We found our tests to yield better results with a replication factor of 2. We conducted the test on replication factors of 1,2,3,4 and 5.

  • Test with replication of 2 :

https://drive.google.com/drive/folders/1k8Zn-6JWkiDfObnwkqzNnLkvOLNe9G-x

  • Test with replication of 3 :

https://drive.google.com/drive/folders/1k8Zn-6JWkiDfObnwkqzNnLkvOLNe9G-x

  • Test with replication of 5

https://drive.google.com/drive/folders/1k8Zn-6JWkiDfObnwkqzNnLkvOLNe9G-x

Tests :- [link](kiali and jaeger tests)

You can confirm our results by downloading jmeter from the following link: http://jmeter.apache.org/download_jmeter.cgi. Execute the jmeter script in the /bin folder. In the jmeter open the 'Graph Results test-1.jmx' file present in the /jmeter directory of our Milestone-2 branch. Under Thread Groups, select the number of concurrent thread to be executed and click the play button. You can view the results in the summary graph.

Contribution

  • Anurag

    • Setting up CD with Jenkins

      • We were advised to shift our CD from Travis to Jenkins. I set up the Jenkins CD for our application running Kubernetes without Isitio. Commit: 2ec64d956f8648835a1e6efe105a66659ff1c31b.

    • Created a new data retrieval service

      • The previous version written in GO had a very complicated implementation of Kafka and debugging was very difficult. I created a python version of the same with much simpler implementation and better error handling. Commit: 1843c73be7cbf1cbb84a018225126ccc6fa215e9

    • Debugging

      • I spent a huge time debugging various errors while deploying our application through Istio. Commits: 3d309598072320f34fe10012a37f00c76b1bc52b, f98860108370c77f0f3f99807f87a4c8a108561f, 1baf67f9e17b384377b054962f2abe70bf3e9860, 42e8293001271971099de9a3a94c5496d316732b, 36797183940bbc1d32f5816a0547889934303ab7.

  • Sathyan

    1. Resolved any issues involving Kubernetes and its deployment.
    2. Performed Testing for different number of replicas of the services which helped identify bottlenecks in the system.
    3. Resolved any issues regarding API-gateway, user-management and session-management involving their deployment on Kubernetes and Istios.

Issues Resolved: https://github.com/airavata-courses/SquirtleSquad/issues/44

Commits https://github.com/airavata-courses/SquirtleSquad/commit/0a2190aa141c998cd8165c7296a5d2c1e21741df https://github.com/airavata-courses/SquirtleSquad/commit/ff844480ca22bc9e21bd88559f663836f1d0d6a4

  • Shanmukha

a) Working with the istio sidecar setup and configuration, and setting up the different graphing software (grafana and jaeger)

b) Configured the istio sidecar setup. although our initial proposal outlined our plan to integrate linkerd as well , but this process served a unique hurdle as the default setup came with a DNS re-routing issue that prevented exposing the dashboard.

c) Worked on linkerd setup and configuration of the injected pods. Discussions with linkerd community

Commits : https://github.com/airavata-courses/SquirtleSquad/commit/bdcce951da8461e761fca07ac91dafb1258c0ad4 https://github.com/airavata-courses/SquirtleSquad/commit/c3acdeac4913ef33b16ca50ff4cc9507ebfa24a9

Issues : https://github.com/airavata-courses/SquirtleSquad/issues/44 https://github.com/airavata-courses/SquirtleSquad/issues/43 https://github.com/airavata-courses/SquirtleSquad/issues/41

Bug Fixes : https://github.com/airavata-courses/SquirtleSquad/commit/5d24c60d0470e5830d181c32fa393bae1e9613af https://github.com/airavata-courses/SquirtleSquad/commit/716fc6d1db068e1d8be61b5e0e0d7526c639aa9c

Clone this wiki locally