Server Side - Services Load Balancing
It is given that software systems will go through iterations and become complex(both team, codebase, service logic) as time goes on. As people move towards service based architectures, one has to design systems for high performance, scalability and availability. Whether these software systems are deployed in private or public clouds, one has to architect the system keeping "service" load balancing in mind.
In my last system I built, I had account , campaign mgmt, micro-moments authoring, reporting/analytics which are all accessible for customers through web application . Then there are bunch of internal micro-services like NLP, recommendation, Data Platform services.
As we modeled the system,
1. We modeled the traffic pattern(to the level, of how much REST API calls a service is expected to serve per minute).
2. Then understood number of "instances" required to "serve" the API calls from consumers of the service. We came up with instance types to host this software.
3. Next thing was, how to make it available for the consumers of it all the time. In our case, since we were using Spring Cloud tech stack with Netflix, we used Eureka to provide discovery service(End of the day, you need a fault-tolerant centralized solution, where clients could discover and fulfill the API request).
4. You can create a Eureka Server and host it(refer to https://github.com/Netflix/eureka/wiki for details). There are several examples.
5. Make all your consumers of the service use this service(There are Eureka Client libraries available or even built-in annotations if you use spring). This will help clients to not be aware of micro-service deployment details and helps with self-healing.
6. Talking of Self-healing, you can use Hysterix as circuit breaker inside the client to recover from all sort of errors and service failures(We all built all these logic few years back, But there are frameworks). You can check Prana, which is sidrecar for all client libraries/services.
https://github.com/Netflix/Prana
To Learn more about, how to build an app with this pattern, contact us.