Microservice and Distributed System infrastructure with Spring Cloud

Introduction

In this article I will provide an overview of the building blocks for creating a Microservice infrastructure and in general a distributed system infrastructure. I will be using the Spring Cloud project an umbrella project under which various sub-projects are available to provide the building blocks for creating a distributed system (Microservice is one such example). If you are looking for a technology that provides all the components out of the box then I suggest you go with Kubernetes (K8s) the most widely used open source containerized platform for Microservices. This article is for folks who want to build these components from ground up.

Building Blocks

The above diagram provides some of the build blocks that go into providing components that are required for a Microservice or distributed system implementation. The individual block description follows

External Configuration Server

In a distributed system there needs to be a way for configuration related details to be centrally available and managed for all the services so that changes can be done from one place. External Configuration server component serves that purpose. Within the Spring Cloud ecosystem, Spring Cloud Config server provides that capability. Consul and etcd are the other two widely used alternatives.
NOTE: If you create a multi-node Kubernetes cluster it uses “etcd” as its central configuration server (runs three pods typically for high availability)

API Gateway

When you are exposing the Services via API (typically REST API) so that they can be consumed by either other services or external systems you need a gatekeeper to control the access of these services. API Gateway is the component that provides you with that capability. With the Spring Cloud ecosystem, Spring provides framework that allows you to use Netflix Zuul as one such API gateway server. Just recently Spring Cloud announced another project initiative called Spring Cloud gateway that is worth checking. There are many other alternatives listed in the above diagram and that list is not even complete. At a minimum some of the capabilities that this component provides include

• Dynamic routing
• Security around access to services
• Rate limiting
• API management
• Acts as a proxy to external clients accessing the internal Services.

Service Discovery

With services build around Domain Driven Design (DDD) or Business capabilities, the need to allow these services to register and discover each other dynamically is there. The Service Discovery component provides that capability. With the Spring Cloud ecosystem, the Netflix Eureka Server provides that capability. Apache Zookeeper and Consul are popular alternatives.

Logging & Monitor Tools

With any distributed architecture the need to have logging, monitoring and tracing capabilities is essential. There are multiple projects/libraries that are providing such functionality a few of which are listed below


Sleuth
Sleuth provides capabilities to capture identifiable logs which can assist in distributed tracing. Spring Cloud provides integration libraries for Sleuth.

ZipKin

Zipkin provides visual tracing of calls made across multiple services. It relies on Sleuth to provide the identifiable logging capabilities upon which it relies to provide a visual representation via a dashboard. Spring Cloud provides libraries that directly integrate with Zipkin.
With Sleuth/Zipkin you can identify Services in your workflow path that are taking longer time to execute and can identify the root cause visually very quickly.


Hystrix
In a distributed system where inter service communication can be complex and where one slow or non-functioning service might bring the entire service flow orchestration or choreography to a grinding halt. It’s essential to have fail-safe mechanism in place. Circuit Breakers provide that fail-safe capability. Netflix Hystrix is a library that allows a fail-safe service to be called if the primary service is not functioning based on expected response times. Spring Cloud provides libraries that integrate with Hystrix and also provides a dashboard that gives you visual insight into a Service health. Hystrix relies on Spring Actuator a capability provided by Spring to monitor and manage any Spring Java based application. Spring Actuator exposes the endpoints via JMX and/or HTTP and Hystrix relies on those exposed endpoints for its functionality.


Turbine
Hystrix provides metrics on a per application basis. We need something that aggregates metric across applications and provides a visual dashboard for the same. Turbine is the library that provides that aggregate capability across applications.

Logical Architecture:

The above diagram provides details regarding the various components that can be utilized through Spring Cloud to provide a distributed system infrastructure that can be used for Microservices architecture implementation. All of the individual components shown above are typically implemented via Spring Boot style applications with Maven controlling which component specific dependencies provided with Spring Cloud can be used

In the following sections I will elaborate on some of these components to explain how they interact with each other in the overall distributed architecture.

Zuul Proxy
A typical user flow will start with user accessing a website URL via the Eureka Zuul proxy. The Zuul proxy server registers itself as a Eureka client with the Eureka Server so that it can discover all the application specific services that it needs to interact with on behalf of the user. Zuul proxy allows you to control which request headers can be passed on to the services invoked by the client. Typically you will pass JWT tokens for single page applications via http headers to the services so they can authorize the client’s requests. The Zuul proxy runs as any other spring boot application and therefore can utilize the Spring Security framework to authenticate the calling user or application. In the diagram I am showing Redis cluster being utilized to externalize the session management capability. Spring Session another Spring sub-project allows sessions to be externalized into Redis cluster which is what you will require to do in a distributed system versus the traditional application server specific session management. If you are using Angular/React or any other Single Page Application (SPA) you can use JWT tokens for authorization. Hystrix (Circuit Break pattern) support is built into Zuul and can be configured to provide fail-safe service implementations.


Spring Cloud Config Server and Eureka Server (Discovery)
All components that need to externalize their properties to a central place will register with Spring Cloud Config Server. The Eureka Server will also via the bootstrap process register as a client with Spring Cloud Config server and the Spring Cloud Config server will register itself as a Eureka Client with the Eureka Server which allows other services to use the Spring Cloud Config server to store and retrieve their service specific configuration details. Basically both the Eureka server and the Config server will act as clients for each other allowing dynamic discovery to occur.

Redis Cluster

Spring Session provides capabilities to externalize any session management to Redis cluster which is needed for a distributed system so session state is not restricted to specific application server instance. You can also use Redis cluster as a cache server for your Spring Services that are encapsulating your business functionality. Also if you plan to use RDBMS (specifically MySQL) as one of your data layer you can consider using Redis as your second level cache to improve the performance of your database calls.

NOTE: If you are building a Single Page Application, you can and should use JWT tokens to invoke Services via the API gateway in a stateless manner versus the traditional way of managing state via session cookies in a typical server-side web application.
Services

Depending on your design approach, your services can be built around business capabilities or DDD. In the diagram I am showing a sample application that is dividing services based on business capabilities. Each of these services in turn are simple Spring Boot applications that register themselves as discoverable Services with Eureka Server. It is through the Eureka server that other services discover each other so does the API Gateway (Zuul). This decoupling allows each of the services to be build and deployed independent of each other and allows us to horizontally scale them for High Availability and increased work load. For example, in order to include multiple instances of Shopping Cart Service, all you need to do is run multiple instances of Spring Boot application and have each of those instance use the same Service-ID that is used by Eureka server to identify those instances to any clients (Discovery Client) that needs to consume them. The client then gets to load balance request between those service instances which typically are stateless because any state that they have is either saved in Redis Cluster (for sessions/cache) or in Data layer (relational like MySQL or NoSQL like MongoDB).

Two project libraries worth mentioning in the context of Services are

Feign

It creates a declarative style REST client that allows services and clients to invoke REST API services quickly with annotation based approach and with very little boilerplate coding. You should use this client library in all places where you are consuming REST API exposed via Services. The other benefit that you get with using Feign is that you can make direct reference to Service names that are registered with Eureka Server as Feign can directly integrate with Eureka Server and can discover services automatically for you based on service name only.


Ribbon
Ribbon provides client side load balancing capabilities with configurable load balancing rules like round robin, availability filtering and weighted response. It also works with Feign to allow service consumers to access services that are load balanced on the client side using one of the pre-defined rules as well as customizable rules if the predefined ones do not fit your needs. Zuul Proxy also uses Ribbon to invoke Services.


Spring Security

As mentioned before, each of these components are simple spring boot applications that can easily take advantage of the Spring Security framework to perform authentication and authorization of the users.
Spring Security provides Authentication Providers for – custom databases, LDAP, SAML 2.0 (as SSO), OAuth2, CAS and a few more and if none meets your need you can always extent the security framework to provide your own authentication/authorization layer.


Data Layer
The choices to pick here are plenty and hence I am not going to elaborate on this component as your business needs will help you determine whether you need a relational database versus a NoSQL type of database like (MongoDB) or an in memory data grid like Redis cluster. One thing worth mentioning here is that, if you plan to use a Microservice architecture and do not want your data layer to be a single point of monolith communication. You can consider using a Database per Service approach which allows you to build individual services in a “silo” manner. One thing that you may then need to consider is how you achieve eventual consistency post that architectural decision. Some of the things to consider for doing that is mentioned below

• Event sourcing – you can use Apache Kafka as an event source. There are other alternatives like using an Event Store database. Conceptually what you are doing here is you are storing events in your database (either Kafka or EventStore) along with Payload of the entity whose state is going to change as a result of that event (or action). The way event sourcing works is you store these events and arrive at the final state of your entity by replaying these events from the start until you arrive at the final state (basically you are doing a replay or simply put as an analogy – you are seeing a cartoon character drawn at different static positions run when you flip through pages. This analogy brings back childhood memories for me). You may have to use techniques like snapshot to improve performance to ensure that overtime you do not start replaying the events from the beginning but start from a snapshot of the data at a more recent point in time.
• Saga patterns to achieve eventual consistency– Two of the saga patterns that are basically built on an Event Driven Architecture needs to be considered - Choreography-based saga and Orchestration-based saga. I am not elaborating on these as these are well defined patterns in an Event Driven Architecture
• Be prepared to design systems for eventual consistency

In a nutshell, your choice of Database per Service will make your implementation of a distributed system a little bit more complex given that you have to design for eventual consistency. If your business need does not require a Database per service approach I recommend using one database to achieve your ACID requirements.

Conclusion

I hope you liked this article as always send me constructive feedbacks.