Home>Articles>Microservices architecture – How does it work?
Articles Kafka Kafka Architecture Kafka as a Service Microservice Architecture Microservices

Microservices architecture – How does it work?

Decoupling microservices architecture with a streaming platform

J.S. Haldane wrote about animals falling down mine shafts: “A rat is killed, a man is broken, a horse splashes.” That means the bigger the object, the greater its fragility. Modern enterprises deal with numerous systems, data applications on a regular basis. Eventually these applications intermingle so much that it looks like a clutter of random things with individual importance. Sometimes it’s causes so much perplexity that a new solution is right in front of us, but due to lack of uniformity & discipline we end up provisioning extra resources (essentially new applications, which we could have avoided by making some correlations from existing applications and reuse already provisioned infrastructure)

Conventional monolithic systems

Creating and managing a distributed and modular system is easier to maintain, easier to replace, repair if in case some service goes down (You don’t have to bring the whole world down, just the sick fish in the lake). Older systems followed monolithic style enterprise applications which would take care of everything, some redundant or unused components were also shipped with the standard package, most of it would be proprietary (And of course black box), there would always be a hard dependency on the proprietary tool vendor for support, any extension (would possibly be very expensive as well).


Microservices Architecture

Benefits of Microservices: Microservices are the modern day crusaders, they are easier to build (brick by brick), you add more components for each new functionality. The system becomes cellular and works together.


The issue with Microservices :Here comes the challenge, the comms. All these microservice based applications may or may not be triggered from external stimuli or other internal applications. The need for a scalable and fault tolerant bus is warranted. Now this communications platform needs to be future ready, ready for high consistency, low latency.


What is microservices architecture ? How does Kafka help ?

Kafka fits the solution like a pro, it decouples all microservices applications from each other, it acts like a message queue which is extremely fast, the data is persisted on brokers. The Kafka platform is already highly available, fault tolerant, distributed, ability to replay messages and extremely fast.  Apart from being cloud native, it brings open source features to the table. This means we can customize it anyway we want (to meet requirements, compliance, security etc). One Enterprise Service Bus (ESB) for all the messaging needs, all applications can have their own pipelines (A specific Kafka topic) for them.



Microservices Architecture Diagram

Fig: Microservices Architecture Diagram

Kafka as a Messaging Streaming Platform | Green dots represent microservices


Now these pipelines or Kafka topics can be highly configured to match/adapt to flavours of communications required (Simply customisations). One can opt for on-prem or cloud based deployment.

Kafka de-couples the enterprise messaging, enabling low latency and persistent messaging. Kafka topics are further partitioned and replicated across brokers.

Here is a block representation of how kafka works as a communication bus for all microservices in an enterprise. Just a single block represents all forms of communications therefore, eases the job of integrating large numbers of microservice applications.

Leave a Reply

Your email address will not be published. Required fields are marked *