Home>Articles>Building Spring Boot application for Apache Kafka Admin operations
kafka logo programing developer
Articles Kafka Architecture Kafka Security Kafka Streams Kafka Use Cases Learning & Development Microservices Scala Microsevices Springboot Microservices

Building Spring Boot application for Apache Kafka Admin operations

Apache Kafka is a distributed streaming platform designed for building real-time streaming data pipelines and applications. Managing Kafka cluster is not an easy task and sometimes confusing. In this article, I will show you how you can implement an application which is capable to do Admin operations in Kafka.

How to implement an application which is capable to do Admin operations in Kafka

In this example of Spring Boot with Kafka we use:

  • Spring Tool Suite 3
  • Spring Boot Framework
  • Zookeeper (version 3.5.6)
  • Apache Kafka (2.11 – 2.3.0)
  • Java 8

In the next part, I will introduce in details how to implement our solution.

Step by step implementation

A. Running Kafka and ZooKeeper

1. You can change the default Zookeeper port in zoo.cfg file (Default port 2181).

2. Open a new cmd, and type zkserver. If the command is not recognized, you should double check the System Environment Variables setup or navigate in cmd to the zookeeper-3.5.6/bin directory and type zkserver there.

 

Congratulations, your ZooKeeper is up and running on port 2181

Thumbs up green emoji

3. Open cmd and navigate to the Kafka home directory.

4. Now type .\bin\windows\kafka-server-start.bat .\config\server.properties and press Enter.

NOTE: If you use any other port than 2181 to your ZooKeeper than go the Kafka_HOME/config directory look for the entry “zookeeper.connect=localhost:2181” and change it to the desired port.

 

B. Creating Project

Create a new Maven project in your IDE.

Have the following dependencies added to your pom.xml:

 

</dependency>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-web</artifactId>

</dependency>

<dependency>

<groupId>io.swagger.core.v3</groupId>

<artifactId>swagger-annotations</artifactId>

<version>2.0.10</version>

</dependency>

<dependency>

<groupId>io.springfox</groupId>

<artifactId>springfox-swagger2</artifactId>

<version>2.9.2</version>

</dependency>

<dependency>

<groupId>io.springfox</groupId>

<artifactId>springfox-swagger-ui</artifactId>

<version>2.9.2</version>

</dependency>

<dependency>

<groupId>org.springframework.kafka</groupId>

<artifactId>spring-kafka</artifactId>

<version>2.3.1.RELEASE</version>

</dependency>

<dependency>

<groupId>org.apache.kafka</groupId>

<artifactId>kafka_2.12</artifactId>

<version>2.3.0</version>

</dependency>

 

Project Structure should look like this:

Apache kafka & spring boot project example

 

In application.properties include “server.port=<port_number>” with the desired port number.

First, we have to make the foundation of our application with the App.java:

 

package com.technaura;


import
org.springframework.boot.SpringApplication;

import org.springframework.boot.autoconfigure.SpringBootApplication;

import springfox.documentation.swagger2.annotations.EnableSwagger2;

 

@SpringBootApplication

@EnableSwagger2

public class App {

              public static void main(String[] args) {

SpringApplication.run(App.class, args);

}

}

 

C. The Service

As the next step, we implement the service methods for the Kafka operations. These methods will directly invoke the AdminZkClient methods.

Have the following imports for KafkaService:

 

import java.util.Properties;

import org.apache.kafka.common.utils.Time;

import org.springframework.stereotype.Service;

import kafka.admin.RackAwareMode;

import kafka.zk.AdminZkClient;

import kafka.zk.KafkaZkClient;

import scala.Option;

 

The key class of this solution is AdminZkClient class which contains multiple methods to operate with Kafka. In this article, I will show how to create a topic.

First, you should have the annotation for your class:

Kafka & spring boot code examples

 

Now we have our client. If we want to do operations in Kafka, we simply just invoke the AdminZkClient’s methods.

For creating topic we introduce the following method:

 

Kafka & spring boot code examples

 

D. The Controller

As a next step, we implement the controller methods to invoke the service methods in these endpoints.

Have the following imports for KafkaController:

 

Then you should annotate your controller class:

Kafka & spring boot code examples

 

How to implement a Kafka controlling operation

So now that we finished coding for Spring boot application with Kafka, it’s time to try out our work.

Run your application from IDE as Spring Boot Application or build it with Maven into a jar file, then run it.

 

When it is done, open your browser and visit:

http://localhost:7775/swagger-ui.html

You will see this:

Kafka & spring boot Api Documentation example

 

Open down Kafka-controller and then the endpoint. Click on “Try it out” give the topic name in the appearing text box then click Execute.

 

Now if you visit your Kafka log directory, a new folder appeared with the name of the newly created topic with a partition number which should be 0 by default.

 

Congratulations, you successfully implemented a Kafka controlling operation!

Thumbs up green emoji

Written by Miklós Takács, Software Developer at Technaura Systems GmbH.

 

For further articles, please read further:

Spring Boot – Apache Kafka

Using Kafka with Spring Boot

How to test Kafka Spring Boot

Kafka Streams

Kafka SSL Client Authentication in Multi-Tenancy Architecture

Deploying Spring Boot Applications

Building web applications with Spring Boot and Kotlin

Spring Boot Tutorial – Bootstrap a Simple Application