Architecting Event-Driven Microservices with Apache Kafka & Consumer Groups

Event-driven architecture allows microservices to communicate asynchronously without synchronous HTTP blocking. Learn how Apache Kafka manages commit logs, topic partitioning, partition rebalancing, and exactly-once processing guarantees (EOS).

1. Kafka Topic Partitioning & Producer Hashing

// Java Spring Kafka Producer with Record Keys
@Service
public class OrderEventProducer {

    private final KafkaTemplate<String, OrderEvent> kafkaTemplate;

    public OrderEventProducer(KafkaTemplate<String, OrderEvent> kafkaTemplate) {
        this.kafkaTemplate = kafkaTemplate;
    }

    public void publishOrderEvent(String orderId, OrderEvent event) {
        // Murmur2 Hash of orderId determines exact Partition assignment
        kafkaTemplate.send("order-events-topic", orderId, event);
    }
}

2. Concurrent Consumer Group Listener

@Component
public class OrderEventListener {

    @KafkaListener(topics = "order-events-topic", groupId = "inventory-group", concurrency = "3")
    public void handleOrderEvent(ConsumerRecord<String, OrderEvent> record, Acknowledgment ack) {
        log.info("Received event key={} from partition={}", record.key(), record.partition());
        // Process event & commit offset manually
        ack.acknowledge();
    }
}

3. Related Resources

👨‍💻

Nagendra Rana

Software Development Engineer 2 (SDE-2) @ Digilytics AI & Founder of WhatInfoTech

Architecting scalable enterprise Angular applications, Spring Boot 3 microservices, Python / FastAPI, PostgreSQL, and AI document extraction solutions.

Support My Work