Apache Kafka Landscape

• Active Apache Kafka projects list: https://bit.ly/kafkaland

• Self-service form to add a project: https://bit.ly/kafkaland-form

Before joining Diagrid to work on the Dapr project, I was a product manager at Red Hat. At Red Hat, I had the opportunity to with with exceptional teams on projects such as Debezium, Data Virtualization, Apicurio, Apache Camel, and Strimzi, and helped build Red Hat's Managed Kafka Service. Kafka is more than just a fast message broker, it is a comprehensive ecosystem of various complementary management and development tools and managed services. Through my learning, writing, and exploration of the vast Apache Kafka Ecosystem, I compiled a list of over 120 active Kafka projects I encountered. Here is a sneak peek some of these projects, with the full data set available in the provided spreadsheet.

Apache Kafka landscape projects

The projects are organized into main categories of the Kafka ecosystem, including Kafka distributions, stream processors, client libraries, connectors, schema registry, replicator, rest proxy, and management tools. Each project includes information on its license, programming language, and deployment types. Additionally, statistics such as GitHub stars or downloads are provided too, but it should be noted that these might have become outdated. A simple report generated from the list is available for reference.


Apache Kafka project stats

Discovering other active projects within the Apache Kafka ecosystem can be time-consuming. However, the data provided here is available for you to explore and use as you desire, saving you time and effort. If you wish to add any additional projects, tools, or services related to Apache Kafka, use the provided self-service form.

If you found this content useful, share and help spread this list to others in the Kafka universe. I hope to see you at Kafka Summit London this year. If my talk is accepted 🤞, I will be discussing the topic of creating "Polyglot Microservices with Dapr and Kafka" in more detail.

Unbundling: The Natural Evolution of Tech Stacks

As the field of IT continually evolves, there is a constant stream of new software architectures, development techniques, and tools. Some examples of these include microservices, micro frontends, zero trust, service mesh, and data mesh, you mesh it. Despite the apparent diversity of these technologies and approaches, they are all connected by a common trend: the unbundling of technology stacks and teams. This trend involves breaking down systems into smaller, independent components and organizing work into smaller, more focused teams in order to achieve greater flexibility, and modularity.

The unbundling of the tech stack

How come each of these represent unbundling?

• Microservices emerged as a response to the limitations of monolithic architecture, which can be inflexible and difficult to scale and maintain as an application grows. By breaking a monolithic application into smaller, independent services, it becomes possible to develop, deploy, and scale each part of the application independently, leading to faster development cycles and greater flexibility.
• Hexagonal architecture was developed to improve the flexibility and maintainability of 3-tier applications by decoupling components and providing standardized interfaces for interacting with them.
• Domain-driven design (DDD) is a software development approach that can help break down a monolithic application into smaller, loosely-coupled modules that represent different business domains or contexts.
• Micro frontends architecture is a design approach in which a large, monolithic frontend application is broken down into smaller, independent modules that can be developed and deployed separately. 
• JAMstack decouples the frontend and backend of an application by separating the HTML, CSS, and JavaScript that make up the user interface from the server-side code and databases that power the application, allowing easier maintenance as changes to one part of the system do not necessarily require changes to the other.
• A service mesh decouples the networking responsibilities of a distributed application, such as routing, load balancing, and service discovery, from the application itself, allowing developers to focus on building business logic and features without worrying about the underlying network infrastructure.
• Similar to microservices, data mesh involves decomposing a large, complex system into smaller, independent components. It decomposes data governance and management practices into smaller, independent components that can be implemented and enforced consistently across different data sources and systems.
• The 2 pizza team model is a strategy for organizing teams and work in organizations that advocates for smaller teams able to respond to changes more quickly, have better communication and collaboration, can make faster decisions and solve problems more efficiently.

The end result of every tech trend is unbundling. Unbundling of technology stacks into independent components and teams into smaller, more focused teams likely to spread into all other areas. After frontend, data, networking, security, what do you think is the next area of unbundling? Join me in working on Dapr and unbundling integration. Also follow me at @bibryam and shout out with any thoughts and comments on the topic of unbundling.