Top 20 Must-Read Software Trends Reports for 2023

In the rapidly evolving software industry, keeping up with new trends, tools, and best practices can be time-consuming. With so much information available, where do you start, and what sources can you trust? I've curated a list of reports that I follow to stay informed and ahead of the curve. These provide insights into everything from programming languages to DevOps, cloud strategy, and security. If you're interested in the latest trends and fascinating posts I come across, follow me or check out my latest writing on industry trends over at the Diagrid blog. I share anything I find insightful and worth reading in the world of cloud and distributed systems. 

Here are the top 20 reports for 2023 I came across so far:

Report	Publisher
Programming Community Index for June 2023	TIOBE
Programming Language Rankings - January 2023	RedMonk
Developer Survey 2023	Stack Overflow
DevOps and Cloud Trends Report – July 2023	InfoQ
Software Architecture & Design Trends 2023	InfoQ
State of the API Report 2023	Postman
State of Application Security Report 2023	DataDog
Technology Radar Vol 28	Thoughtworks
Radar Trends to Watch: June 2023	O'Reilly
State of Cloud Strategy Survey 2023	HashiCorp
State of Kubernetes Security Report 2023	Red Hat
The State of Platform Engineering Report	PuppetLabs
State of Kubernetes 2023	Vmware
The State of Edge Functions 2023	Deno
State of Data 2023	AirByte
State of Data + AI 2023	Databricks
The State of Open Source Software 2022	Github
State of APIs 2022	RapidAPI
CNCF Annual Survey 2022	CNCF
State of DevOps Report 2022	Google

While these reports offer valuable insights, it's important to keep in mind that they can be opinionated. The key to effectively leveraging these resources lies in cross-verifying trends from multiple sources and using them only as a guide for direction rather than absolute truths.

Are there any reports that should be on this list? Tag me on Twitter and I'll include them, subject to my checks I'm always keen to explore new sources! Found this list helpful? Go ahead, Share it.

Call to action: Are you a Dapr user? Your experience is valuable! Contribute your insights and shape the State of Dapr Report.

Updates:

State of Database Survey by Basedash

The State of Data Engineering 2023 by lakeFS

State of Serverless Report by DataDog

The 2023 State of OSPOs and OSS Initiatives

2023 State of DevOps Report: Culture is everything

Datadog’s container reports / Nov 2023

State of AI report 2023

Octoverse 2023

State of DevRel 2023

State of Serverless by DataDog

State of AWS 2024

State of AI 2024

DataDog 2024

Life in First Principles

Let's express life through its finite ingredients. There are three resources in life that everything else depends on. Every time you waste any of them, they're gone forever. These are time, energy, and focus.

Time

Time is the one variable that nobody has any control over. There are 24 hours in a day, 365 days in a year. Time offers the same equal consistency to everybody, but at different lengths. We are born, we live, and we die. All you can do is maximise the other variables in the time given to you, in a way meaningful to you.

Energy

Being alive is a prerequisite, but not sufficient to reach happiness. Given a lifetime, to maximise your purpose and happiness, depends on your energy levels. Energy is the ability to do things, physically or mentally. Having a body sufficiently healthy that will enable you to pursue your purpose. For some, this can be a physically strong body, having a good sleep, healthy diet, and regular exercise. For others, it can be sufficient to be able to get up from the bed and hold a pen. And for some even the ability to express your thoughts through a computer device (such as Stephen Hawking). Energy levels vary from person to person, but so are the energy needs. Energy is not a "have or have not" constant like life is, it is rather a variable that tends that changes every moment, and tends to go down with age. Energy is the multiplier that lets you get the best out of the time given to you.

Focus

Being lucky to be alive, and having sufficient energy, gives the optionality to spend your attention in many ways. Focus is about how we use our time and energy in a directed way. It is the ability to concentrate our attention in a direction that makes us happy, or spread and waste in the universe in a way benefiting others or nobody.


Focus is the variable that we have most control over, and the variable that has the biggest power to change our life. Used in a purposeful way, even in short lifespan and limited energy levels, it has led to personal fulfilments, or human achievements that are remembered throughout millennials. Used purposelessly, can lead to wasted long life full of energy, and many regrets in a death bed.

The happiness formula

If time is a yes/no constant, and the energy level is a multiplier for every moment, then focus is the exponent of all. Every moment, we are alive, we have a certain energy level that we can use for something purposeful or waste for nothing. Then we have recharge again. Every moment we have the ability to focus our energy to things that matter to us, or waste it aimlessly. Life happiness is the sum of all moments we had, with sufficient energy to help us focus on things that makes us happy.

Happiness is the sum of all finite moments where
we focus our energy towards our purpose

Every time we waste these finite resources, they're gone forever. Make sure you are alive first, healthy and energised second, and also focused on what makes you happy. These variables build on top of each other and require a delicate balance. Focus too much on one thing, and you may lose your life in an instant. Ignore your health, and your energy levels will suffer hindering the ability to focus. Do everything right, and still a meteor can hit you and end it all. There is no guarantee, or fairness in any of these, only the awareness of its working. This formula is re presentation of how these finite resources can be transformed into happiness in the equation called life.

How does Knative project compare to Dapr?

How does Knative project compare to Dapr?

Both Dapr and Knative projects help create and run cloud-native applications on Kubernetes, but differ in important aspects. I thought I'd quickly share where these projects overlap and complement each other from a user point of view. If you prefer, you can read this post as a twitter thread too.

Dapr vs Knative

TLDR:
Knative extends Kubernetes with serverless containers (scaling to and from 0) and helps you connect applications declaratively. Dapr helps developers implement reliable connected distributed applications quickly. 

Community

Knative originated from Google, whereas Dapr from Microsoft. Today both projects are incubating at CNCF. Both projects have growing communities and are within top 20 active CNCF projects (Dapr #10 and Knative #17)

Community statistics

See the full CNCF project statistics here.

Primary focus area

➤Knative extends Kubernetes with serverless containers by taking care of runtine networking (sync/async), autoscaling (to/from zero), and app revision tracking.

➤Dapr helps developers create reliable connected distributed applications quickly. It doesn’t manage the lifecycle of the application, instead it runs next to the applications.

Target user

➤Knative serving can be used by Ops to auto-scale and release apps (with traffic splitting). Knative eventing and functions can be used by devs to build, deploy apps, and connect external systems and event-driven containers. 

➤Dapr is toolkit designed primarily for developers. Developers use APIs & SDKs to interact with Dapr and offload responsibilities such as: pub/sub, state access, stateful service invocation, resiliency, etc. There are design time and runtime benefits for architects and operations teams respectively as described here.

Supported platforms

➤Knative runs only on top of Kubernetes and a network layer such as Kourier, Istio, Contour.

➤Dapr can run on Kubernetes, as well as on-premises and edge devices (such as the International Space Station).

For local dev, Knative requires Kubernetes, whereas Dapr can also run on Docker, or as a single binary only.

Deployment model

Both projects have operator, helm chats, CLI that help with installation and operating the control planes on Kubernetes. 

➤On the data plane side, Dapr is a sidecar that gets injected into the application pod. The application interacts with Dapr over well-defined APIs...

➤Knative dictates how the application is defined and run on Kubernetes by creating deployments, pods, configmaps, and networking configurations. It injects a transparent sidecar into every pod to measure network activity. And has an activator to hold off requests while scaling from zero.

Developer experience

➤Knative uses Kubernetes CRDs for defining an app (called Knative Service) composed of container, configuration, revision. It also offers CRDs that can define how events (CloudEvents) flow between these services, subscribe to a broker. And functions-centric CLI.

➤Dapr offers HTTP and gRPC APIs for Service invocation, Pub/Sub, State, Workflows, Bindings, Configuration, Secrets, Distributed lock, called building blocks. Devs use an HTTP/gRPC client, or SDKs for 8+ languages to interact with the above APIs.

Operational experience

➤Knative serving helps Ops with releasing, auto-scaling, configuring services. Knative eventing with abstracting the broker. 

➤Dapr helps Ops with monitoring, securing, and increasing resiliency of services, as well as cloud infrastructure abstraction.

Top features

➤Knative Scale to 0 Autoscaling Traffic splitting App definition Pub/sub Connectors Function CLI

➤Dapr Pub/sub Service-to-service interaction with resiliency, key/value access, Actors, Connectors, Security (mTLS, Auth), Config/Secrets, Workflows, Distributed Lock APIs.

Feature overlap

The primary overlap is around pub/sub capabilities. Both projects offer async interactions between applications by abstracting the broker & using CloudEvents format. Both projects have connectors and ability to subscribe apps to the broker. 

➤Knative defines these w/ CRDs & over HTTP. 

➤Dapr supports HTTP/gRPC, using CRDs and code.

Sweet spot

➤Knative: autoscaling containers (to and from zero).

➤Dapr: event-driven and stateful service interactions.

Upcoming hot feature: 

➤Knative: function development. 

➤Dapr: workflows based orchestration.

Summary

Those are the key differences between Dapr and Knative I'm aware of.  If you know other differences, share those on the twitter thread. We @diagridio are actively working on both projects and exploring how they complement each other and integrate with the greater CNCF ecosystem.

Apache Kafka Landscape

• Active Apache Kafka projects list: https://bit.ly/kafkaland

• Self-service form to add a project: https://bit.ly/kafkaland-form

Before joining Diagrid to work on the Dapr project, I was a product manager at Red Hat. At Red Hat, I had the opportunity to with with exceptional teams on projects such as Debezium, Data Virtualization, Apicurio, Apache Camel, and Strimzi, and helped build Red Hat's Managed Kafka Service. Kafka is more than just a fast message broker, it is a comprehensive ecosystem of various complementary management and development tools and managed services. Through my learning, writing, and exploration of the vast Apache Kafka Ecosystem, I compiled a list of over 120 active Kafka projects I encountered. Here is a sneak peek some of these projects, with the full data set available in the provided spreadsheet.

Apache Kafka landscape projects

The projects are organized into main categories of the Kafka ecosystem, including Kafka distributions, stream processors, client libraries, connectors, schema registry, replicator, rest proxy, and management tools. Each project includes information on its license, programming language, and deployment types. Additionally, statistics such as GitHub stars or downloads are provided too, but it should be noted that these might have become outdated. A simple report generated from the list is available for reference.


Apache Kafka project stats

Discovering other active projects within the Apache Kafka ecosystem can be time-consuming. However, the data provided here is available for you to explore and use as you desire, saving you time and effort. If you wish to add any additional projects, tools, or services related to Apache Kafka, use the provided self-service form.

If you found this content useful, share and help spread this list to others in the Kafka universe. I hope to see you at Kafka Summit London this year. If my talk is accepted 🤞, I will be discussing the topic of creating "Polyglot Microservices with Dapr and Kafka" in more detail.

Unbundling: The Natural Evolution of Tech Stacks

As the field of IT continually evolves, there is a constant stream of new software architectures, development techniques, and tools. Some examples of these include microservices, micro frontends, zero trust, service mesh, and data mesh, you mesh it. Despite the apparent diversity of these technologies and approaches, they are all connected by a common trend: the unbundling of technology stacks and teams. This trend involves breaking down systems into smaller, independent components and organizing work into smaller, more focused teams in order to achieve greater flexibility, and modularity.

The unbundling of the tech stack

How come each of these represent unbundling?

• Microservices emerged as a response to the limitations of monolithic architecture, which can be inflexible and difficult to scale and maintain as an application grows. By breaking a monolithic application into smaller, independent services, it becomes possible to develop, deploy, and scale each part of the application independently, leading to faster development cycles and greater flexibility.
• Hexagonal architecture was developed to improve the flexibility and maintainability of 3-tier applications by decoupling components and providing standardized interfaces for interacting with them.
• Domain-driven design (DDD) is a software development approach that can help break down a monolithic application into smaller, loosely-coupled modules that represent different business domains or contexts.
• Micro frontends architecture is a design approach in which a large, monolithic frontend application is broken down into smaller, independent modules that can be developed and deployed separately. 
• JAMstack decouples the frontend and backend of an application by separating the HTML, CSS, and JavaScript that make up the user interface from the server-side code and databases that power the application, allowing easier maintenance as changes to one part of the system do not necessarily require changes to the other.
• A service mesh decouples the networking responsibilities of a distributed application, such as routing, load balancing, and service discovery, from the application itself, allowing developers to focus on building business logic and features without worrying about the underlying network infrastructure.
• Similar to microservices, data mesh involves decomposing a large, complex system into smaller, independent components. It decomposes data governance and management practices into smaller, independent components that can be implemented and enforced consistently across different data sources and systems.
• The 2 pizza team model is a strategy for organizing teams and work in organizations that advocates for smaller teams able to respond to changes more quickly, have better communication and collaboration, can make faster decisions and solve problems more efficiently.

The end result of every tech trend is unbundling. Unbundling of technology stacks into independent components and teams into smaller, more focused teams likely to spread into all other areas. After frontend, data, networking, security, what do you think is the next area of unbundling? Join me in working on Dapr and unbundling integration. Also follow me at @bibryam and shout out with any thoughts and comments on the topic of unbundling.