Microservices: Have you thought this through?

Regardless of the traction that microservices architectures have, there are technical areas and essential matters that you should take into consideration before developing in a microservices structure.

After working with Microservices, I have radically changed my way of thinking about Software, Scaling, and problem-solving. For the problems I was called to tackle, Microservices were indeed the way to go. The latest piece of code I wrote was for the 50nth Microservice in a cluster, which is ready to expand to accommodate a few hundreds of them. There seems to be a lot of buzz around microservices and for a good reason.

But what are microservices? According to microservices.io:

Microservices – also known as the microservice architecture – is an architectural style that structures an application as a collection of services that are, highly maintainable and testable, loosely coupled, independently deployable, organized around business capabilities.

To be more specific, microservices are autonomous deployable entities that cover only one aspect of your entire business. Each deployable entity is a complete server by itself – with its own necessary stack for running the operations specified.

A development team’s paradise?

If you have a developed a big web application which includes all aspects of your business like user information, payments, product catalogs, you will start breaking it into smaller components/servers and you would deploy those servers inside the same virtual network – where virtual network is an abstraction that enables communication between microservices deployed inside it. You can easily create such kind of virtual network configurations very easily using Kubernetes and Docker Swarm.

Each server would have its own deployment cycle, development cycle, and scaling. This has enabled development teams to:

Have faster and simple development cycles. A team can now develop a microservice as an independent component, without worrying about the implementation of the other components of the business application.
Deploy parts of an application independently.
Lower downtimes. An exception in a server (or a memory leak) will only affect one part of your whole business, as the rest of the servers will continue to work.
Independent scaling. If you see that a certain domain of your business (like getting the user’s information or authorizing users, for example) is getting much more traction than others, you can scale that part alone without affecting the rest of the deployed components. You can usually do that either by adding new replicas (horizontal scaling) or by increasing the allocated memory/CPU to this specific microservice.

Everyone that has read a bunch of other articles about microservices has probably identified the cost to be an additional bullet to the good parts. Well, the cost is a very complicated issue which we will tackle a bit later. Let’s suffice to say that cost-effectiveness applies only to a subset of the microservices appliances I have had the luck to see during my career.

Problems in every paradise

So, should you begin now breaking down your monolithic server into smaller components? You can. I would also dare to say that you have already made your mind about that. I am not here to change anyone’s mind (I love microservices, anyway). But I would like to share some of my thoughts about your routine if you decide to do down the microservice road – and to share with you stuff I have realized the hard way.

DevOps

DevOps is always an essential aspect of the development process – one that is very easily missed/forgotten. There are those who support the idea that developers should also take care of the DevOps themselves since they possess knowledge of their development stack and their resource needs. Others believe that DevOps should be handled by specified and experienced administrators who have access to 24/7 monitoring tools and should take care of everything non-development. For me, the truth is somewhere in the middle.

Regardless of which side of the Developer / Develops party you stand with, you will have a hard time with DevOps. No matter the size of your business and cluster, the following is a non-exhaustive list of your daily tasks regarding DevOps:

Ensuring versioning semantics are appropriately followed in all microservices of your cluster.
Ensure the building scripts for your microservices are working correctly. If you have 50 microservices and you wish to do a change in a Jenkins file (or any build step in any CI/CD tool), you will have to make the change in all your scripts manually.
Manage your Git repository. There are many ways to structure this, but usually, you will probably have one repo per microservice. As soon as you start reaching 30+ microservices, managing source repositories/versioning and setting CI/CD hooks is going to become an increasingly important but also a very tedious process, and there is no way around it. Managing repositories will add complexity and costs for you, whether you are alone or act as the CEO of a company.
Keeping track of your namespaces. Depending on your network policies, this can be hard or harder. You may have more than one namespaces on K8s, and each namespace will probably be dedicated to an environment (“dev,” “staging,” “production,” etc.). Each one of those is probably going to have deployed a copy of your images created by your CI/CD if you have 30+ distinct microservices that means that you will likely have 90 microservices up and running depending on their completion stage.
Resource monitoring. You should look into your microservices often. Notice their RAM usage, their restarts, see if you need to increase or decrease the allocated RAM, figure out how many replicas you need. Grafana will probably be your best friend.
Monitor your logs. Just the console logs will never be enough to debug a microservice properly. Consider this scenario: A request reaches microservice A, which in turn needs to draw information from B, C, and D, and you need to see what happens in this “request journey.” To achieve this, you will either need to pass information about the request to all the dependent microservices (such as a common request-id). You can also implement some advanced logging in your service discoverability mechanism and make all your rest calls pass through there.
Managing your Service Discoverability: How services (your microservices) will talk to each other. Remember, each microservice is a distinct server which exposes an interface (usually a REST API, but that’s not an absolute rule) for accepting information. If you have a big cluster, you may need to add a service discoverability mechanism such as Istio or something equivalent.
Manage networking and network policies in your namespaces (to prevent a breach in one microservice taking over your entire cluster).
If you are building your own cluster, then you will have to do all of the above, plus manage User Access, Bare-Metal provisioning (add more nodes to the K8s cluster).

For the reasons mentioned above, I would recommend that any company (regardless of the size) should have at least one dedicated DevOps specialist that can handle all those matters. And also provide solutions to future problems that may arise when your cluster is growing.

Writing APIs

The chances are that if you are using microservices, you are using REST APIs to make those services communicate with each other. In terms of coding, that means that you must exercise extreme discipline in creating reusable rest interfaces for your microservices to interact.

As your microservices evolve, so will your REST APIs. You have to use proper versioning, keeping track of your microservices, and their business rules. If one call to your service means that you have to gather information from multiple microservices, then the maintenance and complexity of each one’s API is increased dramatically and will undoubtedly demand much of your time in the long run.

This process is vastly more complicated from monolithic architectures where you have the luxury of looking at the inputs of your functions in code. When developing a monolith, refactoring does not always mean that you are going to change the coding interface between different parts of your application. When developing a microservice, however, you will most certainly do.

Development

The API Gateway

In order to expose your apis to the public, you will want an API gateway.

API Gateways are a vast subject, but in short, the API gateway is the entry point to your cluster. Depending on what you need it to do, you would want it to perform one or more of the following tasks:

Request routing & enriching requests from the outside world towards your microservices
Load balancing / Request limiting
API standardization between your frontend (whether it’s a web app, desktop app, or a mobile app) and your cluster.
Manage your APIs

An API Gateway is not a component; it’s a pattern found in microservice clusters – and considered one of the core components of a Kubernetes based solution. If the API gateway doesn’t perform 100% correctly, all of your users will suffer the consequences.

There are solutions provided by major players in the Industry, such as Apigee on Google Cloud, Red Hat’s 3scale, Kong, and many others. Depending on where your service is hosted (and your budget), you will need to customize your deployments so that your services are discoverable by the API Gateway, and also to write code for the API Gateway itself. The process of maintaining and evolving your API Gateway is not something you want to take lightly, regardless of how simple it may seem at first. Your gateway implementation will grow with each API you expose to the outside world.

Code duplication

Each one of your microservices will need to request information from the database and will map this information to models. Those models will most probably be reused across multiple microservices, so you will either need to have to write numerous times the classes used, or you will need to put them in a library and reuse it across your repositories. Same goes with the technical business that your microservice will also need to adhere to (like security and other business-specific business logic.

Technology Stack

You have probably heard and understood that you leverage the microservices architecture’s capabilities to develop your cluster into different technologies.

This belief is true. An example of this is that you can have a performance-heavy part of your business and use Golang or C++ for this part while leaving the rest of your cluster with another technology.

As a general practice, however, you will want to develop microservices in a technology that follows the trends of the company you work for. Do not underestimate the task of maintaining your code as your cluster grows larger. You should follow the “good conventions always win” practice to keep services maintainable. You don’t need to choose the absolute best technology for specific parts of your business – you have to choose the one that can serve 90% of your tasks with ease.

Costs

One of the most important matters is this one.

The cost in microservices clusters has been the subject of debates in many places around the internet – and it is one of those that cannot be quickly answered without knowledge and analysis of your specific use case.

How you break your business into smaller components matters – and very much so. Imagine this: Under normal circumstances, adding just one little new REST API Endpoint to a monolithic microservice will not add any significant overhead to your server, since all your endpoints share most of the library dependencies and code. However, if you develop an entire microservice to accommodate this new small API, you will also add the overhead of a whole stack on top of it – that is perhaps a new JVM with a small system (alpine, for example) that will live inside your docker machine.

Some APIs are going to be used much more than others – and those need to be scaled independently while other microservices need much lower resources allocated to operate. This distinction and flexibility of independent scaling are what is going to save you money – so plan your cluster accordingly.

Based on this knowledge, you alone are always the best person to identify what will be the costs of your cluster – I am just going to say that I have seen clusters exploding in terms of resource usage by not analyzing how components are going to be broken down in advance. Proper analysis is the key before determining how to split your business model into smaller parts – and will affect your cost dramatically.

However, in your cost calculations, do not also forget to add the costs of the Human Factor. That is the human resources that you are going to need when maintaining your cluster, be it DevOps, technical coordinators, etc.

Conclusion

I have written so many things, and I still feel I have barely touched the surface. Microservices are such a broad concept that cannot be covered in an article such as this one.

Microservices are incredibly flexible and exciting. Even if you use technologies well-established in the monolithic world (like Java), you will find out that those are gaining traction again with the advent of frameworks like Quarkus and Micronaut.

But as it happens with any other technology, you should go into this with both eyes open, whether you are a developer or not. Careful planning is always the key – as is avoiding the common pitfalls that software trends sometimes force us to fall in.