A high-level security blueprint for modern apps based on APIs and microservices – part 1
COMBINE EXISTING SECURITY CONCEPTS AND BEST PRACTICES TO BUILD MORE SECURE DISTRIBUTED APPLICATIONS
Originally posted by Farshad Abasi of Forward Security on the IBM Developer blog on May 31st, 2019.
A common approach to modernizing applications is to use APIs and decompose them into smaller units that typically live in containers. Modernizing applications involves many concepts and technologies that are not always well understood, leading to poor application security postures. In addition, solution architects and developers who create the applications often lack the knowledge and expertise to select and apply the required security controls.
This two-part series brings together existing ideas, principles, and concepts such as end-to-end trust, authentication, authorization, and API gateways, to provide a high-level blueprint for modern API and microservices-based application security.
Read both parts of this series to learn the following skills:
Gain a high-level understanding of modern API-, services-, and microservices-based application architectures.
Become aware of key security concerns with these application architectures.
Understand how to best secure application microservices and their APIs.
This series is for you if you are a solution architect, a software developer, or an application security professional who is faced with securing APIs and their microservices.
Prerequisites
Some exposure and previous knowledge of APIs and microservices-based architectures help you better grasp the security aspects discussed. However, it is not necessary.
Estimated time
Take about 30 to 45 minutes to read both parts of the series. Part 1 should take about 20-25 minutes.
What are services and microservices?
According to Wikipedia, “Microservices are a software development technique – a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. In a microservices architecture, services are fine-grained, and the protocols are lightweight.”
For decades, monolithic applications were the order of the day. An application contained a set of several different services in order to deliver business functionality. The monolithic application’s services and functions had the following characteristics and challenges:
They were tightly coupled together.
They did not scale well (especially when different components had different resource requirements).
Monolithic applications were often large and too complex to be understood by a lone developer.
They slowed down development and deployment.
They did not protect components from the impact of issues from other components.
The applications were difficult to rewrite if you needed to adopt new frameworks.
Consider the following illustration comparing the architectures of monolithic applications and microservices-based applications.
A movement began in the late 1990s that began to focus on service-oriented architecture (SOA). Companies, particularly larger enterprises, started analyzing systems deployed across their organizations, looking for redundant services within different application systems. They then created a unique set and exposed legacy systems as services that could be integrated to deliver applications through new or existing channels.
Part of this process involved decomposing existing application systems into a set of reusable modules and facilitating development of various applications by combining those services. Each service in turn contained several tightly coupled functions related to a particular business area exposed through application programming interfaces (APIs), most commonly as SOAP and XML-based web services that communicated over an enterprise service bus. The following illustration shows an enterprise service bus connecting SOA and microservices-based applications and services:
The SOA movement was further propelled into popularity with the rise of web services in the early 2000s. (See the Service-Oriented Architecture Scenario article published by Gartner in 2003.)
Over the past decade, another movement started that further breaks these service “monoliths” into yet smaller microservices by decoupling the tightly connected functions of each service. Each microservice typically resides in its own container and focuses on a specific functional area that was previously provided by the service and tightly coupled with other functions. The individual microservices typically have their own datastore to provide further independence from other components of the application system. However, data consistency needs to be maintained across the system, which brings new challenges.
APIs are still used to expose the functionalities provided by microservices. However, lighter protocols and formats such as REST and JSON are used instead of the heavier SOAP, XML, and web services. This further decomposition does not necessarily lead to additional APIs exposed to service consumers, if the overall set of functions provided by the service or application remain the same. In addition, some microservices might only provide internal application functionality to the other microservices that are part of a specific application’s group of APIs. They might not expose any APIs for consumption by other applications or services.
Microservices offer a few advantages over monolithic architectures and lead to more flexible and independently deployable systems. In the past, when several developers worked on different functions that made up an (often) large service or application, an issue with one function prevented successful compilation and roll-out of the entire service or application. By decoupling those functions into microservices, the dependency on other functions of the application system is removed. Issues with development and roll-out in one area no longer affect the others. This approach also lends itself well to DevOps and agile models, shifting from a need to design of the entire service at one time and allowing for continuous design and deployment.
Using microservices address the issues related to monolithic applications described earlier, but this architecture brings about a new set of challenges. Examples include the complexities of distributed systems development (such as inter-service communication failure and requests involving multiple services), maintenance of data consistency across the system, deployment, and increased memory consumption (due to added overhead of runtimes and containers).
What about APIs?
Let’s visit Wikipedia again to understand an application programming interface, or an API: “In general terms, it is a set of clearly defined methods of communication among various components. A good API makes it easier to develop a computer program by providing all the building blocks, which are then put together by the programmer.”
APIs are a set of clearly defined communication methods between software components. APIs have been around for quite a while and have made it easy for independent teams to build software components that can work together without knowledge of the inner working of other components. Each component only needs to know what functions are exposed as APIs by the other component it interacts with. This architecture creates a nice separation layer and promotes modularity. Each component in a system can focus on a certain set of functionalities and expose the useful features to the outside world for consumption, while hiding the complexities inside, as shown in the following illustration:
The focus of this article is on modern web APIs provided by many of today’s web enabled applications. RESTful APIs, or those that conform to Representational State Transfer (REST) constraints, are the most common. These APIs communicate using a set of HTTP messages, typically using JavaScript Object Notation (JSON), or in some cases Extensible Markup Language (XML), to define the structure of the message. Additionally, the shift in the recent years to use REST web resources and resource-oriented architecture (ROA), gave rise to the popularity of RESTful APIs.
The API gateway and the post-monolithic world
In the world of monolithic applications, all functions reside in the same walled garden, protected by a single point of entry, which is typically the login functionality. After end users authenticate and pass this point, access is provided to the application and all of its functionality, without further authentication. This authentication approach is because the application architecture tightly couples all functions inside a trust zone, and they cannot be invoked by outsiders to the application. In this scenario, each function inside the application can be designed to either perform further authorization (AuthZ) checks or not, depending on the requirements and granularity of entitlements scheme.
When you break these monoliths into smaller components such as distributed microservices, you no longer have the single point of entry that you had with the monolithic application. Now each microservice can be accessed independently through its own API, and it needs a mechanism to ensure that each request is authenticated and authorized to access the set of functions requested.
However, if each microservice performs this authentication individually, the full set of a user’s credentials is required each time, increasing the likelihood of exposure of long-term credentials and reducing usability. In addition, each microservice is required to enforce the security policies that are applicable across all functions of the application that the microservice belongs to (such as JSON threat protection for a Node.js application).
This is where the API Gateway comes in, acting as a central enforcement point for various security policies including end-user authentication and authorization. Consider the following diagram:
After the users go through a “heavy” verification process involving their full set of long-term credentials (such as user name, password, and two-factor authentication), an access token is issued that can be used for “light” authentication and further interaction with downstream APIs from the API gateway. This process minimizes exposing users’ IDs and passwords, which are considered long-term credentials to the system components. The credentials are only used once and exchanged for a token that has a shorter lifespan.
The API gateway typically uses an identity and access management (IAM) service to handle verifying end-user identity, issuing access tokens, and other token-related activities (such as token exchange, which is described later). The API gateway acts as a guard, restricting access to the microservices’ APIs. It ensures that a valid access token is present and that all policies are met before granting access downstream, creating a virtual walled garden.
In addition to the previously described security benefits, the API gateway has the following non-security features:
It can expose different APIs for each client.
The gateway routes requests from different channels (for example desktop vs. mobile) to the appropriate microservice for that channel.
It allows for creation of “mash-ups” using multiple microservices that might be too granular to access individually.
The API gateway abstracts service instances and their location.
It hides service partitioning from clients (which can change over time).
It provides protocol translation.
The importance of user-level security context and end-to-end trust
In many multi-tiered systems, the end-users authenticate through an agent (such as a browser or a mobile app) to an external-facing service endpoint. The other downstream components in turn enforce mutual authentication either by using service accounts over Transport Layer Security (TLS), or mutual TLS, to establish service-level trust.
A major problem with this design is that the service provider gives access to all data provided by the set of functions that the service account is permitted to use. It does not consider the authenticated user’s security context. This approach is too permissive and is against the principle of least privilege.
For example, in a system where user account details are provided by a user account service, the service consumer could ask for and gain access to any user’s account details by simply authenticating to the provider using a service account. An attacker with an agent capable of connecting to a service provider (using a compromised service account credentials or accessing through a compromised system component that established service trust with the provider) can connect and access data belonging to any user. Another problem is this design does not allow for comprehensive auditing of actions.
These problems highlight the importance of establishing an authenticated user’s security context before allowing access to data provided by a service. An access token should be required by the service provider to determine the user’s security context before servicing the request. These tokens must be signed so your system can verify both the authenticity of the issuer and the integrity of the token data. If there is a confidentiality concern related to data in the token (for example, account numbers), make sure that the token is encrypted.
In addition, an API invocation might involve calls across several microservices downstream from the API gateway. End-to-end (E2E) trust means communicating the authenticated user’s security context to all the involved parties across the entire journey and allowing each party to take appropriate action. As noted at the start of this section, the user’s security context often ends at the external-facing service endpoint, and all downstream components rely on service-level trust. There are two ways that you can establish E2E trust across all microservices belonging to a given API group or application. One uses a token-exchange service, and the other relies on E2E trust tokens.
When establishing E2E trust with a token-exchange service, downstream components use that service to exchange a token with another token that has the required scope and protocol. At the API gateway, the user’s security token is verified, security policies are enforced, and the token is sent to the token-exchange service to receive a token that can be used by the immediate downstream microservice. If there are additional microservices across a journey, each one verifies the access token that it receives and uses the token-exchange service to swap it for one that can be used by the next downstream service provider. The token exchange service ensures that the new token has the scope required by the downstream service to both limit the amount of access granted and use the appropriate protocol. (For example, a JWT token can be exchanged for a SAML one.) With token exchange allows for some microservices to use a system of heterogenous protocols such as OAuth 2.0 and others to use SAML. Each microservice validates the token received and enforces further AuthZ, if required, based on the claims provided. More details about AuthZ are discussed in Part 2.
Another way to achieve end-to-end trust is to use one E2E trust token issued at the gateway across the entire journey. In this scenario, the gateway verifies the user’s token and performs the security policy checks required by the application or API group (as in the previous approach). Then, it also issues an E2E trust token after all validations are successful. In this scenario, all microservices involved must have the same scope and protocol requirements. Upon receiving the E2E trust token, a microservice verifies the signature (and performs decryption if required) and processes the request based on the claims provides. If there are other microservices in the journey, the token is passed on, and the same verification process takes place at each downstream microservice. This model removes the round trip to the token-exchange service for each call across the journey, but it does not provide the same level of flexibility and access control as using a token-exchange service.
With both of these models, E2E trust is established between microservices that belong to the same trust zone (for example, they require the enforcement of the same set of security policies) and that use the same identity provider (IdP). If you need to access microservices in another trust zone that have a different IdP, the path must go through the gateway where the API for the microservice is presented, which enforces the required set of security policies for that trust zone. E2E trust tokens that are issued for one trust zone should not be accepted by microservices in another.
Although these mechanisms provide user-level security context across the journey, they do not remove the need for service-level trust. Apply security at all layers and follow the defense-in-depth model. Lack of service-level trust could lead to using a compromised token (or one that is generated by an attacker and signed using a compromised token-signing key) from agents that are not authorized to access microservice components that make up a specific application. Modern API systems allow secure overlay networking to make it easier to establish service-level trust than service accounts or mutual TLS between the microservice components involved.
Summary
Today’s modern service-oriented applications are often decomposed into microservices exposed by APIs. It is important to understand what microservices and APIs are and the role they play in the application system. When adopting an architecture based on APIs and microservices, use API gateways where possible to provide one point of security policy enforcement and to establish end-to-end user-level trust on top of service-level trust. This approach ensures security is applied at multiple layers.
In Part 2 you learn about how to use protocols such as OpenID Connect, OAuth 2.0, and SAML to facilitate authentication (AuthN) and authorization (AuthZ). These protocols aid in designing a system that handles security at the right place and the right time, guaranteeing end-to-end trust across the entire journey. Part 2 covers when and why AuthZ is needed and what to do when applications and services outside a trust boundary invoke APIs. You also learn about additional security policies to consider beyond AuthN and AuthZ (for example JSON threat protection and rate limiting), logging and monitoring considerations, and how group policies can help you build a more secure application based on APIs and microservices.
