1. Product Introduction

• 1: 1.1 Introduction and Use Cases
• 2: 1.2 Industry Background
• 3: 1.3 Architecture Introduction
• 3.1: 1.3.1 Architecture Principles

1 - 1.1 Introduction and Use Cases

Introduction

Koupleless is a modular Serverless technology solution that enables ordinary applications to evolve into Serverless development mode at a relatively low cost, allowing code and resources to be decoupled and easily maintained independently. At the same time, it supports capabilities such as second-level build deployment, merged deployment, and dynamic scalability, providing users with the ultimate development and operation experience, ultimately helping enterprises achieve cost reduction and efficiency improvement. With the increasing digital transformation across various industries, enterprises are facing more and more pain points in terms of development efficiency, collaboration efficiency, resource costs, and service governance. Let’s explore these pain points one by one and see how they are addressed in Koupleless.

Use Cases

Pain Point 1: Slow Application Build and Deployment or Tedious SDK Upgrades

Traditional application image building usually takes 3-5 minutes, and deploying code to start completion also takes 3-5 minutes. Developers have to undergo multiple 6-10 minute build and deployment waits each time they verify or deploy code changes, severely impacting development efficiency. Additionally, each SDK upgrade (such as middleware frameworks, RPC, logging, JSON, etc.) requires modifying all application code and re-building and deploying, causing unnecessary disruption to developers.
By using Koupleless Universal Base and accompanying tools, you can cost-effectively split applications into “base” and “module”. The base encapsulates common SDKs of the company or a specific business department, and base upgrades can be handled by specialists without affecting business developers, who only need to write modules. In our currently supported Java technology stack, a module is a SpringBoot application code package (FatJar). However, the SpringBoot framework itself and other enterprise dependencies are pre-loaded and warmed up by the base at runtime. Each time a module is deployed, it finds a base with pre-heated SpringBoot for hot deployment, similar to AppEngine, enabling users to achieve 10-second build and deployment for applications and seamless SDK upgrades.

Pain Point 2: High Resource Costs for Long-tail Applications

In enterprises, 80% of applications serve less than 20% of the traffic. Along with business changes, enterprises have many long-tail applications. These long-tail applications have CPU utilization rates of less than 10% for an extended period, leading to significant resource waste.
By using Koupleless merged deployment and accompanying tools, you can cost-effectively achieve the merged deployment of multiple applications, solving the problem of excessive fragmentation of enterprise applications and the resource waste caused by low-traffic businesses, thus saving costs.

Here, “Business A Application 1” is called a “module” in Koupleless terminology. Multiple module applications can be merged into the same base using SOFAArk technology. The base can be a completely empty SpringBoot application (Java technology stack), or it can sink some common SDKs to the base. Each time a module application is deployed, it restarts the base machine. In this way, module applications maximize the reuse of the base’s memory (Metaspace and Heap), reduce the size of build artifacts from hundreds of MB to tens of MB or even more aggressively, and effectively improve CPU usage.

Pain Point 3: Low Collaboration Efficiency in Enterprise R&D

In enterprises, some applications require multi-person development collaboration. In the traditional development mode, each person’s code change requires the entire application to be released, leading to a fire-fighting approach to application development iteration. Everyone needs to develop iteratively within a unified time window and release online at a unified time, resulting in a large number of demand release waits, environment machine preemptions, iteration conflicts, and other situations.
By using Koupleless, you can easily split applications into a base and multiple functional modules, where each functional module is a group of code files. Different functional modules can be developed and deployed iteratively simultaneously, and they are independent of each other, eliminating the traditional application iteration fire-fighting approach. Each module has its independent iteration, greatly improving the efficiency of demand delivery. If you enable hot deployment for modules (or restart the entire base each time a module is deployed), the single build + deployment time for modules will also be reduced from 6-10 minutes of normal applications to seconds.

Pain Point 4: Difficulties in Accumulating Business Assets to Improve Middle Platform Efficiency

In some medium and large enterprises, various business middle platforms are accumulated. Middle platforms generally encapsulate the common API implementation and SPI definitions of the business. The SPI definition allows plugins on the middle platform to implement their own business logic. After the traffic enters the middle platform application, it calls the corresponding SPI implementation component to complete the corresponding business logic. Components in the middle platform application have relatively simple business logic. Deploying them as independent applications will incur high resource and operation costs and slow build and deployment speeds, significantly increasing the development burden and affecting development efficiency.
With Koupleless, you can easily split the middle platform application into a base and multiple functional modules. The base can accumulate relatively thick business dependencies, common logic, API implementations, SPI definitions, etc. (referred to as business assets), and provide them to the modules above. The modules can use the capabilities of the base through direct calls between objects or beans, with almost no code modification. Moreover, multiple modules can be developed and deployed iteratively simultaneously without affecting each other, greatly improving collaborative delivery efficiency. Additionally, for relatively simple modules, you can also enable hot deployment, and the single build + deployment time will be reduced from 6-10 minutes of normal applications to within 30 seconds.

Pain Point 5: High Cost of Microservice Evolution

Different businesses in enterprises have different development stages, so applications also have their own lifecycles.

Startup Phase: A startup application generally adopts a monolithic architecture.
↓
Growth Phase: As the business grows, the number of application developers also increases. At this time, you may be uncertain about the future prospects of the business and do not want to split the business into multiple applications prematurely to avoid unnecessary maintenance, governance, and resource costs. Therefore, you can use Koupleless to cost-effectively split the application into a base and multiple functional modules, allowing different functional modules to be developed, operated, and iterated independently in parallel, thus improving the collaboration and demand delivery efficiency of the application in this stage.
↓
Mature Phase: As the business further expands, you can use Koupleless to cost-effectively split some or all functional modules into independent applications for development and operation.
↓
Long-tail Phase: Some businesses may gradually enter a low-activity or long-tail phase after experiencing growth or maturity phases. At this time, you can use Koupleless to easily convert these applications into modules, merge them, and deploy them together to achieve cost reduction and efficiency improvement.

It can be seen that Koupleless supports enterprises to smoothly transition between the startup, growth, mature, and long-tail phases of applications at a low cost and even switch back and forth, thereby easily keeping the application architecture synchronized with business development.
Application lifecycle evolution.

2 - 1.2 Industry Background

Issues with Microservices

As application architectures evolved from monolithic to microservices, combined with the development of software engineering from waterfall models to the current DevOps model, various problems such as scalability, distribution, and collaborative work have been addressed, providing enterprises with better agility and execution efficiency, bringing significant value. However, despite solving some problems, the microservices model has gradually exposed some issues that are currently receiving continuous attention:

Complex Infrastructure

High Cognitive Load

To fulfill a business requirement, there are actually many dependencies, components, and platforms providing various capabilities behind the scenes. If any component below the business layer encounters an exception that is perceived by the business, it will impose a significant cognitive burden and corresponding time cost on the business development personnel.

Various types of exceptions.

Heavy Operations Burden

The dependencies included in the business application also undergo continuous iterative upgrades, such as frameworks, middleware, various SDKs, etc. When encountering situations such as:

1. Major feature releases
2. Urgent bug fixes
3. Encountering significant security vulnerabilities

These dependencies’ new versions need to be upgraded as quickly as possible by the business. This leads to two problems:

For Business Development Personnel

If these dependency upgrades occur only once or twice, it’s not a problem. However, a business application relies on many frameworks, middleware, and various SDKs, and each dependency upgrade requires the involvement of business developers. Managing so many dependencies becomes a significant operational burden for business development personnel in the long term. Additionally, it’s important to note that the business’s common layer also imposes a significant burden on business developers.

For Infrastructure Personnel

Similarly, like the developers of various dependencies, each release of such a new version requires the business applications using them to be upgraded as quickly as possible. However, business development personnel are more concerned with delivering business requirements, so pushing them to complete upgrades quickly is not very realistic, especially in enterprises with many developers.

Slow Startup

Each business application startup process involves many steps, resulting in long waiting times for functionality verification.

Low Release Efficiency

Due to the aforementioned issues of slow startup and numerous exceptions, the deployment process takes a long time, and encountering exceptions that cause delays requires recovery and handling. In addition to platform exceptions, the probability of machine exceptions increases with the increasing number of machines. For example, if the probability of a machine completing a release without any issues (without encountering exceptions) is 99.9%, meaning the success rate in one attempt is 99.9%, then for 100 machines, it becomes 90%, and for 1000 machines, it decreases to only 36.7%. Therefore, applications with many machines often encounter deployment delays, requiring developer intervention, leading to low efficiency.

High Collaboration and Resource Costs

Monolithic/Large Applications are too Big

Blockage in Multilateral Cooperation

As businesses continue to grow, applications become larger, mainly reflected in the increasing number of developers, resulting in blockages in multilateral cooperation.

Large Impact of Changes, High Risk

As businesses continue to grow, online traffic increases, and the number of machines grows. However, a single change can affect all code and machine traffic, resulting in a large impact and high risk from changes.

Too Many Small Applications

During the evolution of microservices, over time, due to factors such as excessive application splitting, some businesses shrinking, or organizational restructuring, there is a continuous accumulation of small or long-tail applications online, resulting in an increasing number of applications. For example, in the past three years, the number of applications at Ant Group has tripled.

High Resource Costs

These applications require several machines in each data center, but in reality, the traffic is not significant, and CPU usage is very low, resulting in resource waste.

High Long-Term Maintenance Costs

These applications also require personnel for maintenance tasks, such as upgrading SDKs and fixing security vulnerabilities, leading to high long-term maintenance costs.

Inevitability of the Problem

A microservices system is an ecosystem, and after several years of evolution within a company, according to the 28 Law, a few large applications occupy a significant portion of the traffic. It is inevitable that problems such as oversized large applications and too many small applications will arise. However, there is no defined standard for what constitutes a large application or how many small applications are too many. Therefore, the pain points experienced by developers due to these problems are subtle, and unless the pain reaches a certain threshold, it is difficult to attract the attention and action of the company’s management.

How to Properly Decompose Microservices

The proper decomposition of microservices has always been a challenging problem, as there are no clear standards. This is also why the issues of oversized large applications and too many small applications exist. The root cause behind these problems is the flexibility of business and organization and the high cost of microservice decomposition, which results in inconsistent agility between the two.

Misalignment between Microservices Decomposition and Business/Organizational Agility

Business development is flexible, and organizational structures are constantly adjusting. However, microservice decomposition requires machine resources and incurs long-term maintenance costs. The misalignment in agility between the two leads to problems such as under-decomposition or over-decomposition, resulting in oversized large applications and too many small applications. If these problems are not fundamentally addressed, microservices governance will continue to encounter issues, causing developers to remain stuck in a cycle of low efficiency and governance challenges.

Problems Faced by Enterprises of Different Sizes

Industry Attempts at Solutions

The industry has many good ideas and projects attempting to solve these problems, such as service meshes, runtime applications, platform engineering, Spring Modulith, and Google ServiceWeaver. These solutions have had some effect but also come with limitations:

1. From the perspective of business developers, only part of the infrastructure is shielded, and the business’s common parts are not shielded.
2. Only some of the problems are addressed.
3. High cost of retrofitting existing applications.

Koupleless is evolving as a development framework and platform capability to address these issues.

3 - 1.3 Architecture Introduction

3.1 - 1.3.1 Architecture Principles

Modular Application Architecture

To address these issues, we have performed both horizontal and vertical splits on applications. Firstly, the vertical split involves dividing the application into base and business layers, corresponding to two layers of organizational responsibilities. The base team, like traditional applications, is responsible for machine maintenance, common logic abstraction, business architecture governance, and providing runtime resources and environments for business. By separating concerns, all underlying infrastructure below the business layer is shielded, allowing the focus to remain on the business itself. Secondly, we horizontally partition the business into multiple modules, allowing independent parallel iteration among them without interference. Since modules exclude the base part, their build artifacts are lightweight, and their startup logic only encompasses the business itself, enabling rapid startup and providing sub-second verification capabilities, thus optimizing module development efficiency to the utmost extent.

Before the split, each developer may have perceived all the code and logic from the framework to middleware to business common parts to business itself. After the split, the collaboration within the team has changed, and developers are divided into two roles: base and module developers. Module developers are not concerned with resources and capacity but enjoy the ability to deploy and verify changes in seconds, focusing solely on business logic.

Here it’s crucial to understand how we perform these vertical and horizontal splits. The split is for isolation, and isolation is for independent iteration, stripping unnecessary dependencies. However, isolation alone, without sharing, is akin to merely relocating deployment positions, which may not yield desirable results. Thus, besides isolation, we also emphasize sharing capabilities. Therefore, it’s essential to focus on understanding the principles behind modular architecture in terms of isolation and sharing.

Module Definition

Before delving further, let’s clarify what modules are in this context. Modules are derived by subtracting the base part from the original application. This subtraction is achieved by setting the scope of dependencies in the module as provided.


A module can be defined by these three points:

1. A jar package generated by SpringBoot packaging
2. A module: a SpringContext + a ClassLoader
3. Hot deployment (no need to restart the process during upgrade)

Isolation and Sharing of Modules

Modules are isolated in terms of configuration and code through ClassLoader and SpringContext. They share configurations and code classes between modules and bases through SOFAArk and between multiple modules through SpringContext Manager.

Within the JVM, this is achieved through:

1. Ark Container providing a multi-ClassLoader runtime environment
2. Arklet managing module lifecycles
3. Framework Adapter associating SpringBoot lifecycle with module lifecycle
4. SOFAArk’s default delegation loading mechanism bridging module and base class delegation loading
5. SpringContext Manager providing bean and service discovery mechanisms
6. Bases essentially being modules with independent SpringContext and ClassLoader

However, modularization technologies in the Java domain have developed for 20 years. Why can modularization technologies be scaled within Ant Group? The core reason lies in the multi-module capabilities based on SOFAArk and SpringContext Manager, which provide a low-cost usage approach.

Isolation Aspect

Compared to other modularization technologies, from an isolation perspective, JPMS and Spring Modulith impose limitations through custom rules, with Spring Modulith requiring verification in unit tests. The isolation capabilities are relatively weak and somewhat tricky, with significant retrofitting costs for legacy applications, and even infeasible for legacy applications. Similar to OSGi, SOFAArk employs ClassLoader and SpringContext for configuration and code, as well as bean and service isolation, maintaining consistency with the native application startup mode.

Sharing Aspect

While SOFAArk shares isolation mechanisms with OSGi, OSGi, JPMS, and Spring Modulith all require defining import/export lists or other configurations between source and target modules, resulting in high usage costs for business modules that need to understand and perceive multi-module technologies. SOFAArk defines a default class delegation loading mechanism and cross-module bean and service discovery mechanisms, enabling business usage of multi-module capabilities without modification.
Additionally, why can the modularization technology based on SOFAArk provide these default capabilities at low cost and emphasize low-cost usage? The main reason is that we have differentiated roles for modules, distinguishing between bases and modules. Based on this core reason, we have also attached importance to low-cost usage and made important design considerations and trade-offs. For specific design considerations and trade-offs, refer to the technical implementation article.

Inter-Module Communication

Inter-module communication relies primarily on the bean and service discovery mechanism provided by SpringContext Manager.

Module Evolution

Looking back at the mentioned major issues, it can be seen that through the isolation and sharing capabilities of modular architecture, problems such as complex infrastructure, collaboration blocking, and high resources and long-term maintenance costs can be solved. However, the issue of inconsistent agility between microservices splitting and business remains unresolved.

Here, we address this by reducing the cost of microservices splitting. So how do we reduce the cost of microservices splitting? The main approach is to introduce modular architecture between monolithic and microservices architectures.

1. Modules do not occupy resources, so splitting incurs no resource costs.
2. Modules do not include business common parts, frameworks, or middleware parts, so modules incur no long-term SDK upgrade and maintenance costs.
3. Modules themselves are SpringBoot, and we provide tools to assist in the low-cost splitting of monolithic applications into modular applications.
4. Modules have flexible deployment capabilities; they can be deployed together in one JVM or separately, allowing modules to evolve into microservices or revert to monolithic application modes at low cost.

The arrows in the diagram are bidirectional. If there are too many microservices currently split, multiple microservices can also be cost-effectively transformed into modules and deployed together in one JVM. So, the essence here is to add a bidirectional transitional modular architecture between monolithic and microservices architectures, reducing the cost of transformation while allowing developers to evolve or roll back according to business needs. This can solve several problems of microservices.

Advantages of Modular Architecture

The main advantages of modular architecture are concentrated in these four points: speed, savings, flexible deployment, and evolvability,

When compared to traditional applications, the data below show more than a 10x improvement in development, deployment, and runtime stages.

Platform Architecture

Just having application architecture is not enough. It is necessary to provide complete supporting capabilities from the development stage to the operation and runtime stages to truly realize the advantages of modular application architecture to developers.

During the development stage, it is necessary to provide capabilities for base access, module creation, and more importantly, local rapid build and debugging capabilities for modules; during the operation stage, provide fast module deployment capabilities, and on top of module deployment, provide A/B testing and second-level scaling capabilities; during the runtime stage, provide reliability capabilities for modules, fine-grained control of observability, traffic, scheduling, and scaling.

Component View

Within the entire platform, four components are needed:

1. Development tool Arkctl, providing module creation, rapid debugging testing, and other capabilities.
2. Runtime components SOFAArk, Arklet, providing module operation, module lifecycle management, and multi-module runtime environment.
3. Control plane components ModuleController
   1. ModuleDeployment provides module deployment and operation capabilities.
   2. ModuleScheduler provides module scheduling capabilities.
   3. ModuleScaler provides module scaling capabilities.