Digital transformation’s last mile: decentralizing software delivery

Sep 28, 2021

How to accelerate your digital transformation with a value stream delivery platform

The pandemic made it crystal clear. Your digital transformation is not a nice-to-have, internally directed effort. It is a strategic project to keep and develop your competitive advantage in a context in which the pandemic accelerated the digital transformation of businesses across the board. As Microsoft CEO Satya Nadella commented last year, “We’ve seen two years’ worth of digital transformation in two months.”

So how do you accelerate? How do you do two years in two months? In the five years that we have been developing our Value Stream Delivery Platform, we talked with CIOs, software architects, and tech leads — whether in large corporations like IBM or in small ambitious enterprises. While their answers varied in their detail, they painted a fairly similar picture.

They told us that companies have long been decentralizing their businesses into nimbler organizations to be closer to their customers. The Internet and cloud technologies enabled new digital products and services, transformed business processes, and put IT and software as critical components in every value chain.

They insisted that to go further in the efficiency, agility, speed, and resilience that they need to merely compete, companies now need to extend their decentralization to IT and software teams. They warned that for all its benefits, that extended decentralization has a complex set of tradeoffs. Ignore those and your digital transformation may fail stupendously.

In that context, self-service DevOps Value Stream Delivery Platforms (DevOps VSDPs) amplify the benefits of decentralization while addressing the tradeoffs of decentralization at scale.

We could not possibly reproduce all the conversations we had with IT and business leaders over the years. We instead condensed the insights into a single interview of a typical CIO.

Read on. This is how businesses large and small are maturing their digital transformation.

 

Let’s start with the business context. What is digital transformation and why is it so high in executives’ agenda?

A digital transformation consists of positioning the company to exploit business opportunities or respond to competitive threats linked to digitalization. Digital technologies have transformed payments, banking, traveling, shopping, logistics, entertainment… Businesses need to adjust to new digital channels, new customer demand and expectations, and increased competitive threats from a new range of digital actors.

It used to be the big overpowering the small. In this brave new world, it is the fast leaving the slow behind. Who remembers Blockbuster? Netflix essentially displaced video rental megastores into bankruptcy. Digital-first bank Simple (later acquired by Spanish bank BBVA) took on large established banks without opening a single branch. Such is the scale of the changes to commit to that business consultancies like IBM or McKinsey go a step further and talk about Digital Reinvention.

This is not new. We’ve been on this digital transformation journey for more than one decade. What is new is that the pandemic accelerated the movement:

Digital transformation has accelerated in the last two years

Digital transformation has accelerated in the last two years

To capture the new opportunities linked to digitization in fast-changing markets, or merely to compete with digitally advanced competitors, companies must fast-forward their digital transformation. With IT and software in every value chain, they must find ways to deliver reliable software faster to drive their transformation.

Let’s talk about software then. A large part of the value created by companies and delivered to customers is through software. We see those fast-changing markets, nimbler competitors getting ahead in their digital transformation and building better capabilities. What should CIOs and CTOs be doing?

Decentralize software operations! We have been there before.

In the last decades, to create more value faster, corporations have decentralized their decision-making processes and organizational structure to give more power to smaller value-creating units (e.g., strategic business units, McKinsey’s value cells).

One core idea is to build the Agile enterprise, which gets immediate feedback from its target markets and has the decisional capability and resources to react faster as a result. As a matter of fact, Agile pushes authority, decision-making, and accountability to those who are closest to the information.

Fast-forward to today, many organizations that decentralized their decisional centers, business processes, and organizational structure find that, in a digital world, they are only as fast as their centralized IT and software delivery allows them.

Let’s take the example of a traditional bank. You have branches. You need software at the branch to serve the customers. You have departments for internet banking, credit risk, core banking, regulatory reporting. They also need software. In a centralized IT context,  these departments address their requests for software to a single point. That single point becomes the bottleneck that every company with centralized IT complains about.

So you have to decentralize software delivery too and put software teams closer to their customers.

But this time around, businesses don’t have a 5-year time frame and a 10MM budget (that eventually snowballs to 25MM) to get some consultancies to think about how to integrate the ERP mammoth. Businesses today have the pressure of having to advance in their digital transformation journey fast. The pandemic was a wake-up call for many.

So what is stopping companies today? The associated budget, operational, and security risks linked to decentralization. According to a pre-pandemic study by the Economist Intelligence Unit (EIU), 2/3 of IT executives report ballooning complexity and increased vulnerabilities paired with higher costs:

two-thirds of IT executives report higher complexity, increased vulnerabilities paired with higher costs

Decentralization increases budget, operational, and security risks

This is not simple. These are real challenges for which companies need real solutions. Many companies delay their transformation because they fear making a bad situation worse. Others are not sure where to start and what to do.

On the other hand, we have companies that have successfully transformed into digital leaders and deliver digital products and services daily to their customers with best-in-class reliability and quality indicators.

So how do digital leaders decentralize software delivery?

Three key initiatives:

  • Decentralize software itself into independently deployable modules. Break down the monolith. Architect around microservices. Adopt cloud-native patterns.
  • Decentralize the structure. Use small, independent teams with end-to-end responsibility for a product/service/value stream. Map teams to microservices.
  • Give full ownership to decentralized units. Adopt DevOps and Agile at scale with build-run teams. You code it. You test it. You deploy it. You are woken up by it at 4 am on a Sunday.

How does this work? Many applications start small: one small team of developers, few key features. As the application’s feature list and team size grow, the development velocity slows down. Even small changes in the monolith may require rebuilding, retesting, and redeploying the entire package. That, in turn, may require the cooperation of multiple teams, with lower flow efficiency and a higher probability of breakage.

With microservices, the monolith’s functionalities are broken down into smaller microservices that are independently operated and deployed. Formerly large teams are split into smaller, decoupled units with fewer, more focused responsibilities. Individual units own a specific set of microservices driven by a business perspective — for instance following a domain-driven design (DDD) approach.

Microservices allow smaller teams with end-to-end ownership on value streams

Microservices allow smaller, specialized independent teams

Decentralized, decoupled teams ship faster to production because they are smaller, tackle a smaller scope, and deploy independently. As Robert C. Martin, one signatory of the influential Agile Manifesto, said:

Decoupling always speeds you up. If there’s one thing we’ve learned about coupling over the last fifty years it is that nothing is better at slowing you down.

Because independent teams deploy faster, they get customer feedback and change requests faster — resulting in faster product/market fit and a better portfolio. Because independent teams deploy faster, they ship smaller increments of software often, instead of one big release every six months. Because they ship often, when an incident occurs, oftentimes they can tie the failure to the latest deployments, which are small. The small surface of code to examine reduces the engineering effort to diagnose the problems and the time to deploy a remedy.

As in a virtuous loop, speed and stability thus reinforce each other: if you can ship faster and often, you also eventually ship higher-quality software. If you increase software quality, you do fewer rollbacks, spend less time correcting failures, and you can then ship more often.

That virtuous loop is enabled by cloud-native technologies and architectural patterns. Microservice architectures enable small, decentralized, decoupled teams. Cloud-native technologies abstract further over infrastructure to provide the illusion of continuously available, scalable, self-healing, zero-downtime, limitless resources. They enable the delivery of digital products and services with the security, high availability, and resilience expected by customers.

Alright, so you decentralized with cloud-native and microservices architectures. You have these small, independent multidisciplinary teams with ownership of value streams. They react faster to changes. They deploy more often. So why would they need a value stream delivery platform (VSDP)?

Remember that you want the speed of decentralized teams without the increase in budget, operational, or security risks. Else you take a bath at scale. You would only run faster into disasters. A DevOps value stream delivery platform (sometimes summarily referred to as a self-service developer portal) effectively mitigates the aforementioned risks through increased efficiency (budget risk), stability (operational risks), and end-to-end control of the delivery (security risks).

Let’s explain those risk mitigation axes one by one.

Efficiency. DevOps VSDPs deliver both flow efficiency (a core Lean concept) and cost efficiency. Flow efficiency comes from the automation of the end-to-end software delivery cycle. It also originates from the elimination of non-value-adding tasks, in particular waiting times. Because DevOps VSDPs act as self-service developer portals, developers can scale services up and down, monitor and trace microservices — and more — without having to ask and wait for somebody else to do it.

Flow efficiency naturally drives down costs. But there is more. With decentralization, companies now have small, independent teams with ownership and decision power. A bank may move from one credit risk team of 150 people to 15 teams of 10 people. There is a tremendous waste potential if those independent teams solve 15 times on their own the same cross-cutting concerns: security, high availability, scaling, reporting, and more.

A good DevOps VSDP addresses cross-cutting concerns in the platform once and for all so teams can focus on the business-specific concerns. A good DevOps VSDP makes a carefully calibrated selection of technologies so teams across the companies do not duplicate that effort independently — while retaining some amount of flexibility to address business-specific situations.

In other words, while individual teams can optimize locally for costs, a careless implementation of decentralization may drive the overall cost up. With a centralized DevOps VSDP, companies instead drive cost efficiency globally. This is glocal cost efficiency.

DevOps VSDPs drive cost efficiency by eliminating cost duplication

DevOps VSDPs mitigate cost duplication

Stability. Good DevOps Value Stream Delivery Platforms support by default cloud-native architectures and patterns (containers, microservices, Infrastructure as Code, distributed tracing, and more). These are the best practices that leverage the unique characteristics of the cloud to provide highly scalable, resilient, and self-recovering services.

Security. Cross-sectional concerns such as security are directly baked in a DevOps VSDP. You don’t want every team figuring out on its own how to secure their customer data and investigate incidents. It is a lot of work, and the chances — and costs — of getting it wrong are high. Platforms strive to provide security capabilities such as centralized audit, security, and incident management.

So DevOps Value Stream Delivery Platforms essentially square the circle by de-risking software and team decentralization. But we described here only the baseline. Any DevOps VSDP should provide the minimum set of capabilities described. Beyond that, DevOps VSDPs are going to differentiate themselves by focusing on certain pain points at the cost of others, or by targeting specific segments.

So how do VSDPs differentiate?

First of all, we should differentiate in-house DevOps platforms, or self-service portals, from DevOps VSDP under a XaaS (anything as a service) model. Some companies will have a platform that is designed and developed as a specific software product line, and maintained by a specific team (PlatformOps team). That formalized platform has generally grown organically from a set of disparate scripts or tooling and has grown to encompass in a unified whole the key software delivery concerns that are specific to the company. 

Other companies will prefer using DevOps VSDP providers like CodeNOW under a PaaS model. Those companies do not need to invest significant CAPEX into creating their own platform and can instead focus on jump-starting their digital transformation. Those companies directly go to the heart of their value creation: the what (digital products and services to deliver to customers) — while the platform automates most of the value delivery.

DevOps VSDPs under a PaaS model provide better value for money

DevOps VSDPs under a PaaS model provide better value for money

Thus the business model is by far the most important differentiator of DevOps VSDPs. You do not see anymore companies that are not pure IT businesses maintaining their own servers. They rent computing capacity with Amazon or any other IaaS provider. In effect, IaaS providers brought inexpensive server infrastructure to the masses.

Similarly, the XaaS model will accelerate the entry of DevOps VSDPs into the mainstream. According to Gartner, DevOps VSDP will become mainstream in 2 to 5 years. There are however not enough platform teams in the market to sustain such quick growth. The XaaS model will prove instrumental in bringing a DevOps VSDP to every company on the digital transformation journey.

Second, VSDPs may target specific pain points and challenges. CodeNOW for instance eliminates vendor lock-in from its platform. CodeNOW leverages a fully open-source stack that consists of carefully selected technologies that are used by millions of developers today (e.g., GitLab, ArgoCD, Kubernetes, SonarQube). As a result, CodeNOW users are not acquiring Azure-specific skills or AWS-specific skills. Instead of proprietary technologies, they learn cloud and cloud-native concepts that they can leverage to operate any vendor tooling.

So when the time comes, they can make the best choice of vendors that optimizes cost and value. So when it makes sense, they can pick and operate multi-cloud environments; or migrate from one cloud to another. So engineers can move to other teams and reuse their knowledge immediately. Large companies told us that rotating developers create value for both developers and the company. It enables the cross-pollination of domain knowledge. It creates happier developers.

In very practical terms, companies that decide to embark on a cloud-native journey face the immediate challenge of upskilling and reskilling their teams. Microservices bring benefits but they significantly increase complexity. There are many more parts to coordinate. In response, some VSDPs provide an environment for self-experimentation, teaching, and learning that speeds up developer onboarding and team productivity.

Companies have many options to choose from, depending on what their core business is, the skills of their teams, and the stage of their digital transformation. Nonetheless, those companies who do not use a DevOps VSDP to increase their delivery KPIs are likely to progressively fall behind their competitors who do. CodeNOW, with its absence of vendor lock-in, is a safe choice for companies that want to explore the value of DevOps platforms.