The hype surrounding Artificial Intelligence (AI) technologies is creating a widening gap between expected business outcomes and actual results. This was highlighted by our survey which showed a startling delta between expectations and reality:

But this is not new when it comes to technology investment, we’ve seen the pattern repeated for decades. Failure to realize how technology investments have completely different risks and returns leads to initiatives that are managed according to unrealistic expectations and destined to disappoint or even fail. In this article, we share our insights on portfolio and delivery management so you can learn how to better align expectations to future outcomes.

“You got to be in it to win it.“

The challenge with any investment is that, on one side, not putting any effort in is not going to yield any returns at all. At the same time, results are not guaranteed, so it’s also correct to say:

 “Don’t confuse effort with results.”

The right amount of effort to invest depends a lot more on how predictable the outcomes are than on the technology.

Risk is not the same as uncertainty

Before any effort is made, a decision is taken on whether to invest in pursuing that opportunity. Regardless of whether this decision is made implicitly or explicitly, an economic decision will have been made when resources are spent¹.  And this decision will have been made based on expected outcomes. From the economic viewpoint, the decision will follow the risk-return trade-off. This principle² states that the potential return rises with an increase in risk. And only by accepting higher risks can you expect higher returns. Note that risk is strictly defined in economic terms; it refers only to known and measurable probabilities. That does not account for the unknowns.

Adding uncertainty to economic risk is what makes technology investments unpredictable

Uncertainty is what you don’t know. This lack of information makes uncertainty, as opposed to risk, impossible to quantify accurately.

The source of uncertainty is an unclear (or unknown) relationship between cause and effect, and the limited control you have over the effects. For a more in-depth description of the management of uncertainty, see the VUCA (Volatility, Uncertainty, Complexity, and Ambiguity) framework.

But uncertainty is inevitably part of the decision-making. While degrees of uncertainty cannot be precisely calculated, different approaches to decisions should be applied based on the degree of predictability.

Unpredictable outcomes

Outcomes are the results (effects) of investments, and evident only in hindsight. As described above, outcomes are subject to both controllable and uncontrollable causes. For example, we can control the size of an investment with the technologies we select. Customer behaviors and competitors’ and regulators’ activities in the marketplace are not controllable.

In our view, it is crucial to manage different technology investments by their degree of predictability. We recommend applying different portfolio management policies to at least three different categories of technology investments:

1. Efficiency improvements that lower costs for the business to operate. Investment in efficiency and productivity has a relatively high degree of certainty but comes with a limited return potential.

2. Experience improvements to products and services. These can both reduce costs and increase revenue and are less predictable as competition adds to complexity.

3. Innovation, launching new products, or expanding into new markets to gain revenue from new sources are the least predictable but also offer the highest potential returns.

Even though potential returns are nominally comparable to ROI (Return on Investment), the way to invest and manage effort should be different as efficiencies and innovations have diametrically different predictability.

Deliver operational efficiency improvements rapidly

While returns are not completely predictable for investments in productivity and cost savings, increased effort can be expected to return bigger savings. Although this is only up to a point, because you cannot remove more than 100% of the costs. 

Examples of efficiency investments we have realized include automating invoicing processes, anonymizing and classifying data, and detecting anomalies for maintenance and energy consumption. Learn about how we help our clients with AI.

Investment decisions

For efficiency initiatives, predictability is high as implementation is typically within the company management’s control. And while the exact state of processes and operations might not be known, information is easily obtained. This is where decisions can be made with discounted cash flow methods, and cost/benefit ratios can be used to compare alternatives.

In our experience, most business cases for IT solutions driving efficiency outcomes will have benefit/cost ratios ranging between three and 20. Investments with estimated pay-back below three times the initial investment are rarely approved, and higher returns than 20x are rarely believed.

Approaches to increase payoffs

In cases where the target outcome is cost savings or productivity improvements, traditional project management approaches can be used to approve and manage initiatives. But there are practices that can significantly increase payoffs if effectively applied. The ones we recommend are:

DO deliver fast to start benefit realization as soon as possible.
The longer the benefit realization period becomes, the bigger the payoffs.

DO analyze flexibility and variants before automation.
Even with AI, technologies do not have common sense, and it can be prohibitively expensive to replicate the flexibility and judgment humans use in edge-case scenarios.

DON’T bundle multiple similar initiatives into one for simplicity.
There are surprisingly few economies of scale in IT investments. It is better to keep things independent and deliver value fast.

DON’T build business cases based on a solution.
Start from the value of the problem the solution is set to solve rather than estimate the benefit of a planned solution.

Improve experiences with product management practices

Creating digital products and services that serve customers and employees is considerably more complex than improving operational efficiencies. Every individual has their own views of what they prefer and what choices they want to make. Competitors also make these ventures a lot less predictable. 

Investment decisions

Cost-benefit ratios are not very useful as a decision-making tool for investing in digital products and services. While costs can be somewhat accurately estimated, the variation and uncertainty of the benefits make the ratio calculation moot. Instead, looking at scenarios and allocating resources using opportunity costing models is a better approach. And make investment decisions periodically, e.g., quarterly, evaluating the current options at that time instead of approving individual opportunities one by one.

Approach to increase payoffs

Once the decision has been made to spend effort pursuing outcomes, the management practices we recommend are needed to provide flexibility to adapt to new information and events to maximize the returns.

DO change plans and projections as new information arrives.
If plans and requirements don’t change based on feedback and new information, the probability of reaching better outcomes does not increase³.

DO deliver in increments.
Uncertainties don’t disappear by deliberate planning; it is far better to test the waters with frequent releases to friends and families instead of a big-bang go-to-market approach.

DO prioritize by business value.
Prioritize and schedule work based on forecasted business value. Opportunity costing models are most effective in ensuring value is maximized under conditions of limited resources.

DON’T manage delivery as a one-time activity with project management.
Instead of starting and stopping projects, manage the lifecycle, view progress, and allocate resources for fixed periods instead of a defined scope.

DON’T outsource your core products or services.
Flexibility and fast adaptation are important to maximize returns. It rarely makes sense to manage product development by contracts.

Drive innovations with experiments

Innovations drive new business from novel products or from new ways of doing business. Innovations are found at the extreme end of the risk-reward spectrum; usually with considerable risk, but it is the level of uncertainty associated with novel concepts that makes innovations highly unpredictable.

Investment decisions

Investments in innovation are not too far off from playing the lottery – the most likely scenario for every initiative is a negative return. But on the flip side, the payoffs are typically asymmetric and can be unlimited for truly disruptive innovations.

In other words, there’s an extremely weak correlation between the effort spent and the potential return. For innovations to pay off, the strategy of investment is to place many small bets rather than “betting the farm” on one idea. Like experience improvements, investing in a fixed capacity provides control over costs while allowing for rapid learning.

Approach to increase payoffs

The approach to increasing the payoffs of investing in innovations is to make it efficient (affordable and fast) to evaluate many ideas.

DO set up a permanent innovation organization.  
It must perform systemic experimentation and evaluate ideas quickly and effectively.

DO follow the patterns of design thinking.
Successively reduce uncertainty about desirability, feasibility, and viability.

DON’T overestimate the probability of success.
If the decision-making processes cannot tolerate uncertainty, change the decision instead of adjusting the estimates.

DON’T punish or try to avoid failures.
A good experiment provides learning both when hypotheses are validated and refuted.

Before you go

Potentially with the exception of “death and taxes^”5, uncertainty is guaranteed. What makes IT investments unpredictable is rarely the technology itself but the outcome the solutions are set to provide

Developing digital products and services is considerably less predictable than pursuing operational efficiencies, regardless of the technology used. Product management practices are better suited to delivering compelling and competitive experiences than project management approaches.

Innovations have long time horizons, are novel solutions, and are very uncertain. They are extremely unpredictable ventures and should not be evaluated and managed in the same way as efficiency and experience improvements.

It’s time to start governing for experimentation, not certainty!

If you found this article interesting, you’ll enjoy reading our insights on the cultural dimension of AI productivity, exploring our key strategies to successfully compete with experiences, or learning about how we support transformation alongside our partner Strategyzer. If you’d like to learn more, get in touch!

Footnotes

¹ Limited to companies depending on technology for their business model. Not companies that are in the business of selling technology.  

² The term ‘Economics’ is derived from the Greek word ‘Oikonomia’ which roughly translates to ‘household management’.

³ Original quote from Bob Brown who hosted the New York Lottery.

⁴ Our reluctance to change plans is well documented in everything from The Cone of Uncertainty to loss aversion bias to my favourite, the Monty Hall problem.

⁵ Benjamin Franklin used this phrase when referring to the US Constitution in a letter in 1789. 

Definitions and terminology

Asymmetric payoff: When the downside is limited, but the upside is unlimited. https://asymmetryobservations.com/definitions/asymmetric-payoff/

Complexity: A system of intertwined forces and issues, making cause-and-effect relationships unclear. https://en.wikipedia.org/wiki/VUCA

Opportunity cost: The potential benefits that a business misses out on when choosing one opportunity over another. https://en.wikipedia.org/wiki/Opportunity_cost

Outcome:  A condition or occurrence traceable to a cause. https://www.merriam-webster.com/thesaurus/outcome

Risk-return trade-off (aka risk-reward spectrum): Principle stating that the potential return rises with an increase in risk. https://www.investopedia.com/terms/r/riskreturntradeoff.asp

Uncertainty: Lack of information about unknown events and issues leading to unpredictability. https://en.wikipedia.org/wiki/VUCA