The DevOps shortcut to AI
In today’s rapidly evolving technology landscape around AI, organizations and leaders face a pressing need to adapt and thrive in the face of rapid innovation. In a keynote at DevTalks Romania, we presented our experience and insights into getting ML and AI solutions operational quickly. The keynote highlighted the benefits we get from applying best practices from DevOps to efficiently deploy, monitor, retrain, and govern ML/AI models in production (commonly referred to as model operations or MLOps).
Cutting across the hype cycle
There’s always a lot of hype and excitement surrounding emerging technologies, and now the time has come for ML and AI. Technologies typically go through stages of inflated expectations before reaching productivity. While hype is neither good nor bad, it’s important to promote learning, processes, and critical thinking within organizations to navigate through the hype to arrive at the best use of technologies in their business environment.
Three approaches to MLOps
Based on our client experience, three approaches to MLOps have emerged among large and midsized enterprises.
- Build in-house with ML specialists
- Buy a platform solution
- Leverage existing DevOps capability
The first involves hiring data scientists and specialists to handle MLOps within the data function.
The second is to invest in a specialized software product or a platform that manages the propagation of data and algorithms.
The third approach is to adapt existing processes and skills from DevOps. By integrating MLOps within the DevOps framework, organizations can build upon familiar practices and accelerate their journey towards successful AI solutions.
Novelty and distinction bias
When options are compared side by side, there’s a tendency to overemphasize differences rather than similarities. This is known to psychologists as distinction bias. And it can possibly explain why MLOps is seen as something completely novel that needs new skillsets as per the first approach and/or new tools to manage.
Investing in a specialized software product or platform solution to manage the lifecycle of AI/ML solutions also suffers from the novelty factor but in a slightly different way. Any large investment in a software solution, while standards and best practices evolve, is a risky proposition.
Instead, we believe in the third approach. It’s best to learn fast and leverage existing practices and experience as the starting point for new practices.
By avoiding the fallacy of setting up MLOps processes and practices as something completely new, we can easily avoid common mistakes and wasted efforts.
But what’s more exciting is what we gain from re-using existing know-how, which in addition to validated processes and practices, are established metrics and stakeholder understanding.
Building MLOps from DevOps
To quickly explain how we approach MLOps at Emergn, we must start with a top-down perspective of DevOps. There are two primary objectives of a business: developing products and services and running and operating them.
We follow a set of three principles for the successful development of digital products and services called Value, Flow, Quality. It brings a common language to our and our clients’ organizations and a set of scalable tools that deliver business results according to each context.
By understanding the commonalities between DevOps and MLOps, organizations can leverage proven tools, practices, and ways of working to accelerate their journey toward successful AI deployments.
Drawing on our Value, Flow, Quality principles with the key attributes of DevOps as illustrated below, we recommend forming MLOps practices on a culture of automation, self-service, continuous delivery, and observability with short feedback cycles.
This is the shortcut to efficient deployment, operation, and improvement of AI models. Let’s look at a couple of practical examples.
Observability and instrumentation
We built and delivered an automation solution for court administrators to anonymize verdicts before they are published.
The solution is integrated with the court office system and allows court staff to automate the redaction and substitutions for real names and other Personal Identifiable Information (PII) before the verdicts are made public. Clearly, the precision of the AI model for entity recognition needs to be very high.
With instrumentation built into the webservice, what we quickly discovered was that user behaviors anonymizing old court documents in test cycles were dramatically different from when the same users anonymized new court documents.
To build user confidence and trust, we built a dedicated UI for the court administrators not just to test but also to teach the model to control and improve the precision in specific model training sessions. From this dedicated UI, we found that:
- Instrumentation and observability are key to capturing benefits
- Fast feedback from real users in production is much more informative than from any QA
- DevOps specialists are faster at setting up environment configurations and backups
Self-service
We built and delivered an automation solution for a legal services firm to automate the processing of claims forms. Using pre-built AI models from Microsoft, the solution reduces the expected time spent on case administration by 70%.
One of the challenges is that forms from claimants and third-party insurers change very frequently. We found the automated solution proved that:
- Self-service is easily achievable
- Power users need a process to configure new versions of forms as they are received
- End-users can judge and decide when to (re-)train the model when the precision is visible and traceable from the UI
Our recommendations
Recognize that AI is on the hype curve. Data scientists have extremely attractive skills, but putting models into operations is not the best use of their skill sets.
In our opinion, with the current rapid development of technology, it’s too early to buy a platform to manage MLOps. Leveraging know-how from DevOps is currently the best shortcut through the hype to establish a solid foundation for ML and AI solutions.
Our recommendations for setting up MLOps are summarized in the table below.
| DOs | DONT’s |
| Automate and instrument | Rush to define new processes |
| Release updates often | Expect data scientists to be good operators |
| Design for self-service and frequent releases | Leave traceability and monitoring |
| Leverage and upskill DevOps expertise | Underestimate how much users rely on UI for users acceptance |
If you want to learn more about our Data & Analytics or DevOps capabilities, please get in touch. For more on this topic, see also: Explaining DevOps to Executives, or watch the full 45-minute keynote video from DevTalks here.
Additional insights
Explaining DevOps to executives
ProductOps is the newest NotNeededOps
How did we help one client out of a technical tight spot?
