Try it free
VIANOPS Introduces Next-Generation Monitoring Platform for AI-Driven Enterprises

VIANOPS Introduces Next-Generation Monitoring Platform for AI-Driven Enterprises

by | Jun 12, 2023 | 2023, Blog All

From traditional ML models to LLMs, VIANOPS Monitors Drift, Data Quality, and Bias-Prone Models to Ensure Reliability and Performance, without the High Cost

By: Dr. Navin Budhiraja, Vianai Systems Chief Technology Officer & Head of VIANOPS Platform

High-Scale Monitoring, On-the-Fly Root-Cause Analysis, and Smarter Alerts – With Unparalleled Affordability

Try VIANOPS Free

In just the last six months, there has been an explosion in the types of models enterprises want to deploy – to improve business processes, power better decision-making, increase customer and partner engagement, and improve employee productivity.

These AI-forward enterprises are companies where data science, machine learning, architecture, infrastructure, IT, and other teams are feeling immense pressure to incorporate new AI technologies within their organizations to support business users and outcomes, but who don’t yet have the tools to fully and reliably monitor them once deployed.

Models are notoriously prone to faltering once deployed to the real world due to the dynamic environment. Models are known to perpetuate bias and unfair practices. Newer types of models such as LLMs introduce new kinds of risks for their hallucinations, fabrications, and other ethical issues.

Once deployed, questions need to be answered (continuously):

With that in mind, we expanded the capabilities of VIANOPS at a similarly rapid pace to meet the emerging and accelerating demands of AI-forward enterprises.

And we’ve made it available to try it free.

Try it Free

The latest release of VIANOPS is now available to try for free. Any ML engineer, data scientist, or other ML practitioner can see and test expanded capabilities, the simplicity and usability of the platform experience, and the ability to create custom plans unique to a particular team or company’s needs.

Users can sign up free directly, or learn more here about our free trial offering before signing up.

Robust, High-Scale Monitoring

The latest iteration of the VIANOPS monitoring platform is engineered to empower data scientists and MLOps teams to manage complex, feature-rich, high-scale ML models that make their businesses run.

What do we mean by “high-scale?” It’s not necessarily about the number of models, although it could be. Instead, our definition of high-scale relates to the complexity of an individual model to properly and confidently monitor all of the details that might cause model performance to drop below what is acceptable.

VIANOPS can monitor models with tens of thousands of predictions per second, hundreds of features and segments, and subsegments, with millions or billions of transactions, across multiple time windows, to find and solve problems degrading model performance.

VIANOPS monitors three critical assets:

    • Input Data – the data used for the prediction
    • Output Data– the prediction or target of the model
    • Ground Truth Data – the real-world data used to compare with the output to determine the prediction’s accuracy

Based on the changes (or drift) in the distributions of the input data, predictions from the model, or the ground truth data, VIANOPS determines when there are significant changes in any of these. VIANOPS can analyze these differences and compare them to either the training data or any production data seen in previous time horizons (e.g. last week or last quarter). Users can also set parameters for the percentage of variation that is acceptable. If the distribution drift goes outside these threshold parameters, smart alerts are triggered to investigate the cause, and to retrain and deploy a better model. This detection and retraining can also be automated using VIANOPS APIs and Python SDK.

Once an ML model is in production, drift risks are inevitable as the models’ training data will differ from real-world data over time. VIANOPS features robust capabilities, including Root Cause Analysis, which can identify patterns at extreme scale in various spliced datasets (arbitrary segments of data) to monitor for minuscule changes and help data scientists and ML engineers take mitigation actions before the model degrades significantly and produces incorrect outcomes.

 

No one is better at this level of scale and complexity.

Low Cost

Another key VIANOPS differentiation is scale at low cost. Our customers can monitor the most complex of AI models – today’s and future types of models (including Large Language Models /LLMs) – and do this without expensive new infrastructure in order to get the scale. This is key to ensuring AI is reliable, and accessible for all types of enterprises that want to take advantage of the benefits but don’t have dramatically larger budgets than before.

VIANOPS is here for enterprises looking to affordably scale their ML operations. For example, if a model does billions of transactions daily, VIANOPS can monitor, splice, and analyze massive amounts of data with scalability.

AI Model Evolution

What was once the land of tabular data-based models has expanded at break-neck speed to include many other types of models including Large Language Models (LLMs), Generative AI models, computer vision models, and many other model types.

On improving employee productivity specifically, LLMs and GAI seem to be driving a desire for radical productivity improvements at scale across an organization, in every role – no longer just in specialized roles where knowledge of AI and AI systems is needed, but for every worker (sales, marketing, HR, finance, legal, and so on).

As these models rapidly descend on the enterprise AI landscape, a new kind of monitoring capability is necessary in order to monitor the performance and reliability of these new, highly error-prone types of models. They hallucinate, learn and perpetuate biases and otherwise introduce a host of ethical concerns and risks for an enterprise.

As more and more companies rely on complex ML models as core to their business processes, the need to monitor and manage the performance of these models becomes increasingly important.

Best-in-Class User Experience

The need for model monitoring is indisputable but what also sets VIANOPS apart from other monitoring platforms is incredible design. Good design is key to invoking trust in the system and technology and ease-of-use is imperative when it comes to collaboration at scale across the enterprise.

The VIANOPS platform offers an unmatched user experience that supports and encourages human oversight and mitigation. Smart alerts are triggered to warn users when things go wrong and allow for rapid involvement and action across teams.

Easily-digestible graphs and dashboards provide multi-dimensional views for the most comprehensive insight to model performance. Clear language and labeling, and health percentage comparisons boosts confidence for those taking charge in model performance.

Ease of Integration into Any Landscape

Easy, secure integration is critical to enterprises as machine learning data is very sensitive and enterprises don’t typically like to release their data to the public cloud. Integrating how businesses can make information secure while automating processes is essential because every enterprise has a different setup and is in another cloud. Since every company has its own structure, platforms like VIANOPS integrate their product into the company’s workflow.

VIANOPS allows for seamless integration for various APIs including:

    • Various data sources like streaming and file systems
    • Other MLOps platforms like AWS Sagemaker and Databricks
    • Collaboration and communication tools like Slack and email

By providing monitoring and alert capabilities beyond clouds and data sources, VIANOPS is an all-in-one platform to observe a variety of data points and technologies to help users save time and energy for higher-level tasks.

Key Capabilities of the VIANOPS Platform – from MLOps to LLMOps

 

  • Monitoring: Monitor AI/ML models across multi-faceted dimensions, tens of thousands of inferences per second, hundreds of features.
  • Root Cause Analysis: Run on-the-fly, ad-hoc analyses and deep dives to find patterns in millions of data points.
  • Mitigation: Mitigate risks, trigger automated workflows to solve problems, know when to retrain models.
  • Validation: Validate a new version after retraining, or a challenger model before promoting to champion.

Looking Toward the Future

The updated VIANOPS platform is built to support companies that run ML models as core to their business operations, have highly complex models, and require scale. The rapid acceleration of enterprises that rely on ML models as core to the entire business model is a new element in today’s landscape. This means not only models running in support of various processes but models that run the business, operationalizing the goods or services themselves.

Sign up for a free 30-day trial to see how VIANOPS can take your business to the next level or ML monitoring and observability for every stage of the ML lifecycle.

And reach out with questions or if you would like to get in touch to learn more about how we can help your business’s ML operations!

As Organizations Look to Scale their ML Operations, the Need for a New Kind of ML Model Monitoring Solution Becomes Increasingly Apparent

by | Apr 12, 2023 | 2023, Blog All

By: Dr. Navin Budhiraja, Vianai Systems Chief Technology Officer & Head of VIANOPS Platform

 

As more and more companies rely on machine learning models as core to their business processes, the need to monitor and manage the performance of these models becomes increasingly important. This need has steadily increased over the last few years with the natural evolution of technology adoption within existing organizations.

What is new is the rapid acceleration of businesses that rely on ML models as core to the entire business model, i.e., it is the business model. Not just models running in support of various processes, but models that run the business, delivering the goods or services themselves. Companies such as financial services firms, payment processing firms, online retailers, video and social media platforms, gaming platforms and others.

Layering onto this picture of a new kind of high-performance ML context is the external environment where we are seeing a radical acceleration of Large Language Models (LLMs) and the desire for organizations to bring advanced AI techniques quickly into their day-to-day business units, directly to users.

With these rapid accelerations, we are also seeing a generational shift in the need for ML model monitoring capabilities to ensure these models are reliable, trustworthy and high-performing – and to do this at an immense scale while keeping costs down.

Where Does the Complexity Come From?

ML model monitoring is not a new concept. Paired with observability, they have both been in play for some time. However, the need for model supervision and retraining increases as companies’ MLOps get more complicated with complex models running at scale. Monitoring tools have been around but were developed in a much different context. Just a few years ago, we didn’t have the proliferation of machine learning models running in organizations we have today. Even the word “model” was primarily associated with rules-based models and other non-ML models that an organization might use for financial forecasting, supply chain predictions and other processes.

Therefore, tools developed at that time were for organizations that needed basic monitoring capabilities on a relatively small scale. These were small-scale because of the low number of models with features at this time, often only in the tens. In addition, the models were interpretable and explainable and hence did not need constant monitoring as there was less risk of drift. Many companies at this time were just beginning to bring on data scientists to build models. Running models in production, even at a low scale, seemed quite far off into the future.

The reality is that the last few years have brought a paradigm shift. Companies have moved beyond experimentation to become more sophisticated in machine learning. With this paradigm shift toward the pervasive use of ML models, we have also seen the size, complexity, and risk of these models dramatically increase – leading to the urgent need for a new kind of ML model monitoring solution, one that can handle the massive scale challenge and power the agility that data science and MLOps teams need.

The complexity is in the details. Complexity doesn’t necessarily come from the number of models running in production. It can come from very few models that each on their own are highly complex or massive in size and scope.

The complexity is also in agility. With less complex models, teams can do scheduled and infrequent runs to observe data and make necessary changes. Less complex models are easier to drill into alerts, as there are fewer layers to sort through.

Today the world looks very different. Many companies today run ML models with 100s or 1000s of features, millions or even billions of data points that need to be analyzed, tens of thousands of inferences per second, and so on.

Traditional tools are simply not designed for this kind of scale and complexity. Even if they were able to scale to larger and more complex models, costs became prohibitive.

What Exactly is Monitoring and Observability at Massive Scale?

High-Performance MLOps teams need solutions that can get into the granular details of model performance, on the fly, in real time, to understand the critical problems that jeopardize the model’s reliability – and to do this on the largest and most complex models we can think of, at minimal cost. Monitoring at scale means not only watching for behavior changes but being able to look into the data from many dimensions – day to day, week to week, month to month, at the desired sensitivity level, into segments and subsegments, and in the context of real-world dynamics and business needs. High-scale monitoring also means drilling into large numbers of features through the noise of alerts to identify drift that matters. Even significant drift may not matter as the impact doesn’t meet the sensitivity thresholds. In contrast, other drift may appear smaller or less noticeable at the surface level but will harm the business if it is not corrected.
High-scale ML model monitoring is not a scheduled “batch job” approach. It’s not the BI of yesteryear. This is real-time, on-the-fly drill-down analytics on input data and output data on extremely high volumes of data, e.g., billions of data points.
The ability to scale at very low costs is essential. Other companies have scaling capabilities, but usually at much higher costs. VIANOPS optimizes our tools to reduce the overall cost of our models with speed 1,000 to 10,000 times faster than some popular, large-scale data processing tools at the same cost. This leads to more tangible business outcomes for those utilizing our tools and makes them more accessible.

VIANOPS High-Performance ML Model Monitoring

In architecting the VIANOPS platform, we approached it with a unique set of design principles:
    • Massive reach and depth in monitoring the most complex, largest, layered, high-volume models we could imagine.
    • Unparalleled flexibility for the user to monitor infinite possibilities, across segments and subsegments, without limitation.
    • Real-time, on-the-fly power to drill into data to see what is happening. Anytime; as-needed; ad-hoc.
    • Unmatched user experience to transform the frustration of data science and MLOps teams working with today’s tools into user delight – and improve productivity.
    • Extreme choice for companies to run on any cloud and integrate any data source.
    • Optimized tools at scale with reduced costs.
With these principles in mind, we are bringing high-performance ML operations at scale to life for businesses worldwide – both for organizations that are just getting started on the high-scale journey and for those organizations already there but struggling with the limitations of today’s tools. The need to monitor and manage the performance of ML models at scale continues to rapidly increase as companies adopt new technologies at the core of their business models. Stay tuned for our next release of VIANOPS, which will include a free trial to test drive our monitoring capabilities.
 

Reach out for questions or if you would like to get in touch to learn more about how we can help your business’s ML operations!