Artificial Intelligence for IT operations or AIOps has exploded over the past few years. As more and more enterprises set about their digital transformation journeys, AIOps becomes imperative to keep their businesses running smoothly.
AIOps uses several technologies like
Machine Learning and Big Data to automate the identification and resolution of
common Information Technology (IT) problems. The systems, services, and
applications in a large enterprise produce volumes of log and performance data.
AIOps uses this data to monitor the assets and gain visibility into the
behaviour and dependencies among these assets.
According to a Gartner publication,
the adoption of AIOps by large enterprises would rise to 30% by 2023.
ZIF – The ideal AIOps platform of choice
Zero Incident FrameworkTM
(ZIF) is an AIOps based TechOps platform that enables proactive detection and
remediation of incidents helping organizations drive towards a Zero Incident
Enterprise™.
ZIF comprises of 5 modules, as
outlined below.
At the heart of ZIF, lies its Analyze and Predict (A&P) modules
which are powered by Artificial Intelligence and Machine Learning techniques.
From the business perspective, the primary goal of A&P would be 100%
availability of applications and business processes.
Let us understand more about thePredict module
of ZIF.
Predictive Analytics is one of the main USP of
the ZIF platform. ZIF encompassesSupervised, Unsupervised and Reinforcement
Learning algorithms for realization of various business use cases (as shown
below).
How does the Predict Module of ZIF work?
Through its data ingestion capabilities, the ZIF
platform can receive and process all types of data (both structured and
unstructured) from various tools in the enterprise. The types of data can be
related to alerts, events, logs, performance of devices, relations of devices,
workload topologies, network topologies etc. By analyzing all these data, the platform
predicts the anomalies that can occur in the environment. These anomalies get
presented as ‘Opportunity Cards’ so that suitable action can be taken ahead
of time to eliminate any undesired incidents from occurring. Since this is ‘Proactive’
and not ‘Reactive’, it brings about a paradigm shift to any organization’s
endeavour to achieve 100% availability of their enterprise systems and
platforms. Predictions are done at multiple levels – application level,
business process level, device level etc.
Sub-functions of Prediction Module
How does the Predict module manifest to enterprise users of the platform?
Predict module categorizes the opportunity
cards into three swim lanes.
Warning
swim lane – Opportunity Cards that have an “Expected
Time of Impact” (ETI) beyond 60 minutes.
Critical
swim lane – Opportunity Cards that have an ETI within
60 minutes.
Processed
/ Lost– Opportunity Cards that have been processed
or lost without taking any action.
Few of the enterprises that realized the power of ZIF’s Prediction Module
A manufacturing giant in the US
A large non-profit mental
health and social service provider in New York
Vasu heads Engineering function for A&P. He is a Digital Transformation leader with ~20 years of IT industry experience spanning across Product Engineering, Portfolio Delivery, Large Program Management etc. Vasu has designed and delivered Open Systems, Core Banking, Web / Mobile Applications etc.
Outside of his professional role, Vasu enjoys playing badminton and focusses on fitness routines.
“Uber, the world’s largest taxi
company, owns no vehicles. Facebook, the world’s most popular media owner,
creates no content. Alibaba, the most valuable retailer, has no inventory.
Netflix, the world’s largest movie house, own no cinemas. And Airbnb, the
world’s largest accommodation provider, owns no real estate. Something
interesting is happening.”
– Tom Goodwin, an executive at the French media group Havas.
This new breed of companies is the
fastest growing in history because they own the customer interface layer. It is
the platform where all the value and profit is. “Platform business” is a
more wholesome termfor this model for which data is the fuel; Big Data & AI/ML
technologies are the harbinger of new waves of productivity growth and
innovation.
With Big data and AI/ML is making a
big difference in the area of public health, let’s see how it is helping us
tackle the global emergency of coronavirus formally known as COVID-19.
“With rapidly spreading disease, a
two-week lag is an eternity.”
DISCOVERING/ DETECTING
Chinese
technology giant Alibaba has developed an AI system for detecting the COVID-19
in CT scans of patients’ chests with 96% accuracy against viral
pneumonia cases. It only takes 20 seconds for the AI to decide, whereas humans
generally take about 15 minutes to diagnose the illness as there can be upwards
of 300 images to evaluate.The system was trained on images and data from
5,000 confirmed coronavirus cases and has been tested in hospitals throughout
China. Per a report, at least 100 healthcare facilities are currently employing
Alibaba’s AI to detect COVID-19.
Ping An Insurance (Group) Company of China, Ltd (Ping An) aims to
address the issue of lack of radiologists by introducing the COVID-19 smart
image-reading system. This image-reading system can read the huge volumes of CT
scans in epidemic areas.
Ping
An Smart Healthcare uses clinical data to train the AI model of the COVID-19
smart image-reading system. The AI analysis engine conducts a comparative
analysis of multiple CT scan images of the same patient and measures the
changes in lesions. It helps in tracking the development of the disease,
evaluation of the treatment and in prognosis of patients.Ultimately it assists
doctors to diagnose, triage and evaluate COVID-19 patients swiftly and
effectively.
Ping
An Smart Healthcare’s COVID-19 smart image-reading system also supports AI
image-reading remotely by medical professionals outside the epidemic
areas.Since its launch, the smart image-reading system has provided services to
more than 1,500 medical institutions. More than 5,000 patients have received
smart image-reading services for free.
The
more solutions the better. At least when it comes to helping overwhelmed
doctors provide better diagnoses and, thus, better outcomes.
MONITORING
AI
based Temperature monitoring & scanning
In
Beijing, China, subway passengers are being screened for symptoms of
coronavirus, but not by health authorities. Instead, artificial intelligence is
in-charge.
Two
Chinese AI giants, Megvii and Baidu, have introduced temperature-scanning. They
have implemented scanners to detect body temperature and send alerts to company
workers if a person’s body temperature is high enough to constitute a fever.
Megvii’s
AI system detects body temperatures for up to 15 people per second andup to 16
feet. It monitors as many as 16 checkpoints in a single station. The system
integrates body detection, face detection, and dual sensing via infrared
cameras and visible light. The system can accurately detect and flag high body
temperature even when people are wearing masks, hats, or covering their faces
with other items. Megvii’s system also sends alerts to an on-site staff member.
Baidu,
one of the largest search-engine companies in China, screens subway passengers
at the Qinghe station with infrared scanners. It also uses a facial-recognition
system, taking photographs of passengers’ faces. If the Baidu system detects a body
temperature of at least 99-degrees Fahrenheit, it sends an alert to the staff
member for another screening. The technology can scan the temperatures of more
than 200 people per minute.
AI
based Social Media Monitoring
An international team is using machine learning to scour
through social media posts, news reports, data from official public health
channels, and information supplied by doctors for warning signs of the virus
across geographies.The program is looking for social media posts that mention
specific symptoms, like respiratory problems and fever, from a geographic area
where doctors have reported potential cases. Natural language processing is
used to parse the text posted on social media, for example, to distinguish
between someone discussing the news and someone complaining about how they
feel.
The approach has proven capable of spotting a coronavirus
needle in a haystack of big data. This technique could help experts learn how
the virus behaves. It may be possible to determine the age, gender, and location
of those most at risk quicker than using official medical sources.
PREDICTING
Data from hospitals, airports, and other public locations are
being used to predict disease spread and risk. Hospitals can also use the data
to plan for the impact of an outbreak on their operations.
Kalman Filter
Kalman filter was pioneered by Rudolf Emil Kalman in 1960,
originally designed and developed to solve the navigation problem in the Apollo
Project. Since then, it has been applied to numerous cases such as guidance,
navigation, and control of vehicles, computer vision’s object tracking,
trajectory optimization, time series analysis in signal processing,
econometrics and more.
Kalman filter is a recursive algorithm which uses time-series
measurement over time, containing statistical noise and produce estimations of
unknown variables.
For the one-day prediction Kalman filter can be used, while for
the long-term forecast a linear model is used where its main features are
Kalman predictors, infected rate relative to population, time-depended
features, and weather history and forecasting.
The one-day Kalman prediction is very accurate and powerful while
a longer period prediction is more challenging but provides a future trend.Long
term prediction does not guarantee full accuracy but provides a fair estimation
following the recent trend. The model should re-run daily to gain better
results.
The Center for Systems Science and
Engineering at Johns Hopkins University has developed an interactive, web-based
dashboard that tracks the status of COVID-19 around the world. The resource
provides a visualization of the location and number of confirmed COVID-19
cases, deaths and recoveries for all affected countries.
The primary data source for the
tool is DXY, a Chinese platform that aggregates local media and government
reports to provide COVID-19 cumulative case totals in near real-time at the
province level in China and country level otherwise. Additional data comes from
Twitter feeds, online news services and direct communication sent through the
dashboard. Johns Hopkins then confirms the case numbers with regional and local
health departments. This kind of Data analytics platform plays a pivotal role
in addressing the coronavirus outbreak.
All data from the dashboard is also
freely available in the following GitHub repository.
One of AI’s core strengths when
working on identifying and limiting the effects of virus outbreaks is its
incredibly insistent nature. AIsystems never tire, can sift through enormous
amounts of data, and identify
possible correlations and causations that humans can’t.
However, there are
limits to AI’s ability to both identify virus outbreaks and predict how
they will spread. Perhaps the best-known example comes from the neighboring
field of big data analytics. At its launch, Google Flu
Trends was heralded as a great leap forward in relation to
identifying and estimating the spread of the flu—until it underestimated the
2013 flu season by a whopping 140 percent and was quietly put to rest.Poor data
quality was identified as one of the main reasons Google Flu Trends
failed. Unreliable or faulty data can wreak havoc on the prediction
power of AI.
Bargunan is a Big Data Engineer and a programming enthusiast. His passion is to share his knowledge by writing his experiences about them. He believes “Gaining knowledge is the first step to wisdom and sharing it is the first step to humanity.”
The Healthcare Industry is going through a quiet revolution. Factors like disease trends, doctor demographics, regulatory policies, environment, technology etc. are forcing the industry to turn to emerging technologies like AI, to help adapt to the pace of change. Here, we take a look at some key use cases of AI in Healthcare.
Medical Imaging
The application of Machine Learning (ML) in Medical Imaging is showing highly encouraging results. ML is a subset of AI, where algorithms and models are used to help machines imitate the cognitive functions of the human brain and to also self-learn from their experiences.
AI can be gainfully used in the different stages of medical imaging- in acquisition, image reconstruction, processing, interpretation, storage, data mining & beyond. The performance of ML computational models improves tremendously as they get exposed to more & more data and this foundation on colossal amounts of data enables them to gradually better humans at interpretation. They begin to detect anomalies not perceptible to the human eye & not discernible to the human brain!
What goes hand-in-hand with data, is noise. Noise creates artifacts in images and reduces its quality, leading to inaccurate diagnosis. AI systems work through the clutter and aid noise- reduction leading to better precision in diagnosis, prognosis, staging, segmentation and treatment.
At the forefront of this use case is Radio genomics- correlating cancer imaging features and gene expression. Needless to say, this will play a pivotal role in cancer research.
Drug Discovery
Drug Discovery is an arduous process that takes several years from the start of research to obtaining approval to market. Research involves laboring through copious amounts of medical literature to identify the dynamics between genes, molecular targets, pathways, candidate compounds. Sifting through all of this complex data to arrive at conclusions is an enormous challenge. When this voluminous data is fed to the ML computational models, relationships are reliably established. AI powered by domain knowledge is slashing down time & cost involved in new drug development.
Cybersecurity in Healthcare
Data security is of paramount importance to Healthcare providers who need to ensure confidentiality, integrity, and availability of patient data. With cyberattacks increasing in number and complexity, these formidable threats are giving security teams sleepless nights! The main strength of AI is its ability to curate massive quantities of data- here threat intelligence, nullify the noise, provide instant insights & self-learn in the process. Predictive & Prescriptive capabilities of these computational models drastically reduces response time.
Virtual Health assistants
Virtual Health assistants like Chatbots, give patients 24/7 access to critical information, in addition to offering services like scheduling health check-ups or setting up appointments. AI- based platforms for wearable health devices and health apps come armed with loads of features to monitor health signs, daily activities, diet, sleep patterns etc. and provide alerts for immediate action or suggest personalized plans to enable healthy lifestyles.
AI for Healthcare IT Infrastructure
Healthcare IT Infrastructure running critical applications that enable patient care, is the heart of a Healthcare provider. With dynamically changing IT landscapes that are distributed, hybrid & on-demand, IT Operations teams are finding it hard to keep up. Artificial Intelligence for IT Ops (AIOps) is poised to fundamentally transform the Healthcare Industry. It is powering Healthcare Providers across the globe, who are adopting it to Automate, Predict, Remediate & Prevent Incidents in their IT Infrastructure. GAVS’ Zero Incident FrameworkTM (ZIF) – an AIOps Platform, is a pure-play AI platform based on unsupervised Machine Learning and comes with the full suite of tools an IT Infrastructure team would need. Please watch this video to learn more.
Artificial intelligence for IT operations (AIOps) is an umbrella term
for the application of Big Data Analytics, Machine Learning (ML) and other Artificial
Intelligence (AI) technologies to automate the identification and resolution of
common Information Technology (IT) problems. The systems, services and
applications in a large enterprise produce immense volumes of log and performance
data. AIOps uses this data to monitor the assets and gain visibility into the
working behaviour and dependencies between these assets.
According to a Gartner study, the adoption of AIOps by large enterprises
would rise to 30% by 2023.
ZIF – The ideal AIOps platform of choice
Zero Incident FrameworkTM (ZIF) is an AIOps based TechOps
platform that enables proactive detection and remediation of incidents helping
organizations drive towards a Zero Incident Enterprise™
ZIF comprises of 5 modules, as outlined below.
At the heart of ZIF, lies its Analyze and Predict (A&P) modules
which are powered by Artificial Intelligence and Machine Learning techniques.
From the business perspective, the primary goal of A&P would be 100%
availability of applications and business processes.
Come, let us understand more about the Analyze
function of ZIF.
With Analyzehaving a Big Data platform under its hood, volumes of raw monitoring data, both structured and unstructured, can be ingested and grouped to build linkages and identify failure patterns.
Data Ingestion and Correlation of Diverse Data
The module processes a wide range of data from
varied data sources to break siloes while providing insights, exposing
anomalies and highlighting risks across the IT landscape. It increases productivity
and efficiency through actionable insights.
100+ connectors for leading tools,
environments and devices
Correlation and aggregation methods
uncover patterns and relationships in the data
Noise Nullification
Eliminates duplicate incidents, false
positives and any alerts that are insignificant. This also helps reduce the
Mean-Time-To-Resolution and event-to-incident ratio.
Deep learning algorithms isolate
events that have the potential to become incidents along with their potential
criticality
Correlation and Aggregation methods
group alerts and incidents that are related and needs a common remediation
Reinforcement learning techniques are
applied to find and eliminate false positives and duplicates
Event Correlation
Data from various sources are ingested
real-time into ZIF either by push or pull mechanism. As the data is ingested,
labelling algorithms are run to label the data based on identifiers. The
labelled data is passed through the correlation engine where unsupervised algorithms
are run to mine the patterns. Sub-sequence mining algorithms help in
identifying unique patterns from the data.
Unique patterns identified are clustered
using clustering algorithms to form cases. Every case that is generated is
marked by a unique case id. As part of the clustering process, seasonality
aspects are checked from historical transactions to derive higher accuracy of
correlation.
Correlation is done based on pattern
recognition, helping to eliminate the need for relational CMDB from the enterprise.
The accuracy of the correlation increases as patterns reoccur. Algorithms also
can unlearn patterns based on the feedback that can be provided by actions
taken on correlation. As these are unsupervised algorithms, the patterns are
learnt with zero human intervention.
Accelerated Root Cause Analysis (RCA)
Analyze module helps in identifying the
root causes of incidents even when they occur in different silos. Combination
of correlation algorithms with unsupervised deep learning techniques aid in accurately
nailing down the root causes of incidents/problems. Learnings from historical
incidents are also applied to find root causes in real-time. The platform
retraces the user journeys step-by-step to identify the exact point where an
error occurs.
Customer Success Story – How ZIF’s A&P
transformed IT Operations of a Manufacturing Giant
Vasu heads Engineering function for A&P. He is a Digital Transformation leader with ~20 years of IT industry experience spanning across Product Engineering, Portfolio Delivery, Large Program Management etc. Vasu has designed and delivered Open Systems, Core Banking, Web / Mobile Applications etc.
Outside of his professional role, Vasu enjoys playing badminton and focusses on fitness routines.
Yes, AIOps the messiah of ITOps is here to stay! The Executive decision now is on the who and how, rather than when. With a plethora of products in the market offering varying shades of AIOps capabilities, choosing the right vendor is critical, to say the least.
Exclusively AI-based Ops?
Simply put, AIOps platforms leverage Big Data & AI technologies to enhance IT operations. Gartner defines Acquire, Aggregate, Analyze & Act as the four stages of AIOps. These four fall under the purview of Monitoring tools, AIOps Platforms & Action Platforms. However, there is no Industry-recognized mandatory feature list to be supported, for a Platform to be classified as AIOps. Due to this ambiguity in what an AIOps Platform needs to Deliver, huge investments made in rosy AIOps promises can lead to sub-optimal ROI, disillusionment or even derailed projects. Some Points to Ponder…
Quality in, Quality out. The value delivered from an AIOps investment is heavily dependent on what data goes into the system. How sure can we be that IT Asset or Device monitoring data provided by the Customer is not outdated, inaccurate or patchy? How sure can we be that we have full visibility of the entire IT landscape? With Shadow IT becoming a tacitly approved aspect of modern Enterprises, are we seeing all devices, applications and users? Doesn’t this imply that only an AIOps Platform providing Application Discovery & Topology Mapping, Monitoring features would be able to deliver accurate insights?
There is a very thin line between Also AI and Purely AI. Behind the scenes, most AIOps Platforms are reliant on CMDB or similar tools, which makes Insights like Event Correlation, Noise Reduction etc., rule-based. Where is the AI here?
In Gartner’s Market Guide, apart from support features for the different data types, Automated Pattern Discovery is the only other Capability taken into account for the Capabilities of AIOps Vendorsmatrix. With Gartner being one of the most trusted Technology Research and Advisory companies, it is natural for decision makers to zero-in on one of these listed vendors. What is not immediately evident is that there is so much more to AIOps than just this, and with so much at stake, companies need to do their homework and take informed decisions before finalizing their vendor.
Most AIOps vendors ingest, provide access to & store heterogenous data for analysis, and provide actionable Insights and RCA; at which point the IT team takes over. This is a huge leap forward, since it helps IT work through the data clutter and significantly reduces MTTR. But, due to the absence of comprehensive Predictive, Prescriptive & Remediation features, these are not end-to-end AIOps Platforms.
At the bleeding edge of the Capability Spectrum is Auto-Remediation based on Predictive & Prescriptive insights. A Comprehensive end-to-end AIOps Platform would need to provide a Virtual Engineer for Auto-Remediation. But, this is a grey area not fully catered to by AIOps vendors.
The big question now is, if an AIOps Platform requires human intervention or multiple external tools to take care of different missing aspects, can it rightfully claim to be true end-to-end AIOps?
So, what do we do?
Time for you to sit back and relax! Introducing ZIF- One Solution for all your ITOps ills!
We have you completely covered with the full suite of tools that an IT infrastructure team would need. We deliver the entire AIOps Capability spectrum and beyond.
ZIF (Zero Incident Framework™) is an AIOps based TechOps platform that enables proactive Detection and Remediation of incidents helping organizations drive towards a Zero Incident Enterprise™.
The Key Differentiator is that ZIF is a Pure-play AI Platform powered by Unsupervised Pattern-based Machine Learning Algorithms. This is what sets us a Class Apart.
Rightly aligns with the Gartner AIOps strategy. ZIF is based on and goes beyond the AIOps framework
Huge Investments in developing various patented AI Machine Learning algorithms, Auto-Discovery modules, Agent & Agentless Application Monitoring tools, Network sniffers, Process Automation, Remediation & Orchestration capabilities to form Zero Incident Framework™
Powered entirely by Unsupervised Pattern-based Machine Learning Algorithms, ZIF needs no further human intervention and is completely Self-Reliant
Unsupervised ML empowers ZIF to learn autonomously, glean Predictive & Prescriptive Intelligence and even uncover Latent Insights
The 5 Modules can work together cohesively or as independent stand-alone components
Can be Integrated with existing Monitoring and ITSM tools, as required
Applies LEAN IT Principle and is on an ambitious journey towards FRICTIONLESS IT.
