How do you know if AI will be the solution to your problem? And are your data even ready for it?
I frequently hear from contacts in the digital business space that they urgently want to get started on AI, to get the first learnings. This is mainly pushed by C-level executives or fortunate individuals in positions of wealth and business acumen who drive their brand new intelligent cars and subscribe to Elon Musks’ vision of…everything.
Alas, we must recognize that the AI programming in a self-driving car has taken billions of gigabytes of data and compute-hours to develop and tune. Alone, the car’s AI’s maintenance is a tremendous task, requiring many people with various skills.
Self-driving cars have three benefits in terms of success for AI.
They generate a lot of high-quality data (to improve and train the next version of AI).
The output result is just four controls (speed, brake, left, right).
The input they consume is “just” video and has a closed domain purpose (in this case, navigating traffic).
Your business data, however, is an entirely different matter!
Some companies have a lot of data. Recently, I was working on a project for a team that had a dataset of 45 million data points. They explained they had a particular error, and they wanted to forecast when this error occurred. Anecdotally, this was supposed to happen every two weeks and with one full year of data (45M), we should be OK.
What happened was that there were issues every week, but not the kind they were looking for. What they were looking for was only happening five times over the one year of data we had, and we had to conclude that we would not be able to make anything meaningful in terms of AI to identify the error early on.
In this case, the outcome was the decision not to do an AI project. The analysis concluded that it would not benefit us to use AI to solve the problem. Instead, good old statistics came to the rescue, and we found an excellent solution to identify and classify the important from the less important errors. We handed over the algorithm, and the project ended.
In another scenario, AI will definitely be the right choice to solve a given problem, but it’s by no mean a given that it will work in any case. To identify this, we want to perform the pre-AI analysis of the data provided. The analysis will focus on the desired goals and what data, with what frequency, can affect the goal. In the car’s case, it’s easy, “just don’t crash.” In business data, identifying this goal is orders of magnitude more difficult.
How do I know if my data are ready for AI?
It can be challenging to tell but, generally, AI works well for finding patterns. The rule of thumb is that if a human cannot provide a relatively large set of examples to identify the situations we want to detect automatically with AI, it will be difficult. Getting a human to classify, e.g., images or sounds is a well-known case of AI working well. However, detecting the historical factors leading up to a customer taking the leap and buying a particular product entails a whole new complexity level. There are promising prospects concerning business data in quality control where the report showing “rejected products” will serve very well as a training set. It could also be in customer segmentation to know who the “bad” customers are, which can serve as a training set.
Using AI, there is also an option to identify the factors affecting the desired outcome. In other words, you can ask, “what makes a good customer?”. What made the best ten productions run the best? What parameters? Was it the employees, the materials, the sunshine, the afternoon cake in the canteen? Well, just add the data, cook on full heat for a couple of hours on one of our supercomputers and get the result. -It’s not that easy, but still, it’s not rocket science with the tools available.
Some time ago, we evaluated a project concerning microscopy imaging of crystalline structures in metal, but again, that’s images and thus on the “easy” end of the scale. That project broke on IT infrastructure. Someone needed to get the data from the microscope to the cloud and back; That can be expensive, but it’s another excellent learning from our past projects. If you decide to take the next step after an AI pre-analysis, you may need to invest in additional software, subscriptions for cloud services, and new processes.
In summary: Image, video streams, as used by cars, mobile phones, and YouTube, is the “easy” part, as a well-known set of requirements and methodologies apply. Corporate transactional data of various kinds are more challenging to work with, but the benefit is also equally high. Imagine being able to do something that may be difficult but will be out of your competitors’ reach for years to come.
If you are unsure and want to know if your data are ready for AI, give us a call. The process is quite simple and will require minimal effort on your part. At MindVectors Innovation Lab, we will crunch the numbers, provide you with feedback on your readiness, and suggest a path forward.
.png)