Structuring AI Projects

The adoption of AI is rapidly moving beyond hype and into reality and is set to have a significant impact on business operations and efficiencies. Taking time to plan the implementation of AI will put your organizations in a far stronger position to enjoy the AI benefits further down the line.

1. Define a use case

It is important, to begin with defining the challenges or specific problems you would like AI to solve. The more precisely we define the goal the higher our chances for successful implementation of AI. For example, stating that the enterprises would like to “rise online sales by 10%”, is not very specific. Defining that we are “aiming to raise online sales by 10% by observing the demographics of site visitors” significantly increases the chances that the goal is properly understood by the stakeholders.

2. Verify the availability of data

The second step is to make sure that the systems and processes are capable of tracking and capturing the data required to train Machine Learning models. We should make sure that the right data is being available with the right variables or features in sufficient volumes.

3. Carry the data exploration

It is tempting to leap directly into a model building, but it is very important that we first carry out a data exploration. In this step, we verify our data understanding and assumptions. This step will help us to understand the data. We discover what kind of features we deal with, the type of data categorizations, all the peculiarities of the data, and anomalies or inconsistencies we have to deal with.

Doing so will help to establish whether the data is telling the right story based on your subject matter expertise and business acumen.

4. Define model-building methodology

In any research, it is important to concentrate on the hypothesis itself rather than focusing on the end goal the hypothesis might achieve. We will run tests in order to determine the most significant variables or features, as well as, models and parameters.

In a perfect scenario, we involve business and domain experts, as their continuous feedback is critical for validation and for ensuring all stakeholders are on the same page. Indeed, as the success of an ML model is dependent on successful feature engineering, a subject matter expert will always be more valuable than an algorithm when it comes to deriving better features.

5. Define model-validation methodology

The performance measures will help us to compare, evaluate, and analyze the results from multiple models which will, in turn, help to further refine the final model. A good practice is to define a single optimization metric used to assess the performance. For example, in the classification task, we can aim as maximizing the accuracy metric. In some cases, we might include additional satisfactory metrics to which we need to comply. For example, the execution time is below 10ms.

Data will need to be divided into three data sets:

• a training set, on which the algorithm will be trained,
• a validation set, which we will use for selecting and tuning a model,
• a test set, against which it will be evaluated.

It is also a good practice to involve business and domain experts to validate the findings and ensure that everything is moving in the right direction.

6. Production and Automation rollout

Once we have built and validated the model, we can move to deploy it into production. We begin with a pilot phase of a few weeks or months, upon which business users can provide continuous feedback on the model behavior and outcome, it can then be rolled out to the wider audience.

Data ingestion should be automated with the correct tools and platforms, to spread results to the audiences. The platform must give various interfaces to account for several degrees of knowledge among the businesses’ end-users.

7. Continue updating the model

An AI system should be continuously monitored as, by understanding its the validity, your organization will be able to update the model as required.

Models can become out of date for a number of reasons. For example, the market dynamics, as well as, your organization and your business model may change. Models are built on historical data in order to predict future outcomes, but as market dynamics move away from the way your organization has always done business, so the model’s performance can deteriorate.

It’s important, to remain mindful of what process must be followed to ensure the model is up to date.

This is my adaptation of the key steps to help AI and machine learning deliver on their full potential outlined by Prentiss Donohue in Information Age.