Fast-Track Instructions

For this project, Roboflow will be the primary tool used for managing and preparing data.

You will get images in your Roboflow account by forking another Project.

Roboflow need you to "publish" the Dataset to be trained by creating a "version".

Now is time to get the details to access this Dataset version. You don’t need to download the files, the images and metadata will be directly gather from Roboflow in this workshop.

Ok, you have your Dataset ready to be used, move into the next task.

This section will guide you through the essential steps of engineering, training, and evaluation to create the first model prototype.

In this step you will be using OpenShift AI and OpenShift Data Foundation.

OpenShift AI is an integrated platform that simplifies the development and deployment of AI workloads. We will use it to create and manage the Jupyter Notebooks for experimentation and model training.

OpenShift Data Foundation is an Open Source High Performance Object Storage where we will save the models and the files with the performance metrics.

Now you have your prototyping Jupyter Notebook ready. In order to start the prototype training you just need to click the "Run all blocks" (>> icon) icon on the top bar menu (click "Restart" button once it appears in the screen).

Even being a Mock Training it could take some time to finish if you are using CPUs instead of GPUs, in the meanwhile you can take a look at the cell’s output.

To facilitate easy verification of files uploaded to the object bucket, the workshop includes a (Web UI to browse the bucket contents. You can go to that console ( https://s3-browser-USERNAME-ai-models-USERNAME-tools.apps.$CLUSTER_DOMAIN ) and open the prototype/notebook folder. You will find a folder with the "model train name" and inside you will have the model with the best performance metrics (best.pt) and the last produced with the last training epoch (last.pt) along with the performance stats for the Test, Validation and Training sets.

At this point you can navigate to "Data Science Projects" and stop your Workbench to save resources in the OpenShift cluster.

In production environments, training machine learning models is not as simple as running a script or experimenting in a notebook. A robust pipeline is essential. In this workshop we will use Kubeflow Pipelines.

In this step you will use the same tools than in the previous one: OpenShift AI and OpenShift Data Foundation.

Before Importing a pipeline you will need to enable the Pipeline server.

Wait until the Pipeline is ready. Then you can import your pipeline.

After the correct import, you will see the Pipeline diagram:

You can view the details of each task while it’s running to monitor important information. Additionally, you can check the POD name generated for the task (top right corner, in a red square in the image below), which is useful for accessing real-time logs in the OpenShift Console (since the Logs tab in the OpenShift AI Pipeline view is only available once the task has completed).

If the Pipeline run POD is scheduled in a node where other workloads using GPUs are located, it could happen that your GPU run out of memory. If it happens try reducing the batch size or launch the run again after the other workloads finished or just import the Kubeflow Pipeline that only make use of CPU (just for testing propouses).

When the Kubeflow Pipeline reaches the create-modelcar you can navigate to the "Pipelines" menu in the OpenShift Console. There you can see the OpenShift Pipeline execution that creates the modelcar container image and push it into the Container Image Registry:

After some time, the pipeline will finish. You can at that point go to the Object Storage and check that:

1- The contents that have been uplaoded to models directory in your "USERNAME-ai-models" bucket.

2- The "modelcar" container image is available in the internal Quay Container Image Registry (user: USERNAME ; password: PASSWORD)

3- The newly trained model in the Model Registry (check the left menu in OpenShift AI console), where it will be available along with all the associated metadata details that were added during the registration process.

One thing to take into account now is that you used a reduced ("Mock Training") dataset and a limited number of epochs for faster training during this workshop, so the model produced won’t perform adequately in the field. It is advisable to discard that model. Instead, you can upload and use the provided pre-trained model going forward to ensure better accuracy and robustness in deployment.

The easiest way to use the provided pre-trained model in your environment is to pull the v1 of this pre-created modelcar container image and push it into the Quay Container Image Registry.

The Model Serving Phase is where a validated machine learning model is prepared for production use.

OpenShift AI provides a Model Serving capability (based on (KServe and ModelMesh) to deploy the AI models inside the OpenShift cluster where OpenShift AI is installed.

We can use those to deploy our models in the same OpenShift cluster where the OpenShift AI platform is running but, as mentioned, these options could not be directly available for your edge use case, remember that for our use case the MLOps workflow diagram is:

If you are using Microshfit or OpenShift at the Edge you can use the Kserve Custom Resource Definitions but if you plan to deploy your model directly on RHEL, without a Kubernetes API, you will need to use a different approach, for example running containers directly with Podman using Quadlets to configure them.

In this workshop we will be using an off-the-self inference server, Triton Server, more specifically the one provided by NVIDIA. Triton uses the KServe community standard inference protocols to define HTTP/REST and GRPC APIs plus several extensions so it is a good fit.

In this section, we will explore how to deploy our model and examine the inference performance. After that you will also deploy the associated microservices to do an end-to-end validation of the architecture used for the workshop use case. This architecture leverages the model’s predictions to create a system that raises alarms when individuals are not wearing hardhats. Lastly you will promote your model to be used in production.

Let’s start by deploying the Triton Inference Server in OpenShift and check our model’s performance on it, but first you need enable the "Single-model serving" capability in one of your OpenShift Datascience Projects (USERNAME-test)

Now deploy the Inference Server with the model in onnx format provided by the modelcar container image that you produced during the training phase.

When the Inference server appears as avialable in the OpenShift AI console, you will be able to copy the internal endpoint that we need to use to access to it (copy the endpoint).

Now that the Inference Server is running and you have the endpoint needed to access it, let’s deploy a test application that uses the server to tag a set of images.

You’ll see a UI with the endpoint of your Inference Server pre-selected. You can now upload images of people wearing or not wearing hard hats to the application (for example, images from the "Test" set of your dataset), then click "Submit".

The inference process may take some time depending on the number of images and whether you’re using a GPU. Once it completes, you should see a result similar to the following:

As you can see, hardhats were detected on the images, so we are good to perform an end-to-end validations of the model using the microservices used in production, but first understand the overall solution architecture, including the microservices involved and how they communicate. In this architecture we use a webcam to detect objects at the edge, and how those detections can trigger messages/alarms that can be visualized in a dashboard on the Core Datacenter/Cloud.

The solution is based on the following microservices, you can clik on the names to get detailed information about each one:

The workflow is the following:

As you can see, there are both edge (local-side) and cloud or data center (cloud-side) environments. Let’s start by deploying the cloud-side applications in OpenShift.

You’ve successfully deployed the cloud-side applications! Now, take the next step by running the remaining applications on your own laptop. In this case you have to deploy the following services:

Inferencing with a CPU can be slow. However, if you have an NVIDIA GPU in your laptop, you can deploy the Inference Server using it for testing. Let’s proceed with the deployment.

The script creates a Podman POD with an init-container (it runs before the "main" container starts) that pull the modelcar container, extract the model files and share them with the Inference Server. That’s the same approach that when you are using the modelcar in OpenShift.

The Inference service will be available at http://localhost:8001 (gRPC) and http://localhost:8000 (REST). You can check that the model has been loaded with the right values:

Let’s move on and deploy the Camera stream manager.

This is the service that will take images from your Webcam and send them to the Inference Server. It will also publish the detections in an endpoint.

Before going ahead check the logs to be sure that there are no errors.

The final step is to deploy the "Actuator". This service will send keepalives to the Dashboard and also an alarm is the Camera stream manager detects no_helmet label in the images.

Before going ahead check the logs to be sure that there are no errors.

Now you are ready to perform the final end-to-end test. As part of the workshop materials, hardhats should be provided. If you don’t have one, you can use a cycling helmet, though this may reduce detection accuracy.

Now that you are sure that the model that you generated covers what will be needed in the field, you need to "promote" it to be used in production.

How do you promote a model to production? The approach depends on how you plan to distribute it.

In our case, we are distributing the model as a modelcar, which is essentially a dedicated container image. Promoting the model becomes straightforward: it’s as simple as assigning the correct container image tag to the image you wish to use. For example, you can use a prod tag for the model intended for production. To do this, you’ll need to select the container image with the model you’ve just created and apply the appropriate tag. Let’s take a look at how to do this in Quay.

At this stage, you are well-positioned to hand over the solution to the Platform Specialist for deployment on Edge Devices. However, if you prefer to skip that step or have already completed it in a previous part of the workshop, you can proceed to the final task for the AI Specialist.

Over time, models deployed in production environments can experience a decrease in performance due to several factors.

In our example use case, the trained model for detecting hardhats on the factory floor had been deployed and working as expected. However, over time, reports started emerging about incidents where people were not wearing helmets, but the system did not trigger any alarms. After investigation, it was found that the individuals in question were wearing cups or hats, which the model did not recognize as something that could interfere with hardhat detection. Since the model was only trained to detect hardhats and not other headgear, these individuals were simply not detected, causing false negatives.

To solve this issue, retraining the model with new data is necessary.

Dataset Update

The first step to correct the problem is to have labeled data of people wearing hat and cup in order to train our model with those as well.

You need to repeat the steps that you performed, but this time you might follow the Data Management section. Remember that this time you will need to add images of hats and cups and labeling those as hat.

Retraining

In this phase you just need to re-run the training pipeline including the last version of you Dataset in the Pipeline Run setup.

Final Testing

Once you have the new model file and associated modelcar container image after retraining, you can deploy again the Inference Server with the new model.

Besides the Inference Server, there is another microservice that you need to update: the "Actuator". This microservice monitors the last predicted class, since you added a new class hat, you need to add that into the code too. In order to do that include the label in the MONITORED_CLASSES variable when you launch the service.

You have reached the end of the AI Specialist Fast-Track. You can proceed to the Platform Specialist Introduction section or if you find it useful, you can revisit the AI Specialist model following the Full Guide.