What Can Your Camera “See”? Train Your Own AI Object Detector with Azure Custom Vision

Harnessing the power of AI to identify objects in images offers endless possibilities. This article takes you through the exciting journey of object detection using Azure Custom Vision, including detailed steps for training, deploying, and testing your very own object detection model.

Prerequisites:

Before you begin, make sure you have the following:

1. An Azure account with the necessary permissions.
2. Images for training, each containing the objects you want to detect.

Step 1: Set Up Azure Custom Vision

1. Navigate to the Azure portal (portal.azure.com).
2. In the left sidebar, select “Create a resource” and search for “Custom Vision.”
3. Click on “Custom Vision” and select “Create” to set up a new Custom Vision resource.

This will create two new resources.

Step 2: Creating Your Project

1. Setting the Stage:

• Head to the Azure Custom Vision portal and log in with your Azure account.
• Click on “New Project” and provide a descriptive name and optional details.
• Choose your target Azure resource and select “Object Detection” as the project type.

2. Selecting the Domain:

• Custom Vision offers pre-trained domains like “General” and “Retail” to leverage existing knowledge.
• If your object is specialized, choose “General” for the best starting point.
• You can explore other domains later for fine-tuning if needed.

Step 3: Training the Model

1. Gathering Data:

• Collect high-quality images containing the objects you want to detect. Aim for variety in backgrounds, poses, and lighting.
• Organize your images into tagged folders, making it easier to manage your data.

2. Uploading and Tagging:

• Back in Custom Vision, click “Add Images” and upload your prepared folders.
• Open each image and use the rectangle tool to draw bounding boxes around your target objects.
• Assign a relevant tag to each bounding box, creating categories of objects your model will learn.

3. Iteration is Key:

• Start with a small set of images and train your model. Review the results, including confidence scores for each detection.
• Add more images, addressing any misidentified or missed objects. Retrain and iterate until you achieve desired accuracy.

Step 4: Deploying Your Model

1. Publishing for Predictions:

• Once satisfied with your model’s performance, click “Publish” to make it available for predictions.
• Choose a prediction endpoint name and resource location.
• Download the prediction configuration files for integration with your chosen platform.

2. Integration Options:

• Custom Vision offers REST APIs and SDKs for various languages, allowing you to integrate your model into applications, websites, or mobile apps.
• The prediction configuration files provide detailed instructions on setting up the connection.
• Following is a python code for example:

from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
from msrest.authentication import ApiKeyCredentials
from matplotlib import pyplot as plt
from PIL import Image, ImageDraw, ImageFont
import numpy as np
import os

def main():
    from dotenv import load_dotenv

    try:
        # Get Configuration Settings
        load_dotenv()
        prediction_endpoint = os.getenv('PredictionEndpoint')
        prediction_key = os.getenv('PredictionKey')
        project_id = os.getenv('ProjectID')
        model_name = os.getenv('ModelName')

        # Authenticate a client for the training API
        credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
        prediction_client = CustomVisionPredictionClient(endpoint=prediction_endpoint, credentials=credentials)

        # Load image and get height, width and channels
        image_file = 'produce.jpg'
        print('Detecting objects in', image_file)
        image = Image.open(image_file)
        h, w, ch = np.array(image).shape

        # Detect objects in the test image
        with open(image_file, mode="rb") as image_data:
            results = prediction_client.detect_image(project_id, model_name, image_data)

        # Create a figure for the results
        fig = plt.figure(figsize=(8, 8))
        plt.axis('off')

        # Display the image with boxes around each detected object
        draw = ImageDraw.Draw(image)
        lineWidth = int(w/100)
        color = 'magenta'
        for prediction in results.predictions:
            # Only show objects with a > 50% probability
            if (prediction.probability*100) > 50:
                # Box coordinates and dimensions are proportional - convert to absolutes
                left = prediction.bounding_box.left * w 
                top = prediction.bounding_box.top * h 
                height = prediction.bounding_box.height * h
                width =  prediction.bounding_box.width * w
                # Draw the box
                points = ((left,top), (left+width,top), (left+width,top+height), (left,top+height),(left,top))
                draw.line(points, fill=color, width=lineWidth)
                # Add the tag name and probability
                plt.annotate(prediction.tag_name + ": {0:.2f}%".format(prediction.probability * 100),(left,top), backgroundcolor=color)
        plt.imshow(image)
        outputfile = 'output.jpg'
        fig.savefig(outputfile)
        print('Results saved in ', outputfile)
    except Exception as ex:
        print(ex)

if __name__ == "__main__":
    main()

Step 5: Testing and Refinement

1. Feeding New Images:

• Use your chosen integration method to send new images to your deployed model.
• Observe the detection results, including bounding boxes, confidence scores, and tag names.

• Analyze any errors or unexpected results to identify areas for improvement.

2. Continuous Learning:

• The beauty of Custom Vision lies in its iterative nature.
• Gather new images addressing shortcomings you observed in testing.
• Retrain your model with this additional data to continuously improve its accuracy and generalizability.

Conclusion

In this article, we’ve covered the entire process of creating an object detection model using Azure Custom Vision. From setting up the project to training, deploying, and testing the model, you now have a comprehensive guide to leverage the power of custom object detection in your applications. Remember to continuously iterate and improve your model for optimal performance.

With this comprehensive guide and your dedication, you’re well on your way to mastering object detection with Azure Custom Vision. Remember, the journey is as rewarding as the destination — have fun exploring the endless possibilities of AI-powered vision!