AI Image Recognition

Resco can integrate with Azure's Custom Vision service to classify images. This essentially allows functions similar to barcode scanning - except you don't need barcodes!

Prerequisites

Mind the following prerequisites for using this function:

• Azure Custom Vision license or subscription and at least one model for classifying images
• Connection to the Internet to access Azure servers that perform the necessary image analysis
• Proper configuration in Woodford (see below)

Change field format to barcode

The entity where you want to use image recognition has to include a field with barcode format.

1. Using Woodford, edit an app project and select the entity from the Project menu.
2. Select the field. It must have the type Single Line of Text.
3. On the Properties pane, set Formatting to Barcode.
4. Click Save.

For more information about managing entities and fields in Woodford, see Managing entities.

Set up connection to Azure services

1. Edit an app project and select Settings > AI Image Recognition from the Project menu.
2. Click New to define a new model.
3. As Name, enter the name of the configuration model.
4. As Entity, enter the entity where the model should be used.
5. As Service type, use Azure.
6. For Prediction key and URL, use the values from your Azure Custom Vision site.
7. Click OK to save the model.

You can define multiple models as needed.

Use case: Find a record in a view

For entities that include a barcode field and have at least one model created, there's an additional button in the views for image recognition. It allows you to take a photo. The picture is then analyzed by Azure. If the resulting tag (with highest probability) equals the primary name of a record in the list, that record is selected. In effect, this is similar to barcode scanning, using image tag instead of a barcode for record identification.

Use case: Place image classifier to a field in a form

Using rules, you can add a custom function to your forms that initiates camera and image classification. If the image matches the model (with a probability above a customizable threshold), the name of the model is saved to a field.

For example, you can add a row button to your form that uses the Scan Image step to fill the value on a form.

1. Edit a form.
2. Click Add Button and set Binding to MyScanButton.
3. Click Button Click to open the rules editor.
4. Add the following rule and save your changes.

If	Condition	ChangedItem	Equals	MyScanButton
Then	Scan:	Action: AI Image Recognition	Save to: Entity.name	Probability: <Pick Value>

Use case: Scan image to answer questions in questionnaires

Using rules in Questionnaire Designer, you can add a custom function to your questionnaire template that initiates camera and image classification. If the image matches the model (with a probability above a customizable threshold), the name of the model is saved as an answer.

This applies to the following question types:

• TBD

See also

• Let the AI do the work Blog