YOLOvX App has reached to a significant number of users and we get asked if it works on our device or not, how will it perform on other devices and how will it perform. To address this, we created a benchmark so that our new and old users can test through a set of tests. This AI benchmark helps determine the app’s capability to run on various mobile devices for iOS and Android. The test contains two main Computer Vision tasks Detection and Segmentation and it is mainly based on three pillars: Speed, Accuracy, and Resource.
Detection Models
Object detection identifies and localises objects in an image by drawing bounding boxes around them. Selection of object detection models are used for testing to check what devices can run smoothly in production.
Segmentation Models
Segmentation goes further by delineating the exact shape of each object, often using pixel-wise masks. A few object segmentation models are used for testing purpose.
AI Score Parameters
Speed: Models time attributes such as initialisation and inference determine the speed score.
Accuracy: As the models goes through quantisation, accuracy differs on mobile devices than original model, and hence mAP is utilised to determine the device performance for accuracy score.
Resource: Device max capability of RAM and CPU versus average utilisation during the benchmark are used to determine resource score.
