Although AI algorithms have achieved quite some progress in plankton image recognition, most of them can only deal with limited number of known classes. In reality, however, the ocean scientists will definitely meet with new (unknown or unseen) plankton species, where the existing algorithms are very likely to fail as this is a known challenge in computer vision field called open-set recognition task. 

In a recent paper published by Dr. Li Jianping from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences and his collaborators from Xiamen University in ICCV (International Conference on Computer Vision 2021), they reported an anomaly detection pipeline and its execution methods trying to help marine plankton observers to deal with this problem.  

The paper named "Anomaly Detection for In Situ Marine Plankton Images" was published in ICCV.

Basically, the pipeline needs training before it can be used, which includes two phases, namely the pre-training phase and the post-training phase. In the pre-training phase, the team proposed a data augmentation technique to generate auxiliary datasets so that the AI model is able to acquire abundant imaginary knowledge of the unseen classes. Additionally, the team proposed a new CKA loss function to better detect the new classes and classify the existing classes simultaneously. In the post-training phase, a detector is further trained to discriminate an input image as known or unknown class based on its features extracted by the pre-trained model. If the image is determined as known class, the detector will further classify it into one of the known classes.  

After training, the AI pipeline can better sort a future image into one of the known classes or unknown class for biologist’s further actions. The team conducts experiments using the DYB-PlanktonNet built by the SIAT team, and compared the proposed method with the state-of-the-art methods. Numerical results show that the proposed method outperforms its competitors in terms of several important evaluation metrics on both classification and anomaly detection.  

The proposed open-set classification methods are expected to help marine biologists to better identify their observation targets of interest, so that the in situ monitoring of marine plankton could become more convenient and efficient. In addition, Dr. Li’s collaborators will continue to investigate the robustness improvement of the approach for both classification and anomaly detection, and the extension of the method for other challenging applications beyond the plankton image classification task. 

The training has AI classified known plankton, unseen plankton and particles and bubbles. (Image by LI Jianping’s team)