Being inherently limited by the wave properties of light, underwater microscope and camera compromise between their imaging resolution and field of view for in situ observations. In order to enlarge the sampling volume, they usually adopt lower imaging magnifications. However, this will sacrifice the imaging resolution so that bringing challenges for subsequent plankton species identification. 

To alleviate this magnification-resolution dilemma, Dr. Jianping Li’s group from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences has developed a novel image super-resolution method for restoring resolutions of in situ plankton images from low to high digitally.

In their recent report: Super-Resolution for In Situ Plankton Images, they demonstrated that the restoration could successfully recover underwater plankton imaging resolution with a scaling factor of 4, which was published in the first workshop on applications of computer vision for oceanic research in ICCV.

To realize this, the team has trained a deep learning model called Enhanced Deep Residual Net-work (EDSR) with a large scale real-world dataset called IsPlanktonSR, which is also built by the team through long-time hard work by using a dual-channel darkfield imaging apparatus invented by Dr. Li. During the training, the team has tried different loss functions and compared the model performance by using traditional downsampled and the IsPlanktonSR data sets. Through extensive experiments, the team has demonstrated that the model trained on real data through the contextual loss has delivered the best visual and quantitative SR performance. Additionally, the model has also been proved to generalize well to images of various plankton species or captured by different instruments.  

In the future, the developed technology is anticipated to enhance the existing plankton imageries and strengthen next-generation plankton imagers for better observation capabilities and hence deeper human beings’ understanding of the mysteries deep ocean. 

Aligned real low-resolution (LR) and high-resolution (HR) underwater plankton image pair after image registration. (Image by LI Jianping’s team)