Generative AI is becoming an increasingly valuable tool, enabling researchers to create new models, simulate complex systems, and explore theoretical hypotheses in a faster and more efficient way.  

One of the exciting applications of generative AI is in the field of 3D brain point generation. By using deep learning algorithms, the 3D point cloud of the patient's brain can be generated for reconstruction during minimally invasive surgery.  

Because minimally invasive surgery and automated robot-guided surgery are likely to be performed under extreme conditions, new visual environments and navigation modalities are required, placing new demands on intraoperative information acquisition capabilities. Minimally invasive surgeries have recently become more widely performed, 3D brain point cloud generation plays an essential role in overcoming the visual limitations of these surgeries.  

However, some conventional existing point cloud reconstruction methods still have challenges when applied directly to real surgical scenarios. 

To address the challenges associated with reconstructing 3D brain imaging, Prof. WANG Shuqiang's research team from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences has proposed a hierarchical shape-perception network (HSPN).  

The result was published in IEEE Transactions on Neural Networks and Learning Systems on May 11. 

In the framework of HSPN, A branching predictor is constructed to generate point clouds that accurately describe the incomplete images and then complete these point clouds with high quality. Hierarchical attention pipelines are constructed to transfer the local geometric features aggregated by each layer of the encoder to the corresponding decoding block. A novel generator via branching graph convolutional network (GCN) is constructed to describe the complex brain microstructure. Meanwhile, attention gate blocks (AGBs) are designed to efficiently aggregate geometric local features of incomplete PCs transmitted by hierarchical attention pipelines and internal features of reconstructing point clouds. The employment of AGB can significantly increase the detailed expression ability while reducing the generation error and enhancing the stability.  

With the proposed HSPN, 3D shape perception and completion can be achieved spontaneously. Comprehensive results measured by Chamfer distance and PC-to-PC error demonstrated that the proposed HSPN outperformed other competitive methods in terms of qualitative displays, quantitative experiments, and classification evaluation.  

"The proposed method has a significantly short inference time," said Prof. WANG, "which enables effective real-time feedback of local image properties and this feedback can guide doctors to find diagnostically valuable surgical locations." 

Fig.1 Architecture of HSPN. (Image by SIAT) 

Media Contact:
ZHANG Xiaomin
Email:xm.zhang@siat.ac.cn