VISUALIZATION OF RECONSTRUCTION OF 3D SCENE USING SEMANTIC REGULARIZATION OF NORMALS TSDF WHEN TRAINING NEURAL NETWORK Russian patent published in 2024 - IPC G06N3/08 G06T17/00 G06T7/55 

FIELD: image processing.

SUBSTANCE: method for reconstruction of 3D scene and its visualization using a neural network consisting of a basic neural network, including a skeleton, which is the main part of the basic neural network, which calculates features, a TSDF head, which performs prediction of TSDF values in each voxel, and a segmentation head, which is 3D sparse convolutional segmentation module, which predicts the segmentation marks in each voxel, comprising steps of: basic neural network is trained to obtain TSDF volume for scene voxels as follows, training data is input, TSDF loss between TSDF prediction is calculated, calculating the segmentation loss between the segmentation prediction and the segmentation reference, calculating the coordinates of the normals, calculating the total loss function as the total loss function TSDF, minimizing the total loss function, using a trained basic neural network to obtain a TSDF volume of an input sequence of RGB frames of a real scene, applying an algorithm to the TSDF volume, which calculates reconstruction of 3D scene, 3D scene reconstruction is rendered to obtain 3D scene visualization.

EFFECT: high accuracy of reconstructing 3D scene.

11 cl, 2 dwg, 3 tbl