Deep Learning Technique Could Reveal Transparent Features in Medical Images

The MIT engineers recently developed neural networks to reconstruct transparent objects in images taken with plenty of light. However, they became the first to use deep neural networks in experiments to reveal invisible objects in images taken in the dark.

In their study, the researchers reconstructed transparent objects from images of those objects, taken in almost pitch-black conditions using a “deep neural network.” This machine-learning technique involves training a computer to associate certain inputs with specific outputs — in this case, dark, grainy images of transparent objects and the objects themselves.

The researchers trained a computer to recognize more than 10,000 transparent glass-like etchings, based on extremely grainy images of those patterns. The images were taken in very low lighting conditions, with about one photon per pixel — far less light than a camera would register in a dark, sealed room. They then showed the computer a new grainy image, not included in the training data, and found that it learned to reconstruct the transparent object that the darkness had obscured.

The researchers repeated their experiments with a totally new dataset, consisting of more than 10,000 images of more general and varied objects, including people, places, and animals. After training, the researchers fed the neural network a completely new image, taken in the dark, of a transparent etching of a scene with gondolas docked at a pier. The once again found that the physics-informed reconstruction produced a more accurate image of the original, compared to reproductions without the physical law embedded. The results demonstrate that deep neural networks can be used to illuminate transparent features such as biological tissues and cells, in images taken with very little light.

“We have shown that deep learning can reveal invisible objects in the dark,” said the study’s lead author Alexandre Goy. “This result is of practical importance for medical imaging to lower the exposure of the patient to harmful radiation, and for astronomical imaging.”

Related Links:
Massachusetts Institute of Technology