Intelligent microscopes for the detection of rare biological events

Imagine you are a Ph.D. Student with a fluorescence microscope and a sample of live bacteria. How can these resources best be used to obtain detailed observations of bacterial division from the sample?

You may be tempted to forego food and rest, sit continuously at the microscope and take pictures as bacterial end division begins. (It can take a bacterium hours to divide.) It’s not as crazy as it sounds, as manual detection and detection control is widespread across many sciences.

Alternatively, you can set the microscope to take pictures randomly and as often as possible. But too much light weakens the sample’s fluorescence faster and can prematurely destroy living samples. Also, since few would contain images of dividing bacteria, you would generate many uninteresting images.

Another solution would be to use artificial intelligence to detect precursors to bacterial division and use that to automatically update the microscope’s control software to capture more images of the event.

EPFL biophysicists have now found a way to automate microscope control for detailed imaging of biological events while limiting sample loading using artificial neural networks. Their technique works for bacterial cell division and for mitochondrial division. The details of their intelligent microscope are described in natural methods.

“An intelligent microscope is something like a self-driving car. It has to process certain types of information, subtle patterns to which it then responds by changing its behavior,” explains study leader Suliana Manley from EPFL’s Laboratory of Experimental Biophysics. “By using a neural network, we can detect much more subtle events and use them to drive changes in acquisition speed.”

Manley and her colleagues first solved how to recognize mitochondrial division, which is more difficult than in bacteria like C. crescentus. Mitochondrial division is unpredictable because it occurs infrequently and can occur almost anywhere within the mitochondrial network at any point in time. But the scientists solved the problem by training the neural network to look out for mitochondrial constriction, a change in shape of mitochondria that leads to division, combined with observations of a protein known to be enriched at sites of division.

When both constrictions and protein concentrations are high, the microscope switches to high-speed imaging to capture many images of division events in detail. When constriction and protein levels are low, the microscope switches to slow imaging to avoid exposing the sample to excessive light.

Using this smart fluorescence microscope, scientists showed that they could observe the sample longer compared to standard fast imaging. Although the sample was more stressed compared to slow standard imaging, they were able to get more meaningful data.

“The potential of smart microscopy includes measuring what would be missed in standard imaging,” explains Manley. “We’re capturing more events, measuring smaller bottlenecks, and can track each split in more detail.”

The scientists are making the control framework available as an open-source plugin for the open microscope software Micro-Manager, with the aim of enabling other scientists to integrate artificial intelligence into their own microscopes.