Dr. Hari Shroff — The Science and Play of Super Resolution Imaging

>> Diego (narration): A couple of weeks ago NASA unveiled the first images captured from the James Webb Telescope that was launched into space on Christmas of last year. The images feature nearby planets and distant galaxies all in new and improved resolution. And they’re pretty awe-inspiring, especially when you put them next to pictures taken by its predecessor, the Hubble Space telescope, and you can actually see how far imaging the extraterrestrial has come in a few short decades.

Well, the same can be said about the advances of imaging biology here on Earth.

>> Dr. Shroff: It's sometimes easy to forget how amazing this is, right? That we're actually zooming in to a single cell and looking inside it. You can see not just the organelles, but you can see like what's inside the organelles.

>> Diego (narration): That’s Dr. Hari Shroff, a senior investigator and chief of the Section on High Resolution Optical Imaging at the National Institute of Biomedical Imaging and Bioengineering. Instead of pointing instruments up to the stars to peer into the vanishingly distant, scientists like Dr. Shroff, are focusing their gaze down to see the invisibly small.

>> Dr. Shroff: The bread and butter of our lab is inventing and building new kinds of microscopes.

>> Diego (narration): But in paradox to the molecular world they set out to map, the microscopes they build start out quite large.

>> Dr. Shroff: What you're looking at in here are arrays of different lenses and mirrors and those things are lasers. This is sort of the guts of a microscope.

>> Diego (interview): This is one microscope?

>> Dr. Shroff: This is one microscope, yeah.

>> Diego (narration): In a room wall-papered with scribbled formulas and sketched schematics, Dr. Shroff showed me a microscope in-the-making—and it’s not what you might think. Laid out across a silver-topped conference table, there’s an assortment of shiny trinkets and gadgets that appear to be strategically arranged. It kind of looks like a very high-tech game of Mousetrap.

>> Dr. Shroff: We build our microscopes from these Tinkertoy-like parts. So, this is not unlike Lego, right, where you buy these kind of components and then we can do things like; we can mount lenses on posts and things. So, we typically buy these component parts from third-party vendors. And then, we assemble them in sort of unique ways. So, you know, this isn’t your garden variety microscope.

>> Diego (narration): It sure isn’t. When most people think of a microscope, I assume they flash back to what they had in their high school biology classroom —you know, the portable type that are only about the size of a shoebox. I’m pretty sure if you tried to fit one of Dr. Shroff’s microscopes in a classroom, there would be no room for the students.

So what’s so special about these larger-than-life microscopes? What does Dr. Shroff use them to see? And how is he hacking them to push the boundaries of what they can image?

For answers, I visited Dr. Shroff in his lab.

[TRANSITION MUSIC]

>> Diego (narration): To start, it’s important to mention that Dr. Shroff largely deals in fluorescence microscopy. That’s when scientists physically tag specific parts of a cell with fluorescent molecular markers. When stimulated with ultraviolet light that is not visible to the human eye, these markers emit light that we can see, often in in brilliant blues, vibrant reds and neon greens. They basically bring the parts of interest into technicolor while everything else stays dark.

This technique allows scientist to pick out whatever is fluorescently tagged and observe how it develops and behaves. But even if the thing you’re looking at has its own flare signal so to speak, it doesn’t mean much if it’s blurry or in low resolution. So, to really see biology at play, the trick is to get images in high-def. And if you’re ambitious enough, maybe even 3D.

>> Dr. Shroff: Resolution in microscopy is sort of about isolating the signal from smaller and smaller regions—not just in X, Y but also in Z. You know, you’re used to thinking about 3D imaging. Most of the effort on improving microscopes has been done like in the plane of the image, right? But there's that third dimension which is super important.

>> Diego (narration): You can think of the Z dimension as the depth. I kind of imagine it like fishing on a lake with a drone—only the drone doesn’t just scan for fish that might be near the surface, it can actually see into the water. And as you can imagine, being able to see at a greater depths lets scientists gather more insight about what’s really going on inside cells.

>> Dr. Shroff: So, this is Xuesong, a postdoc in my lab. Xuesong built a system that improves the resolution compared to a conventional microscope about a factor of five in the Z-dimension

So, what are we looking at here, Xuesong?

>> Xuesong: So, here basically is a mitochondria ultra-membrane.

>> Diego(narration): To the untrained eye, the image almost resembles the view of a city from an airplane window on a very clear night. You can easily make out the individual mitochondria glowing like cars stuck on busy a highway. I’ve included a link to Xuesong’s paper on our website so you can see for yourself.

>> Xuesong: In general, it's a live cell, we're going to use very low laser power to avoid the phototoxicity and the photo lesion.

>> Dr. Shroff: To keep it alive.

>> Xuesong: To keep it alive.

>> Dr. Shroff: The more resolution you need, the harder it is to keep it alive. So, lower the laser is the healthier the cell is. But the lower the laser is the less signal you get. And so, there's this sort of tradeoff, right, between signal to noise and keeping the cell alive. And Xeusong’s microscope is sort of unique in the resolution it can produce for how long it can produce it.

>> Diego (narration): There are other microscopy techniques like electron microscopy that can see in finer detail but those only capture a snapshot in time because the sample has to be frozen still. A big advantage here is that you can study how live cells behave relatively undisturbed and in real-time… and now in higher resolution.

>> Xuesong: Right, right. You can see the difference here. The actual resolution compared to this one, not only the background but also the actual resolution is dramatically increased.

>>Diego (interview): Yeah.

>> Dr. Shroff: You can actually see the mitochondria, not just in X-Y but in Z sort of cross-section of it.

>> Xuesong: Yeah, this is the actual view.

>> Dr. Shroff: We call this isotropic imaging because the image looks equally good no matter how you look at it, right? So that's the main selling point of what he's done. And that's essential, right? If you're looking at organelles, in this case mitochondria inside the cell, you want to be able to see them clearly in all dimensions, right? Biology is sort of 3D. And so, you know, you can see static shots, but in this case, he also did like what we call 4D imaging, so volumes over time.

>>Diego (interview): Oh wait, so you could actually like make a movie out of that?

>> Dr. Shroff: Yes. You have that movie actually?

>> Xuesong: I have the movie.

>> Dr. Shroff: He’ll show you the movie.

>> Diego (narration): Xuesong brought up a similar airplane-window view, only this time the mitochondria sprung to life, as if the roadblock had been cleared from the fluorescent highway.

>> Dr. Shroff: Yeah. So, there's like sort of the shape of the cell. And then you can see the individual shapes of each of these mitochondria, these organelles as they're moving around.

>> Diego (interview): What scale is this?

>> Dr. Shroff: So, this should be maybe a few microns and then inside here would be like a micron.

>> Diego (interview): And the micron is?

>> Dr. Shroff: Micron is a millionth of a meter. I don't know, the width of a human hair on the order of a micron, 1000 times bigger than this, right? So, it's small. It's sometimes easy to forget how amazing this is, right? That we're actually zooming in to a single cell and looking inside it. With high enough resolution you can see not just the organelles but you can see like what's inside the organelles.

>>Diego (interview): Yeah, that's crazy.

>> Dr. Shroff: It’s crazy.

>> Diego (interview): So how does it work?

>> Dr. Shroff: Ok, so we shape the light from the laser into a pattern of light. And that pattern of light illuminates the sample and elicits fluorescence and then the fluorescence get collected back along the same way—that’s the actual signal emitted by the sample. That gets siphoned off into what we call the processing arm of the microscope to form an image.

>> Diego (narration): On the way from the laser to the sample and then back up to the detectors that render the image, the light ping pongs off of lenses and mirrors based on a bunch of fancy optical physic. Calibrating all the parts to align and work the way they should is a delicate process, that’s why new microscopes start on such large scale.

>> Dr. Shroff: Oftentimes when you start building a microscope, it's big. We need this whole optical table to kind of figure out how to do it. And so, we use all of the real estate to make sure that we do a good job. We're not thinking about kind of making everything small in the first iteration. We're thinking about making it work. But one of the advantages of this setup is that we can hack it to make it better because we know what everything does. If we want to modify something, we can do that. So, getting back to the lab, Tim is a guest from California,

>> Diego (interview): Hi Tim.

>> Tim: Hi there.

>> Dr. Shroff: He's a professor. He actually invented a concept for a new kind of microscopy and he's building it here. So, this is sort of a good example of like, I guess you could call it research and action where Tim had an idea in his mind, he basically then conceptualized it, you know, on paper and in a computer. And then, he actually came here and built it. And so, it's not quite done yet but it's, you know, it's getting close to the point where we can put a sample on and actually image it. He's doing some alignment right now. But you can sort of see, right, like the sort of homebuilt components on the optical table, some light source. You can see some blue laser light there.

>> Tim: It’s really expensive Lego. That’s what this stuff is.

>> Diego: Yeah (laughter). That’s a good way to put it. So this is kind of the beta, the test---

>> Dr. Shroff: Exactly, yeah. So, this is real research in the sense that like we have to figure out how to do it, the principle. Then we have to actually build it. And then if we do a good job, you know, maybe or maybe not it gets commercialized. And then that's sort of the second stage, right, where you can make it more compact or user-friendly.

To give you a concrete example, if we look at some of the rooms here, you’ll actually see microscopes that look like they’re more boxed. They’re more kind of commercialized. This is called a dual-view light sheet microscope where we use one lens to kind of illuminate the sample with a sheet of light. And the other objective collects the fluorescence from that sheet. The advantage is that in like delicate samples, you don't want to illuminate anything more than what you're looking at. And since your camera looks at a plane, you want to illuminate with a plane of light. And so, for instance our C. elegans embryos are very delicate. And so, we use this light sheet microscope to look at, you know, delicate samples. But, everything on that table that you saw has been condensed, into this little box.

>> Diego (interview): That’s crazy. This is what, like two feet by three feet?

>> Dr. Shroff: At most yeah, it’s very compact. You know, the concept was useful and so it got commercialized. So, NIH holds the patent. And in fact, I wanted to show you this facility; This is a trans-NIH facility called the Advanced Imaging and Microscopy Resource that I and a bunch of other PIs founded. And it's adjacent to my lab. The intention of this Trans-NIH resource was to try and make some of these microscopes more accessible to other folks here at the NIH.

We're very fortunate to be surrounded by very target-rich environment of biologists. We're a little bit the minority. You know, the tinkerers, the tool builders, and the engineers, but we're surrounded by many people that need our tools badly. And it sort of keeps us inspired and on our toes to be surrounded by these folks that need the better tools.

>>Diego (interview): Yeah, and you’re side by side—literally, the clinical center is right outside the window. So, what do people come here to look at?

>> Dr. Shroff: All kinds of stuff. So, they look at heart in disease models of mice. They look at organoids. They look at organs from different samples, typically mouse but also other, you know, we've had people look at drosophila in this thing, like drosophila larvae for instance. So,

this is a particular light sheet microscope that we use for looking at cleared tissue which is becoming very in demand here at NIH and elsewhere. So, the idea is you might have a section of a mouse or something, right, where you want to look where all the cells are inside that section of mouse tissue. But, some tissues are too big for the light to penetrate. And so, the idea is that you can shave off slices of the tissue and just image the slices slice by slice to look at really big. And so, the biologists would come mount the fluorescent sample on this microscope and then scan it through the light sheet at high speed, to kind of reconstruct a tissue volume or something.

>> Diego (interview): Yeah, so it's like a cross-section of one sample.

>> Dr. Shroff: Exactly.

>> Diego: Like a 3D version.

>> Dr. Shroff: Yeah, exactly. You're building up a 3D image of the sample by imaging it in cross-sections.

>> Diego (narration): Up to this point, you’ve heard a lot about all the mechanics of super resolution imaging. How microscopes are conceptualized, the tricks to building them, how they work and what they can reveal once they’re up a running, but the hardware is only half the story.

>> Dr. Shroff: There's the whole image analysis side. And that’s the really interesting stuff, which is like, how do you actually extract meaning from the images, right? So the biologists might want to know, where is a protein or where is a mitochondria, or a nucleus inside that data. And historically, that's been a very manual affair, right? It takes some user, you know, looking at the data maybe with a computer and trying to identify things.

>> Diego (narration): But with the advent of deep learning, Dr. Shroff hopes to teach the computer to identify things in the data by itself.

>> Dr. Shroff: One way to think about these deep learning networks are as basically extremely fancy, pattern recognition devices. So there's something that repeats or is regular. And that's the thing that you're training the computer to pull out. So how do you train it? Well, you have to give it examples of the pattern you're trying to find in the data. In the beginning, the computer has no idea, but as you give it things that look like sort of these blobby shapes, these nuclear-like shapes, eventually it starts to form sort of a mathematical sense as to what is a nucleus in this data. And so, you know, we call that training the neural network. And by neural network, I just mean a bunch of complicated mathematical relationships between the data and the thing you want from the data. And so, the deep learning part comes in just feeding it lots and lots of examples of the input and the output. Given enough examples, this mathematical architecture can learn to automatically find a pattern. And so that pattern could be pulling out the signal from the noise, right, at a very abstract level or it could be at a more biological level of finding the nucleus in an electron microscopy data set. And it's not just object recognition, like where's the nucleus, it might be like, how does like the gene activity in a microscopy image series change over time? And so that is where I think a lot of the sort of fun computation comes in, is like once you have the data for the microscope, what does it all mean and how do you -- how does that help you construct a model of the biological process you're studying?

>> Diego (interview): Yeah. Well now that you said fun, I’ve kind off gathered that everything you’ve shown me has a lot of room for play. Like in trying to drive progress and innovate on what already exists, there’s a lot to play around with and try things both on the mechanics and computational sides, and in a way that’s the essence of science right.

>> Dr. Shroff: Yeah, definitely. I mean, you know, the creativity, in some ways, is tied to the play, right? I mean, there is this element of play, I think, in almost any scientific endeavor because you're trying to push the boundaries of knowledge further. A lot of times, you get stuck if you just try things that have worked. And so the sort of art and play of it is figuring out what to tweak in a smart way to kind of push the boundary. That's often where the new science is, right, the new discovery. And I think that’s one of the most practical and best things about the intramural program; the freedom to try something out. Funding is sustained here, and that is just a huge benefit. And reviews are retrospective, so you don’t always have to say that you know what you’re going to do—maybe you don’t know what you’re going to do to. You can explore and I think that a key advantage of being here at the NIH. So there is this element of play.

>> Diego(interview): Well, there's also like a sense of beauty when you look at these things. I mean, it kind of looks like you're staring at one of those like majestic satellite images of like distant galaxies in a way.

>> Dr. Shroff: Absolutely.

>> Diego (interview): It’s kind of like the opp—not the opposite, but opposite in scale, at least.

>> Dr. Shroff: Yeah.

>> Diego (interview): You're looking at something really, really, really small, that I think like inspires wonder and curiosity. So do you see it like that, kind of like art?

>> Dr. Shroff: Absolutely, yeah. In fact, I mean, sometimes I get so used to seeing those images that it’s not until I talk to someone like you, right, that I'm reminded of like the visceral beauty in these recordings, right? It really is kind of like a galaxy of information writ small, you know, and at that sort of microscopic level. And, you know, one the interesting things that I think about sometimes is that even within a single cell, this sort of beauty and the curse of microscopy is you just never see it all. You know, like there is a whole world in there. And in our case, we're labeling parts of that world with fluorescent dyes, right? So we only see part of the story at any time, but the image is life, right? I mean, sort of unfolding in the microscope before your eyes.

>> Diego (interview): Well, to draw like, I guess, a further space parallel; so the James Webb Telescope was launched last year as a new frontier in space tech. Is there something in the horizon for like microscopy that you're really excited about or that you're doing, I guess?

>> Dr. Shroff: I think like -- you know, this has been like an amazing few decades for microscopy with the advent of like a Nobel Prize for super-resolution microscopy, seeing really small stuff. I think, we're sort of seeing a similar renaissance with the application of deep learning, right, to the raw data. So that's like something that I'm very excited about. And I'm excited about in both in terms of pushing past the optical physics boundaries of microscopes. But I'm also excited to like better understand the limitations of these techniques. You may have heard of bias in AI when looking at humans, right?

>> Diego (interview): Yes.

>> Dr. Shroff: But I think there's also the potential for bias even in looking at, you know, images of cells, right? Like, I mean, if the network is exposed to something it's not trained on, how do we interpret that in sort of a reasonable way?

There's a whole side to this field that is – you know, it's kind of like the wild west right now. People are just trying things. And so that is also kind of exciting, right? It'll be exciting to see how we'll sort of get there at the end. We'll transform this from being kind of a – you know, throw something on the wall and see what sticks to like a more prospective kind of science.

>> Diego (interview): Yeah, yeah, yeah. Well, kind of shifting gears a little bit to like your personal journey to NIH—and as the last kind of space reference—you often hear this origin story of astronomers looking up at the night sky and wondering what's out there and that being like the impetus, I guess, for their career track. Is there something like that for you, instead of looking up, where you look down and you were like, "I wonder what's really, really going on down there?"

>> Dr. Shroff: Yeah. I mean, with respect to like the inspiration for looking down, I can sort of trace a lot of that back to this course that I took in Woods Hole in Massachusetts, where I was taking this physiology course, and it just struck me how much of discovery at the microscopic level is determined by what you can see or not see, right? And so if you can make a better microscope, you really have the potential for seeing something nobody has seen before. And that's kind of a very inspiring thought. I mean, I have the sense that like some of the tools we build, they're not just useful at NIH. They could be useful all over the world, right, to look and see some biology. And having a piece of that, even at the very beginning, is kind of inspiring, right? I mean, it's something that I think about.

>> Diego (interview): One hundred percent. Well thank you so much for the insightful tour.

>> Dr. Shroff: Yeah, thank you. Thanks for taking the time.