Immersing Data into Virtual Reality

Anoushka Handa spoke to Alex Spark and Alex Kitching, the co-founders of a nanotech company, called LUME, to demonstrate their new use of Virtual Reality to view super-resolution micoscopy data...

Anoushka - I met up with the founders of LUME, a VR company that's based in London to have a look at the human T-cell that Ed and I had imaged using a VR headset. LUME is a nano imaging software company, specifically working on super resolution microscopy and the visualization of 3D super resolution microscopy data. But what is virtual reality?

Alex S - VR is where you're in a completely virtual environment. And you interact with everything virtually. You have two screens in front of your eyes, which gives you 3D stereoscopic vision and projected onto that as a completely digital environment.

Alex K - Instead of playing a video game in the traditional sense where you have a controller and you're looking at your character through a TV screen, you are actually in the video game. So when you look right your character is looking right.

Anoushka - Can I step into this human T-cell that was imaged for me earlier this week,

Alex S - Step right this way,

Anoushka - Right, I'm gonna put the headset on. That's on, where are the controllers. Okay. I think I've got them. So what do I have to do first?

Alex S - One controller goes in your left hand, the other one, your right. They have some tactile thumb feedback and triggers on the back. So it's sort of like an Xbox controller, but split in half.

Anoushka - I'll load the cell up. I'm guessing, menu. And then this cell?

Alex S - Yeah. Load data. And then yeah, click that cell there.

Anoushka - Oh, wow. That was really quick. That took like a couple of seconds to get up. And there's about a million localizations in this one. What can I do in this zone now?

Alex S - First thing to always do is just move around it, check it out. We've done a lot of work to make sure there's absolutely no motion sickness you'll feel with VR. So you have complete smooth movement.

Anoushka - I'm gonna try and walk through this cell. This is incredible. I can see the fingers like Ed was telling me these microvilli that are coming out the cell and it looks absolutely beautiful.

Alex S - So you can grab, move the cell around, rotate your way around it. You can color the cell by any dimension you want, not just X, Y, and Z dimensions, but also intensity or frames. And you can also get into the more analytical parts and you can select, for instance, the fingers of the data. And then you can go into the scripts and then run the certain analytical scripts on those areas that you selected.

Anoushka——我会和捕获这一个finger. And how easy is that? Because if I'm doing that on a screen, it's relatively difficult to get a single finger.

Alex S - Why don't you have a try?

Anoushka -奥卡河y. Right. I'm gonna navigate my way on this menu and see if I can figure this out. I think I've got it. Cool. Okay. And now can I analyze just the isolated data?

Alex S - Yes. And it'll be super fast as well.

Anoushka - There's so much I can do with this. There's visualization, segmentation, exploration and analysis. Those are the four. So I can basically hold this cell in the palm of my hand as well as stand well in the midst of it.

Alex S - And I think what's interesting is because you're in a virtual reality environment, you can make this data as small or as large as you please.

Anoushka - That's wonderful. So you can literally just throw around the cell that we imaged... Bit dangerous.

Alex S - Yeah you don't want to throw around valuable data.

Anoushka - How did you guys get into using VR as a visualization technique?

Alex K - We met with Professor Steven Lee at the science museum where he was presenting their work in super resolution microscopy. We saw their work, and thought it was quite underwhelming to just look at these massive three-dimensional point cloud structures of cells, just using posters. We thought we could actually help with our VR experience to bring those to life a little bit.

Anoushka - What can it show that a screen can't?

Alex S - If you have three dimensional data, then looking on a 2d screen where you can end up with an obscurance so you could have data overlaying data, so you can't see behind it. Whereas in VR you'd be able to see that because you'll have two slightly different perspectives and you'd be able to move around it so you can get a rotation around it so that you could perceive its three dimensional nature.

Anoushka - How can we use virtual reality to our scientific advantage?

Alex K - I think you can use it in two ways. In our case, we used it very much to try and reveal hidden insights within the data. So by analyzing it and using the VR as a medium to really dig deeper into the data. But I think another approach is also to bring people closer together, to collaborate around and over this incredibly complex data. And I think there's a plethora of different applications from education collaboration, and just VR is a great medium for, or those connections to happen in a digital way.

Anoushka - I found it pretty amazing to try the VR kit on and to walk around this incredible cell, but I'm always a little skeptical of VR. Is it just a buzzword or people actually become immersed into the concept?

Alex S - I think there are definitely some people on the warpath to VR. I mean famously Mark Zuckerberg, he did a press release internally to his company in 2015 that said we're gonna completely transition the whole company and everything's gonna be in virtual reality by say 2025.

Alex K - The world that maybe is portrayed by Mark Zuckerberg of like wearing the VR the time. I don't think we're quite just there yet.

Anoushka - But Apple are now bringing out a VR headset later this year I think. Do you think that that will have a big selling point for the general public?

Alex K - It's the first step towards widespread adoption. It's going to be great for us and everyone who already has a presence in that space.

Julia - Okay. I get the boasting now that is incredibly cool.

Anoushka - I know!

Julia - What were those four things that you could do?

Anoushka - So the four things were visualization. So you can look at the data, you can walk around the data, you can have a look around it and share that experience with someone else. Exploration, which is again, walking in the cell, playing around with the cell, making it bigger, making it smaller, in the palm of your hand, throwing it around. Not that much though. Segmentation, so you are cropping, maybe specific fingers that Ed had mentioned, from the cell membrane and then finally analysis, where you are analyzing those fingers, the isolated data, or the whole cell as well. And you can do that with a various number of scripts.

Julia - And you were obviously standing still in like a static image of a cell. Do you think in the future, there could be a way that we could be in a cell that's sort of live and in action and moving and we can look at the cell as it's functioning.

Anoushka - Well, possibly. I think what we can do with the super resolution data, is we can load the cell up in time. So you can see these fluorescent dyes that have come on in time. So I think Alex and Alex, from LUME had both mentioned that there's different dimensions in which you can load the cell up in X, Y, Z time, intensity and that's for people to play around with.

Julia - Nice. And, what specimen would you like to walk in next? You've done the T-cell. What would be next on your list?

Anoushka - I think there's a number of things, bacteria would be great to have a look at neurons, but I think first of all, I'd love to be able to walk around in some brain tissue. I think that would be really powerful, especially for lots of neurological diseases that are affecting a lot of the population. So Alzheimer's, Parkinson's and I think it'd be really exciting to know a little bit more about how we function.

Julia - Yeah, definitely. And especially people who have genetic conditions where that affects proteins on cells and you can maybe tag them and see what's exactly different in the brain and be fully immersed and that, I think that's amazing, but for you, how did it feel to be fully immersed?

Anoushka - To be honest, it was an amazing experience and also really exciting for understanding the future, of our bodies and how we work. Just like Leeuwenhoek did, but in even finer detail.