Genes in 3-D

Visual artist Andy Lomas used Unreal - a virtual reality gaming engine usually used to make life-like shoot-em-ups and other games - to turn all this scientific data into 3-dimensional virtual reality. But how did he actually start doing it? Kat Arney found out.

Andy - That’s a good question. I was wondering the same myself when I started this project. So we got all these files, just genuine files and simulations that Csilla has been running. I haven't used Unreal that much but I just thought, okay, this game engine technology, it should be able to read in this data. It should be able to write a code that can directly read the simulation files and what can we then do with it and if we put it in that environment, we can hopefully look at things in different ways, try things out and basically, we’ve almost got like an environment to experiment with things.

So, I was learning to be honest, doing it. I thought it should work. I wanted to have something that interactive. I guess most of the work I've done in the past has been in film and visual effects where you really know what you're doing – you’ve got a storyboard, you know exactly what the target is, and you're often doing things that take very, very long amount of time.

I wanted to have something that allowed us to try sort things out really quickly which is why we wanted to try to use this game technology. So it’s a very big experiment and we wrote all the codes which just reads these files, reads them into memory, and builds the geometry based out of the data.

Kat - So, this isn’t like you’ve got a string of DNA and you're animating it to do what you want. You're actually taking real data from the experiments and just throwing it into this computer engine and seeing what it throws back at you?

Andy - Yeah, pretty much. It’s real data from Csilla’s research work on trying to reconstruct the shape of the nucleus, the DNA, and the chromosomes. So it’s actually real files from her which amount from the simulations she’s running which is three-dimensional data. It’s like positions of various locations on the chromosomes and positions over time and space. So it is already three-dimensional but I was having to turn that into visible geometry and also animated geometry.

我想当我们来到研究所和罗ok at their work, what we they were showing us was the final shapes which is what most researchers are really trying to reconstruct. What we’re trying to explore in this work was, how were they getting to those shapes? How were they reconstructing this geometry? Actually, I don’t think before they'd actually looked at the data during the simulation, they were always looking at things at the end of the simulation.

Kat - It’s sort of, how did we get here?

Andy - Exactly, yeah – the sort of the process, as if it was almost like musical because they're doing these simulations that go through different phases. So just visually, it’s interesting. I mean, it just tells a story of almost like a discovery. You're going from something which is just this perfect mathematical, but almost this arbitrary structure to start with, because we really don’t know what the real shapes of the nucleus and the chromosomes are.

And then through all these forces, these constraints – things pulling things towards each other which need to be close to each other by the end – and just simulating with those literally, you heat it up to a ridiculous temperature so everything moves around with all this thermal noise and then gradually cools down. As it cools down, it solidifies hopefully into the right shape and so, there is beauty to that. You'll see it as a work in different movements.

Kat -是一个不可思议的事情,因为当n you look at the video or indeed in the virtual reality simulation, you see this big column of almost like strings stacked up in a spiral, and then it starts to go crazy – it’s dancing all over the place and then slows down and solidifies into a sort of a ‘fluffy blob’, I guess would be the best way of putting it. How did you decide to turn that into a virtual reality environment where people could actually step inside and explore?

Andy - It was almost just because we could. I mean, our big thing with this was, “Let’s try this out in Unreal so we can experiment with it quickly, so that we can actually try things out, so we can look at what's in the data in a three-dimensional space, interactively.”

Unreal just has the opportunity to do things in virtual reality because it’s one of the two main game engine pieces of software you use to create to top end games including virtual reality games. So, it just has a VR preview mode and I have a virtual reality headset which I've been playing with and trying things out with and we just tried it out. Just like, “Okay, it should work. We should be able to hit VR preview and see.” Just how much you spatially understand the structure by being in VR actually really surprised me, even though I've done quite a number of things in VR before.

And then we worked it and turned this into a more interactive thing thinking like, how would you want to maybe visualise things, understand things, look at chromosome colours, which can influence colours off, look at activation zones? It was like, what would I want to explore in this set? But the first thing was, we could just hit that button, VR preview and have a look at it, and we looked at it and okay, that’s great!

Kat - That’s cool! As an ex-scientist whose field was this kind of thing, I was really fascinated by going from the straight line of DNA to the writhing library of genes and how they're all interacting, thinking about it in a three-dimensional way. When I was a scientist, there weren’t these kind of tools and what you’ve done here is an art project. But could we develop this kind of tools for scientists? I think it would be incredible to sit inside my data and virtually pick it apart, see it in space around me?

Andy - I think absolutely, yeah. I mean, we have started talking with people at Babraham Institute exactly about that. First, wouldn’t this be great to take much further as an art and science project for science fairs and things like that? And also, for all the genuine research – if you're actually generating structures and particularly interesting questions like, where are the locations of activation? Is the location really on the interphases between the different chromosomes and things like that?

Actually, spatially understanding the structure when it’s really complex – it’s this fluffy, really complex shape. It’s a very, very difficult structure to understand by just looking at like a picture on a computer screen. If you see it in VR, it feels like it is there in the room with you. It gives you just intuitively so much deeper understanding of that structure. I think particularly, it’s those really complex forms like these folded chromosomes where that’s really important.

凯特的course, the sci-fi nerd in me loves the idea. If you see the films, the scientists kind of go, 'bzzz' and the data appears like hovering over this table. Are we going to get there? Are we going to get to be able to probe our scientific data in three dimensions? I’d love that.

Andy - I guess so. I guess one thing is because we are genuinely using the real science data, it does take about 2 minutes to start up at the beginning as it loads all the data files. So, big data takes quite a long time and game technology, and these graphics cards in computers now have incredible amount of power to visualise really, really complex difficult spatial datasets. So it’s almost like you're riding on top of the people playing first person shooters to help you do real science.

Kat - Andy Lomas, and the piece you heard earlier was Chromos, by Max Cooper - the title track of his new EP inspired by the project, which isavailable now on iTunes. There are also a couple of amazing videos on Youtube, combining Max’s music and Andy’s visuals, so head to the podcast page at nakedcientists.com/genetics to find the links or search online for Max Cooper Chromos.
Videos:
https://www.youtube.com/watch?v=oyxwoLHFv_U
https://www.youtube.com/watch?v=gq7YwOIVW1Y