Jump to content

Welcome to FutureTimeline.forum
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!
Photo

Volumetric VR - Future of Cinema

volumetric virtual reality volumetric vr volumetric virtual reality video recording intel studios holograms

  • Please log in to reply
12 replies to this topic

#1
quantumdoc

quantumdoc

    Member

  • Members
  • PipPip
  • 37 posts
  • LocationUnited States

Has anyone heard or seen the recording in what is known as volumetric VR? The best example of what is to come for the cinematic experience I believe is what Intel is doing. They have created "Intel Studios" where they constructed a brand new and revolutionary studio and method of video capture of which allows the viewer to experience performance from any angle, not just in 2D.

 

Imagine watching a movie with VR goggles and being able to navigate around the actors from any angle as they act their parts. Imagine watching a "movie" and as you walk around and past a scene, you notice another scene occurring in the distance only seen by walking around. Volumetric VR recording is just that. It is a new immersive method of storytelling and film making of capturing action of whats inside a volume of cameras, not just recording as a flat image. 

 

 

Anyone have any experience or knowledge of this? If so I'd love to hear about it!

 

 

 


"what we observe is not nature itself, but nature exposed to our method of questioning" WH


#2
wjfox

wjfox

    Administrator

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPip
  • 12,912 posts
  • LocationLondon

Sounds like an early precursor to holodecks.



#3
starspawn0

starspawn0

    Member

  • Members
  • PipPipPipPipPipPipPipPip
  • 2,170 posts

Well, there are various groups that have worked on light field displays that cover an entire wall, and produce "holographic" images a lot better than ordinary holograms:

 

https://uploadvr.com...eld-lab-urbach/

 

Holograms work by interference of light; but light field displays are even more amazing, as they give you the full spectrum of light in every direction, and every point on the screen -- so are, literally, like looking through a giant window onto another world.  You can also induce the displays to have objects protrude out of the screen.

 

As far as the "recording" and "rendering" aspects, the future here will probably involve the use of deep learning.  With deep learning, you can produce models that take in the image from a small number of different angles (so, not many cameras needed; maybe fewer than 10 will suffice), and then reconstruct it from any angle and any position.  At the moment, it's slow, requiring too many computations -- but give it about 2 or 3 years, and it will run in better than real time:

 

http://www.matthewtancik.com/nerf

 

You could apply this technology to next-gen Google Maps data, and then could walk around in VR in any place in the U.S., as though you were actually there.  

 

Addendum:  I forgot to say that you can probably already at least do the viewing of the scene in real time or almost real time.  It's just that the training phase takes a very long time.  In principle, you ought to be able to take as input a movie, filmed at 10 different locations, and construct a giant, multi-billion parameter (or even trillion) neural net, so that if you input (x,y,z,phi, theta, psi, t) coordinates (6df + time = 7 df), it will paint that scene at that place, time and orientation, and do it in real-time or near real-time (maybe using a lot of computing hardware!).  You might even be able to change the lighting and other aspects.  Training such a model would take a very long time; but running it would work pretty fast, if you have enough hardware.



#4
quantumdoc

quantumdoc

    Member

  • Members
  • PipPip
  • 37 posts
  • LocationUnited States

Well, there are various groups that have worked on light field displays that cover an entire wall, and produce "holographic" images a lot better than ordinary holograms:

 

https://uploadvr.com...eld-lab-urbach/

 

Holograms work by interference of light; but light field displays are even more amazing, as they give you the full spectrum of light in every direction, and every point on the screen -- so are, literally, like looking through a giant window onto another world.  You can also induce the displays to have objects protrude out of the screen.

 

As far as the "recording" and "rendering" aspects, the future here will probably involve the use of deep learning.  With deep learning, you can produce models that take in the image from a small number of different angles (so, not many cameras needed; maybe fewer than 10 will suffice), and then reconstruct it from any angle and any position.  At the moment, it's slow, requiring too many computations -- but give it about 2 or 3 years, and it will run in better than real time:

 

http://www.matthewtancik.com/nerf

 

You could apply this technology to next-gen Google Maps data, and then could walk around in VR in any place in the U.S., as though you were actually there.  

 

Addendum:  I forgot to say that you can probably already at least do the viewing of the scene in real time or almost real time.  It's just that the training phase takes a very long time.  In principle, you ought to be able to take as input a movie, filmed at 10 different locations, and construct a giant, multi-billion parameter (or even trillion) neural net, so that if you input (x,y,z,phi, theta, psi, t) coordinates (6df + time = 7 df), it will paint that scene at that place, time and orientation, and do it in real-time or near real-time (maybe using a lot of computing hardware!).  You might even be able to change the lighting and other aspects.  Training such a model would take a very long time; but running it would work pretty fast, if you have enough hardware.

 

 

yes, and with this it seems to be able to create almost a new from of communication. if everyone had some sort of portable handheld recording device capable of recording you, then the deep learning software will compute and fuse/render the real-time scene and people will actually be able to "travel" in VR. 


"what we observe is not nature itself, but nature exposed to our method of questioning" WH


#5
Kynareth

Kynareth

    Member

  • Members
  • PipPipPipPip
  • 198 posts

It won't become popular with live-action movies. Too much hassle to make one. More popularity will be achieved with computer generated stuff (hopefully not necessarily photorealism oriented like the new Lion King, because it was rather boring, lacked emotion). I think it's going to be one of many forms of entertainment (and also learning) in the future (in 25 years): VR gaming, monitor/TV gaming, 2D movies, 3D movies, volumetric movies, volumetric photography (using small drones?). It took about 20 years for 3D animated movies to become very popular after Toy Story appeared and there is not even one full feature volumetric VR movie.



#6
Erowind

Erowind

    Anarchist without an adjective

  • Members
  • PipPipPipPipPipPipPip
  • 1,563 posts
The current issue too is wearing these cumbersome low res headsets for long periods of time. We're probably looking for 16k full FOV displays at 240hz+ in a wireless headset around the weight of eyeglasses before the the tech will really hit home. That sounds out there, but in a decade or two it shouldn't be too hard to pull off. Valve Index is cool, but it's an IBM 5150 in terms of timescale here. The Oculas DK1 is an Altair 8800.

#7
Poncho_Peanatus

Poncho_Peanatus

    Member

  • Members
  • PipPip
  • 20 posts

Has anyone heard or seen the recording in what is known as volumetric VR? The best example of what is to come for the cinematic experience I believe is what Intel is doing. They have created "Intel Studios" where they constructed a brand new and revolutionary studio and method of video capture of which allows the viewer to experience performance from any angle, not just in 2D.

 

Imagine watching a movie with VR goggles and being able to navigate around the actors from any angle as they act their parts. Imagine watching a "movie" and as you walk around and past a scene, you notice another scene occurring in the distance only seen by walking around. Volumetric VR recording is just that. It is a new immersive method of storytelling and film making of capturing action of whats inside a volume of cameras, not just recording as a flat image. 

 

 

Anyone have any experience or knowledge of this? If so I'd love to hear about it!

 

 

I was actually thinking about this the other day, wanted to make a small movie for my little bro. I used four cameras all placed around. But I could not get the navigate between people and obstacle. Well, this was just the first attempt.....more research are needed.



#8
Poncho_Peanatus

Poncho_Peanatus

    Member

  • Members
  • PipPip
  • 20 posts

The current issue too is wearing these cumbersome low res headsets for long periods of time. We're probably looking for 16k full FOV displays at 240hz+ in a wireless headset around the weight of eyeglasses before the the tech will really hit home. That sounds out there, but in a decade or two it shouldn't be too hard to pull off. Valve Index is cool, but it's an IBM 5150 in terms of timescale here. The Oculas DK1 is an Altair 8800.

 

True, as much I love VR (I almost only use VR now, even for study) the technology is not there yet. to heavy, clumsy and low res. More investment are needed, maybe with the help of Darpa we could cut some time to a Hi-res vr headset?

 

I also like compare it to consolles and gaming tools of bygone days. VR games are getting more and more complex than just a vr tech demo. Kinda like 70's games like pong or break out vs 80's videogames like Mario and pacman. Hopefully we soon move to 90's games.



#9
Kynareth

Kynareth

    Member

  • Members
  • PipPipPipPip
  • 198 posts

Half-Life: Alyx feels to me like a NES or GBC game in terms of advancement, let's say Contra or Double Dragon. It's good enough to be entertaining. Many less advanced VR games are rather boring compared with some high quality traditional games. To move to SNES or GBA level we need much better hardware and then software. It'll be a long ride. Recently hardware innovations slowed down, I hope they speed up.



#10
quantumdoc

quantumdoc

    Member

  • Members
  • PipPip
  • 37 posts
  • LocationUnited States

Completely agree regarding the need for full view 16K lightweight glasses before the tech really takes off. I think also, it'll take just one or two highly applicable AR apps integrated into current society of which everyone will feel the need to use before the push, money and time, will be driven to speed up the AR wearable tech innovation. I do think this will be in the next 5-10 years. Good point about Toy Story, but it indeed pave the way for how all kids movies are made today. So I think a headset similar to a nice pair of glasses is inevitable, and absolutely will replace the cell phone. With this tech, we will be able to make 'AR calls' live, and fully interact, walk around and 'hang out' in an AR space with anyone we wish. We will not need to hold a clunky 2D device to our ears. We will simply be able to have a pair of glasses on and fully interact in 3D with our 'AR calls', hands free and integrated within our normal full field of view. Further down the road, evidenced by patent filings I've seen, companies will evolve this into contacts, instead of glasses, then even further, implantable Intraocular Lens Implants, fully upgradable wirelessly. Bionic !


"what we observe is not nature itself, but nature exposed to our method of questioning" WH


#11
quantumdoc

quantumdoc

    Member

  • Members
  • PipPip
  • 37 posts
  • LocationUnited States

 

Has anyone heard or seen the recording in what is known as volumetric VR? The best example of what is to come for the cinematic experience I believe is what Intel is doing. They have created "Intel Studios" where they constructed a brand new and revolutionary studio and method of video capture of which allows the viewer to experience performance from any angle, not just in 2D.

 

Imagine watching a movie with VR goggles and being able to navigate around the actors from any angle as they act their parts. Imagine watching a "movie" and as you walk around and past a scene, you notice another scene occurring in the distance only seen by walking around. Volumetric VR recording is just that. It is a new immersive method of storytelling and film making of capturing action of whats inside a volume of cameras, not just recording as a flat image. 

 

 

Anyone have any experience or knowledge of this? If so I'd love to hear about it!

 

 

I was actually thinking about this the other day, wanted to make a small movie for my little bro. I used four cameras all placed around. But I could not get the navigate between people and obstacle. Well, this was just the first attempt.....more research are needed.

 

What cameras did you use ? There are several ways to record volumetric video, extremely expensive ways using many cameras, and less expensive ways, albeit, still pretty expensive. Microsoft Kinect Azure DK is one of the better ones. You can use 1-4 cameras here with some software and create some cool holograms.


"what we observe is not nature itself, but nature exposed to our method of questioning" WH


#12
quantumdoc

quantumdoc

    Member

  • Members
  • PipPip
  • 37 posts
  • LocationUnited States

It won't become popular with live-action movies. Too much hassle to make one. More popularity will be achieved with computer generated stuff (hopefully not necessarily photorealism oriented like the new Lion King, because it was rather boring, lacked emotion). I think it's going to be one of many forms of entertainment (and also learning) in the future (in 25 years): VR gaming, monitor/TV gaming, 2D movies, 3D movies, volumetric movies, volumetric photography (using small drones?). It took about 20 years for 3D animated movies to become very popular after Toy Story appeared and there is not even one full feature volumetric VR movie.

 

 

I disagree here. I think it will become popular with live-action movies, in particular live anything such as musical performances, opera, anything on stage. Intel studios was the first to invest in a 10,000 square foot studio to do just that, make live-action movies. I don't know how to add links here but youtube intel studios and watch how they made a small movie clip and re-created Grease. This is just the beginning and its revolutionary. The viewer can now go 'inside' the movie and walk around every character and literally choose the angle and view. The viewer becomes part of the movie. I see though that it looks like you do agree that in 25 years it will become popular. I do agree that it will be a long while before it will replace regular 2D movie experiences. I think it will roll out like iMax and 3D did, as an option. The wearable, view-finder needs to have a better full field of view as well. The small drone method you mentioned seems really cool as well, it each drone , around 60-100 could be outfitted with 8K video recording tech, battery life of 8 hours of full flying and recording time, and ability to be fully synchronized together, and wireless data transfer or live streaming. I don't think all of that will come before the first full length volumetric movie however. 


"what we observe is not nature itself, but nature exposed to our method of questioning" WH


#13
Kynareth

Kynareth

    Member

  • Members
  • PipPipPipPip
  • 198 posts

 

I disagree here. I think it will become popular with live-action movies, in particular live anything such as musical performances, opera, anything on stage.

 

 

By live-action movies I meant full-length movies you go to cinema for, not performances you watch on YouTube. Yes, orchestra may be volumetrically fully captured in the future, true. I am not sure if people even want movies like that, I think most just want to sit and see what the directors wanted to show. It's also an artistic expression to show only what the viewer needs to see, not everything around.







Also tagged with one or more of these keywords: volumetric, virtual reality, volumetric vr, volumetric virtual reality, video recording, intel studios, holograms

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users