Well, there are various groups that have worked on light field displays that cover an entire wall, and produce "holographic" images a lot better than ordinary holograms:
Holograms work by interference of light; but light field displays are even more amazing, as they give you the full spectrum of light in every direction, and every point on the screen -- so are, literally, like looking through a giant window onto another world. You can also induce the displays to have objects protrude out of the screen.
As far as the "recording" and "rendering" aspects, the future here will probably involve the use of deep learning. With deep learning, you can produce models that take in the image from a small number of different angles (so, not many cameras needed; maybe fewer than 10 will suffice), and then reconstruct it from any angle and any position. At the moment, it's slow, requiring too many computations -- but give it about 2 or 3 years, and it will run in better than real time:
You could apply this technology to next-gen Google Maps data, and then could walk around in VR in any place in the U.S., as though you were actually there.
Addendum: I forgot to say that you can probably already at least do the viewing of the scene in real time or almost real time. It's just that the training phase takes a very long time. In principle, you ought to be able to take as input a movie, filmed at 10 different locations, and construct a giant, multi-billion parameter (or even trillion) neural net, so that if you input (x,y,z,phi, theta, psi, t) coordinates (6df + time = 7 df), it will paint that scene at that place, time and orientation, and do it in real-time or near real-time (maybe using a lot of computing hardware!). You might even be able to change the lighting and other aspects. Training such a model would take a very long time; but running it would work pretty fast, if you have enough hardware.