Stereoscopic 3D Workshop 05.01.01 CRAIG LEES RAVENSBOURNE MA BROADCAST FUTURES Stereoscopic 3D production Difference between 3D and S3D Stereoscopic 3D is based on the illusion of recreating Human binocular Vision by using two sets of images captured with slight positional disparities to create perceived depth. This effect is known as Stereopsis. 3D describes real or computer-generated three-dimensional objects and is also the term more often used with CGI, 3D modelling and VFX. Natural Depth Cues Our brain interprets depth through a number of differing two-dimensional depth cues such as occlusion. When objects are covered by other objects, our brain registers them as being nearer/further away to us. Other natural depth cues include relative size of objects, saturation (objects’ colours desaturate the further away they are). Depth Perception Convergence/ Divergence Parallax – Negative and Positive The business case for 3D With the success of Avatar at the Box Office and the Sky Pub 3D experiment, 3D movies are noticeably attractive as ticket prices are elevated and 3D in the home naturally has extra revenues for income for manufacturers.
Transcript
Stereoscopic 3D Workshop
05.01.01
CRAIG LEES RAVENSBOURNE MA BROADCAST FUTURES
Stereoscopic 3D production Difference between 3D and S3D Stereoscopic 3D is based on the illusion of recreating Human binocular Vision by using two sets of images captured with slight positional disparities to create perceived depth. This effect is known as Stereopsis. 3D describes real or computer-generated three-dimensional objects and is also the term more often used with CGI, 3D modelling and VFX. Natural Depth Cues Our brain interprets depth through a number of differing two-dimensional depth cues such as occlusion. When objects are covered by other objects, our brain registers them as being nearer/further away to us. Other natural depth cues include relative size of objects, saturation (objects’ colours desaturate the further away they are). Depth Perception Convergence/ Divergence Parallax – Negative and Positive The business case for 3D With the success of Avatar at the Box Office and the Sky Pub 3D experiment, 3D movies are noticeably attractive as ticket prices are elevated and 3D in the home naturally has extra revenues for income for manufacturers.
Aside from monies from 3D film ticket sales at the cinema, there are a plethora of 3DTV channels set to hit our screens over the next year.
3D storytelling S3D has a great opportunity to produce different creative tools. Not only can we make things appear come out of the screen and into the screen, we can also make objects appear smaller or greater than what they are (dwarfism and giantism).
Stereoscopic Technology
3D or not 3D? As of yet there is NO universal standard or format. Public adoption of 3DTV is an unknown entity. The screens are relatively expensive with many around the sum of £2000. Examples of differing formats:
Circular polarising Passive technology
Active Shutter Technology
Glasses-free Autostereoscopic Displays using Lenticular or parallax barrier technology from Toshiba, expected to be released at Christmas
Stereoscopic 3D laptop
Live 3D – The engineering challenge of getting it right in real-time
Telegenic UK Outside Broadcast provider commissioned by Sky 3D to deliver 3D broadcast using Sony cameras and 3ality systems Digital technology innovation has made Live 3D possible. Sony System: SIP, P1 / HDC 1500 camera systems, Passive 3DTV monitoring
3ality Digital systems
TS-2 Studio Beamsplitter Rig
Motorised Lens system
TS-2 Studio Beamsplitter Rig on Ped
Sony HD CAM SRW-1 Deck – records both video feeds onto one multiplexed format
3D Village and Video Village
3ality 3D monitoring on a 2D screen using luma difference 2% parallax grid
3ality 3D monitoring metadata
3ality 3D monitoring alignment and depth budget
Fusion Pace Rig – Engineered by Vince Pace. This system was used on Avatar, The Final Destination plus recently used on the upcoming TR2N 3D.
James Cameron on Pace Fusion 3D rig
Pace On Location
Arri Alexa Cameras mounted on Pace Fusion 3D Segway Rig on Martin Scorcese’s The Invention of Hugo Cabret at Sheperton Studio
The Sky 3D Broadcast Model
Sky 3D Pub Football
The Stereoscopic encode format is Side by Side compressed within a 1080i25 frame To enable the 3D program to retain the highest quality throughout, a minimum of 90% must be native 3D footage.
Stereoscopic Alignment These guidelines are for the final Program content being displayed on screen sizes in the range of 46" to 70":
Main subject point should nominally be the screen focus point or convergence point of the two images
Positive disparity or image separation at distant points (into the screen) should not exceed 2% for majority of shots
Negative disparity Image separation at close points (Out of Screen) should not exceed 1% for shots. Record format should be at a bit rate no less than 100Mbit/s 4:2:2 for acquisition. S3D Cinematography Theory The Stereo Window – think about shooting within a window to avoid edge violations. Monitor BOTH cameras. Calculating 3D depth for a scene – the variables of screen size Converged cameras Vs Parallel cameras – weigh up the pros and cons for what is best for your shoot Pre-visualisation – weigh up the best solution for your production – DOES NOT HAVE TO BE EXPENSIVE! Controlling depth – The Depth Bracket and depth script. How much depth do you want in the scene to express your storytelling? What should be the Inter axial distance: Hyper stereo, Ortho-Stereo or Hypo stereo? Pre-production is VERY important in Stereo 3D. The effect of screen size upon the 3D effect – What is your final output? This impacts on whether you should employ 1/30th / 1/100th laws for stereoscopic cinematography.
The importance of Metadata – When using VFX, it is very important that you record stereoscopic camera data for your VFX artist to match effects. What is the IA? What is the distance to nearest/furthest distances? What is the focal length? What is the F-stop? Twice the number of cameras is required for a native 3D stereoscopic 3D camera system. This results in TWICE THE DATA. Zoom Lens problems: telecentricity errors, also known as zoom wander. The effect of different focal lengths. Wider lenses create a greater 3D effect whereas longer lenses tend to flatten the 3D effect. Lighting. Reflections, high contrast imagery shadows and specular highlights often create visual artefacts in stereo 3D The importance of using shutter-sync cameras Cameras with Genlock sync
Figure 1, R Cam 1 frame before clap L Cam 1 frame before clap
Figure 2, R Cam at clap L Cam at clap
Cameras with Temporal sync (shutters NOT synced)
Figure 3 L Cam 1 frame before clap R Cam 1 frame before the clap
Figure 3, L Cam at noise of clap R Cam at noise of clap
Fast moving objects in temporally synced cameras can cause Retinal Rivalry artefacts, as we are seeing different images between our left eye and our right eye, which can cause discomfort. The greater the size of the screen, the more retinal rivalry is amplified.
S3D Pre-Production
Depth Budget and Depth Script Each scene will have a designated depth budget associated with it for a stereographer to work with on set. This budget will be created using a variety of factors including feel, continuity, and broadcast/cinema specifications. A depth script will be constructed in the post-production process and are likely to have decisions regarding Inter-axial and convergence settings. Note that though errors in convergence can be easily corrected in post, altering the inter-axial distance is very difficult. To date, only highly expensive systems such as Mistika
and Occula are capable of changing inter-axial distance in a scene, furthermore, the large computational data required by this interaxial operation means that these advanced programs cannot render these actions in real time.
The depth script is constructed in the form of a graph that tracks the filmmakers emotional intended impact of each scene. These depth decisions can be created
in the script with signifying notations such as; C-5 (lenses strongly converged so that the action very much happens in the viewer’s lap – lots of negative parallax); C-0 (placing key subjects on the screen plane); and C+5 with moderately converged lenses pulling the viewer into the screen space.
As well as convergence decisions, another major component of the depth budget is the required interaxial distances that each scene should contain, which again is very much part of the pre-production process. Due to inter-axial distance being the single major influence over perceived 3D depth, attention should be noted upon interaxial distances and that they do not jump from one extreme to the other. Such extremes produce uncomfortable depth cuts upon the viewer. Also, interaxial distances have an effect upon scale. This sense of scale, due to exaggerated inter-axial distances, brings filmmakers the opportunity to enhance the emotional impact of their stories (Alice in Wonderland, where Alice grows taller and shrinks being a great example). Narrow inter-axial distances invoke gigantism effects (making the subject appear larger than what they are). Wider interaxial distances cause miniaturisation effects (making subjects appear smaller than what they are). These scale decisions can be notated in the script much in the same way convergence notations do such as I-5; I-0; I+5 , where I-0 indicates orthostereo (human interpupillary distance – 65mm or thereabouts), I-5 indicates the interaxial where the lenses are narrowly separated (hypostereo) and I+5 where the lenses are widely separated (hyperstereo). The depth budget follows the decisions made within the depth script and is maintained by live 3d pre-visualisation measuring tools such as the 3ality Stereo Image Processor (SIP 2100) and the Sony MPE-200 SIP, or by simply using a grid overlay on the multiplexed stereoscopic imagery.
Things to consider 3D uses double the bandwidth of usual video editing as two video tracks are used to create every S3D shot. Spec your machine for a faster stereoscopic pipeline. Depth Cuts Floating Windows. Slower cuts for shots with lots of perceived 3D depth Multiple S3D viewing and output modes are very useful Organise your media.
Products Cineform (Mac & Windows) Neo 3D around £2000 and Neo HD (under £400) Stereo 3D Toolbox (Mac) Final Cut Pro Plugin created by Tim Dashwood (around £800) Avid Nitris around £10K Mistika around £100K Quantel Pablo 3D around £100K Smoke (Mac & Linux) around £10K After Effects CS5 scripting and expressions Premiere CS5 – now has stereo functions and can handle 4K Red footage (with Nvidia Quadro Graphics Card) – VERY EXCITING! Shows how powerful non-linear edit systems (NLE’s) are becoming.
DIY 3D - Stereoscopic 3D on a budget Build your own rig
There are benefits in shooting parallel Stereoscopic player – available from 3tv.at Dual firewire Neo Player – Cineform’s Player allows you to play the Cineform S3D Codecs on your machine using a 3D display. Adapt your existing HD kit Vizard – Build your own 3D theatre – Wired magazine article Zalman Monitor – Full colour Passive S3D under £300 Anaglyph monitoring – 3D monitoring on a 2D screen (VERY cost effective) NVidia 3D vision solution iZ3D – stereoscopic 3D gaming on a PC
Stereo 3D Toolbox by Tim Dashwood Today we will learn the basics of the FCP plugin Stereo 3D Toolbox. Why? It is VERY stable and has lots of fine extensive stereo tools, which can produce everything you need for your S3D productions. Exercise One Making a Stereopair Sync your clips together for your Left Eye Clip and Right Eye Clip. Make sure they have the same start and finish points. If they don’t, trim them.
Once you have ensured they are temporally synced and are the same duration, double click each clip and then Make a Subclip (Command U for you hot key freaks out there – like me!). This will make a subclip in your browser.
Do the same for your Right Eye temporally synced clip. NOTE – Why do we have to make subclips?: Well, FCP recognises start media and end media only in your video clips. It is very difficult to get two cameras two record at EXACTLY the same moment of time. Making subclips is the fail-safe way of ensuring your clips start and end at the same point in time. Setting In and Out points alone does not work! Once you have made a Left and Right Subclip, which you can see in your browser, we can now make our stereopair.
NOTE – your clips will be named depending on how your media is captured. When using tapeless, make sure you use Log and transfer to name your clips - TRUST ME - this will save you lots of pain later!!
Then, drag your Left eye clip into your timeline. Once there, double-click your Left eye clip, then go to Effects – Video Filters – Dashwood Stereo 3D Toolbox (NOT the LE lite edition) – Stereo 3D clip coupler.
Click the Filter Tab. On the S3D Input, Select Clip= Left eye & Image Well = Right Eye.
This will enable your Left Eye Clip to be the Master Clip and it will create an Image Well that you can drag and drop your Right Eye Subclip into to create your stereopair. Keep the S3D Output Mode as Side by Side at this stage. If you have your Left and Right Eyes the wrong way round, You can select the Swap Output tab, but getting it right first time is VERY GOOD PRACTICE!
Once you have created your stereopair using Clip Coupler, you then want to geometrically align your clips in order to produce the desired stereoscopic effect. To do this we double click our clip – Effects – Video Filters – Stereo 3D Toolbox – Stereo 3D Geometry.
Click on the filters tab for the stereopair clip, if not already, to reveal the Stereo 3D Geometry Controls.
Once the Stereo 3D Geometry Controls Tab is up, we can move the images around to create the aspired stereoscopic effect. This is the fun part as this is where the S3D comes alive. Depending on how you have shot your stereoscopic material, whether it be parallel or converged, the amount of applied geometrical alignment will change. If the material has been shot using the parallel method, we will need to apply what is called Horizontal Image Translation (HIT) to the two images, which basically involves sliding the two images
horizontally over each other until they slightly overlap to the point of convergence that best suits the material.
Anaglyph Stereo pair with no HIT applied
Anaglyph Stereo pair with HIT applied
To adjust HIT, use the Convergence Slider in the Stereo 3D Geometry panel and change the S3D Output mode to the appropriate 3D viewing mode you are using, this may be interlaced for passive screens such as the Zalman, side by side for 3DTV’s such as the JVC or anaglyph if you are using a standard 2D screen. Use the Convergence % Course slider first to get as close to the required convergence point possible, then use Convergence % Fine to make finely tuned, more accurate alignment adjustments. If you have vertical misalignments, use twirl down the Disparity Correction triangle to release the vertical alignment controls.
Other misalignment errors that you may wish to fix include Rotation, Zoom, Keystone (these are all quite common), can all be fixed here in the Stereo 3D Geometry controls.
How do I measure my 3D effect? There is a great tool in the Video Generator tab which can be accessed by selecting - Dashwood Stereo 3D Toolbox – Stereo 3D Parallax guides.
These Parallax Guides are REALLY useful as a means to measure how much depth you have coming OUT of the screen (Negative Parallax) and back into the screen (Positive Parallax) within each scene. And this is where good old anaglyph monitoring is really useful. TIP – Negative Parallax (coming out of the screen), the disparity ‘fringing’ will be GREEN on the left.
Positive Parallax (going back into the screen), the disparity fringing will be RED on the left. If it’s not, you have got your eyes the WRONG WAY ROUND! Don’t stress though, this is a common mistake when first starting out!
You can set the guides to a matter of percentage, which is good, but make sure you are monitoring on a screen the same size of your final output screen. The Sky 3D Guidelines are 1% negative parallax and 2% positive parallax, which is a pretty good rule to stick to though you may want to push this further when displaying on smaller screens below 50”.
To select parallax guide controls, click the controls tab, then select custom parallax %, then change the slider (or type in the value) to 1% custom parallax.
Then drag this video generator onto your timeline above your stereopair.
Now you can measure your 3D! Measure the fringing disparity of the object closest to the camera to measure your maximum negative parallax (furthest out of the screen object which has green fringing on the left), and measure your maximum positive parallax (the furthest behind the screen object which has red fringing on the left). What is the measured 3D Depth in this scene?
Yes you can change the colour of the parallax guides – Very useful!
Expanded view of the open window. The yellow marks show how the
Ma ximum p ositiv e p a ra lla x/d isp a rity 0.8 %
Ma ximum ne ga tiv e p a ra lla x/d isp a rity 1%,
Based on: IAD 20mm, focal length 7.6mm, convergence point actress
(2m), closest point shaven-headed actor's shoulder (1.6m) , furthest point
tree-line above house outside (50m)
……………………………………………………………………………………..........
Ok, that is the end of this introduction to the Stereo3D Toolbox plugin and Stereoscopic fundamentals. There’s more than enough here to keep you going for some time in your beginning adventures with S3D. There are plenty of other useful tools within the plugin such as creating 2D text in 3D space and Displacement Maps but for now concentrate on these tools to build your stereoscopic stories. And questions or help required, email me at [email protected] and I will try to answer your questions as quickly as possible. Thanks and good luck new 3D movie-makers! Craig Lees Ravensbourne MA Broadcast Futures Further research material Bernard Mendiburu’s 3D Movie Making from Script to Screen.
Lenny Lipton’s classic “Foundations of the StereoScopic Cinema. A Study in Depth.”
