Post on 24-Mar-2018
transcript
Image Generators and DisplaysStefan Seipel
Interactive Graphical Systems Fall 2002
Historical Overview
1960 First random scanned display available 1
1964 A few applications using computer graphics ($100.000) 100
1968 Phosphor storage tubes increasingly available ($5000) 1000
1978 Raster scanned displays become increasingly popular 100.000
1984 First PC with graphical user interface (Macintosh) 1000.000
1994 Graphical user interfaces replaces increasingly text interface 50.000.000
1999 Almost no computer available without GUI 200.000.000
(Source: E.Bengtsson, Uppsala University)
Random scan display
• based on phosphor storage tube• cathode ray is controlled arbitrarily in XY direction• analog control of ray implemented in electronic circuits• sets of vector primitives available (points, lines )
• very smooth lines / high resolution• line drawing only• no filled surfaces• no block transfers• limited time frame to draw graphics • up to 100.000 short vectors per refresh cycle
Random scan display - computer control
• computer keeps a list of graphical primitives
• computer issues rendering and parameters
• very little memory required to store display list
• no management of graphics buffer required
• display/engine and computer strictly separated
Raster scan display
• based on TV technology - short glowing phosphors
• screen is scanned in a line-by-line order
• pixels within a line are switched on/off while line is scanned
• two-dimensional memory matrix (frame buffer) contains pixel value
• filled areas can be drawn
• screen must be refreshed in short intervals
• memory must be read out -> video signal must be generated
Raster scan display - computer control
• computer manipulates values in the frame buffer
• discrete frame buffer matrix -> rasterization
• arbitrary graphical objects can theoretically been displayed
• need for rasterization algorithms
• graphical primitives must be computed
• host computer calculates and accesses frame-buffer
Frame buffer readout - RAMDAC
host DAC(digital to analog converter)
Frame buffer RAM(random access memory)
memoryread
memorywrite
create videosignal
• readout of 1280x1024 pixels at 60 Hz -> memory access time 12.7 nanoseconds !
• “normal” RAM have an average access time of 60ns
• special purpose frame buffer required
• must allow for simultaneous read and write operations
Video Controller
Simple Raster Scan Display Architecture
CPU
SystemMemory
VideoMemory
VideoController
System Bus
other periphery
(frame buffer operations)
Monitor
More or less typical configuration with common graphics cards
Typical frame buffer operations:
2D - Graphics operations
• BitBlt (bit block transfer)
• Draw Pixel
• Draw Line
• Draw Text
• Fill Area
Frame buffer operationsare a heavy burden fora single CPU system !
2D graphics subsystem
CPU
SystemMemory
RasterEngine
VideoController
System Bus
other periphery
Monitor
Frame (color)Buffer
2D graphics subsystem
(2D graphical primitives)
Color buffer standards (common resolutions)
Spatial resolutions:CGA - 320x200EGA - 640x350VGA - 640x480SVGA - 800x600XGA - 1024x768SXGA - 1280x1024 or 1600x1280HDTV - 1900x1200ATC - 2000 x 2000Human eye - 10.000 x 10.000 (not evenly distributed)
Color resolutions:palletized - 256 colors lookup-table (8 bit/pixel)grayscale - 256 shades (8 bit /pixel)grayscale - 4096 shades (12 bit /pixel) e.g. in medical apps.Hi color - 5:5:5 RGB, or 5:6:5 RGB (15 or 16 bit/pixel)true color - 8:8:8 RGB (24 bit/pixel)true color - 8:8:8:8 RGBA (32 bit/pixel)
Memory requirements and signal bandwidths
Spatial Resolution SXGA 1280x1024Color Resolution True color 24 bitScreen Refresh 72 Hz
Video Memory Requirement : 1280*1024*24 bit = 3.75 Mbyte (1.3Mio pixel)
Video readout : 94 Mio./pixel per sec.
Memory readout : 226 Mio. bits per sec. !
-> very special memory readout required(e.g. 240 MHz RAMDAC)
3D graphics adds another dimension
2D color buffer only is not sufficient
The standard 3D graphics pipeline
Geometric processing- thousands of flops- vertex transform- normal transforms- lightning calculation
Rasterization- operates on pixels (framebuffer)- millions of iops- alpha compares- depth buffer test- stencil test- alpha arithmetic- texture addressing
DisplayList
Traversal
ModelingTransfor-mation
ViewingTransfor-mation
Clipping Projection Rasteri-zationLightning
Features of 3D image generators
Geometric processing (per vertex operations):
• one or several floating point geometry engines• perform matrix and vector operations
hardware supported graphics operations:• transformation of vertices• rotation of vectors• normalization of vectors (after scale of object)• calculation of lightning • projection & clipping
Features of 3D image generators
Rasterization (per pixel operations):One or several raster engines feature:
• discrete line drawing and polygon fill
• z-Buffer test (depth buffer test)
• z-Buffer blending (fog)
• blending with alpha-channel (transparency)
• color interpolation (e.g. Gouraud shading)
• texel addressing
• tri/bilinear interpolation of textures
• anti-aliasing of edges
• stencil test
Advanced 3D graphics subsystem
CPU
SystemMemory
System Bus
other periphery
Monitor
Graphics subsystem
GeometryProcessor(s)
RasterEngine(s)
VideoController
Frame Buffer(s)Other Buffers
(Stencil,Depth,Alpha…)
Local Memory(caches, textures)
Buffers configuration example (3D system)
red-color buffer 8 bitgreen-color buffer 8 bit
blue-color buffer 8 bitalpha buffer 8 bit
depth buffer 24 / 32 bit
accumulation buffer 8 b
stencil and/or overlay buffer 8 bit
double buffer(front and back)
Memory required per pixel: 2*24+8+8+32+8 = 128 bit = 16 byte
For a resolution of 1280x1024 -> 20 MB frame buffer
Buffers configuration example (3D stereo system)
red-color buffer 8 bitgreen-color buffer 8 bit
blue-color buffer 8 bitalpha buffer 8 bit
depth buffer 24 / 32 bit
accumulation buffer 8 bit
stencil and/or overlay buffer 8 bit
Memory required per pixel: 4*24+8+8+32+8 = 176 bit = 22 byte
For a resolution of 1280x1024 -> 27,5 MB frame buffer
left right
front
back
Quad-buffer
Depth-Buffer Aliasing
Fixed number of bitsin the z-Buffer limitsresolution of the scene depth
Example:16 bit depth buffer allows only65536 discrete steps in depth
Arithmetical rounding operationof floating point depth valuescauses ambiguous z-values !
Visual artifacts in rendering of objects which are very close toeach other(see picture)
Performance parameters – History (1999)
Pixel Fillrate:• refers to rasterization performance• number of shaded/textured/buffered pixels per second• common: 5-40 million/sec. (low and medium cost game accelerators)• quite good: 100 million/sec. (graphics workstations)• high end: 500-2000 million/sec. (e.g. SGI top of the line)
Geometry Performance:• refers to throughput of graphical primitives• number of 3D shaded triangles/sec. (of certain size e.g. 25 pixel)• number of 3D shaded lines/sec. (of certain length e.g. 10 pixels)• common: <0.5 million triangles/sec. (low cost game accelerators)• quite good: <1-2 million triangles/sec. (graphics workstations)• high end: <50 million/sec. (e.g. SGI Octane)
Nowadays (2002) performance doubled with ten even on PC hardwareThe gap between professional systems and consumer products is closing
Performance parameters - What do they say ?
Problem:• stated performance values are often only achieved under specific circumstances• often these values refer to the most simple rendering modes (no shading, no z-buffer)• performance is often achieved with native implementations• quality of driver implementation is essential• comparability is a tricky issue, since various new technical features
Therefore:• don´t trust vendor supplied specifications !• if you are lucky, you reach 10%-20% of the stated performance !• graphics sub-systems should be tried before bought !
• in the target system (depending largely on CPU configuration)• using your application (depending on typical graphics operations)
• Always check : is there driver support ? are there drivers at all ?
Bottlenecks in 3D graphics systems
Geometry bound systems- too many polygons in the scene- too complex lightning calculation- too complex model transformations
Fill bound systems- too big areas to fill (number of pixels per polygon)- too many large polygons- high overdraw ratio- too many textures in scene- too complex alpha arithmetic (blending, fog)
Depends on your application and graphics system
Example bottleneck evaluation
User requirements for a flight simulation:• terrain model 1000 textured polygons, will fill most of the screen background• airplane model 5000 polygons, average size 200 pixels• for quality reasons, rendering should appear in SXGA resolution• frame-rate >40/sec.
There is a graphics subsystem available:• 60 Mio. textured, lit, shaded pixels/sec.• 2.000.000 triangles/sec. (25 pixels)• cost: 30.000 SEK
Is it advisable to buy this graphics subsystem ?• (1280x1024+5000x200)x40 = 92.428.800 pixels/sec.
• (1000+5000)x2x40 = 480.000 triangles/sec
Fill bound!
API´s (application programmers interface)
Avoid to develop hardware oriented software
Minimize turnaround costs and time
Use well established and standardized 3D API´s
Use graphics accelerators which support those API´s- full functional support- optimized performance for these drivers
The best graphics hardware is worth nothing without appropriate API and driver support !
API´s continued
Different standard 3D API´s:
Many different available today. The most renowned are:
PHIGS and GKS (old DEC machines, not common any longer)
OpenGL (SGI, Microsoft, many others)
QuickDraw 3D (Apple)
Direct3D (Microsoft)
Glide (3Dfx)
Heidi (used by many CAD programs)
Quake (sort of standard for games)
Additional Reading
Roy S. Kalawsky: The Science of Virtual Reality and Virtual EnvironmentsAddison-Wesley Publishing Company, 1993, ISBN 0-201-63171-7
Perception: pages 50-59
Displays techniques: pages 98-107
DisplaysPhysiological Aspects
Spatial Retinal Resolution : 1´
Visual Field : approx. 200o, with 120o binocular overlap
Limits of depth perception from lateral disparity
Temporal Resolution : approx. 50 Hz, increasing with luminance
Visual Displays - Basic Technologies
• Cathode Ray Tubes
• Flat Panel Displays
• Electroluminiscence Displays
• LCD Displays
• Active Matrix TFT
• Light Valves
• Laser Scanners
• Micro Mirror Devices
Basic Technologies - Cathode Ray Tubes (CRT)
CathodeCathode
AccelerationAccelerationAnodeAnode
FocussingFocussing
xx--/y/y--DeflectionDeflection((elstatelstat // magnmagn.).)
Phosphor CoatingPhosphor Coating
Electron gun
Advantages:- high resolution- easy to control- reliable technology- low cost
Basic Technologies - Cathode Ray Tubes (CRT)
Operation modes of CRTs:
Random scan mode Raster scan mode
+ no aliasing, smoth lines+ transformation in hardware possible+ little memory required
- only line drawing possible- complex control hardware for the beam- flickers if scene becomes complex
Basic Technologies - Random Scan CRT
How it works:
Display primitives are stored in a display list(ellipses, circles, lines...)
Electron beam is controlled continuously betweenvertice -> analog line drawing
+ uses old TV technology (simple approach)+ filled and shaded surfaces are possible+ guaranteed screen refresh rate
- aliasing problems “staircase effect”- electronics required for raster memory readout - drawing algorithms more complex
Basic Technologies - Raster Scan CRT
How it works:
Display is rasterized into pixels which are stored in a two-dimensional memory array (raster memory)
Electron beam is traversing the screen line by linein a regular time frame and scheme
Content of the raster memory controls beam intensity
Basic Technologies - Raster Scan CRT
Display ControllerDisplay Controller
HostHostHost00000000011100000000000000100010000000001010100000010101000110000000000000000000010010001111100000000000000000000010010100001001000000000000101010001001111110000000000000000101100111100101100000000000000000000000101001010011
Frame buffer
VideoController
VideoController
Converts analog primitiveinto discrete representation Converts discrete frame buffer
into analog video signal
scan line
horizontretrace
verticaretrace
Basic Technologies - Cathode Ray Tubes (CRT)
The role of the phosphor coating :- Fluorescence (glowing when hit by electron beam)- Phosphorescence (after glowing while being activated)- Persistency (time until glowing phosphorescence decreases below 10%)typically 5-60 milliseconds.
- Persistency is important. Short persistency requires high update rates otherwiseflicker. Long persistency causes stabile but smeary images.
- Granularity of the phosphor -> spot size, image resolution
- Type of phosphor defines color:p1 : green, average persistencyp4 : white, short persistencyp12 : orange, average persistencyp31 : green, short persistency
Basic Technologies - Cathode Ray Tubes (CRT)
How is color accomplished in displays ?
Most usually, colors are mixed by additive composition of base colors
1. Spatially modulated color composition
see page 99, in Kalawsky
2. Temporally modulated color composition
see next slide
Basic Technologies - Cathode Ray Tubes (CRT)
Color shutter technology.
- Sequential display of color fields on monochromatic CRT- Synchronization of the fields with color filters
- Filters can be : a) LCD filters (electronically controlled)
b) Optical filters (mechanically coupled)
- Advantages : No color convergence errors
- Disadvantages : High frequent oscillations in the visual field decompose colors
How is color accomplished ?
Parameters for display assessment
- dot size (mm)- dot pitch (mm)- resolution (lp/mm)- brightness (cd/m2)- contrast ratio- display size- addressability- refresh rate- color range- convergence- weight / power consumption
1mm 1mm
1mm
1mm
dot size ?dot pitch?resolution?
Basic Technologies - Typical parameters for CRTs
Screen Diagonal Size (14”-26”)
Shadow masks (Triple holes, Strips)
Dot pitch (0.24 - 0.30 mm)
Video bandwidth (50 -250 MHz)
Horizontal Sync. Frequency (30 - 170 kHz)
Vertical Sync. Frequency ( 48 - 170 Hz)
max. Resolution (1280x1024 - 4800 x 4000)
Basic Technologies - Flat Panel Display - LCD
Liquid Crystal Displays
Light sourceor mirror
hor.pol.filter
vert. pol.filter
vertical electrodes
horizontaltransparentelectrodes
liquid crystallayer
No electric field = light passes through
Basic Technologies - Flat Panel Display - LCD
Addressing of Pixels:
Deposit an electronic charge on intersections between horizontaland vertical electrodes sequentially row by row.
Limited speed, since certain minimum time is required to is deposited (depending on the capacity of the intersections)
When last row has been addressed, first rows have already losttheir electric charge
Poor contrast image
Basic Technologies - Flat Panel Display
Thin Film Transistor Matrix (TFT):
An array of transparent transistors isdeposited on the LCD
Pixels can be switched on and off
Pixel keep their electrical state andoptical properties
=> Significant contrast and intensity enhancements
Basic Technologies - Flat Panel Display
Color reproduction / Optical Efficiency:
Groups of adjacent pixels are forming one effective color pixel
Sub-pixels are covered with color filters
Common sub-pixels configuration are RGB stripes, triads or quads
Efficient resolution is reduced
Light intensity is diminished significantly when passing throughpolarizes, liquid crystals, and color filters (poor optical efficiency)
(see also page 96)
Basic Technologies - Flat Panel Display
Electro Luminescence Displays (plasma panel displays):
S
D
t
Strike VoltageStrike Voltage
Discharge VoltageDischarge Voltage
Plasma generates light Plasma generates light
PotentialGas is encapsulated between electrodes
When a certain amount of voltage is applied(striking voltage) the plasma discharges andglows until the potential drops below thedischarge voltage.
Plasma cell keeps luminating for a whilewithout being refreshed.
Active luminance, high intensity display
Basic Technologies - Flat Panel Display
Electro Luminescence Displays (plasma panel displays):
glas
vertical transparent electrodes
horizontal transparent electrodes
glass
glass substrate with plasma cells (Neon, Argon)
SandwichSandwich--TechniqueTechnique
Basic Technologies - Light Valves
see page 103
Application : Projection Displays
High resolution light modulation (>1600 x 1200)
High refresh rates possible (>130 Hz)
Usable for high light output projection displays
The first choice for stereo projection systems
Tricky problems : Ghost images with slow phosphor
Basic Technologies - Laser Scanner
Used for:
large screen projection displays
direct retinal displays
(very high resolution)
for color displays, several lasersrequired (convergence ?)
Screen
horizontaldeflection
mirror
verticaldeflection
mirror
laser source
Basic Technologies - Micromirror Devices (MMD)
Matrix of micro mirrors
Addressable and electronically controllable
Used for Light Reflection and Projection Display Systems
Extremely high optical efficiency
Basic Technologies - Micro Mirror Devices (MMD)
MMD System Working Principle
Visual Displays - 3D Displays and Optical Systems
3D Projection Systems Front ProjectionRear / Retro Projection
Autostereoscopic DisplaysSlot MaskLenticular / Double Lenticular Arrays
Volume Displays
Other Optical Coupled Displays Fiber coupled displays (HMD see chapter 4.2.1)Lens/Mirror coupled displays (HMD see chapter 4.2.1)
3D Displays and Optical Systems - Projection Systems
3D Projection Display Systems require:
Very high intensity image source• Transmission LCD/TFT Panel + Light Source• Reflection LCD/TFT Panel + Light Source• Projection CRT (specialized high intensity CRT tube )• MMD • Light Valve• (Laser)
Focusing optics/color splitter• Wide / narrow angle optics, fixed or variable
Projection screen• Transparency / Diffusion / Specular Properties
Means of splitting left/right channel• Time Multiplexing / Color Separation / Polarization
3D Displays and Optical Systems - Projection Systems
3D Front Projection Systems:Image source and observer are located on the same side of the projection surface( - user may interfere with projection beam)
Screen
Image Source + Optics
Observer
Stereo 3D with active shutter glasses• requires very fast image source (>120 Hz)• light valve / projection CRT• extremely expensive• single graphics pipeline• screen with good diffuser properties
Stereo 3D with passive polarizing glasses• requires two image sources• image alignment problems • dual graphics pipeline required• requires special silver screen which
preserves polarization
Stereo 3D with color field separation (red/green)• one color capable projector (cheap)• poor image result• screen with good diffuser properties
3D Displays and Optical Systems - Projection Systems
3D Rear / Retro Projection Systems:
Image source is positioned behind the projection screen
No interference between user and image source
Requires transparent screen material
No polarized 3D stereo possible sincepolarization is disturbed in transmission
Screen
Image Source + Optics
Observer
Stereo 3D with active shutter glasses• requires very fast image source• light valve / projection crt• expensive• single graphics pipeline
Attention has to paid to mirror effects
3D Displays and Optical Systems - Projection Systems
3D Rear / Retro Projection Systems: Examples
Caves Virtual Planes Viewing wands
3D Displays / Optical Systems - Autostereoscopic Displays
Autostereoscopy - stereoscopic perception with the “naked” eye
Pixe
l Col
umn
R
Pixe
l Col
umn
R
Pixe
l Col
umn
R
Pixe
l Col
umn
L
Pixe
l Col
umn
L
Pixe
l Col
umn
L
Display surface
Image Splitter (Sanyo)
• Display divided in vertical stripes
• Alternate stripes display left and right image
• Slit-mask is blocking out the view of the left
eye onto the right picture and vice versa
• Only a single user
• Dedicated observer position
• Horizontal resolution decreased
Slit mask
3D Displays / Optical Systems - Autostereoscopic Displays
(Double) Lenticular Lens Arrays
• Display divided in vertical stripes
• Alternate stripes display left and right image
• Half-Cylinder shaped lenses project the
stripes to the corresponding eye
• Several viewing zones
• Dedicated observer distance
• Horizontal resolution decreased
LCD-Projektor LCD-Projektor
Double-lenticular retro-projection system
Lenticular flat panel display system
LCD/TFT Panel
3D Displays / Optical Systems - Autostereoscopic Displays
Examples (Heinrich-Hertz Institute, Berlin)
• Allows for user movements
• Uses head-tracking
• Screen is automatically positionedcorrectly with a robot arm
• Allows for user movements
• Uses head-tracking
• Lenticular array is shifted
3D Displays / Optical Systems - Volumetric Displays
The display creates a real volumetric representation which is perceived as a 3Dstructure without the need for glasses or other aids.
The idea is to project dynamic images onto oscillating or rotating surfaces in order to createthe sensation of a volumetric object.
Prototypes have been build using:- rotating LED matrices- rotating helical projection surfaces with laser projection laser- lasers projecting into fog- experiments are underway to bring a solid crystal to illumination on addressable positions
All these systems can only show transparent/monochrome objects
Mechanical problems and limits, dead viewing areas
Commercial systems are far ahead
3D Displays / Optical Systems - Volumetric Displays Choice of VR Displays - Evaluation of Requirements
How many observers are watching at the same time ?
What resolution and color fidelity requirements are there ? -> basic display technology
Is wide field of view desirable ?
Is immersion an important issue for the application ?
Is stereoscopic 3D rendering required ?
If yes, decide which type • one screen polarized -> take care for optical properties of the system• one screen time multiplexed -> display must tolerate high refresh rates• dual screen (HMD) -> check for resolution• autostereoscopic ?
Does the application require interaction with haptic stimuli ?
Displays Technologies - Features
C R T L C D T F T M M D L i g h tw a l vA d d r e ssa b i l i t y 4 k P i x e l 2 k P i x e l 2 k P i x e l 1 . 3 k P i x e l 4 k P ix e lC o n t r a s t h ig h lo w h ig h h ig h h ig hC o l o r s ve r y g o o d m e d iu m g o o d ve r y g o o d ve r y g o o dD im e n s i o n s h u g e s m a l l / m e d iu m s m a l l / m e d iu m s m a l l m e d iu mR e f r e sh < 1 8 0 H z 6 0 H z 6 0 H z 6 0 ( 1 8 0 H z ) 1 4 0 H zC o s ts l o w a ve r a g e h ig h h ig h ve r y h i g h
3D Displays and Optical Systems - Projection Systems
Considerations with regard to stereo image projection
Time-multiplexing with active shutters:• both front and retro projection possible• active glasses are quite expensive (if many are required)• very high speed projector is required (light valve technology, expensive)
Polarized filtered images:• projection screen must preserve polarization (aluminized silver screen)• retro projection not yet possible (no suitable screen material available)• glasses are very cheap• two projectors are required (can cause image alignment problems)
FULLY IMMERSIVE SPHERICAL PROJECTION SYSTEM(THE CYBERSPHERE)
http://www.ndirect.co.uk/~vr-systems/sphere1.htm
Contact:David Trayner or Edwina Orr.RealityVision Ltd. 6 Yorkton St. London E2 8NH, UK: +44 (0)171 7399700F: +44 (0)171 739 9707E: reality@augustin.demon.co.uk
RealityVision´s autostereoscopic displayusing holographic optical elements