Post on 22-Mar-2022
transcript
ITE Trans. on MTA Vol. 5, No. 3, pp. 78-87 (2017)
78
1. Introduction
Holography is a promising method to realize a three-
dimensional display and imaging 1) 2). Holograms directly
encode the light field by addressing both the amplitude
and phase information 3)-5). Recording and projecting
three-dimensional (3D) images using holography
technology provides natural and realistic 3D images,
unlike stereographic techniques which display two
different images simultaneously on the left and right
eyes exploiting binocular parallax of human eyes 6) 7).
In holography, it is crucial to precisely measure and
reconstruct the phase information of light fields. The
direct measurement of optical phase information is
currently impossible because the optical temporal
frequency is several orders faster than the electronics-
based device. Alternatively, conventional holography
methods exploit interference to record both the
amplitude and phase information of an optical field. A
signal beam of interest is combined with a well-defined
reference beam, resulting in the formation of
interference patterns which encode the light field
information. After the first demonstration by Gabor 4) 8),
various holographic techniques have been proposed to
record and reconstruct static optical fields 9)-11).
Recently, the developments of electronic devices such as
a charge-coupled device (CCD) or a spatial light modulator
(SLM) have enabled digital holography techniques 12)-14).
Since digital holography takes advantages in recording,
reading, and transferring of dynamic optical field
information, it has been extended into various fields such
as biomedical optics 15) 16), holographic microscopy 17)-19), 3D
display 20), data storage 21) and security 22).
Although digital holography for recording and
displaying of 3D optical information has potentially
interesting applications, its implementation has been
limited to mostly laboratory-level demonstrations
because of the requirement for a bulky interferometric
setup and a limited space-bandwidth product (SBP) of
SLMs in holographic displays.
In 3D holographic imaging technique, portability
would be crucial for practical applications. However, the
need for a reference beam in conventional interferometry
significantly limits the realization of a portable device.
In 3D holographic display, 3D optical fields are
generated by a spatial light modulator. The product of
the image size and the viewing angle is directly
proportional to the number of controllable optical modes,
or an SBP of an SLM 23). Since the current state-of-art
Abstract Digital holography has high potentials for future 3D imaging and display technology. Due to the
capability of recording and projecting realistic 3D images, holography has been extensively studied for decades.
However, the requirement of a reference beam in interferometric systems and a limited number of pixels in
existing spatial light modulators have been major obstacles for the practical applications of 3D holography
technology. Recently, the field of wavefront shaping, or the study of controlling multiple scattering of light, has
emerged with numerous interesting applications in digital holography. In this review, we introduce the
principles of multiple light scattering in complex media and highlight recent achievements to overcome the
limitation in conventional 3D holography by exploiting multiple light scattering. The complexity of multiple light
scattering, which had been regarded as a major barrier for conventional optical systems, can provide reference-
free 3D holographic imaging and 3D holographic display with several advantages.
Keywords: 3D holographic imaging, 3D holographic display, complex optics, multiple light scattering, wavefront shaping.
Received March 18, 2017; Revised May 20, 2017; Accepted June 7, 2017†Department of Physics, KAIST(Republic of Korea)
Review: 3D Holographic Imaging and Display ExploitingComplex Optics
Hyeonseung Yu†, YoonSeok Baek†, Jongchan Park†, SeungYoon Han†,
KyeoReh Lee† and YongKeun Park†
Copyright © 2017 by ITE Transactions on Media Technology and Applications (MTA)
technology is incapable of addressing a large number of
optical modes, the performances of 3D holographic
display techniques are still limited to a small size with a
narrow viewing angle range 24).
In order to overcome the limitations of such holographic
imaging and display technology, various methods have
been proposed. For example, diffractive optical elements
can enlarge the image size of a 3D holographic screen by
encoding a large-curvature lens 25). Holographic optical
elements 26) are also used for delivering images in head-
mount displays. Freeform optics 27) or deformable
reflecting surfaces 28) have been used to enhance
controllability of optical information as in the case of a
multifocal projection system. However, still much
advancement is required to project viewer-comfortable
3D images.
In this review, we introduce digital holographic
imaging and display methods exploiting multiple light
scattering. When coherent light transmits through
scattering media, speckle patterns are formed as a result
of interference of multiply scattered light paths. It seems
the multiple light scattering events are undesirable for
imaging and display applications because it results in
the loss of optical information. However, in multiply
scattered light, the optical information is scrambled
rather than lost 29) 30), and the events of multiple light
scattering are deterministic processes although it seems
stochastic. With appropriate characterizations of the
optical transfer property of a disordered medium, it can
be utilized as a powerful optical element, even for
holographic imaging and display.
2. Principles
Conventional optical components, such as a lens or a
mirror, offer great possibilities of controlling the light.
Incident and transmitted rays through a conventional
optical lens can be described by a simple linear matrix
[Fig. 1(a)], which can be readily adapted to an optical
imaging system. Despite their simplicity, there are
certain limitations of conventional optics. For example,
the numerical aperture of an imaging and a display
system is highly limited unless a high-magnification
objective lens is used.
Light scattering in scattering media exhibits
extremely complex behaviors, compared to the
conventional optics. However, the linearity and the
deterministic nature of propagation of optical fields are
still preserved in multiple light scattering [Fig. 1(b)].
Therefore, the light transmission process can also be
expressed in a linear matrix formalism, so called a
transmission matrix (TM) 30). The TM is an operator that
maps an incident (input) field to a transmitted (output)
field. Assuming the linearity of a system and the coherency
of a light source, this relationship is expressed as,
where Ein and Eout are an input and an output field,
respectively. When a TM is measured, the light can be
controlled as desired by exploiting the large degrees of
the freedom contained in scattering media.
Theoretically, this linear and deterministic nature of
multiple light scattering has been known for decades,
the experimental control of multiple scattering was
demonstrated very recently. Vellkoop et al. showed that
an optical focus could be made through a scattering
layer consist of TiO2 nanoparticles 31). Although this
study did not explicitly use TMs, it exploited the
deterministic and linear response properties of multiple
light scattering of coherent light. This pioneering work
has initiated various interesting experiments.
Complex optics enables access to and manipulate
optical information unless cannot be addressed with
conventional optics. For instance, the evanescent near-
field information of light can be coupled to propagating
E TEout in= ( )1
79
Paper, Invited Paper » Review: 3D Holographic Imaging and Display Exploiting Complex Optics
Fig.1 (a) (above) The photograph of an imaging system with a
conventional lens. The input USAF target image is clearly
projected through the lens. (lower) The image transfer is
well described by a simple linear relationship according to
the lens equation. (b) (upper) the photograph of an output
speckle pattern when the image transmits through
scattering media. (lower) the input image is highly distorted
due to complex scattering. The image transfer process is
still described by a deterministic linear relationship.
far-fields though scattering layer. This near-to-far field
conversion is described by a TM, which had been used for
focusing and imaging at the subwavelength scales 32) 33). A
scattering layer, whose TM was calibrated, was used to
extend the field of view with extended resolutions in
microscopic imaging 34).
2.1 The measurement of transmission matrices
TMs can solve technical challenges in holographic
imaging and display. First, a TM of a turbid layer can be
used so that the turbid layer can be exploited as a lens.
When an incident beam transmits through a scattering
layer, the output field exhibits complex speckle patterns
due to multiple scattering. Popoff et al. have
demonstrated that an incident field can be retrieved
from a field passing through a turbid layer, by using the
inverse of a TM of the layer 30). Mathematically, when an
output field Eout is measured, the input beam is
retrieved using the TM of a turbid layer as,
Alternatively, one can tailor output fields by actively
controlling the incident fields if the TM information of
an optical system is known 35). Given the desired output
field Eout, projection, the required input field is calculated by
Experimentally, the input field Eout, required can be
produced by using an SLM, such as a LCoS (Liquid
Crystal on Silicon) SLM, a digital micro-mirror device
(DMD), or a deformable mirror (DM).
In order to measure a TM of a turbid layer, essentially
two processes are required: modulating of incident fields
and recording of corresponding output fields. In the TM
formalism, the m-th column of the TM corresponds to
the output field response of the m-th basis of an incident
field [Fig. 2(a)], which is given as
Thus, by measuring the output fields corresponding to
each input basis field, the whole TM elements can be
experimentally obtained.
The basis of input fields can be arbitrarily chosen in
the spatial domain, the spatial frequency domain, or
independent patterns (such as Hadamard bases). In the
spatial domain, the input bases correspond to point
sources at different spatial locations 32). In the spatial
frequency domain, the input bases correspond to plane
waves with different propagation angles 36). Hadamard
basis is also widely used due to the simplicity of
encoding the basis fields using a digital modulator such
as a LCoS SLM or a DMD 37) 38).
For recording of a TM, output fields responding to
various input bases, complex amplitudes of output fields
must be measured. It should be noted that a TM
describes a field-field relation between the input and
output light. Thus, to fully calibrate a TM, typically
interferometric imaging systems, in which a well-defined
reference beam is used, have been utilized 36). However,
one can also measure a partial TM without using an
external reference beam 30) 39). The TMs measured in
these works only have a relative phase information
between different optical modes. Thus they are lack of
the information about a global phase value. The global
phase information is critical in imaging through a
scattering layer. However, for the projection of tailored
output images, the global phase information is not
necessary. This is because the transmitted light is fully
described by the relative phases between speckles.
2.2 Wavefront shaping techniques
In certain applications, one does not need to know
about the full TM information about a scattering layer.
For example, if focusing at several positions is sufficient
for certain applications, a partial information about a
TM is enough. Optical techniques for coherent control of
light transport in turbid media, without the need for full
characterizations of TMs, is also known as wavefront
shaping.
To generate an optical focus at the m-th position in
the output fields through a turbid media, the required
incident fields is given as,
Then the m-th element of output fields is calculated by
E e e eint t tm m mn=
′− − −arg( ) arg( ) arg( ) .1 2 … (( )5
E
E TE T
in m m m nm
out in m
,
,
.= ′
= =
δ δ δ1 2
0 0 1 0 0
…
… … ′
= ′t t tm m nm1 2
4
… .
( )
E T Ein required out projection, ,= −1 3. ( )
E T Ein imaging out, = −1 2( )
ITE Trans. on MTA Vol. 5, No. 3 (2017)
80
Fig.2 (a) Transmission matrix contains full optical information
about the field-field relationship between input and output
fields. (b) The wavefront shaping technique only utilize a
single row in the transmission matrix, corresponding to the
single output channel.
Therefore, the process for focusing a spot through a
turbid layer in wavefront shaping is identical to
characterize a specific row of the TM of the turbid layer
Fig. 2(b). This process could be performed through
various optimization procedures 31), instead of measuring
a whole TM.
An important parameter in wavefront shaping
techniques is the enhancement factor, defined as the
peak to background intensity ratio, after the
optimization. The enhancement factor EF is a function
of the number of controllable optical modes N in an
SLM. The equations for estimating EF in the field
modulation 40), the phase-only modulation 31), and the
binary amplitude modulation 41) are given as follows,
2.3 Practical consideration of transmission
matrices
Scattering layer exhibits different responses
depending on the wavelength and the polarizations of
light. This enables the use of complex optical elements
as a dynamic wave plate 42), a spectral filter 43), and the
control of spatiotemporal profile of light 44)-46). A TM has
different elements depending on the frequency and
polarization of the light. For reconstructing color images
through a turbid layer, a TM of the layer should be
measured for each wavelength. For color display
purposes, the input pattern corresponding to the desired
output pattern should be found separately for each color
channel.
The number of optical modes contained in scattering
media is also an important issue. If scattering media is
interpreted as a 2-D optical waveguide, the number of
optical modes is proportional to the incident area 47). For
this reason, acquiring a TM for large size scattering
materials requires long acquisition time and heavy
computation. Thus, it is important to appropriately
design a measurement system and measure a TM of the
system, considering the tradeoffs between the
information capability and the optimization load.
3. 3D camera exploiting complex optics
3.1 Conventional holography and its
shortcomings
In order to acquire 3D image information, the phase,
as well as the amplitude information of an image, should
be determined. Previously, several holographic imaging
techniques based on interferometry have been utilized in
order to record the wavefront of light 16) 48) 49). It
measures and analyzes intensity patterns generated by
the interference between a sample and a reference
beam.
The realization of an interferometry-based imaging
system is complicated, compared to a photographic
camera. Because coherence light must be divided into a
sample and a reference beam to construct an
interferometer, an optical instrument becomes large and
complicated, and precise and stable experimental
conditions are strongly required 50).
To overcome these limitations, simplified imaging
techniques which can access 3D information have been
proposed such as Shack-Hartmann wavefront sensors 51) 52),
imaging systems based on the transport-of-intensity
equation 53), ptychographic scanning 54) 55), and iterative
algorithms to extract phase information from measured
intensity patterns 56)-58). However, in order to achieve an
appropriate guess of the incident wavefront without
using a reference beam, these techniques require a
priori knowledge (or assumptions) on an object or
incident field has to be provided as a trade-off.
3.2 An optical diffuser as a holographic lens
Recently, Lee et al. proposed a 3D camera technique
using an optical diffuser as an effective solution to the
phase problem 59). In this method, diffused light through
a diffuser or speckle patterns paradoxically helps in
reconstructing holographic images (Fig. 3).
Light passing through an optical diffuser is scrambled
and produces a speckle pattern. In general, speckle is
known to deteriorate imaging quality 60). Because an
optical diffuser scrambles the input optical information
in a highly complex manner, an image cannot be simply
reconstructed from the diffused field. This is the reason
why an optical diffuser is often called as opaque glass.
As addressed in Section 2, if the TM of a diffuser is
known, the optical diffuser is a diffractive optics, similar to
conventional optics. The calibration of a TM is useful for
imaging systems which inevitably include diffusive media
such as scattering media 30) and multimode fibers 61) 62).
Indeed, in conventional imaging systems without
EFNNN
field= − +
− +
( / )( )( / )( )
,,π
π4 1 1
1 2 1 1pphasebinary,
. ( )7
E t t e eout m m mnt tm mn
,arg( ) arg( )=
′
− −1
1… … ′
= ∑ abs tmii( ).
( )6
81
Paper, Invited Paper » Review: 3D Holographic Imaging and Display Exploiting Complex Optics
diffusive obstructions, there is no reason to introduce a
diffuser and complicate the system, even if the TM of the
diffuser can be obtained or calibrated.
3.3 Speckle-correlation scattering matrix
method
For 3D imaging purpose, the use of an optical diffuser
took a principal role in solving the phase problem 59). In
the approach named 'speckle-correlation scattering
matrix (SSM),' the insertion of an optical diffuser in
front of a 2-D image sensor results in the formation of
highly scrambled speckle patterns at the sensor plane
(Fig. 4).
Exploiting the random nature of a speckle pattern, an
intriguing mathematical relation called Isserlis' theorem
(or Wick's theorem) was utilized to extract the phase
information directly. The Isserlis' theorem holds for
random variables, X1, X2, X3, and X4 as,
where ⟨·⟩ indicates the ensemble average. This relation
has been widely used in quantum field theory as a
mathematical tool.
Using Eq. (8), it was theoretically shown that the
incident field information could be extracted from a
single intensity measurement of a speckle pattern
without any additional requirement. Based on the
proposed theory, experimental demonstrations of a
reference-free holographic camera utilizing an optical
diffuser (Fig. 5) was also performed.
The important advantage of the SSM approach is that
it does not require for any assumption about a target or
incident light. However, the SSM technique still has
several limitations to overcome. First, dealing with the
TM of diffusive media is very challenging. Because
image quality is determined by the number of optical
modes, which is limited by the size of the TM, heavier
calculations and handling of enormous matrices are
required for high-resolution image reconstruction. In
Ref. 59), the actual implementation needed 12 GB of data
storage for about 4, 000 optical modes, and it took 1 (2
minutes to compute the input field. Fortunately, this
problem is a matter of computing ability of digital
devices, so that advancements in devices and computers
can resolve such problems. Another restriction of this
technique is the calibration of the TM, which is achieved
through very complicated measurements. However, this
difficulty also can be alleviated by carefully
manufacturing scattering objects. Generation of exact
replicas of a disordered layer would provide the same
TM. It means that the TM of one layer can also be used
for other replicas.
3.4 Discussion
The single-shot 3-D imaging using the SSM method,
promises a number of interesting future works. A typical
digital camera can function to obtain 3-D image
information by attaching a scattering object in front of
its image plane. As the SSM technique does not need
sophisticated apparatus, it can be applied to mobile
smartphone cameras. In addition, the frequency range of
light for 3-D reconstruction can be expanded to an X-ray
regime. This is a strong benefit because 3-D imaging
with a reference beam in an X-ray regime has extremely
limited due to the limitation on X-ray optics. As the SSM
method enables a generalized single-shot 3-D imaging
for any wavelengths, various new applications in X-ray
X X X X X X X X
X X X X X X X X
1 2 1
1 1
3 4 2 3 4
3 2 4 4 2 3
8=
+ +( )
ITE Trans. on MTA Vol. 5, No. 3 (2017)
82
Fig.3 The schematic of proposed idea: an optical diffuser as a
holographic lens. The incident field is converted into the
diffused field by the diffuser whose transmission matrix is
known. Exploiting the random nature of the diffused field,
the incident field and be reconstructed from the speckle
intensity snapshot captured by a camera. This figure is
modified from Reference 59).
Fig.4 The photograph of the first demonstrated holographic
camera using as optical diffuser (left). The optical diffuser is
directly installed in front of the camera using conventional
C-mount thread. The schematic shows the detailed
composition of system (right). The aperture and polarizer is
added to block the ambient light. This figure is modified
from Reference 59).
are anticipated in fields such as medicals or molecular-
level researches.
4. 3D display exploiting complex optics
4.1 Conventional 3D displays and its
shortcomings.
Recently 3D display has drawn significant interest
due to emerging technologies in virtual and augmented
reality. Most types of current commercial 3D displays
mainly employ binocular disparity of human eyes: by
projecting two different images on a viewer, they
produce the perception of 3D effects. This approach
exploits the limitation of the human visual system, but
only 2D projection is controlled in physical space.
A holographic display is an ultimate type of 3D
display because it exactly replicates 3D optical fields of
real objects. For projecting 3D images, SLMs are utilized
in general, in order to control the light fields.
If viewing conditions of 3D holographic displays are
analyzed with two parameters, viewing angle and image
size, then the current 3D holographic display only offer
3D images with a centimeter scale with a very narrow
viewing angle up to a few degrees 24). This limitation is
mainly due to the small number of pixels, or space-
bandwidth product of current SLMs 23). Using a
conventional optical lens, one can tune the image size
and viewing angle of 3D holograms generated with an
SLM, as shown in Fig. 6(a). However, as long as the
space-bandwidth product of the SLM is fixed, the
product between the image size and the viewing angle
remains constant regardless of the transfer lens.
To overcome this limitation, various multiplexing
methods have been proposed to enhance the number of
controlled pixels 23) 63)-68). However, these multiplexing
approaches require complex and expensive systems.
Recently large optical degrees of freedom can be encoded
using metasurfaces 69)-71) or graphene surface 72), but
these new types of wavefront modulators only offer the
generation of static images.
4.2 3D display exploiting volume speckles
In Chapter 3, we introduced that complex light
scattering can solve the phase problem in holographic
imaging. Similarly, in 3D displays, Yu et al.
demonstrated that random light scattering could also be
exploited to overcome the conventional limitation of 3D
holographic displays 73).
In this study, two holographic diffusers were inserted
83
Paper, Invited Paper » Review: 3D Holographic Imaging and Display Exploiting Complex Optics
Fig.5 Diffused field retrieval. The reflected field from the diffusive surface of dices are reconstructed by the proposed method. Two dices are
separated by 40 cm from each other (upper middle). The measured intensity speckle image (upper left), and the corresponding reconstructed
field result are shown (upper right). From the measured field result, the focal plane can be freely refocused for desired focal plane (lower
part, the first row). For the clearer visualization, 25 holograms are compounded into one clear image (lower part, the second row). This
figure is modified from Reference 59).
Fig.6 (a) Conventional Fourier holographic display system. The
viewing angle and the image size can be adjusted by the
focal length of the transfer lens. The product between the
viewing angle and the image size is fixed given the number
of pixels in the SLM (b) 3D holographic image projection
exploiting complex light scattering. The light scattered from
holographic diffusers forms volume speckle fields with large
viewing angle and image size. Inside this volume, 3D
images can be projected by properly modulating the
incident field. This figure is modified from Reference 73).
after the DM to ensure sufficient mixing of light
scattering paths. Then, the constant product of image
size and viewing angle was readily broken: the highly
scattered light forms volume speckles fields which
produce both large viewing angle and large image size,
as illustrated in Fig. 6(b).
Due to the randomness of volume speckle fields, no 3D
image is projected in the initial state. However, this
complex light pattern can be controlled to produce 3D
images by using the wavefront shaping techniques or
transmission matrices as introduced earlier. As shown
in the Fig. 7, volume speckle patterns are controlled to
generate '3DHD' letter images consisting of micrometer-
size optical foci with 35° viewing angle in a volume of 2
cm x 2 cm x 2 cm.
4.3 Viewing conditions of scattering display
For 3D images projected inside volume speckle, one
needs to characterize the image size and viewing angle.
In the current situation, 3D images are projected as a
point cloud. Therefore, the viewing angle is determined
by the physical property of single focus forming the point
cloud and the image size is also governed by the
projection area where the wavefront optimization of
single focus successfully works.
Firstly, the viewing angle of the 3D focus depends on
the contributing angle range of the diffusers. When 3D
images are decomposed into point clouds, the viewing
angle of a single point is governed by a supporting range
of spatial frequencies in the focus formation. For
example, if the volume speckle fields are formed after a
holographic diffuser with 60° diffuse angle, then the
viewing angle is the same as this angle range.
Experimentally, the viewing angle is measured by the
size of the focus. For an example, numerical aperture of
an optimized focus is NA = sin 30° = 0.5, so the focus size
d is 0.63 µm (d = 0.51 λ/NA) for 633 nm. In Ref. 73), the
focus size was measured as 1.05 µm corresponding to the
viewing angle of 35°.
Secondly, the image size of the 3D volume is
determined by the working range of wavefront shaping.
Because the wavefront shaping is based on the TM
information, which has no restriction on the spatial
position, the working range can be extended arbitrarily.
In Ref. 73), it was demonstrated that the focus could be
created throughout the diffuser area of 4.2 cm. If larger
diffusers are utilized, and the sufficient randomness is
guaranteed, the image size can be easily extended.
Based on the viewing angle and image size, we can
calculate the product of the viewing angle and image
size. The scattering display showed a striking
enhancement factor of 2, 600 in the product over the
DM-only case without the diffusers. However, it should
be noted that the amount of controllable information in
the SLM is not enhanced. As the information capacity
remains same, the restriction on the viewing angle and
the image size is converted into the limitation on the
contrast and the number of projected points.
4.4 Image quality of scattering 3D display
Since new restriction on the image projection arises in
the scatting display, it is also important to consider the
image quality of projected 3D scenes. Because 3D image
projection is based on a point cloud, the image quality
should also be discussed in the context of a single focus.
For the quality of point cloud images, we can mainly
consider two parameters: the contrast of a single focus
and the number of projected points.
The contrast of a single focus is determined by the
enhancement factor achieved in the wavefront shaping
algorithm. In Ref. 73), the EF of a single focus ranged
from 500-1000 for the use of a DM. However, this EF is
not exactly the same with the actual contrast because
the pixel unit in the calculation of the EF was set to the
size of the physical focus ~ 1 µm. In human perception,
the smallest size of the human eye can resolve much
larger than 1 µm 74). Therefore the actual focus contrast
perceived by viewers would be lower than the EFs. This
consideration of human visual system must be studied
for the future study.
Another important image quality metric is the number
of projected points. For a given the number of controlled
pixels in an SLM, the product between the number of
points and the contrast of single point is fixed. In Ref. 73),
the maximum number of projected points in a single
ITE Trans. on MTA Vol. 5, No. 3 (2017)
84
Fig.7 '3DHD' letters are sequentially projected inside a volume of
2 cm x 2 cm x 2 cm. Each letter consists of 15 points with
35°viewing angle. This figure is modified from Reference 73).
frame was 15 for the use of the DMD. However, it is still
far from high-quality 3D image projection. In order to
increase the number of points, the number of controlled
pixels should be enhanced. Recently a new wavefront
shaping system was introduced, which utilizes the half
of the full pixels in a DMD 75). In this system, the
enhancement factor of the single focus was measured as
10,531. Again, the unit pixel of the single focus is 10 µm,
which is still much smaller than PSF of human eyes, so
the contrast reduction is expected for the human
perception. We expect that further developments of this
ultrahigh enhancement system may enable high-quality
image projection in the near future.
4.5 Discussion
In order to generate realistic 3D images with the
scattering display, the image quality should be further
improved. Because the limitation on the number of
pixels in SLMs directly affects the image quality,
increasing the number of controlling pixels or spatial
multiplexing of multiple SLMs would be possible
solutions. Considering human perception factor can be
another solution to overcome the limitation of the
scattering display. For example, the speckle background
is effectively suppressed by temporal multiplexing of
independent light patterns 76), so the contrast can be
effectively increased for the same number of controlled
pixels.
One of the major hardships in the current system is
long optimization time. The measurement of a TM and
the calculation of the optimal incident patterns are
required for 3D image projection. However, the
optimization time is linearly proportional to the number
of controlled pixels in SLMs. In the current wavefront
shaping system, total optimization time for a single spot
is 73 min for Megapixel control 75). To generate a point
cloud with a large number of point elements, further
improvement on the wavefront shaping technique is
highly desired. We envision that the development of
high-speed optimization system may enable fast and
practical 3D projection through scattering media.
5. Conclusion
In this review, we introduced recent progress on
overcoming the limitation of conventional imaging and
display devices by exploiting the complexity of multiple
light scattering. It may seem to contradict to the
common belief that only well-designed optical
components can be used for imaging and projection
purposes. However, multiple scattering can offer new
possibilities otherwise inaccessible with conventional
optics.
Although we discussed the applications of imaging
and display separately, these two topics are closely
related to each other. For example, the hologram is used
in both recording and projection. The underlying concept
bridging these two applications is optical phase
conjugation 10). Recently Lee et al. demonstrated that
scattering media could serve as a medium for recording
the phase-conjugated light 77). Extending this concept to
the macroscopic scale, i.e. recording and replaying back
the light fields originating from real-world objects, is
expected to bring new possibilities in 3D imaging and
display using complex optics.
We expect that the advancement of the optics
applications exploiting complex optics will be
accompanied by the progress in device technologies. As
discussed earlier, the current systems still lack high
speed and large data calculation, fast modulation and
acquisition devices. Therefore, further investigation on
optimizing devices would significantly enhance the
capabilities of complex optics.
References
1) B. Lee: "Three-dimensional displays, past and present", PhysToday 66, 36-41 (2013)
2) P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey,V. Sainov and C. von Kopylow: "A survey of 3DTV displays:techniques and technologies", Circuits and Systems for VideoTechnology, IEEE Transactions on 17, 1647-1658 (2007)
3) R. Collier, Optical holography (Elsevier, 2013) 4) D. Gabor: "A new microscopic principle", Nature 161, 777-778
(1948) 5) E.N. Leith and J. Upatnieks: "Reconstructed wavefronts and
communication theory", JOSA 52, 1123-1130 (1962) 6) I.P. Howard and B.J. Rogers, Binocular vision and stereopsis
(Oxford University Press, USA, 1995) 7) C. Wheatstone: "Contributions to the physiology of vision.--Part the
first. On some remarkable and hitherto unobserved, phenomena ofbinocular vision", Philosophical transactions of the Royal Society ofLondon, 371-394 (1838)
8) D. Gabor: "Microscopy by reconstructed wave-fronts", inProceedings of the Royal Society of London A: Mathematical,Physical and Engineering Sciences (The Royal Society, 1949) , 454-487
9) S.A. Benton: "Hologram reconstructions with extended incoherentsources", in Journal of the Optical Society of America (AMER INSTPHYSICS CIRCULATION FULFILLMENT DIV, 500 SUNNYSIDEBLVD, WOODBURY, NY 11797-2999, 1969) , 1545-&
10) P. Hariharan, Basics of holography (Cambridge university press,2002)
11) H. Kogelnik: "Coupled wave theory for thick hologram gratings",Bell Labs Technical Journal 48, 2909-2947 (1969)
12) J.W. Goodman and R. Lawrence: "Digital image formation fromelectronically detected holograms", Appl Phys Lett 11, 77-79 (1967)
13) M. Takeda, H. Ina and S. Kobayashi: "Fourier-transform methodof fringe-pattern analysis for computer-based topography andinterferometry", J. Opt. Soc. Am. 72, 156-160 (1982)
14) I. Yamaguchi: "Phase-shifting digital holography", in Digital
85
Paper, Invited Paper » Review: 3D Holographic Imaging and Display Exploiting Complex Optics
Holography and Three-Dimensional Display (Springer, 2006) ,pp.145-171
15) G. Popescu, Quantitative Phase Imaging of Cells and Tissues(McGraw-Hill Professional, 2011)
16) K. Lee, K. Kim, J. Jung, J.H. Heo, S. Cho, S. Lee, G. Chang, Y.J.Jo, H. Park and Y.K. Park: "Quantitative phase imagingtechniques for the study of cell pathophysiology: from principles toapplications", Sensors-Basel 13, 4170-4191 (2013)
17) P. Marquet, B. Rappaz, P.J. Magistretti, E. Cuche, Y. Emery, T.Colomb and C. Depeursinge: "Digital holographic microscopy: Anoninvasive contrast imaging technique allowing quantitativevisualization of living cells with subwavelength axial accuracy",Optics Letters 30, 468-470 (2005)
18) G. Popescu, L.P. Deflores, J.C. Vaughan, K. Badizadegan, H. Iwai,R.R. Dasari and M.S. Feld: "Fourier phase microscopy forinvestigation of biological structures and dynamics", Optics letters29, 2503-2505 (2004)
19) Y. Park, G. Popescu, K. Badizadegan, R.R. Dasari and M.S. Feld:"Diffraction phase and fluorescence microscopy", Opt Express 14,8263-8268 (2006)
20) T.-C. Poon, Digital holography and three-dimensional display:Principles and Applications (Springer Science & Business Media,2006)
21) N. Kukhtarev, V. Markov, S. Odulov, M. Soskin and V. Vinetskii:"Holographic storage in electrooptic crystals.I. Steady state",ferroelectrics 22, 949-960 (1978)
22) S.P. McGrew: "Hologram counterfeiting: problems and solutions",in OE/LASE'90, 14-19 Jan. , Los Angeles, CA (InternationalSociety for Optics and Photonics, 1990) , 66-76
23) J. Hahn, H. Kim, Y. Lim, G. Park and B. Lee: "Wide viewing angledynamic holographic stereogram with a curved array of spatiallight modulators", Optics express 16, 12372-12386 (2008)
24) F. Yaras, H. Kang and L. Onural: "State of the art in holographicdisplays: A survey", Journal of display technology 6, 443-454(2010)
25) K. Wakunami, P.-Y. Hsieh, R. Oi, T. Senoh, H. Sasaki, Y.Ichihashi, M. Okui, Y.-P. Huang and K. Yamamoto: "Projection-type see-through holographic three-dimensional display", Naturecommunications 7, 12954 (2016)
26) H.-j. Kim, S.-K. Lee, M.-L. Piao, N. Kim and J.-H. Park: "Three-dimensional holographic head mounted display using holographicoptical element", in Consumer Electronics (ICCE) , 2015 IEEEInternational Conference on (IEEE, 2015) , 132-133
27) X. Hu and H. Hua: "High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics", Opticsexpress 22, 13896-13903 (2014)
28) D. Dunn, C. Tippets, K. Torell, P. Kellnhofer, K. Aksit, P. Didyk,K. Myszkowski, D. Luebke and H. Fuchs: "Wide Field Of ViewVarifocal Near-Eye Display Using See-Through DeformableMembrane Mirrors", IEEE Transactions on Visualization andComputer Graphics (2017)
29) A.P. Mosk, A. Lagendijk, G. Lerosey and M. Fink: "Controllingwaves in space and time for imaging and focusing in complexmedia", Nature photonics 6, 283-292 (2012)
30) S. Popoff, G. Lerosey, R. Carminati, M. Fink, A. Boccara and S.Gigan: "Measuring the transmission matrix in optics: an approachto the study and control of light propagation in disordered media",Physical review letters 104, 100601 (2010)
31) I.M. Vellekoop and A. Mosk: "Focusing coherent light throughopaque strongly scattering media", Optics Letters 32, 2309-2311(2007)
32) C. Park, J.-H. Park, C. Rodriguez, H. Yu, M. Kim, K. Jin, S. Han,J. Shin, S.H. Ko and K.T. Nam: "Full-Field SubwavelengthImaging Using a Scattering Superlens", Physical review letters113, 113901 (2014)
33) J.-H. Park, C. Park, H. Yu, J. Park, S. Han, J. Shin, S.H. Ko, K.T.Nam, Y.-H. Cho and Y. Park: "Subwavelength light focusing usingrandom nanoparticles", Nature photonics 7, 454-458 (2013)
34) Y. Choi, T.D. Yang, C. Fang-Yen, P. Kang, K.J. Lee, R.R. Dasari,
M.S. Feld and W. Choi: "Overcoming the diffraction limit usingmultiple light scattering in a highly disordered medium", Physicalreview letters 107, 023902 (2011)
35) D.B. Conkey and R. Piestun: "Color image projection through astrongly scattering wall", Optics express 20, 27312-27318 (2012)
36) H. Yu, T.R. Hillman, W. Choi, J.O. Lee, M.S. Feld, R.R. Dasari andY. Park: "Measuring large optical transmission matrices ofdisordered media", Physical review letters 111, 153902 (2013)
37) X.D. Tao, D. Bodington, M. Reinig and J. Kubby: "High-speedscanning interferometric focusing by fast measurement of binarytransmission matrix for channel demixing", Optics Express 23,14168-14187 (2015)
38) D.B. Conkey, A.M. Caravaca-Aguirre and R. Piestun: "High-speedscattering medium characterization with application to focusinglight through turbid media", Optics express 20, 1733-1740 (2012)
39) J. Yoon, K. Lee, J. Park and Y. Park: "Measuring opticaltransmission matrices by wavefront shaping", Optics Express 23,10158-10167 (2015)
40) I. Vellekoop and A. Mosk: "Universal optimal transmission of lightthrough disordered materials", Physical Review Letters 101,120601 (2008)
41) D. Akbulut, T.J. Huisman, E.G. van Putten, W.L. Vos and A.P.Mosk: "Focusing light through random photonic media by binaryamplitude modulation", Optics Express 19, 4017-4029 (2011)
42) J.-H. Park, C. Park, H. Yu, Y.-H. Cho and Y. Park: "Dynamicactive wave plate using random nanoparticles", Opt. Exp. 20,17010-17016 (2012)
43) J.H. Park, C.H. Park, H. Yu, Y.H. Cho and Y.K. Park: "Activespectral filtering through turbid media", Opt. Lett. 37, 3261-3263(2012)
44) J. Aulbach, B. Gjonaj, P.M. Johnson, A.P. Mosk and A. Lagendijk:"Control of light transmission through opaque scattering media inspace and time", Physical Review Letters 106, 103901 (2011)
45) O. Katz, E. Small, Y. Bromberg and Y. Silberberg: "Focusing andcompression of ultrashort pulses through scattering media",Nature photonics 5, 372-377 (2011)
46) D.J. McCabe, A. Tajalli, D.R. Austin, P. Bondareff, I.A. Walmsley,S. Gigan and B. Chatel: "Spatio-temporal focusing of an ultrafastpulse through a multiply scattering medium", NatureCommunications 2, 447 (2011)
47) D. Gabor: "IV Light and Information", Progress in optics 1, 109-153 (1961)
48) P. Hariharan, Optical Holography: Principles, techniques andapplications (Cambridge University Press, 1996)
49) M. Tegze and G. Faigel: "X-ray holography with atomicresolution", Nature 380, 49-51 (1996)
50) K. Lee and Y. Park: "Exploiting the speckle-correlation scatteringmatrix for a compact reference-free holographic image sensor",Nature Communications 7(2016)
51) X.Q. Cui, J.A. Ren, G.J. Tearney and C.H. Yang: "Wavefront imagesensor chip", Optics Express 18, 16685-16701 (2010)
52) R.V. Shack and B.C. Platt: "Production and Use of a LenticularHartmann Screen", J Opt Soc Am 61, 656-& (1971)
53) D. Paganin and K.A. Nugent: "Noninterferometric phase imagingwith partially coherent light", Phys Rev Lett 80, 2586-2589 (1998)
54) R. Hegerl and W. Hoppe: "Phase evaluation in generalizeddiffraction (ptychography) ", Proc. Fifth Eur. Cong. ElectronMicroscopy, 628-629 (1972)
55) J.M. Rodenburg, A.C. Hurst, A.G. Cullis, B.R. Dobson, F. Pfeiffer,O. Bunk, C. David, K. Jefimovs and I. Johnson: "Hard-x-raylensless imaging of extended objects", Phys Rev Lett 98(2007)
56) J.R. Fienup: "Phase Retrieval Algorithms - a Comparison", Appl.Opt. 21, 2758-2769 (1982)
57) R.W. Gerchberg: "A practical algorithm for the determination ofphase from image and diffraction plane pictures", Optik 35, 237(1972)
58) G.A. Zheng, R. Horstmeyer and C.H. Yang: "Wide-field, high-resolution Fourier ptychographic microscopy", Nature Photonics 7,739-745 (2013)
ITE Trans. on MTA Vol. 5, No. 3 (2017)
86
59) K. Lee and Y. Park: "Exploiting the speckle-correlation scatteringmatrix for a compact reference-free holographic image sensor",Nature Communications 7, 13359 (2016)
60) J.W. Goodman: "Statistical properties of laser speckle patterns", inLaser Speckle and Related Phenomena (Springer BerlinHeidelberg, 1975) , pp.9-75
61) Y. Choi, C. Yoon, M. Kim, T.D. Yang, C. Fang-Yen, R.R. Dasari,K.J. Lee and W. Choi: "Scanner-Free and Wide-Field EndoscopicImaging by Using a Single Multimode Optical Fiber", Phys RevLett 109(2012)
62) T. Cizmar and K. Dholakia: "Exploiting multimode waveguides forpure fibre-based imaging", Nature Communications 3(2012)
63) N. Fukaya, K. Maeno, O. Nishikawa, K. Matsumoto, K. Sato andT. Honda: "Expansion of the image size and viewing zone inholographic display using liquid crystal devices", in IS&T/SPIE'sSymposium on Electronic Imaging: Science & Technology(International Society for Optics and Photonics, 1995) , 283-289
64) K. Maeno, N. Fukaya, O. Nishikawa, K. Sato and T. Honda:"Electro-holographic display using 15mega pixels LCD", inElectronic Imaging: Science & Technology (International Societyfor Optics and Photonics, 1996) , 15-23
65) Y.-Z. Liu, X.-N. Pang, S. Jiang and J.-W. Dong: "Viewing-angleenlargement in holographic augmented reality using time divisionand spatial tiling", Optics Express 21, 12068-12076 (2013)
66) Z.M. A. Lum, X. Liang, Y. Pan, R. Zheng and X. Xu: "Increasingpixel count of holograms for three-dimensional holographic displayby optical scan-tiling", OPTICE 52, 015802-015802 (2013)
67) Y. Matsumoto and Y. Takaki: "Improvement of gray-scalerepresentation of horizontally scanning holographic display usingerror diffusion", Optics Letters 39, 3433-3436 (2014)
68) Y. Takaki and K. Fujii: "Viewing-zone scanning holographicdisplay using a MEMS spatial light modulator", Optics Express 22,24713-24721 (2014)
69) S. Larouche, Y.-J. Tsai, T. Tyler, N.M. Jokerst and D.R. Smith:"Infrared metamaterial phase holograms", Nature materials 11,450-454 (2012)
70) L. Huang, X. Chen, H. Mühlenbernd, H. Zhang, S. Chen, B. Bai, Q.Tan, G. Jin, K.-W. Cheah, C.-W. Qiu, J. Li, T. Zentgraf and S.Zhang: "Three-dimensional optical holography using a plasmonicmetasurface", Nat Commun 4(2013)
71) X. Ni, A.V. Kildishev and V.M. Shalaev: "Metasurface hologramsfor visible light", Nature communications 4(2013)
72) X. Li, H. Ren, X. Chen, J. Liu, Q. Li, C. Li, G. Xue, J. Jia, L. Cao,A. Sahu, B. Hu, Y. Wang, G. Jin and M. Gu: "Athermallyphotoreduced graphene oxides for three-dimensional holographicimages", Nat Commun 6(2015)
73) H. Yu, K. Lee, J. Park and Y. Park: "Ultrahigh-definition dynamic3D holographic display by active control of volume speckle fields",Nat Photon 11, 186-192 (2017)
74) M. Yamaguchi and T. Utsugi: "Speckle suppression incomputational hologram generated using ray-sampling plane", in2014) , 91170A-91170A-91110
75) H. Yu, K. Lee and Y. Park: "Ultrahigh enhancement of lightfocusing through disordered media controlled by Megapixelmodes", arXiv preprint arXiv:1701.02933 (2017)
76) J.G. Manni and J.W. Goodman: "Versatile method for achieving1% speckle contrast in large-venue laser projection displays usinga stationary multimode optical fiber", Optics Express 20, 11288-11315 (2012)
77) K. Lee, J. Lee, J.H. Park, J.H. Park and Y. Park: "One-WaveOptical Phase Conjugation Mirror by Actively Coupling ArbitraryLight Fields into a Single-Mode Reflector", Physical ReviewLetters 115(2015)
87
Paper, Invited Paper » Review: 3D Holographic Imaging and Display Exploiting Complex Optics
YongKeun Park is Associate Professor ofPhysics at KAIST. He earned a Ph.D. from Harvard-MIT Health Science and Technology. Dr. Park's area ofresearch is wave optics and biophotonics.
KyeoReh Lee received his BS degree in physicsfrom KAIST. He is currently a PhD student in theDepartment of Physics at KAIST.
SeungYoon Han is currently a BS student inthe Department of Physics at KAIST.
Jongchan Park received his BS degree inphysics from KAIST. He is currently a PhD student inthe Department of Physics at KAIST.
YoonSeok Baek received his BS degree inphysics from KAIST. He is currently a PhD student inthe Department of Physics at KAIST.
Hyeonseung Yu received his PhD degree inPhysics from KAIST in 2017. He received his BSdegree in physics and mathematical science fromKAIST. He is currently a postdoctoral associate in MaxPlanck Institute for Informatics.