MEMS-based lidar for autonomous drivingH.W. Yoo, N. Druml, D. Brunner, C. Schwarzl, T. Thurner, M. Hennecke, G. Schitter OVE
Lidar, the acronym of light detection and ranging, has received much attention for the automotive industry as a key component forhigh level automated driving systems due to their high resolution and highly accurate 3D imaging of the surroundings under variousweather conditions. However, the price and resolution of lidar sensors still do not meet the target values for the automotive marketto be accepted as a basic sensor for ensuring safe autonomous driving. Recent work has focused on MEMS scanning mirrors as apotential solution for affordable long range lidar systems. This paper discusses current developments and research on MEMS-basedlidars. The LiDcAR project is introduced for bringing precise and reliable MEMS-based lidars to enable safe and reliable autonomousdriving. As a part of development in this project, a test bench for the characterization and performance evaluation of MEMS mirror isintroduced. A recently developed MEMS-based lidar will be evaluated by various levels of tests including field tests based on realisticscenarios, aiming for safe and reliable autonomous driving in future automotive industry.
Lidar, ein Akronym für Light Detection And Ranging, erhielt als Schlüsselkomponente für autonome Fahrsysteme in der Automobil-industrie viel Aufmerksamkeit, da es hochauflösende und hochgenaue 3D-Bilder der Umgebung bei verschiedensten Wetterbedin-gungen liefert. Derzeit entsprechen die Preise und die Auflösung der kommerziell verfügbaren Lidar-Sensoren jedoch noch nicht denZielanforderungen, um als Basissensor zur Gewährleistung der Sicherheit während des autonomen Fahrens akzeptiert zu werden.MEMS-basierte Scan-Spiegel waren als potentielle Lösung für ein leistbares weitreichendes Lidar-System Fokus einer kürzlich durch-geführten Arbeit. Diese Arbeit erörtert die derzeitige Entwicklung und Forschung von MEMS-basierten Lidar-Systemen und stellt dasLiDcAR-Projekt vor, dessen Ziel es ist, ein präzises und verlässliches MEMS-basiertes Lidar-System für autonomes Fahren zu entwickeln.Als Teil-Entwicklung dieses Projekts wird ein Prüfstand zur Charakterisierung und Evaluierung von MEMS Scan-Spiegeln vorgestellt.Das kürzlich entwickelte MEMS-basierte Lidar-System wird auf verschiedenen Testebenen evaluiert, inklusive Feldtests, basierend aufrealistischen Szenarien, mit dem Ziel, in Zukunft sicheres und verlässliches autonomes Fahren zu gewährleisten.
1. IntroductionLidar is the acronym of light detection and ranging, in an analogy toradar. The basic structure of any pulsed time of flight (TOF) lidar canbe described as follows (see Fig. 1): A laser module generates a shortlaser pulse and the light hits the target object. Then the receiverdetects the reflected pulse from the target. From the elapsed timefrom the transmitted pulse to the received pulse, a data acquisitionunit calculates the distance.
Since its first use for highly accurate lunar ranging, many lidarsystems have been developed for various applications as a high res-olution and highly accurate measurement technique, e.g. for earthsurface measurements [1], wind speed measurements [2], buildingconstruction operations [3], mining industry [4], forestry [5], androbotics [6].
Recently lidar sensors have received much attention for the auto-motive industry as a key component for high level automated drivingsystems [7–9]. Compared to other 3D sensing techniques, such asstereo cameras and radar, lidar sensors can provide high resolutionand highly accurate 3D measurements of the surroundings undervarious weather conditions [10]. The expected function from auto-motive lidar sensors is to ensure reliable and safe automated drivingsuch as collision detection, blind spot monitoring, object and pedes-trian recognition [11, 12], terrain mapping [13]. Currently Google[14], BMW [15], Volvo [16] and other autonomous car developersinclude lidar sensors in their development to ensure safe driving [17].
The commercially available lidar sensors can be categorized intotwo types according to the maximum range and the resolution ofthe system: short range and long range lidar sensors. The shortrange lidar sensors usually measure less than 50 m distance with anarrow angle for forward collision warning and blind spot detection[18]. They usually implement a flash lidar structure, shown in Fig. 1.In a flash lidar, the transmitter illuminates the whole scene and anarray of detectors measures the distance of each pixel of the im-age. Continental (Hanover, Germany) and LeddarTech (Quebec City,Canada) provide affordable flash lidar sensors to the market [19,20]. Short range lidar sensors are low-priced due to the simplicityof the setup, but the measurement distance is fairly short since thelight intensity from the transmitter is dispersed with a relatively largeangle, and is also limited by eye-safety considerations [21]. In con-
Yoo, Han Woong, Automation and Control Institute, TU Wien, Gußhausstraße 27–29,1040 Vienna, Austria (E-mail: [email protected]); Druml, Norbert, Design CenterGraz, Infineon Technologies Austria AG, Babenbergerstraße 10, 8010 Graz, Austria;Brunner, David, Automation and Control Institute, TU Wien, Gußhausstraße 27–29,1040 Vienna, Austria; Schwarzl, Christian, Virtual Vehicle Research Center, Inffeldgasse21a, 8010 Graz, Austria; Thurner, Thomas, Design Center Graz, Infineon TechnologiesAustria AG, Babenbergerstraße 10, 8010 Graz, Austria; Hennecke, Marcus, DesignCenter Graz, Infineon Technologies Austria AG, Babenbergerstraße 10, 8010 Graz, Austria;Schitter, Georg, Automation and Control Institute, TU Wien, Gußhausstraße 27–29,1040 Vienna, Austria
ORIGINALARBEIT H. W. Yoo et al. MEMS-based lidar for autonomous driving
Fig. 1. Structure of lidar system: flash lidar (left) and scanning lidar (right)
trast, scanning lidar sensors, also shown in Fig. 1, focus their lighton a very small area, making them suitable for long distance andhigh resolution imaging. A scanning lidar sensor diverts a collimatedbeam to the target and illuminates a small fraction of the target ob-ject at one shot. Then the reflected light from the small area of thetarget object is focused on the detector, recording a pixel. By movingthe beam position, the distance of the next pixel is measured and a3D image of the entire scene can be generated. Currently Velodyne(Morgan Hill, CA, US) and Quanergy (Sunnyvale, CA, US) provide en-tire view (360◦) lidars in the market and Ibeo (Hamburg, Germany)provides lidars with a full front range (145◦ ) [22–24]. Scanning li-dars can provide rich 3D maps of the surroundings, which allowsfurther analysis such as identification and avoidance of objects andpedestrians.
The currently popular scanning solution in the automotive lidarmarket is the rotating scanner [7, 25]. The rotating scanner is cur-rently most popular for many commercial lidar sensors because itprovides straight and parallel scan lines with a uniform scanningspeed over a wide field of view. Furthermore, it provides less stressand low vibrations on the mirror surface [26]. As a drawback, it isgenerally bulky and less scalable due to the limiting duty cycle by theedges between facets for polygon mirrors [27]. In addition, a largeinertia of the rotating module limits the frame rate of lidar sensorsand requires rather high power consumption.
As an alternative solution, optical phase arrays and microelec-tromechanical system (MEMS) mirrors receive much attention aspromising solutions for affordable lidars [18, 28] and compete witheach other for the future automotive lidar market. An optical phasearray (OPA) is a solid-state device that steers the beam by micro-structured waveguides. Since there is no mechanical moving part,OPA allows extremely high scanning speed over 100 kHz over largeangles. However the insertion loss of the laser power is a drawback[29, 30]. As a developing technology with high potential, the inter-ests on OPA for automotive lidar is growing in academia and industryeven though OPAs are still not proven for long range lidar yet. Micro-electromechanical systems (MEMS) mirrors carry a high potential asa future automotive lidar solution since both the form factor and thecost in a large volume can be drastically reduced. The lidar manufac-tures are keen to test these two promising technologies. Quanergyannounced a solid state lidar system (S3) based on optical phasearray (OPA) scanner at the Consumer Electronics Show (CES) 2016,which has 120◦ optical scanning angle for horizontal and verticaldirection and with 0.05◦ resolution [31]. Innoviz (Kefar Sava, Israel)also released a MEMS-based solid state lidar (InnovizPro), which al-lows ranging up to 150 m distance with a 73◦ × 20◦ field of view(FOV) with 0.15◦ horizontal resolution [32].
This work discusses the current research on MEMS-based lidarsensors and introduces a MEMS-based lidar sensor. In Sect. 2, thecurrent state of the art about MEMS-based lidar sensors is reviewed.
A MEMS-based lidar approach in LiDcAR project is introduced and itschallenges are elaborated in Sect. 3. A MEMS test bench to evaluateMEMS mirror for lidar application is presented in Sect. 4. Verifica-tion and validation of automotive functions using lidar sensors areexplained in Sect. 5. Then Sect. 6 concludes the paper.
2. Review on current development of MEMS-based lidarMEMS-based scanning mirrors have recently received much interestfor their use in automotive lidar applications as a light-weight, com-pact and low power consumption scanning solution. Several MEMS-based lidar sensors have already been developed in various applica-tion such as space applications [33, 34], robotics [35, 36] and windvelocimetry [37], showing the feasibility of the technology. For au-tomotive applications, Hoffman et al. developed a 360◦ scanning2D MEMS scanner lidar with 60◦ of azimuth angle [38] in the EUproject of Minifaros [39]. Stann et al. also developed a short rangelidar sensor for the small unmanned ground vehicle (UGV) and airvehicle (UAV), whose range is up to 160 m with a 42◦ × 21◦ FOVand 6 fps [40]. Ito et al. developed MEMS-based lidar with singlephoton CMOS focal plane detectors, which can measure distancesup to 25 m with a 45◦ × 11◦ FOV [41]. MEMS lidar sensors are alsoused for testing of the lidar technique itself, e.g. digital modulation[42] and optical code division multiple access (OCDMA) [43].
MEMS scanning mirrors can be mainly categorized into resonantMEMS mirrors and non-resonant MEMS mirror according to the op-erating frequency with respect to their mechanical mode [44]. Theresonant MEMS mirrors provide a large scan angle at a high fre-quency and a relatively simple control design while the scan tra-jectory is sine-like, i.e. non-uniform scan speed. In addition, MEMSscanner as a nonlinear oscillator can cause softening and or stiff-ening, limiting the operation frequencies [45]. Non-resonant MEMSmirrors, also called quasi-static MEMS mirrors, provide a large de-gree of freedom in the trajectory design. Although rather complexcontroller is required to keep the scan quality, desirable scanningtrajectories such as triangular or saw tooth scanning with constantscan speed at large scan ranges can be generated by an appropriatecontroller design [46, 47]. Quasi-static MEMS mirrors, however, usu-ally have a smaller scanning angle compared to the resonant MEMSmirrors. To enlarge the scan angle, an additional optics and lens areused [40, 46, 48, 49]. An immersion of the MEMS mirror packagealso increases the scan angle by Snell’s window effect with additionaldamping as a tradeoff [50].
Actuation and scanning principles get diversified, looking for thebest technique in the future lidar market. The most common actua-tion principle is currently the electrostatic comb drive while electro-magnetic actuation [51] and piezoelectric actuation [52] have alsobeen reported for lidar applications. Not only flat mirror based scan-ning, a 1D refractive MEMS scanner has been developed for a ±5◦
optical scan angle based on the rotation of a convex lens with comb
H. W. Yoo et al. MEMS-based lidar for autonomous driving ORIGINALARBEIT
drive. A commercially available digital micro mirror device (DMD) hasbeen applied for a scanning lidar sensor as a programmable grating[53]. Wang et al. developed a new concept of optical phase arraybased on MEMS approach using vertically actuating comb drives,which can steer the laser beam ±11◦ [54].
The figure of merit (FoM) for the MEMS mirror for automotive li-dar is different from a typical MEMS mirror for other applications.A FoM for MEMS mirrors is proposed for pico-projector application,which is the product of the scan angle, mirror size, and resonantfrequency (i.e. scanning frequency), which defines the resolution ofthe projector [44]. For long range lidar, scan angle and mirror sizeare accepted as important parameters. Especially for the bistatic lidarstructure, a larger aperture at receiver is important to ensure enoughsignal to noise ratio (SNR) for long distance detection. Sander et al.propose a dedicated MEMS mirror structure, which consists of a res-onant MEMS mirror for the transmitter and 14 identical MEMS mir-ror for the receiver, operating in a synchronized manner [55]. Besidesthe mirror size, robustness to the harsh environment conditions suchas extremely high or low temperature and large vibrations should beconsidered [56]. A trade-off between the mirror size and the vibra-tion rejection performance is discussed while an immersion mediumin the MEMS mirror package is reported to provide enhanced ro-bustness against external vibrations and shocks [57].
3. LiDcAR project: evaluation of MEMS-based lidarThe LiDcAR project, which is a collaborative (Infineon TechnologiesAustria AG, TU Wien – Automation and Control Institute (ACIN), andthe Virtual Vehicle Research Center) FFG funded research project,aims to explore and to assess the two most promising lidar tech-nologies in order to pave the way for the long sought automotive-qualified, long-range, robust, and low-cost Lidar solution. This willbe accomplished in a three-step process:
1. LiDcAR’s first step is to perform a comprehensive laboratory-based evaluation and comparison of the most promising tech-nologies (such as MEMS mirror, rotating polygon mirror, opticalphased array, flash Lidar, and fixed multi-beam). Based on theidentified strengths and weaknesses, the two most promisingtechnologies are selected and two experimental prototypes arerealized.
2. The second step of LiDcAR is to research and develop method-ologies that will enable essential lidar-metrology and verificationmethodologies. This effort will pave the way towards a properlidar evaluation system and will thus enable the path towards anautomotive-qualified lidar system.
3. In a third step, in tight collaboration with ALP.Lab, the two exper-imental lidar prototypes will be integrated into an autonomoustest vehicles. Extensive in-field tests at Styria’s autonomous driv-ing test-region will provide valuable data that will be analyzedand assessed according to the developed lidar-metrology and ver-ification methodologies. The gained results and insights will fi-nally pave the way towards the future automotive-qualified andlow-cost key lidar technology made in Austria.
The LiDcAR project and its consortium identified the micro-scanning 1D MEMS-mirror lidar as one of the currently most promis-ing lidar technologies. Therefore, the consortium is currently inten-sively working on a very first MEMS-based lidar prototype. Its fun-damental scanning concept is depicted in Fig. 2. A 1D MEMS mirroroscillates and deflects a laser pulse into the scenery. It is controlledand actuated by a dedicated driving circuitry. Laser pulses reflectedoff objects in the scenery are focused by optics and detected byan array of photo diodes. The received analog signals are amplified
via transimpedance amplifiers and high-speed analog-to-digital andtime-to-digital conversions are carried out. Finally, a 3D point cloudcan be computed and forwarded to a central sensor fusion ECU. Inthis particular approach to lidar sensors the MEMS mirror oscillatesabout only one axis, horizontally scanning the scenery left-right. Atthe same time, the laser beam fans out and paints a vertical linein the distance, resulting in a full scan of the scenery. This uniqueapproach results in a number of key features: The horizontal reso-lution of the system is entirely determined by the timing accuracyof the MEMS driving circuit and can reach 0.1◦ or better. Since allangles in the vertical direction are scanned in parallel with the samelaser pulse, the scan speed is very high. With the laser, light focusedinto one thin line the pulse energy is concentrated in a small areaenabling long range distance measurements. And finally, a MEMSmirror with only one axis is far less complex to manufacture andto operate than a two axis mirror and it is also far more robust tovibration and shock.
A major LiDcAR result achieved so far is the realization of a firstfeasibility study. This feasibility study includes the MEMS mirror anda very first version of a future MEMS driver circuitry, as depicted inFig. 3. Together both components form the central and most crucialpart in the micro-scanning 1D MEMS-mirror lidar concept. Prelim-inary results already proved successfully the feasibility of this lidarconcept. Furthermore, this platform enables rapid technology explo-rations and rapid prototyping, which is crucial in the light of systemcomplexity and the fast-paced lidar research arena.
4. Test bench development for MEMS scanning mirrorThe characterization of MEMS scanning mirrors is crucial in the de-velopment of the MEMS mirror, in order to measure and evaluatethe performance and reliability of the developed mirror prototypes inorder to ensure to meet all requirements needed to be use in the au-tomotive environment as essential part of automotive lidar systems.In our 1D scanning lidar approach the MEMS mirror is used to steerthe laser beam in a deterministic and reproducible manner. The char-acterization of such MEMS mirror needs the real-time measurementof the mirror deflection together with actuation voltage signal andthe implemented current sensing approach, but with guaranteed ac-curacy and resolution in a calibrated measurement setup. The char-acterization environment thus should provide the scanning trajecto-ries in degree within a required measurement uncertainty, resolutionand possibly appearing distortion. Repeatability and reproducibilityof the developed setups should considered with included geomet-ric calibration procedures, to correct for all relevant systematic errorcontributors to the scan trajectory measurement. The signal qualityfrom all sensor readouts should be high to meet the required chal-lenging resolution and measurement uncertainty requirements, re-gardless of noise and external disturbances such as electromagneticinterference (EMI) from high voltage operation of the MEMS scan-ning mirror. For such a purpose, a test bench for MEMS scanningmirrors has been developed and implemented.
Figure 4 describes schematics of the mirror test bench. The testbench mainly consists of a laser unit, a position sensitive device unit,and adjustable mounts for the direct measurement of the mirror de-flection, including calibration and alignment. The laser unit consistsof an adjustable fiber collimator from an external fiber laser sources.The laser is focused via the central region of the MEMS mirror ontoa position sensitive device (PSD) (S5991-01, 2D PSD, Hamamatsu).This PSD element allows for real-time high bandwidth tracking ofthe reflected laser beam position on the active PSD area. For theaccuracy of geometrical alignment, the laser collimator is attachedon a 5 DOF mount, correcting position and direction of the laser
ORIGINALARBEIT H. W. Yoo et al. MEMS-based lidar for autonomous driving
Fig. 2. Working principle of the micro-scanning 1D MEMS-mirror lidar approach
Fig. 3. MEMS mirror and driver feasibility study
Fig. 4. Schematics and a picture of the test bench. The laser is equipped with a 5 DOF optics mount, the mirror is attached with a 6 DOF mirrormount, and the PSD is mounted on a 1 DOF motorized stage for the alignment
H. W. Yoo et al. MEMS-based lidar for autonomous driving ORIGINALARBEIT
beam. The MEMS scanning mirror is placed on a 6 DOF mount tocorrect all possible device orientation and tilt errors of the MEMSmirror in the given package prototype. Proper positioning of theMEMS mirror after replacement and inclusion in the setup. The po-sition error during the mirror exchange procedure is measured bya high resolution digital camera with attached telecentric objective.An image processing software provides the position and directionerror of the mirror, allowing an intuitive support for proper manualadjustment of the MEMS mirror mount. The PSD unit is attached ona fixture mounted on a 1D motorized linear translation stage, whichenables the measurement and of the geometrical correction factorsas needed by the geometrical calibration process. The enable max-imum range for mirror deflection sensing the PSD is tilted by 45◦on the mount, having the scan trajectory aligned with the diago-nal of the quadratic-shaped active PSD area. On the electrical sidethe PSD unit is connected to a dedicated preamplifier module withtransimpedance amplifiers (TIA) and a following analog processingmodule. All analog processing circuits have optimized for robust-ness against the EMI generated by the high voltage operation of theclosely located MEMS mirror.
Systematic errors from the alignment of the individual test benchcomponents are the major contributors to the achievable accuracyof the mirror deflection measurement. One of the major errorssource is the distance between the PSD and the MEMS mirror, whichneeds to be determined with an uncertainty of less than tens of mi-crometers to ensure the required accuracy in the measurement. Adedicated calibration process has been developed to estimate theneeded geometrical correction factors for enabling a highly accu-rate mirror deflection measurement.
Figure 5 provides an example of a measurement result from thedeveloped test bench, showing a typical response curve as one ofthe main characteristics of a MEMS mirror prototype operated inopen loop. The diagram shows the evolution of MEMS mirror de-flection angle amplitude over frequency when starting the mirrorand operating up to the high angle operation point – in this case at15◦ mechanical angle amplitude.
By providing accurate mirror angle data the developed setup –including the geometrical calibration procedure – allows for reliablehigh accuracy characterization of MEMS scanning mirrors with highrepeatability and reproducibility.
5. Verification and validation of MEMS-based lidar forautonomous driving
Before a new vehicle function can go into serial production a thor-ough verification of its functional correctness is necessary not onlyto ensure customer satisfaction, but also to avoid safety threats tothe vehicle inhabitants or other traffic participants in case of safetycritical functions. This is achieved by performing tests at differenttest levels, where the developed function parts are integrated andtested using real or simulated stimulations. With increasing test levelthe test environments become closer to reality, where in the lowestlevel the function is tested using purely simulated inputs and in thehighest level only real sensor data is used obtained by during testdrives on public roads.
An example of the main test levels considered in the automo-tive industry is shown in Fig. 6, where also a commonly used testenvironment is depicted for each test level. The shown test levelsreflect the integration steps of the developed function into the vehi-cle, where in level one and two its hardware and software parts aretested and in level three to five the function is tested in combinationwith other vehicle functions.
Since testing at test levels three to five is very costly and timeconsuming, the LiDcAR consortium cooperates tightly with ALP.Lab
Fig. 5. Frequency response of the MEMS mirror, measured by the de-veloped test bench. The response curve of the MEMS mirror consistsof top response curve (red solid line) and bottom response curve(blue dashed line). When sweeping down the excitation frequency onthe mirror along the bottom response curve, a jump (labeled 1, withan up arrow) to the top response curve at higher angle amplitudes isobserved. The scan angle increases further by an following up sweepafter the jump, following the top response curve. At a certain driv-ing frequency, the angle amplitude falls back (labeled 2, with a downarrow) to the bottom response curve (Color figure online)
(Austrian Light Vehicle Proving Region for Automated Driving) in or-der to get access to wide range of test hardware, proving groundsand permissions for public road tests. In particular, the access tolarge number of test tracks from AVL, Magna Steyr, Zentrum amBerg, Testregion Lungau, or the Redbull race track in Spielberg al-lows the simulation of a wide range of environment conditions anddriving situations. On these test tracks automated robots will beused to move objects like pedestrians, cars and cyclists into the fieldof view of the lidar sensors. This approach allows the execution oftests under a wide range of weather conditions with known ob-jects and their position, which ensures the reproducibility of the testresults and ensures that the sensor measurement data can be auto-matically verified for correctness. The simulation environments pro-vided by ALP.Lab can also be used to specify the tests performed onthe proving ground, increasing efficiency during test development.
In addition, ALP.Lab provides highly accurate sensor systems,which can be mounted on the test vehicle, allowing a performanceassessment of the used lidar sensors. Moreover, ALP.Lab providesa cloud-based data management solution for seamless data acqui-sition, storage, and automated analysis allowing global access tothe obtained measurement data and test results. By using this cloudbased storage and analysis framework it is possible to process a largenumber of test drives within a short time by leveraging the massiveparallel computation power provided by such systems. The cloudbased storage approach, where each ALP.Lab customer has its owntenant ensuring protection against unauthorized access, is capableof handling many terabyte of data. This large amount of storageis necessary because the different sensors, especially lidar systems,produce measurement data at a very high rate, making the use oflatest hard disks infeasible.
6. ConclusionThis paper discusses the current developments and research onMEMS-based lidar systems. To ensure safe autonomous driving so-
ORIGINALARBEIT H. W. Yoo et al. MEMS-based lidar for autonomous driving
Fig. 6. Test levels ranging from 1 to 5 and their typical test environments used for testing automotive functions
lutions in future mobility, besides 2D cameras and radar sensors anautomotive qualified lidar system is seen essential to enable properperception of the car environment during all driving situations. Strin-gent requirements on such lidar systems are long range, high resolu-tion and accuracy in distance but also high angular resolution in lat-eral and vertical direction w.r.t. sensing direction, secured long life-time in the automotive environment conditions (temperature range,temperature changes, vibration and shock robustness), and, alto-gether at an acceptable low price. MEMS-based lidar is regarded asone of the most promising solutions for affordable automotive sys-tems, and many rising but also well-established companies as wellas research institutions are currently working with high intensity tobring it into the future automotive lidar market.
To cope with the challenging tasks during lidar system develop-ment, the LiDcAR project aims in investigation and validation of au-tomotive lidar solutions. The aims and underlying main work pack-ages of the LiDcAR project have been described throughout the pa-per, targeting in development on characterization environments forlidar components but also lidar systems, and the validation of lidarsystem prototypes in the automotive environment with the ALP.Labautomated driving test roads. Currently the concept of the feasibil-ity study is successfully finished, resulting in a working MEMS mirrorbased scan system where a 1D MEMS mirror is interfaced by anFPGA-based evaluation platform. This evaluation platform is basedon the current version of a MEMS driver ASIC sensing and controlconcept – implemented on the FPGA board as a part of the evalua-tion system. First results have proven suitability for the chosen mir-ror and driver concept for use in automotive scanning lidar applica-tions. As another part of the project work, a test bench for detailedcharacterization of scanning MEMS mirrors has been developed. Thetest bench includes a real-time 2D PSD based optical readout of themechanical deflection of the MEMS mirror for all operation modes,enabling reproducible high accuracy electromechanical characteriza-tion of MEMS mirror devices.
As next steps, the developed test benches will be further opti-mized and extended towards the robustness evaluation of scanninglidar components and systems. Lidar system prototypes will be real-ized and these MEMS-based lidar solutions will be evaluated for usein automotive applications, including field tests based on realisticdriving scenarios.
AcknowledgementsOpen access funding provided by TU Wien (TUW). This work wassupported by Mobilität der Zukunft program of the Austrian Re-search Promotion Agency (FFG).
Open Access This article is distributed under the terms of the CreativeCommons Attribution 4.0 International License (http://creativecommons.org/
licenses/by/4.0/), which permits unrestricted use, distribution, and reproduc-tion in any medium, provided you give appropriate credit to the original au-thor(s) and the source, provide a link to the Creative Commons license, andindicate if changes were made.
References
1. Glennie, C. L., Carter, W. E., Shrestha, R. L., Dietrich, W. E. (2013): Geodetic imagingwith airborne LiDAR: the Earth’s surface revealed. Rep. Prog. Phys., 76(8), 086801.
2. Manninen, A. J., O’Connor, E. J., Vakkari, V., Petäjä, T. (2016): A generalised back-ground correction algorithm for a Halo Doppler lidar and its application to data fromFinland. Atmos. Meas. Tech., 9, 817–827.
3. Zhang, K., Yan, J., Chen, S.-C. (2006): Automatic construction of building footprintsfrom airborne LIDAR data. IEEE Trans. Geosci. Remote Sens., 44(9), 2523–2533.
4. Schumann, A., Arndt, D., Wiatr, T., Götz, A. E., Hoppe, A. (2011): High-resolutionterrestrial laser scanning and 3D modelling of a mineral deposit for extraction man-agement optimisation. Z. Deutsch. Gesellschaft Geowissenschaft., 162(4), 435–442.
5. Kelly, M., Di Tommaso, S. (2015): Mapping forests with Lidar provides flexible, accu-rate data with many uses. California Agricult., 69(1), 14–20.
6. Gouveia, B. D., Portugal, D., Silva, D. C., Marques, L. (2015): Computation sharing indistributed robotic systems: a case study on SLAM. IEEE Trans. Autom. Sci. Eng., 12(2),410–422.
7. Lidar: driving the future of autonomous navigation. Frost & Sullivan, CA, USA, 2016.8. Rasshofer, R. H., Gresser, K. (2005): Automotive radar and lidar systems for next
generation driver assistance function. Adv. Radio Sci., 3, 205–209.9. Gotzig, H., Geduld, G., (2015): LIDAR-sensorik. In Handbuch Fahrerassistenzsysteme:
Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort (S. 317–334). Berlin: Springer.
10. Laux, T. E., Chen, C.-I. (2014): 3D flash lidar visiton systems for imaging in degradedvisual environments. In Degraded visual environments: enhanced, synthetic, and ex-ternal vision solution. Proc. SPIE (Vol. 9087).
11. Wang, H., Wang, B., Liu, B., Meng, X., Yang, G. (2017): Pedestrian recognition andtracking using 3D LiDAR for autonomous vehicle. Robot. Auton. Syst., 88, 71–78.
12. Himmelsbach, M., Müller, A., Lüttel, T., Wünsche, H.-J. (2008): LIDAR-based 3D objectperception. In Proceedings of 1st international workshop on cognition for technicalsystems.
13. Hata, A., Wolf, D. (2014): Road marking detection using lidar reflective intensity deataand its application to vehicle localization. In IEEE17th international conference onintelligent transportation system, Qingdau, China.
14. Poczter, S. L., Jankovic, L. M. (2014): The Google car: driving toward a better future?J. Business Case Stud., 10(1), 7–14.
15. Chauffeur Inklusive: BMW Active Assist, BMW [Online]. Available:http://www.bmw.com/ [Accessed 3 Feb. 2017].
16. Ross, P. E. (11 May 2016): Volvo’s self-driving program will have redundancy for ev-erything. IEEE Spectrum.
17. Ackerman, E. (17 August 2016): Ford and Baidu invest $150 million in velodyne foraffordable lidar for self-driving cars. IEEE Spectrum.
18. Thakur, R. (2016): Scanning LIDAR in advanced driver assistance systems and beyond.IEEE Cosum. Electron. Mag., 5(3), 48–54.
19. LeddarTech Inc., (2015): Leddar M16 multi-element sensor module.20. Continental, SRL 1 + SRL 1C infrrot short range lidar sensor (2009).21. Toth, C. K. (2009): R&D of mobile lidar mapping and future trends. In Annual confer-
ence ASPRS, Baltimore, Maryland, US.22. Velodyne (2017): Data sheet: puck real time 3D LiDAR sensor.23. Quanergy Systems, Inc. (2015): 360◦ 3D LIDAR M8-1 sensor.24. Ibeo Automotive Systems GmbH (2014): User manual Ibeo ScaLa B2® laserscanner.
H. W. Yoo et al. MEMS-based lidar for autonomous driving ORIGINALARBEIT
25. Stone, W. C., Juberts, M., Daglakis, N., Stone, J., Gorman, J. (2004): Performance anal-ysis of next generation LADAR for manufacturing, construction, and mobility. Unitedstates department of commerce.
26. Ullrich, A., Pfennigbauer, M., Rieger, P. (2013): How to read your LIDAR spec – acomparison of single-laser-output and multi-laser-output LIDAR instruments. In Riegllaser measurement systems GmbH.
27. Stutz, G. E. (2012): Polygon scanners: components, performance, and design. In G. F.Marshall, G. E. Stutz (Eds.), Handbook of optical and laser scanning (pp. 247–279).Boca Raton: CRC Press.
28. Fersch, T., Weigel, R., Koelpin, A. (2017): Challenges in miniaturized automotive long-range lidar system design. In Three-dimensional imaging, visualization, and display,Anaheim, CA, USA. Proc. SPIE (Vol. 10219).
29. Yaacobi, A., Sun, J., Moresco, M., Leake, G., Coolbaugh, D., Watts, M. R. (2014):Integrated phase array for wide angle beam. Opt. Lett., 39(15), 4575.
30. Sun, J., Timurdogan, E., Yaacobi, A., Hosseini, E. S., Watts, M. R. (2013): Large-scalenanophotonic phase array. Nature, 493, 195–199.
31. Ackerman, E. (7 January 2016): Quanergy announces $250 solid-state LIDAR for cars,robots, and more. IEEE Spectrum.
32. Innoviz Technologies (2018): Data sheet of InnovizPro – high performance solid-stateLiDAR.
33. Mizuno, T., Mita, M., Kajikawa, Y., Takeyama, N., Ikeda, H., Kawahara, K. (2008):Study of two-dimensional scanning LIDAR for planetary explorer. In Sensors, systems,and next-generation satellites XII.
34. Park, I. H., Jeon, J. A., Nam, J., Nam, S., Lee, J., Park, J. H., Yang, J., Ebisuzaki, T.,Kawasaki, Y., Takizawa, Y., Wada, S. (2009): A new LIDAR method using MEMS mi-cromirror array for the JEM-EUSO mission. In Proceedings of the 31st ICRC.
35. Moss, R., Yuan, P., Bai, X., Quesada, E., Sudharsanan, R., Stann, B. L., Dammann, J.F., Giza, M. M., Lawler, W. B. (2012): Low-cost compact MEMS scanning ladar systemfor robotic applications. In Laser radar technology and applications XVII. Proc. SPIE(Vol. 8379).
36. Sudharsanan, R. (2013): Low cost scanning lidar imager. LIDAR Mag., 3(2).37. Hu, Q., Pedersen, C., Rodrigo, P. J. (2016): Eye-safe diode laser Doppler lidar with a
nant 7 mm-MEMS mirror for automotive LIDAR application.39. E. T. P. o. S. S. I. (EPoSS) (2015): European roadmap smart systems for automated
driving.40. Stann, B. L., Dammann, J. F., Giza, M. M. (2016): Progress on MEMS-scanned ladar.
In Laser radar technology and applications XXI. Proc. SPIE (Vol. 9832).41. Ito, K., Niclass, C., Aoyagi, I., Matsubara, H., Soga, M., Kato, S., Maeda, M., Kagami,
M. (2013): System design and performance characterization of a MEMS-based laserscanning time-of-flight sensorbased on a 256x64-pixel single-photon imager. IEEEPhotonics J., 5(2), 6800114.
42. Giese, T., Janes, J. (2015): 2D MEMS scanning for LIDAR with sub-Nyquist sampling,electronics, and measurement procedure. In Three-dimensional imaging, visualization,and display. Proc. SPIE (Vol. 9495).
43. Kim, G., Park, Y. (2016): LIDAR pulse coding for high resolution range imaging atimproved refresh rate. Opt. Express, 24(21), 23810–23828.
44. Holmström, S. T. S., Baran, U., Urey, H. (2014): MEMS laser scanners: a review. J.Microelectromech. Syst., 23(2), 259–275.
45. Agarwal, M., Park, K. K., Candler, R. N., Kim, B., Hopcroft, M. A., Chandorkar, S. A.,Jha, C. M., Melamud, R., Kenny, T. W., Murmann, B. (2006): Nonlinear characterizationof electrostatic MEMS resonators. In IEEE international frequency control symposiumand exposition, Miami, FL, US.
46. Milanovic, V., Kasturi, A., Yang, J., Hu, F. (2017): Closed-loop control of gimbal-lessMEMS mirrors for increased bandwidth in LiDAR applications. In Laser radar technol-ogy and applications, Anaheim,CA, US. Proc. SPIE (Vol. 10191).
47. Milanovic, V., Kasturi, A., Yang, J., Su, Y. R., Hu, F. (2018): Iterative learning control(ILC) algorithm for greatly increased bandwidth and linearity of MEMS mirrors in Li-DAR and related imaging applications. In MOEMS and miniaturized systems XVII, SanFrancisco, CA, US. Proc. SPIE (Vol. 10545).
48. Lee, X., Wang, C. (2015): Optical design for uniform scanning in {MEMS}-based 3Dimaging lidar. Appl. Opt., 54(9), 2219–2223.
49. Yang, B., Zhou, L., Zhang, X., Koppal, S., Xie, H. (2017): A compact MEMS-basedwide-angle optical scanner. In 2017 international conference on optical MEMS andnanophotonics.
50. Zhang, X., Koppal, S. J., Zhang, R., Zhou, L., Butler, E., Xie, H. (2016): Wide-anglestructured light with a scanning MEMS mirror in liquid. Opt. Express, 24(4), 256226.
51. Ataman, Ç., Lani, S., Noell, W., Rooij, N. D. (2013): A dual-axis pointing mirror withmoving-magnet actuation. J. Micromech. Microeng., 23(2), 025002.
52. Ye, L., Zhang, G., You, Z. (2017): 5 V compatible two-axis PZT driven MEMS scanningmirror with mechanical leverage structure for miniature LiDAR application. Sensors,17(3), 521.
53. Smith, B., Hellman, B., Gin, A., Espinoza, A., Takashima, Y. (2017): Single chip li-dar with discrete beam steering by digital micromirror device. Opt. Express, 25(13),14732–14745.
54. Wang, Y., Kyoungsik, Y., Wu, M. C. (2016): MEMS optical phased array for LIDAR. InThe 21st microoptics conference (MOC), Berkeley, CA, US.
55. Sandner, T., Wildenhain, M., Gerwig, C., Schenk, H., Schwarzer, S., Wölfelschneider,H. (2010): Large aperture MEMS scanner module for 3D distance measurement. InMOEMS and miniaturized systems IX. Proc. SPIE (Vol. 7594).
56. Wolter, A., Hsu, S.-T., Schenk, H., Lakner, H. K. (2005): Applications and requirementsfor MEMS scanner mirrors. In MOEMS and miniaturized systems V, San Jose, CA, US.Proc. SPIE (Vol. 5719).
57. Milanovic, V., Kasturi, A., Yang, J. (2016): Novel fluidic packaging of gimbal-less MEMSmirrors for increased optical resolution and overall performance. In SPIE defense andcommercial sensing conference, Baltimore, MD, US.
Authors
Han Woong Yoowas born in Seoul, Republic of Korea. Af-ter his bachelor study at Yonsei University, hefinished the master study in electrical engi-neering at Seoul National University in 2007.He worked in Samsung Advanced Instituteof Technology (SAIT) and Samsung Electron-ics co. LTD, semiconductor business for lowpower digital RF and algorithms for reliabilityof multi-level non-volatile memories. In 2015
he received a Ph.D. from Delft University of Technology, Nether-lands, about adaptive optics and optomechatronics for confocal mi-croscopy. Currently, he is working at the Institute auf Automation anControl (ACIN) at Vienna University of Technology, Austria, as a post-doctoral researcher. His main research interests are optical metrol-ogy, precision mechatronics, and biomedical imaging.
Norbert Drumlwas born in Klagenfurt, Austria in 1980. Af-ter being self-employed for ten years in thefield of embedded systems development, hereceived a master’s degree in telematics anda doctoral degree in electrical engineering,both from Graz University of Technology inAustria. In 2014, he joined Infineon Technolo-gies Austria AG, where he was leading sev-eral industrial research projects (in the fields
of embedded systems, sensors, and security) and where he workedas a concept engineer for the next generation sensor chips. His re-search interests include hardware/software co-design, secure em-bedded systems, and automotive sensing technology.
ORIGINALARBEIT H. W. Yoo et al. MEMS-based lidar for autonomous driving
David Brunnerwas born in Waidhofen an der Thaya, Austriain 1990. After working 5 years at SiemensAG as a working student and 1 year at Vi-enna University of Technology as a studentassistant, he received the master’s degree inenergy engineering and automation technol-ogy. In 2017 he started his doctoral studiesin electrical engineering at TU Vienna, wherehe is working on the control concept of a
MEMS-based LiDAR system for autonomous driving. His research in-terests include advanced identification and control concepts, opto-mechatronic systems and system integration.
Christian Schwarzlstarted his career after finishing the informa-tion and computer engineering studies as re-searcher at the Virtual Vehicle Research Cen-ter in Graz, Austria. He began his computersciences dissertation in parallel, which he fin-ished with honours in 2012. He became LeadResearcher in 2013 and managed the activi-ties in the field of verification and validation.Since 2014 he has been head of the Depend-
able Systems Group, which primarily focusses on the verification andvalidation of autonomous vehicles, functional safety and security.Since 2016 he has been active member of the ISO/TC22/SC 32/WG8 committee for functional safety and contributes to standards likeISO 26262 and ISO PAS 21448 – Safety of the intended functionality.
Thomas Thurnerfinished his studies in electrical engineeringat the Graz University of Technology in 1999,focusing on optical metrology and measure-ment signal processing during the followingPh.D. studies in technical sciences and post-doc academic research at the Institute of Elec-trical Measurement and Measurements Sig-nal Processing at the Graz University of Tech-nology. From 2008 to 2015 Dr. Thurner was
heading the Fatigue Testing Facility at the Graz University of Technol-ogy with strong involvement in research on measurement principlesfor mechanical quantities and with the development and optimiza-tion of multi-channel mechanical test control systems. In May 2015Dr. Thurner joined Infineon Technologies Austria at the Graz Devel-opment Center, where he heads the Component Verification groupfor the Automotive Sense & Control Graz Department.
Marcus Henneckestudied electrical engineering at the Technis-che Universität Darmstadt, Germany, com-pleting his studies in 1991 with a diplomadegree. In 1992 he graduated from StanfordUniversity with a Master of Science degree inthe area of neural networks and continued asdoctoral student on the topic of computer-aided lip reading. In 1996 he completed histhesis and moved to Ulm in Germany where
he worked on an automated spoken language translator at theDaimler-Benz research center until 1998. In the years to come un-til 2006 he worked on automated speech recognition for automo-
tive dialog systems for the companies Daimler-Benz Aerospace AG,TEMIC Telefunken and Harman Backer as head of the departmentsalgorithms, research and speech recognition, respectively. In 2006he then moved to Graz, Austria where he headed the business seg-ment Pattern Recognition and Image Processing (PRIP) at the Bio-metric Center of Siemens up to 2009.From 2009 to 2016 he worked for EFKON AG as head of thedepartments Frontend and Machine Vision. Since November 2016Mr. Hennecke has been heading the Concept Engineering depart-ment at the Design Center Graz of Infineon Technologies. In hisprofessional career Mr. Hennecke has initiated and executed vari-ous successful national (German and Austrian) as well as Europeanfunded projects. He is (co-)author of numerous publications andpatents.
Georg Schitteris Professor for Advanced Mechatronic Sys-tems at the Automation and Control Institute(ACIN) of TU Wien. He received a M.Sc. inelectrical engineering from TU Graz, Austria(2000) and an M.Sc. and Ph.D. degree fromETH Zurich, Switzerland (2004). His primaryresearch interests are on high-performancemechatronic systems, particularly for applica-tions in the high-tech industry, scientific in-
strumentation, and mechatronic imaging systems, such as AFM,scanning laser and LIDAR systems, telescope systems, adaptive op-tics, and lithography systems for semiconductor industry. He re-ceived the IFAC Mechatronics best paper award (2008 to 2011) and2013 IFAC Mechatronics Young Researcher Award, and served as anAssociate Editor for IFAC Mechatronics, Control Engineering Prac-tice, and for the IEEE Transactions on Mechatronics.
