Date post: | 15-Jan-2016 |
Category: |
Documents |
Upload: | elwin-clark |
View: | 216 times |
Download: | 0 times |
October 5, 2006. 1Memorial University of Newfoundland
Memorial University of Newfoundland
Faculty of Engineering & Applied Science
Engineering 6806
Electrical & Computer Engineering
Design Project
INTRODUCTION TO MACHINE VISION II
Prof. Nick Krouglicof
October 5, 2006. 2Memorial University of Newfoundland
Presentation Outline
1. Machine vision systems for mechanical metrology
• A systematic approach to the calibration of machine vision systems for industrial metrology
• An automatic, machine vision based system for robot calibration
2. Industrial Applications of Machine Vision
• High speed, line scan camera-based inspection system for the food processing industry
• Vision based inspection of liquid crystal display (LCD) modules
3. CPLD based interfaces for sensors and actuators
October 5, 2006. 3Memorial University of Newfoundland
A systematic approach to the calibration ofmachine vision systems for industrial metrology
• In the field of machine vision, camera calibration refers to the experimental determination of a set of parameters which describe the image formation process for a given analytical model of the machine vision system.
• Ideally, camera calibration is performed without specialized optical equipment, without modifications to the hardware, and without a priori knowledge of the vision system.
• Most calibration techniques are based on the observation of planar (2D) targets with a large number of control points.
October 5, 2006. 4Memorial University of Newfoundland
A systematic approach to the calibration ofmachine vision systems for industrial metrology
The machine vision parameters which must be identified include :
a) The scale factor b) The frame buffer coordinates of the image center c) The effective focal length of the lens-camera assemblyd) The radial lens distortion coefficiente) The pose (position and orientation) of the camera
Parameters a) through d) are classified as intrinsic, e) as extrinsic.
October 5, 2006. 5Memorial University of Newfoundland
A systematic approach to the calibration ofmachine vision systems for industrial metrology
October 5, 2006. 6Memorial University of Newfoundland
A systematic approach to the calibration ofmachine vision systems for industrial metrology
October 5, 2006. 7Memorial University of Newfoundland
A systematic approach to the calibration ofmachine vision systems for industrial metrology
In general, most camera calibration techniques fail because:
•They fail to account for the mathematical dependency between the image center and the X-Y position of the camera.
•The determination of the intrinsic parameters assumes a priori knowledge of the extrinsic parameters (i.e. position and orientation of the camera). Therefore, the extrinsic parameters are generally determined first.
•In order to accurately evaluate the intrinsic parameters, the extrinsic parameters must be know to a degree of accuracy which is practically unattainable (6 significant digits).
October 5, 2006. 8Memorial University of Newfoundland
A systematic approach to the calibration ofmachine vision systems for industrial metrology
In order to compensate for the deficiencies of existing calibration techniques, a special fixture was designed to evaluate the critical parameters (i.e. image center and effective focal length) before the remaining parameters.
October 5, 2006. 9Memorial University of Newfoundland
• To locate the edge with sub-pixel accuracy, the gradient in the vicinity of the edge is reconstructed using a continuous Gaussian function.
• The coordinates of the pixel nearest to the edge are denoted by i and j. The location of the edge with sub-pixel accuracy is determined by calculating the offsets x and y which maximize the function s(x,y) below over a 5 by 5 kernel :
en)+j m,+G(i = y)s(x, )n-(y+)m-(x2
1-
2
-2=n
2
-2=m
22
A systematic approach to the calibration ofmachine vision systems for industrial metrology
October 5, 2006. 10Memorial University of Newfoundland
An automatic, machine vision basedsystem for robot calibration
Objective: To develop a low cost, non-contact robot calibration system that can be applied on a routine basis without significantly affecting production schedules.
Based on a CCD camera mounted on the robot’s wrist. The camera is used to determine the position and orientation of a fixed, passive target in six degrees-of-freedom.
The system employs the Hayati and Mirmirani modeling convention for closed kinematic chains and non-linear least squares analysis to determine the real kinematic parameters specific to a particular robot.
October 5, 2006. 11Memorial University of Newfoundland
An automatic, machine vision basedsystem for robot calibration
October 5, 2006. 12Memorial University of Newfoundland
An automatic, machine vision basedsystem for robot calibration
October 5, 2006. 13Memorial University of Newfoundland
An automatic, machine vision basedsystem for robot calibration
October 5, 2006. 14Memorial University of Newfoundland
Industrial Machine Vision Applications
October 5, 2006. 15Memorial University of Newfoundland
High Speed, Line Scan Based Inspection System
October 5, 2006. 16Memorial University of Newfoundland
•Objective: To remove defects (I.e. visible, dark particle larger than 0.007”) from apple sauce
•System must be able to handle 12 metric tons per 8 hour shift
•System must remove 95% of visible defects
Line Scan Based Inspection System: Specifications
October 5, 2006. 17Memorial University of Newfoundland
•2 distinct challenges:
•Detection
•Removal
High Speed, Line Scan Based Inspection System
October 5, 2006. 18Memorial University of Newfoundland
High Speed, Line Scan Based Inspection System
October 5, 2006. 19Memorial University of Newfoundland
High Speed, Line Scan Based Inspection System
October 5, 2006. 20Memorial University of Newfoundland
•Detection system based on a high performance line scan camera; 4096 pixels per line at 4800 lines per second.
•Image acquisition and processing functions implemented on a Complex Programmable Logic Device (CPLD) as opposed to a microprocessor or Digital Signal Processor (DSP).
•The objective is to implement image processing functions with dedicated logic gates (i.e. hardware) for real-time performance.
Line Scan Based Detection System
October 5, 2006. 21Memorial University of Newfoundland
What are CPLDs?
•Complex Programmable Logic Devices (CPLDs) are a class of programmable logic device that are commonly used to implement complex digital designs on a single integrated circuit.
•Applications of CPLDs in the field of computer engineering include the implementation of bus controllers, address decoders, priority encoder and state machines
October 5, 2006. 22Memorial University of Newfoundland
Line Scan Camera-Based Detection System
October 5, 2006. 23Memorial University of Newfoundland
Line Scan Camera-Based Detection System
October 5, 2006. 24Memorial University of Newfoundland
Line Scan Camera-Based Detection System
Typical Particle
Typical section of apple sauce recorded with an area scan camera
October 5, 2006. 25Memorial University of Newfoundland
LINESCAN DATA
20
40
60
80
100
120
1 101 201 301 401 501 601 701
Pixel Number
Lig
ht
Lev
el
Line Scan Camera-Based Detection System
October 5, 2006. 26Memorial University of Newfoundland
Line Scan Camera-Based Detection System
October 5, 2006. 27Memorial University of Newfoundland
Removal System
October 5, 2006. 28Memorial University of Newfoundland
Removal System
October 5, 2006. 29Memorial University of Newfoundland
Removal System: Flow Characterization
•Rheological nomenclature and associated velocity profiles for steady flow through tubes with circular cross section.
October 5, 2006. 30Memorial University of Newfoundland
Viscosity Measurement
October 5, 2006. 31Memorial University of Newfoundland
Viscosity Measurement
Velocity profile can be characterized by power law!
October 5, 2006. 32Memorial University of Newfoundland
Flow Profile of Apple Sauce
October 5, 2006. 33Memorial University of Newfoundland
Particle Removal Window
October 5, 2006. 34Memorial University of Newfoundland
System Timing Diagram
October 5, 2006. 35Memorial University of Newfoundland
Aspiration Valve Characterization
October 5, 2006. 36Memorial University of Newfoundland
Aspiration Valve Characterization
October 5, 2006. 37Memorial University of Newfoundland
Particle Detection Rate Versus Flowrate
October 5, 2006. 38Memorial University of Newfoundland
High Speed, Line Scan Based Inspection System:Current Status
•Industrial partner is currently developing the production version of the system.
•Packaging of the principle components (i.e., lenses, cameras, electronics, light sources) remains a major challenge given the environment.
•One possible solution is to integrate all the electronics in the camera enclosure.
•Partner is anxious to explore applications in the pulp and paper industry.
October 5, 2006. 39Memorial University of Newfoundland
Vision Based Inspection of LiquidCrystal Display (LCD) modules
•Objective: to automate the inspection of LCD modules in order to improve quality control
•One step in the implementation of a Six-Sigma Program (“3.4 defects per million opportunities”)
•The inspection must be completed within 30 seconds for 10 predetermined LCD patterns
•System can “learn” new LCD modules without modifying software
October 5, 2006. 40Memorial University of Newfoundland
Vision Based Inspection of LCD modules: System Components
•Pulnix camera with macro lens
•High frequency fluorescent light sources
•Coreco Bandit integrated image acquisition and VGA accelerator
•Software developed using with WiT graphical programming environment in combination with Microsoft VB
October 5, 2006. 41Memorial University of Newfoundland
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 42Memorial University of Newfoundland
Original Image Showing Error in Alignment
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 43Memorial University of Newfoundland
Thresholding Operation – Image Subtraction with respect to an image with no segments
illuminated
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 44Memorial University of Newfoundland
Blob Analysis – Reference Points are Identified
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 45Memorial University of Newfoundland
Image Rotation and Translation
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 46Memorial University of Newfoundland
Pixel by Pixel Image Subtraction from Reference Image – Thinning Operator
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 47Memorial University of Newfoundland
Blob Analysis
Vision Based Inspection of LiquidCrystal Display (LCD) modules
October 5, 2006. 48Memorial University of Newfoundland
Blob Analysis
Vision Based Inspection of LiquidCrystal Display (LCD) modules: Conclusions
•Inspection system was installed at BAE Systems Canada Ltd. where it was used to test between 200 and 600 LCD displays per day.
•Number of defective modules that passed inspection was basically reduced to zero.
•Occasional “false positives” proved to be technical problems with the devices that previously went unreported.
•Applications for this technology are numerous given the number of LCD displays produced annually.
October 5, 2006. 49Memorial University of Newfoundland
CPLD based interfaces for smart
sensors and actuators
October 5, 2006. 50Memorial University of Newfoundland
Introduction
•Several novel “mechatronic” applications for CPLDs are presented.
•CPLD basically provides reconfigurable digital I/O that permits the implementation of interfaces for smart sensors and actuators.
•Applications that will be discussed include:
•Stepper motor controller
•Quadrature decoder/counter for optical encoder
•Pulse-Width Modulation (PWM) motor drives
•By incorporating a CPLD that supports In-System Programmability (ISP) the target device can be reprogrammed by the user for a variety of applications without removing it from the host system.
October 5, 2006. 51Memorial University of Newfoundland
Mechatronic Applications of CPLDs – RSG System
October 5, 2006. 52Memorial University of Newfoundland
Design Objectives of RSG System
To develop a low cost, microcontroller-based system that allows the user to interface a variety of sensors and actuators to a laptop computer. In addition to being “user- friendly”, the system must be capable of functioning as either:
1. A portable data acquisition system,2. An autonomous, stand-alone controller in support of
mechatronic design projects,3. A data logger that can acquire and store data at
regular intervals over extended periods of time.
October 5, 2006. 53Memorial University of Newfoundland
Design Philosophy
Embedded Controller• System incorporates an 80C52-BASIC
Microcontroller• Simplifies the management of the various
onboard peripherals (Analog-to-Digital converter, LCD display, serial interface)
• Incorporates a powerful, imbedded BASIC interpreter
• Can also be programmed in assembler or C (Small Device C Compiler: SDCC)
• Assembled/compiled code can be “called” from Basic
October 5, 2006. 54Memorial University of Newfoundland
Design Philosophy (continued)
Programmable Logic• System incorporates a user configurable
Complex Programmable Logic Device (CPLD)• CPLD supports in-system programmability by
the user• Mechatronic applications envisioned for the
CPLD on the UC2 system include quadrature decoder/counter interfaces for optical encoders, stepper motors controllers, PWM motor drives, sequential controllers, event counters/timers, Serial Peripheral Interface (SPI), etc.
October 5, 2006. 55Memorial University of Newfoundland
80C52-BASICMICROCONTROLLER
32K X 8 EEPROM
32K X 8 SRAM
12 / 16 BIT ATOD CONVERTER
COMPLEX PROGRAMMABLE LOGIC
DEVICE
16 CHARACTERLCD DISPLAY
RS232PORT
JTAGPORT
DIGITAL I/O(18 LINES)
ANALOG IN(4 CHANNELS)
DATA BUS
ADDRESS BUS
CONTROL LINES
Block Diagram of RSG System
October 5, 2006. 56Memorial University of Newfoundland
Stepper Motor Controller: Synchronous Counter Design
October 5, 2006. 57Memorial University of Newfoundland
Stepper Motor - Full-Stepping
October 5, 2006. 58Memorial University of Newfoundland
Stepper Motor – Half Stepping
October 5, 2006. 59Memorial University of Newfoundland
Stepper Motor Half-Step Switching Sequences
STEPTRANSISTORS
1 2 3 4
1 ON OFF ON OFF
2 ON OFF OFF OFF
3 ON OFF OFF ON
4 OFF OFF OFF ON
5 OFF ON OFF ON
6 OFF ON OFF OFF
7 OFF ON ON OFF
8 OFF OFF ON OFF
October 5, 2006. 60Memorial University of Newfoundland
Truth Table for Half-Stepping Switching Sequence
October 5, 2006. 61Memorial University of Newfoundland
Stepper Motor: Synchronous Counter Design
October 5, 2006. 62Memorial University of Newfoundland
Schematic of stepper motor controller
Stepper Motor: Synchronous Counter Design
October 5, 2006. 63Memorial University of Newfoundland
Quadrature Decoder/Counter Interface for Optical Encoder: State Machine
October 5, 2006. 64Memorial University of Newfoundland
Optical Encoder
October 5, 2006. 65Memorial University of Newfoundland
Optical Encoder - Decoding
October 5, 2006. 66Memorial University of Newfoundland
Optical Encoder – State Diagram
October 5, 2006. 67Memorial University of Newfoundland
Optical Encoder – Edge Detection Circuitry
October 5, 2006. 68Memorial University of Newfoundland
Summary of Quadrature Decoding Logic
State TransitionEncoder Signals
Flipflop outputs: enc_decX.q
PastState
PresentState
CountChan
AChan
B0 1 2 3
1 2 UP 1 1 1 1 0
2 3 UP 1 0 1 1 1
3 4 UP 0 0 0 0 1
4 1 UP 0 1 0 0 0
1 4 DOWN 0 0 1 0 0
4 3 DOWN 0 0 0 1 0
3 2 DOWN 1 1 0 1 1
2 1 DOWN 1 1 1 0 1
Note: and indicate a rising and falling edge respectively.
October 5, 2006. 69Memorial University of Newfoundland
Truth Table Implementation of Quadrature Decoder
TABLEenc_dec[3..0].q => dwn_cnt, up_cnt;B"0000" => 0, 0;B"0001" => 0, 1;B"0010" => 1, 0;B"0011" => 0, 0;B"0100" => 1, 0;B"0101" => 0, 0;B"0110" => 0, 0;B"0111" => 0, 1;B"1000" => 0, 1;B"1001" => 0, 0;B"1010" => 0, 0;B"1011" => 1, 0;B"1100" => 0, 0;B"1101" => 1, 0;B"1110" => 0, 1;B"1111" => 0, 0;
END TABLE;
October 5, 2006. 70Memorial University of Newfoundland
AHDL Implementation of Counter Module
IF up_cnt THENencoder[].d = encoder[].q + 1;
ELSEIF dwn_cnt THEN
encoder[].d = encoder[].q - 1;ELSE
encoder[].d = encoder[].q;END IF;
END IF;
October 5, 2006. 71Memorial University of Newfoundland
Pulse-Width Modulation: VHDL/AHDL Programming
October 5, 2006. 72Memorial University of Newfoundland
AHDL Implementation of PWM Controller
totaltime[].clk = clk;totaltime[].clrn = VCC;totaltime[].prn = VCC;totaltime[].ena = io_space_2 & (add_latch[3..0].q == B"0110") & !write/; totaltime[].d = add_data[];
lowtime[].clk = clk;lowtime[].clrn = VCC;lowtime[].prn = VCC;lowtime[].ena = io_space_2 & (add_latch[3..0].q == B"0111") & !write/;lowtime[].d = add_data[];
October 5, 2006. 73Memorial University of Newfoundland
cntr[].clk = clk;cntr[].clrn = !(cntr[].q == totaltime[].q);cntr[].prn = VCC;cntr[].ena = VCC;cntr[].d = cntr[].q + 1;
switch.clk = clk;switch.clrn = VCC;switch.prn = VCC;switch.ena = VCC;pwm_out = switch.q;switch.d = (cntr[].q < lowtime[].q);
AHDL Implementation of PWM Controller
October 5, 2006. 74Memorial University of Newfoundland
Pedagogical Applications
October 5, 2006. 75Memorial University of Newfoundland
Pedagogical Applications
October 5, 2006. 76Memorial University of Newfoundland
Conclusions
•CPLDs are well suited to the implementation of interfaces for smart sensors and actuators.
•Incorporating a CPLD that supports In-System Programmability (ISP) into a data acquisition system provides reconfigurable digital I/O that allows the target system to be reprogrammed by the user for a variety of applications without hardware modifications.
•This is particularly useful for those applications that require hardware interfaces for several different sensors simultaneously.
October 5, 2006. 77Memorial University of Newfoundland
QUESTIONS
?