Control software for positioning and autofocus in an ... · Control software for positioning and...

UPTEC X 11 041

Examensarbete 30 hpSeptember 2011

Control software for positioning and autofocus in an optical system for biomolecule analysis

Anders Vestberg

Bioinformatics Engineering Program Uppsala University School of Engineering

UPTEC X 11 041 Date of issue 2011-09 Author

Anders Vestberg Title (English)

Control software for positioning and autofocus in an optical system for biomolecule analysis

Title (Swedish) Abstract Q-linea AB develops biomolecular technologies and analysis instruments for high-performance DNA and protein analysis. Based on an ASMD-platform (Amplified Single Molecule Detection), such an analysis instrument is able to perform rapid and sensitive detection and identification of microorganisms (e.g. biological warfare agents). The work presented in this master thesis is an implementation of control software for the detection unit of the analysis instrument. In particular, the software will control a motorized positioning system and apply image analysis for autofocusing of an optical system.

Keywords biomolecule analysis, positioning, image analysis, autofocus, object-oriented design

Supervisors Johan Stenberg

Q-linea

Scientific reviewer Olle Eriksson

Department of Information Technology, Uppsala University Project name

Sponsors

Language

English

Security Secret until 2013-09

ISSN 1401-2138

Classification

Supplementary bibliographical information Pages 83

Biology Education Centre Biomedical Center Husargatan 3 Uppsala Box 592 S-75124 Uppsala Tel +46 (0)18 4710000 Fax +46 (0)18 471 4687

Control software for positioning and autofocus in an opticalsystem for biomolecule analysis

Anders Vestberg

Popularvetenskaplig sammanfattning

Q-linea AB utvecklar molekylarbiologiska tekniker och hogpresterande instrument for analys avDNA- och protein. Q-lineas analysinstrument baseras pa en egenutvecklad molekylarbiologiskplattform, ASMD (Amplified Single Molecule Detection), som mojliggor en precis detektionav oerhort sma mangder biomolekyler. I forlangningen innebar detta att man kan, med hjalp avanalysinstrumentet, identifiera specifika mikroorganismer i vatten- och/eller luftprov. En sadananalys kan tillampas inom en rad olika omraden, exempelvis diagnostik eller identifiering avbiologiska stridsmedel.

I detta examensarbete utvecklas ett styrsystem till detektionsenheten i Q-lineas analysinstru-ment. Detektionsenhetens uppgift ar att registrera och analysera det ljus som reflekteras franett (fluorescens-preparerat) prov nar det belyses med laser. Detta arbete innefattar huvudsakli-gen utveckling av viss mjukvara vars uppgift ar att styra ett motoriserat positioneringssystemoch analysera bilddata for autofokusering av ett optiksystem. Mjukvaran baseras pa en objekt-orienterad design och implementeras som en server da systemet maste kunna fjarrstyras.

Civilingenjorsprogrammet Bioinformatik, 2011-09

Contents

Acknowledgements iii

List of Figures iv

List of Tables v

List of abbreviations vi

1 Introduction 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Background 32.1 Analysis Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 Amplified Single-molecule Detection . . . . . . . . . . . . . . . . . . 32.1.2 Optical System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Autofocus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2.1 Image Processing/Analysis . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Implementation 153.1 Software Requirements Specifications . . . . . . . . . . . . . . . . . . . . . . 15

3.1.1 Product Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.1.1.1 System Interfaces . . . . . . . . . . . . . . . . . . . . . . . 163.1.1.2 User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . 163.1.1.3 Hardware Interfaces . . . . . . . . . . . . . . . . . . . . . . 163.1.1.4 Software Interfaces . . . . . . . . . . . . . . . . . . . . . . 173.1.1.5 Communications Interfaces . . . . . . . . . . . . . . . . . . 17

3.1.2 Product Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.1.2.1 Logging System . . . . . . . . . . . . . . . . . . . . . . . . 183.1.2.2 Daemon Process . . . . . . . . . . . . . . . . . . . . . . . . 18

i

3.1.2.3 Configurable . . . . . . . . . . . . . . . . . . . . . . . . . . 183.1.3 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2 Software Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.2.1 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2.1.1 Presentation Layer . . . . . . . . . . . . . . . . . . . . . . . 193.2.1.2 Business Object Layer . . . . . . . . . . . . . . . . . . . . . 203.2.1.3 Hardware Communication Layer . . . . . . . . . . . . . . . 203.2.1.4 Common Application Framework . . . . . . . . . . . . . . . 21

3.3 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.4 Implementation Technicalities . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Evaluation 234.1 Autofocus Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Application Screenshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5 Summary and Conclusions 30

References 32

Appendices 35

A Client-Server Protocol 35A.1 DCP TCP Communication Protocol . . . . . . . . . . . . . . . . . . . . . . . 35A.2 Commands and responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

B Server-MC Protocol 37B.1 Control of detection cell positioning control . . . . . . . . . . . . . . . . . . . 37B.2 Commands and responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

C Implementation Details 40

ii

Acknowledgements

I would like to thank my supervisor Johan Stenberg for guidance and support throughout theproject. I would also like to thank all the other people involved in this master thesis, especiallyMagnus Elgh and Carl Rosenfeld at Q-linea, for technical feedback on specific software andhardware issues.

iii

List of Figures

2.1 Mechanism of padlock probe ligation . . . . . . . . . . . . . . . . . . . . . . 42.2 Schematics of the Amplified Single-Molecule Detection technology . . . . . . 52.3 Florescence excitation pathway . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Florescence emission pathway . . . . . . . . . . . . . . . . . . . . . . . . . . 62.5 Comparison of pixel intensities . . . . . . . . . . . . . . . . . . . . . . . . . . 72.6 Schematics of the autofocus system in the detection unit. . . . . . . . . . . . . 82.7 Light is focused to a microchannel on the CD. . . . . . . . . . . . . . . . . . . 82.8 Focused image obtained by the autofocus module. . . . . . . . . . . . . . . . . 102.9 a) Defined ROI. b) Result after thresholding. . . . . . . . . . . . . . . . . . . . 112.10 a) As a result of a close/erode operation, the inner region of our AF structure has

been filled. b) Border-connected ojects have been removed. . . . . . . . . . . . 122.11 Resulting image after performing the edge detection operation. . . . . . . . . . 13

3.1 Simplified overview of DCP module, of which the DCPCS is a part. . . . . . . 163.2 DCPCS System architecture overview. . . . . . . . . . . . . . . . . . . . . . . 193.3 Concept of deadline for socket timeouts. . . . . . . . . . . . . . . . . . . . . . 21

4.1 Focus curve, showing the positions at which best focus is achived. . . . . . . . 244.2 Screenshots from startuping the server application and the test client. . . . . . . 254.3 The user issues a focus command in the test client and the server responds by

performing the operation (forwarding a translated command to hardware). . . . 264.4 The user issues a snapimage command in the test client and the server triggers

the camera to take a snapshot of the sample and send the image data back to theclient. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

iv

List of Tables

A.1 Command set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

B.1 Fault codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38B.2 Error codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38B.3 Command set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39B.4 Focus positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39B.5 Clamp positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

v

List of abbreviations

AF Autofocus

ASCII American Standard Code for Information Interchange

ASMD Amplified Single Molecule Detection

CCD Charge-coupled device

CR Carriage Return

COM COMunication (Serial Port)

DCP Detection Cell Positioning

DCPCS Detection Cell Positioning Control Software

RCA Rolling Circle Amplification

RCP Rolling Circle Product

ROI Region of Interest

SDD Software Design Document

SRS Software Requirements Specifications

TCP/IP Transmission Control Protocol/Internet Protocol

vi

1 Introduction

1.1 Introduction

The work presented in this master thesis has been performed at Q-linea AB in Uppsala. Q-linea,a spin-out of Olink Bioscience (and Uppsala University) is a biotechnology company specializedon development of technologies and systems for high-performance detection of biomolecules. Ofparticular interest for this thesis is an analysis instrument able to rapidly detect very low amountsnucleic acids and proteins. The great capability and potential of this instrument come from the Q-linea developed ASMD-technology[10] (Amplified Single Molecule Detection), which will bediscussed in section 2.1.1. Being able to effectively detect and count molecules, this technologyis useful in a range of fields, such as defense applications where infectious agents in water or aircan be detected, or in diagnostics.

1.2 Problem Description

In this master thesis, we implement control software for the detection unit of the analysis in-strument. In particular, the software will control a motorized positioning system and performimage processing for autofocusing in an optical system. The implementation should be basedon an object oriented design and utilizes a client-server model as the system must able to becontrolled remotely. The goal is to develop a stable and efficient software which will eventuallyreplace an existing LabView implementation of the control system. This is beneficial in manyways, including reduced licensing costs, greater flexibility and control.

1.3 Objectives

The primary goals of this thesis are:

• Implement a control software for the detection unit. Specifically, software controlling po-sitioning and autofocus. This includes:

1

1.4. THESIS STRUCTURE CHAPTER 1. INTRODUCTION

• Give a fully qualified description of all significant software requirements (Software Re-quirements Specifications).

• Describe architecture and system design (Software Design Document).

• Optimization, thorough testing and verification.

1.4 Thesis Structure

This thesis is outlined as follows. Chapter 2 provides a review of the analysis instrument, includ-ing some theory on the underlying biotechnology, necessary to get the big picture. The detectionunit and its optical system are described more in-depth and the chapter is concluded with someimage analysis and theory behind autofocus of cameras.

Chapter 3 first gives an overview of the required functionality of the control software, followedby the elaborated software design. Certain implementation details and technicalities are includedin the last section.

Evaluation results of the software’s performance and some testing of the autofocus algorithmare included in chapter 4.

The last chapter, chapter 5, summarizes the thesis with concluding remarks, possible enhance-ments and any related future work.

2

2 Background

2.1 Analysis Instrument

The analysis instrument introduced in section 1.1 consists of many units and different parts,of which the detection unit is of particular interest. The work of this thesis solely concernssoftware for the detection unit, thus there will be no in-depth descriptions of other aspects ofthe instrument. However, a brief introduction on the workflow of the instrument, from sample toresult, is beneficial for our understanding of the project.

2.1.1 Amplified Single-molecule Detection

In the detection process, a well-defined series of biotechnological steps, referred to as AmplifiedSingle-Molecule Detection [10], are performed. This technology provides very high sensitivityand efficiency for biomolecule detection and analysis. As described by Jarvius et al., this schemeallows for conversion of nanometer-scale molecules to micrometer-scale DNA molecules, highlysuitable for optical detection by using fluorescence techniques. The very first step is to extractDNA or protein from an air or water sample. Then, either through padlock probe ligation (fornucleic acids) or proximity ligation (for proteins), circular DNA molecules are generated uponrecognition of the target sequences.

When considering the padlock probe ligation mechanism, a padlock probe is a synthetic linearoligonucleotide, with both its 3’ and 5’ end sequences complementary to the target sequence.Upon hybridization of the probe to the correct target sequence, the probe ends will be positionedin a specific way1 that allows for DNA ligase to join the ends together, circularizing the probearound the target [14, 13, 1]. Figure 2.1 shows the principle of padlock probe ligation.

1The target-complementary ends of the probe are brought into juxtaposition, placing the ends close to each other.

3

2.1. ANALYSIS INSTRUMENT CHAPTER 2. BACKGROUND

Figure 2.1: Mechanism of padlock probe ligation

The requirement of two hybridization reactions, one at each end of the probe, for a success-fully circularized product, makes the padlock probe ligation technique very specific. A singlenucleotide mismatch between the target and one of the extreme ends of the probe may be suf-ficient to create a topology-distortion large enough to inhibit the enzymatic ligation reaction.For protein targets, the proximity ligation technique is utilized. It works in a similar fashion, butis slightly more sophisticated by using two affinity probes equipped with oligonucleotide arms[9, 5].

In the next step, we perform rolling circle amplification (RCA). RCA is based on the fundamen-tal property of a circle; the endlessness of a circular line. In essence, this means that for a ring ofDNA, DNA polymerase (initiated by a primer) will recurrently synthesize a linear replica of theDNA circle until the process is stopped by some kind of external factor [4]. Here, our intermedi-ate circular DNA products, of approximately 20 nanometer in diameter, act as perfect templatesto the rolling circle amplification reactions. The products of the RCA, rolling circle products(RCPs), are large single-stranded DNA molecules, spontaneously folded into random micron-sized DNA coils, consisting of repeated complementary copies of the probes sequences.

The individual RCPs are then labeled with certain oligonucleotides, tagged with fluorophores(fluorescent molecules). This is done by only allowing the fluorophore-tagged oligonucleotidesto attach to specific binding sites; namely the binding sites present in the replicated DNA circles.Thus, the fluorophore-tagged oligonucelotides can hybridize to the repeated sequences of theRCPs, giving a high fluorophore concentration within the DNA coils. In fact, the concentrationof fluorophores in the RCPs can be 100 times higher than in the surrounding solution (containingfree fluorophores) [2].

Finally, the individual RCPs are passed through a microfluidic channel and illuminated by laserlight. The RCPs, now with a higher local concentration of fluorophores than in the surround-ing solution, will fluoresce and the emitted light is registered for image analysis (thresholdingfor binarization) and counting of RCPs. Figure 2.2 below outlines the the described steps theASMD-technology.

4


Figure 2.2: Schematics of the Amplified Single-Molecule Detection technology.Reprinted by permission from Macmillan Publishers Ltd: NATURE METHODS, copyright 2006.

2.1.2 Optical System

The detection unit consists of a complex optical system to enable both detection and enumerationof RCP entities, as well as performing autofocus. The autofocus part of this system is describedlater in section 2.2. Here, we give a brief description of the light sources, optics and detectorsthat enable RCP detection/enumeration.

The light pathway can be divided in two parts; from the lasers to the microchannel (fluorescenceexcitation pathway), and from microchannel to the line scan detectors (fluorescence emissionpathway), as illustrated by figure 2.3 and 2.4. First, two lasers, of wavelength 532 nm (green)and 671 nm (red) respectively, emit beams of light. By letting the beams pass through beamexpanders, mirrors and beam shaping optics, the beams are transformed to lines. The lines con-tinue through a laser reflection dichroic mirror, and finally the objective, in order to illuminatethe microchannel.

As the RCP-rich solution passes through the illuminated microchannel, emitted light from theRCPs returns through the objective and the laser reflection dichroic mirror. Continuing througha notch filter (band stop filter, with narrow stop band), some of the light goes into a channelmonitoring CCD (Charge Coupled Device) camera via a beam pickup, whilst the most of thelight goes through a tube lens which makes sure to focus the light. Finally, the light passesthrough another dichroic mirror which separates the light, and the two lines go through specificband pass filters and into to the CCD line detectors.

5


Figure 2.3: Florescence excitation pathway. a) 532 nm laser b) 671 nm laser c1, c2) Beam ex-panders d) Laser mirrors e) Laser multiplex dichroic mirror f) Beam shaping optics g) Laserreflection dichroic mirror h) Objective i) Detection channel.

Figure 2.4: Fluorescence emission pathway. a) CCD line detector b) 582/75 band pass filter c)648 LP dichroic mirror d) 710/40 band pass filter e) Tube lens f) Beam pickoff g) Channel mon-itoring tube lens h) Channel monitoring CCD camera i) Notch filter 532 nm j) Laser reflectiondichroic mirror k) Objective l) Detection channel.

6

2.2. AUTOFOCUS CHAPTER 2. BACKGROUND

2.2 Autofocus

Obtaining crisp and sharp digital images is often crucial to successful image analysis, and is theresult of several optimized optical parameters. One such parameter of great importance is focus2.Autofocus in an optical system is the process of automatically enhancing the focus of a subject,the region of interest (ROI). This may be achieved by moving the lens back and forth until themost clear image is formed. Generally, optimal focus in an image is calculated by algorithmsbased on finding the extremes of a function that represents a ”figure-of-merit” [8]. A numberof important criteria applies to such a focus function, for instance speed and accuracy (must beable to find extreme values if image is in focus).

There are essentially two types of autofocus; active and passive autofocus.

Active autofocus systems detect the distance to the subject by emitting signals, usually infraredlight or ultrasonic sound waves, and measuring the delay of the reflected signals. This dif-ference in outbound and inbound signals is then used by the camera processor to determinehow far and in which direction the focus motor should move the lens [6].

Passive autofocus works by directly analyzing acquired image data and moving the lens ac-cording to the output of some focus function. There are two types of passive autofocus:phase detection and contrast measurement. In this thesis, passive autofocus is used, and isachieved by using contrast measurement. In contrast measurement, the idea is to comparethe difference in intensity among adjacent pixels [6]. An image acquired with incorrectfocus tends to have very similar intensities between adjacent pixels, as illustrated by fig-ure 2.5. Thus, a focused image can be obtained by iteratively searching for the maximumintensity difference among adjacent pixels.

Figure 2.5: Comparison of pixel intensities. a) Apple in focus. b) Apple out-of-focus.Original apple image: Master isolated images / FreeDigitalPhotos.net

2Technically, focus is a point at which rays or waves meet at a single point (converge) [15].

7


The traditional way of performing autofocus is by moving the lens back and forth. In our anal-ysis instrument however, the lens is fixed. Instead, a motorized positioning system supports thefocusing mechanism. When illuminating our microfluidic channel with laser light, the reflectedlight passes through a fixed lens which focuses the light on a USB camera. The acquired imagedata is forwarded to a software module performing passive autofocus. This module controls amotor which in turn moves the actual subject, our microfluidic channel. More precisely, the in-strument can be loaded with several samples in separate channels of a CD, and focus is on oneof the two microstructures surrounding the channels, as shown in the figures below.

Figure 2.6: Schematics of the autofocus system in the detection unit.

Figure 2.7: Light is focused to a microchannel on the CD.

The task for our autofocus module is thus to obtain image data, run an algorithm performing

8


image processing/analysis, and control the CD holder motor accordingly. In this module, we usea simple hill climbing algorithm for the passive autofocus. Listing 1 below presents pseudo codefor such an algorithm.

Listing 1 Autofocus psuedo-algorithmy← stepSizexold ← startContrastValueMove y steps in one directionwhile stopCondition is not met do

xnew← currentContrastValueif xnew > xold then

Move y steps in same directionelse

y←y/2Move y in opposite direction

end ifxold ← xnew

end while

2.2.1 Image Processing/Analysis

We have not yet defined how to actually determine how well focused an image is, compared toanother image. In the simplest form of contrast measurement (passive autofocus), it naturallyboils down to analyzing pixel intensities, as contrast is defined as the difference in intensitybetween the highest and lowest level in an image [7]. For our special set of images however,it is beneficial to perform some initial processing steps in order to enhance the accuracy ofobtaining a good image focus quality. This involves thresholding, mathematical morphology,edge detection and image quantification. These concepts require some fundamental knowledgein digital image processing. Figure 2.8 shows a typical image sent to the autofocus module. Bystarting from this image, we will walk through the image processing steps, briefly describing theunderlying theory.

9


Figure 2.8: Focused image obtained by the autofocus module.

The red rectangular area in figure 2.8 shows the autofocus structure we are interested in. Thefirst step is to locate and define our region of interest, which is shown in figure 2.9a3. The restof the image can now be discarded as the valuable information is solely contained in our definedROI. Next, a thresholding function is applied, producing a two-level, binary, image. This worksby first defining two threshold values in our greyscale image and then go through the indiviualpixels: if the pixel value is within our threshold range, the pixel is considered to be an objectpixel, otherwise it is a background pixel. All pixels labeled as being an object pixel are thengiven the value 1 (white)4 whilst the background pixels are given the value 0 (black), as shownin figure 2.9b.

3This ROI image does not originate from the image shown in figure 2.8, but from another image obtained fromthe AF module. The same concept is still valid though.

4Shown in red the here for clarity

10


(a) ROI Image. (b) Thresholding.

Figure 2.9: a) Defined ROI. b) Result after thresholding.

In the next step we are interested in filling the hole within our AF structure. This may be achievedin several ways, but a standard technique is to utilize mathematical morphology, a powerfulapproach based on set theory. Simply put, one uses a structuring element (SE), a small predefinedarray, to probe the image of interest. Two fundamental morphology operations are erosion anddilation. If we let A denote a set in the euclidian space (in this case our binary image is in the2-D integer space Z2), z a vector, B a structuring element, Bz the translation of B by z and B thereflection of B, the erosion and dilation operations are defined as follows

AB = {z|Bz ⊆ A} (2.1)

A⊕B = {z|Bz∪A 6=∅} (2.2)

Definition 2.1 tells us that A eroded by B is the set of all points (vectors) such that B translated byz is a subset of A. Similary, definition 2.2 says that A dilated by B is the set of all displacements,z, such that B and A overlap by at least one element. [7]. What erosion effectively does is toshrink or thin objects, while dilation on the other hand expands or thickens objects.

By having defined these two operations, we can form a third operation called closing, definedas

A•B = (A⊕B)B (2.3)

The closing operation is thus an erosion by B on the result of A dilated by B. Now, the closingoperation has just the properties we are looking for: it tends to smooth the contour of the objectby fusing narrow gaps and filling holes. By performing a closing operation on our image (figure2.9b), we may eliminate the long thin gap in our AF structure. This may seem like a naiveapproach, but if choosing an appropriate structuring element (and possibly adjust the result withan additional erode/dilate operation), a good result is easily acquired (other more precise andslightly more complicated methods exist, such as conditional dilation). The result is shown infigure 2.10a.

Further more, we would like to remove the border-connected object in the image shown in figure2.10a. This artifact comes from the shadow of the camera objective, and objects such this one,

11


connected to the image border, is not suitable for image analysis. Even if this object was areal property we were interested in, it would still be best to remove it since we cannot not obtaincomplete information about it anyway. This can be achieved by a using a technique called borderclearing, which is based on morphological reconstruction. In morphological reconstruction, animage, called the marker, is being processed by the charateristics of a second image, called themask. This is a recursive process in which, for instance, a series of dilations are performed onthe marker image as long as the marker image fits under the mask, i.e. the mask limits the growthof the marker. In the border clearing algorithm, we define our original image as the mask I, andthe marker image F as the image in which we set every border pixel (i.e. borders in the originalimage) to value 1, and everywhere else to value 0, i.e.

F(x,y) ={

I(x,y) if(x,y) is on the border if I0 otherwise

(2.4)

The border clearing algorithm first calculates the morphological reconstruction by dilation ofthe mask F from the marker image I, denoted as RD

F (F). This gives us an image of all border-touching objects. Then, to finally obtain a result similar to the image in figure 2.10b, we simplytake the set difference I−RD

F (F).

(a) Reulst after fillingholes.

(b) Result after clear-ing border-connectedcomponents.

Figure 2.10: a) As a result of a close/erode operation, the inner region of our AF structure hasbeen filled. b) Border-connected ojects have been removed.

Now at this stage, we may apply some more simple morphological operations in order to removeany small particles or other noise. Such operations are not included here. The last step is toperform edge detection. By doing this we can significantly reduce the amount of data and filterout useless information[12]. An edge is the connected set of edge pixels, which are pixels atwhich the change of intensity is abrupt.

A vast amount of different algorithms and techniques for edge detection exist, and we will not gointo the very details here. The most basic methods, however, are based on the fact that changes inintensity can be detected using derivates (especially suitable for this is the first- and second-orderderivates, where the first-order derivate can be approximated by a taylor expansion). Basically,the derivate of a 2-D function f (x,y) will have a large magnitude at locations where the functionf changes abruptly. In other words, the gradient ∇ f vector points in the direction where wehave the greatest rate of change, naturally corresponding to edges in an image. The length of

12


the gradient of f, ‖∇ f‖, then gives us the strength of the edge at a particular location of theimage. This simple concept is the fundamentals of all gradient-based edge detection methods.The computed derivates are then compared to a given threshold value. The most simple methodsfor estimating the derivates often produces a noisy result. This can be corrected for by applyingcertain gradient operators. In our application we performed the edge detection step using thePrewitt operator, resulting in image shown in figure 2.11.

Figure 2.11: Resulting image after performing the edge detection operation.

Now after all these preprocessing steps, we are ready to perform some statistics calculations. Weare calculating the standard deviation of intensity values in the processed ROI image and usesthis value as a measure of the current sharpness or focus quality, aiming to obtain the image withthe highest standard deviation.

13


14

3 Implementation

This chapter describes the software requirements specifications, the software design and anytechnical discussions regarding the software.

3.1 Software Requirements Specifications

The following sections will give an overview of the required functionality of the software interms of actions, communications, interfaces to other software and hardware, and any other con-straints or dependencies. The Software Requirements Specifications, SRS, designed as a sim-plification of the IEEE 830-1998 standard recommendations for Software Requirements Spec-ifications, was originally composed as an external document as part of the development pro-cess.

3.1.1 Product Perspective

The software component to be developed is a part of the Detection Cell Positioning module thatis a controller subsystem for the detection unit. In order to distinguish our software from othercomponents, it will be referred to as the DCPCS component - Detection Cell Positioning ControlSoftware. A simplified diagram, presented in figure 3.1 below, shows the DCPCS component andits interactions with other components.

In the standard workflow, user input, from a connected client, is validated and processed on theserver, a certain operation is performed, and a response is issued back to the client. In manycases this simply means that incoming, valid, commands are translated to specific commandsthat are forwarded to the microcontroller and/or the camera. The following subsections includea description of involved components and their relationship.

15

3.1. SOFTWARE REQUIREMENTS SPECIFICATIONSCHAPTER 3. IMPLEMENTATION

Figure 3.1: Simplified overview of DCP module, of which the DCPCS is a part.

3.1.1.1 System Interfaces

DCPCS The DCPCS will act as a server and communicates with an external front-end. It con-stitutes the interface between client and hardware components.

Client Any kind of user interface (graphical or command line based) software issuing com-mands to the DCPCS.

Microcontroller The DCPCS is connected to a microcontroller that controls the motors andvarious proximity sensors.

Camera Taking snapshots of the sample.

3.1.1.2 User Interfaces

There is no specific interface between end users and the DCPCS. Interaction between user andthe system is done via a client. Note that multiple clients connected simultaneously are not to besupported.

3.1.1.3 Hardware Interfaces

Apart from the external hardware listed in 3.1.1.1, i.e.. microcontroller and camera, there areno specific hardware requirements on the computer where the DCPCS should run, other thanstandard computer components, including Ethernet card and USB 2.0 port(s). However, there isa requirement of a serial port for communication with the microcontroller. This requirement canbe neglected by using a USB to RS-232 serial adapter.

There are no major constraints or limits on primary and secondary memory affecting perfor-mance of the DCPCS. It should be able to run fine on a small amount of resources, using any”standard set” of computer hardware.

16

3.1. SOFTWARE REQUIREMENTS SPECIFICATIONSCHAPTER 3. IMPLEMENTATION

3.1.1.4 Software Interfaces

The system is tended to run on a Microsoft Windows platform, either Windows NT-based orWindows Embedded. This is due to the fact that other systems (e.g. Sapera image acquisitionsoftware) running in the same environment, requires Windows. The DCPCS will be developedaiming for platform independence. However, this is not a strict requirement, merely a gooddesign principle.

The Boost C++ libraries [3] might be used to make the software transparent to other operat-ing systems, as well as providing convenient functionality for communication. The Boost C++libraries consist of several individual high quality libraries that may speed up the initial de-velopment process and avoid ”reinvention-of-the-wheel”. By providing easy-to-use frameworksfor handling common programming scenarios, Boost actually brings C++ closer to other high-level programming languages such us .NET C# or Java, yet maintaining the C++ low-level con-trol.

Depending on what camera to use in the system, certain software libraries provided by the cam-era manufacturer will be used by the DCPCS (for communication). The camera currently beingused in the development environment is a uEye LE USB camera [16].

3.1.1.5 Communications Interfaces

The client will communicate with the DCPCS over TCP/IPv4 (Transmission Control Proto-col/Internet Protocol), using a user-specified port. Support for IPv6 is not a strict requirement,but should be easy to implement in a correct design. The two parties will communicate using awell defined ASCII protocol. Refer to appendix A for details on this protocol.

The DCPCS communicates with the microcontroller using RS-232 serial communication. Asnoted previously, the device can be connected to a USB port using an on-board or cable mountedUSB to Serial interface circuit. The communication protocol to be used is a simple ASCII pro-tocol. Refer to appendix B for this protocol. The camera hardware is connected using a standardUSB 2.0 port and communication is done via manufacturer-provided libraries.

3.1.2 Product Functionality

The major functionality of the DCPCS is to communicate with client, microcontroller and cam-era as outlined in section 3.1.1.5. The DCPCS must be able to correctly parse and interpretincoming commands, perform command-specific logic and return correct response. Availablecommands, parameters and status codes are defined in the protocols presented in appendices Aand B. Other functional requirements are specified in the following subsections.

17

3.2. SOFTWARE DESIGN CHAPTER 3. IMPLEMENTATION

3.1.2.1 Logging System

The system should be able to log requests and responses from client sessions, system events,etc., with different severities. One of many available open source logging libraries on the Internetmight be suitable for this.

3.1.2.2 Daemon Process

The DCPCS should preferably be able to run as a daemon background process, in particular asa Windows service. This simplifies scheduling and starting/stopping of the DCPCS.

3.1.2.3 Configurable

The DCPCS must be easy to configure by utilizing a global configuration text file or similar. Thisconfiguration file may include parameters such as server port number, COM port and logginglevel etc.. Possibly, a user may be able to modify some of these settings from client side. Further,the configuration can also include specifications of what hardware to use; e.g. if multiple cameratypes are to be supported, the current camera in use can be specified here.

3.1.3 Constraints

The system is to be written in standard C++ (ISO/IEC 14882), based on a object oriented design.Regarding modularity/maintainability, different components or subsystems, identified at a laterstage in the design process, should be as independent as possible in order to be able to easilyreplace parts of the system if needed. However, this should automatically follow from a logicaldesign.

Other aspects that might limit the developer’s options are reliability and availability of the sys-tem. This might be rather intuitive, and not a limitation, but the software must include robust ex-ception handling and error checking in order to minimize, or ideally to eliminate, unexpected be-havior and crashes. A common mechanism for this may be used throughout the software.

3.2 Software Design

The following sections describe the architecture and software design of the DCPCS. The soft-ware requirements, identified in 3.1, are translated into a representation of software components,interfaces and data necessary for implementation. These section originates from a Software De-sign Document, SDD, composed during the development process. The content presented hereinserves as the primary source of information for the actual code implementation and testing. First,

18


we present a high-level design in which the overall system architecture and information flow isdefined. Then, a more detailed and precise design is presented.

3.2.1 System Architecture

The following diagram shows a high-level overview of how the system can be partitioned intocomponents, and how these components collaborate with each other in order to achieve therequired functionality of the system.

Figure 3.2: DCPCS System architecture overview.

This diagram shows a layered architecture of the DCPCS system. Conceptually, each layer rep-resents a distinct aggregation of functionality. It might not be a truly n-tiered architecture, butshould still provide sufficient separation and flexibility. Here, the information flows in a horizon-tal fashion. The architecture components and their interactions might not be crystal clear just bylooking at this diagram, and deserve some explanations. Thus, the next subsections will describeeach layer and its components. Note that these components do not necessarily correspond toclasses and objects in code, they are to be thought of as logical partitions.

3.2.1.1 Presentation Layer

The presentation layer, or client layer, represents the top-most tier of the system, at which anend user interacts with a user interface and provides user input to the system. Communicationis based on a TCP/IP protocol. Note that the Client component is not considered as a part of theDCPCS system to be developed. It is an external independent component.

19


3.2.1.2 Business Object Layer

The business layer is the heart of the DCPCS system and will contain all business logic requiredfor the supported operations of the application. It provides the interface between an externalclient and the hardware. The interaction with the hardware is done via communication objectsfrom the Hardware Communication Layer, refer to section 3.2.1.3.

Server The main component of the DCPCS is the Server component. It is the base componentof this layer (containing the application main entry point) that initializes other components andsets up the server to listen for and accept incoming client connections. More over this componentwill provide support for the DCPCS to run as a daemon process, as well as controlling any threadmanagement and so on.

Session When a client connects to the system, the Server component will ”spawn” a newunique session (an active TCP connection), which is represented by the Session component. Theresponsibility of this component is to receive data from the client, and send processed data backto the client. The communication skills of this component is inwitialized by the Server compo-nent. The actual data processing is managed by its neighboring component, the DCPController.The Session component will merely act as a messenger.

DCPController This is the component that will process data, both from the client and thehardware layer. This component will contain all the business logic that is necessary to completeoperations issued by commands from the client. Part of the logic will be a parser that validatesand/or decomposes incoming data. The behavior of the parser is defined by different protocolsspecified. Incoming data are validated, processed and forwarded to different communication ob-jects in the Hardware Communication layer for appropriate actions. Outgoing data, i.e. responsesfrom the hardware, are forwarded to the current Session component. More specifically, it triggersmovement of motors, performs autofocusing etc.

3.2.1.3 Hardware Communication Layer

The responsibility of this layer is to manage the actual data communication with the hardware. Itprovides a set of communication providers, each specialized on a specific communication type.The intention of relying on these providers is to encapsulate and hide the complexity and detailsof low-level data transfer operations.

The Serial Communication component provides general support for RS-232 serial communica-tion, but is specifically designated to communication with a microcontroller unit.

The Camera Communication component manages the communication with the camera. In itsessence, it is standard USB 2.0 communication, but this component will rely on software li-braries provided by the camera manufacturer.

20

3.3. IMPLEMENTATION DETAILS CHAPTER 3. IMPLEMENTATION

3.2.1.4 Common Application Framework

The components of this ”framework” provide utility classes that are used across the system,spanning over several layers. This includes a Configuration component that provides the systemwith various settings (default or user overridden). Another component is a logging system that isused to log events and possible failures to disk or standard output. There is also an error/excep-tion mechanism used throughout the system. The idea is to propagate any exceptions throughthe system and let this central mechanism take care of it.

3.3 Implementation Details

Refer to Appendix C for a structured and thorough decomposition description of all softwarecomponents presented above.

3.4 Implementation Technicalities

Stability in communication is essential for the overall performance of this software. A particularproblem may arise, especially for serial communication, when expected data transfers are notinitiated, never terminated, or interrupted. This may occur if we somehow lose connection withthe COM port or the cable is plugged out etc. For instance, if trying to read data from a socket,and no data is immediately available (or worse, if no data is ever available) the thread will block.Essentially, this means that we must monitor the data operations and utilize timeouts. Our serialcommunication class manages socket timeouts by applying the concept of a deadline. Someasynchronous operations, e.g. async write, async read, are given deadlines by which they mustcomplete. Further, the deadlines are enforced by an ”actor” that persists for the lifetime of theobject. The idea is outlined in figure 3.3.

Figure 3.3: Concept of deadline for socket timeouts.

For example, when reading data from the socket, we define a deadline and setup the asyn-chronous async read until operation, together with a callback function (for updating possibleerror codes and length variables). Then we trigger it to run and block until the asynchronousoperation has completed. In this way, we can perform the operation in blocking mode (whichis what we want), and at the same time continuously checking if the deadline has passed. If the

21

3.4. IMPLEMENTATION TECHNICALITIES CHAPTER 3. IMPLEMENTATION

deadline actor determines that the deadline has expired, the outstanding asynchronous opera-tion is canceled so that the blocked read or write operation will return. By using the Boot Asiolibrary1, we can easily implement this concept.

A similar reasoning can by applied to the TCP sockets in the communication between our servercomponent and a client. Here however, the server is asynchronously accepting incoming clientconnections, and a session manager is used to handle the open connections. If a serious com-munication error occurs, the session manager can simply close this particular session (clientconnection), without affecting other server components or interfering with other clients. Notethat it is not a software requirement to handle multiple clients, but in theory, this server imple-mentation should be able to handle this (although other conflicts may arise when simultaneouslyissuing commands to hardware).

1Boost Asio is a cross platform C++ library for network and low-level I/O programming that provides developerswith a consistent asynchronous model using a modern C++ approach [11].

22

4 Evaluation

The objective for this thesis work was primarily to design and implement control software forthe detection unit of the analysis instrument. The most difficult part in the implementation pro-cess was to develop good support for autofocus, which will be discussed in section 4.1.As the main work of this thesis concerns pure software development, we do not have any spec-tacular results to present herein. The software was tested and performs according to the softwarerequirements specifications. Besides referring to the source code and implementation details inthe appendices, the only valuable information to show here is a series of screenshots showingapplication in action. This is presented in section 4.2 .

4.1 Autofocus Evaluation

The most tricky part in the implementation work was to acquire a good representation of thefocus function. That is, for each of focus motor positions, an acquired image will have a specificfocus value. The behavior of this function is essential for determining parameters to the autofo-cus algorithm. Figure 4.1 below shows the focus value as a function of the focus motor position.Here, the position range is from -37500 to -34500 steps relative to the WORKPOS1 position, aninterval large enough to guarantee that the true focus position is included.

In this figure (4.1) the first peak represents where the best focus is achieved. If moving to theleft or right from this position (∼35800±50 ), you will miss the optimal focus point and theacquired frame will be out-of-focus. The autofocus algorithm performed well on this type ofcurves, being able to fairly accurately pin-point the best focus position.

One initial flaw in the algorithm was its sensitivity to sudden changes in the focus value. Refer-ring again to the focus curve in figure 4.1, one will notice that, on either side of the first peak,the derivative rapidly changes sign within small regions (this is particularly obvious when zoom-ing in to the individual data points in these regions), before the actual focus peak arises. Thismade the autofocus algorithm unstable, giving rise to “random” results throughout some runs.What happens is that the algorithm gets trapped in local maxima, instead of iterating towards

1The CD is initially moved/rotated to a pre-defined work position.

23

4.2. APPLICATION SCREENSHOTS CHAPTER 4. EVALUATION

Figure 4.1: Focus curve, showing the positions at which best focus is achived.

the global maximum. In order to overcome this peculiarity, a threshold for the focus value wasintroduced. As we know the approximate shape of the focus function in advance, we can safelyinstruct the hill-climbing algorithm to continue in the same direction as long as we are belowthis threshold value, even though a drop in the focus value is encountered. This addition to thealgorithm drastically improved the results.

As noted previously, the autofocus algorithm has been proven working on a well-defined testdataset. However, a series of complications in the lab environment made it difficult to run a final“proof-of-concept” test. Examples of such complications are non-static ROIs, changes in the de-sign/depth of the autofocus structures, varying light conditions and different timing parametersof the camera (pixel clock, frame rate and exposure time). Nevertheless, when all components(and other dependencies) are finalized, this should only be a matter of fine-tuning optical param-eters.

4.2 Application Screenshots

Below follows a series of screenshots from the implemented software. The figures illustrate theissuing of commands between a testing client and the DCPCS, communicating over TCP/IP.Figure 4.2 displays the startup phase, figure 4.3 illustrates how to move the focus motor andfigure 4.4 shows how to acquire a single frame from camera and display it in the client.

24


(a) Client startup.

(b) Server startup.

Figure 4.2: Screenshots from starting up the server application and the test client.

25


(a) Client issuing the focus command.

(b) Server performs the move focus command.

Figure 4.3: The user issues a focus command in the test client and the server responds by per-forming the operation (forwarding a translated command to hardware).

26


(a) Client snapimage command.

(b) Server snapimage command.

(c) Preview of snapshot on clientside.

Figure 4.4: The user issues a snapimage command in the test client and the server triggers thecamera to take a snapshot of the sample and send the image data back to the client.

27


Note that the previous screenshots shows the server component of the DCPCS as a standaloneapplication. In the software design we stated that the DCPCS shall preferably be able to run as adaemon background process. This functionality was implemented as a separate Windows serviceproject. This Windows service starts a separate process and spawns the DCPCS server within anew thread. This has been tested successfully - the DCPS server may be installed as a serviceand run as a background process, still being able to serve clients and perform logging.

28


29

5 Summary and Conclusions

In section 1.3 we identified the objectives and goal of this master thesis. In its essence, theprimary objective was to design and implement control software for the detection unit of theanalysis instrument. This started with identification of software requirements, followed by astructured design and lastly the key work of writing and testing the software.

Our short evaluation shows that the software is working as expected. However, when it comes tothe autofocus functionality, the final optimization and verification aspects must be left as futurework, mainly due to the reasons mentioned in section 4.1. Thus, the DCPCS as a whole, maynot be fully verified until a complete and finalized lab environment is available.

30

CHAPTER 5. SUMMARY AND CONCLUSIONS

31

References

[1] D. O. Antson. Genotyping RNA and DNA Using Padlock Probes. PhD thesis, UppsalaUniversity, Department of Genetics and Pathology, 2001.

[2] G. A. Blab, T. Schmidt, and M. Nilsson. Homogeneous detection of single rolling circlereplication products. Analytical Chemistry, 76(2):495–498, January 2004.

[3] Boost c++ libraries. http://www.boost.org, February 2011.

[4] V. V. Demidov. Rolling-circle amplification in dna diagnostics: the power of simplicity.Expert Rev. Mol. Diag, 2(6):542–548, November 2002.

[5] S. Fredriksson, M. Gullberg, J. Jarvius, C. Olsson, K. Pietras, S. M. Gustafsdottir,A. Ostman, and U. Landegren. Protein detection using proximity-dependent dna ligationassays. Nature Biotechnology, 20(5):473–477, May 2002.

[6] N. Goldberg. Camera Technology: The Dark Side of the Lens. Academic Press, April1992.

[7] R. C. Gonzalez and R. E. Woods. Digital Image Processing. Pearson Education, Inc, thirdedition, 2008.

[8] F. C. Groen, I. T. Young, and G. Ligthart. A comparison of different focus functions foruse in autofocus algorithms. Cytometry, 6(2):81–91, March 1985.

[9] S. M. Gustafsdottir, A. Nordengrahn, S. Fredriksson, P. Wallgren, E. Rivera,E. Schallmeiner, M. Merza, and U. Landegren. Detection of individual microbial pathogensby proximity ligation. Clinical Chemistry, 52(6):1152–1160, 2006.

[10] J. Jarvius, J. Melin, J. Goransson, J. Stenberg, S. Fredriksson, C. Gonzalez-Rey, S. Bertils-son, and M. Nilsson. Digital quantification using amplified single-molecule detection.Nature Methods, 3(9):725–727, August 2006.

[11] C. Kohlhoff. Boost asio library. http://www.boost.org/doc/libs/1_46_1/doc/html/boost_asio.html, February 2011.

[12] R. Maini and D. H. Aggarwal. Study and comparison of various image edge detectiontechniques. International Journal of Image Processing (IJIP), 3(1):1–3, February 2009.

32

http://www.boost.org

http://www.boost.org/doc/libs/1_46_1/doc/html/boost_asio.html

http://www.boost.org/doc/libs/1_46_1/doc/html/boost_asio.html

REFERENCES REFERENCES

[13] J. Melin. Single-Molecule Detection and Optical Scanning in Miniaturized Formats. PhDthesis, Uppsala University, Department of Engineering Sciences, 2006.

[14] M. Nilsson, H. Malmgren, M. Samiotaki, M. Kwiatkowski, B. Chowdhary, and U. Lande-gren. Padlock probes: circularizing oligonucleotides for localized dna detection. Science,265(5181):2085–2088, September 1994.

[15] C. Soanes and A. Stevenson. Oxford Dictionary of English. OUP Oxford, new ed of 2revised ed edition edition, August 2005. focus.

[16] I. I. D. Systems. Usb ueye le-1545le manual. http://www.ids-imaging.de/frontend/files/uEyeManuals/Manual_eng/uEye_Manual/.

33

http://www.ids-imaging.de/frontend/files/uEyeManuals/Manual_eng/uEye_Manual/

http://www.ids-imaging.de/frontend/files/uEyeManuals/Manual_eng/uEye_Manual/

Appendices

34

A Client-Server Protocol

A.1 DCP TCP Communication Protocol

The client will establish a new connection for initialization, settings and testing. Several requestsmay be sent during each connection. Each command line sent by the client is an ASCII-string ofmaximum length 256 bytes. The bytes CRLF (carriage return, 0x0D, line feed, 0x0A) denotesthe end of the command. CR or LF must not be present except at the end of the command. Eachcommand line is divided into one or several words, separated by single spaces. The first wordconstitutes the command word. The remaining words are interpreted on a command-specificbasis.

A.2 Commands and responses

Each request should generate a single response. The response should be sent once the operationis completed. A response message is an arbitrary length ASCII-string, formatted in the sameway as a request message, and ended in the same way by CRLF. The first word is a responseword, status, which can be OK, FAULT or ERROR.

• OK indicates that the requested operation completed successfully. Status string is OK.Payload strings depend on the request message being responded to.command was suc-cessfully received and executed. Any subsequent words are interpreted on a commandspecific basis.

• FAULT indicates a problem with the component hardware. Status string is FAULT, fol-lowed by an error code payload string.

• ERROR indicates a communication error or an invalid request message. Status string isERROR, followed by a fault code payload string.

The following tables show the preliminary command set.

35

A.2. COMMANDS AND RESPONSES APPENDIX A. CLIENT-SERVER PROTOCOL

Command Arguments Description Expected ResponseSENSOR FOCUS/CLAMP/CD/ROTATIONReturns the value of a sensor OK valueCLAMP OPEN/SEMI/CLOSE Opes, semi opens or closes the

clampOK, no payload. FAULT if the fo-cus motor is not at home when run-ning OPEN.

FFC - Finds the first channel on a CD OK, no payload. FAULT if no chan-nel was found.<CR>

FNC - Finds the next channel on a CD OK, no payload. FAULT if no chan-nel was found

VERSION - Returns the current version of theDCP

OK DCP Vx.xx.

WORKPOS - Moves the CD to work position OK, no payload. FAULT if clamp isopen.

CHANGEPOS - Move the CD to change CD posi-tion

OK, no payload

GETPOS FOCUS/CLAMP/ROTATION Returns the current position of amotor

OK position

HOMEFOCUS - Returns the focus motor to its homeposition

OK, no payload. FAULT if homingfailed.

MOVEX FOCUS/CLAMP/ROTATIONx

Moves x steps on a motor OK, no payload. FAULT if movefailed.

SETMODE FOCUS/CLAMP/ROTATIONmode

Sets motor mode OK, no payload

SETRES FOCUS/CLAMP/ROTATIONres

Sets motor resolution OK, no payload

SETSPEED FOCUS/CLAMP/ROTATIONspeed

Sets motor speed OK, no payload

SETALL FOCUS/CLAMP/ROTATIONmode res speed

Sets motor mode, resolution andspeed

OK, no payload

AF FAST/SLOW Autofocusing on a CD channel OK, no payload. FAULT if focusingis not possible.

SNAPIMAGE - Returns a snapped image from cam-era

OK image data

LED ON/OFF Turns the LED on or off

Table A.1: Command set

36

B Server-MC Protocol

B.1 Control of detection cell positioning control

The detection cell positioning control device (microcontroller) should be controllable from acomputer. The device will be connected to a USB port using an on-board or cable-mountedUSB to Serial interface circuit. At a later stage, this may be switched to a CAN (controllerarea network) bus, an RS-485 bus or similar to reduce the amount of cabling required. The ini-tial communication protocol will be a simple ASCII-based protocol with the computer sendingrequests and receiving responses consisting of single lines terminated by CR (carriage return,0x0D).

Each command line will consist of at most 32 bytes. The CR byte (0x0D) denotes the end ofthe command. CR must not be present except at the end of the command. Each command line isdivided into one or several words, separated by single spaces.

B.2 Commands and responses

The first word is the command word. The following words are arguments that are interpreted ona command-specific basis. Integer arguments are sent as decimal strings.

command [argument1] [argument2]<CR>

For example, the MOVE command could be sent as follows:

MOVE FOCUS 500<CR>

Each request should generate a single response. The response should be sent once the operationis completed. A response message is formatted in the same way as a request message, andended in the same way by CR. The first word is a response word, which can be OK, FAULT orERROR.

• OK indicates that the command was successfully received and executed. Any subsequentwords are interpreted on a command specific basis.

37

B.2. COMMANDS AND RESPONSES APPENDIX B. SERVER-MC PROTOCOL

• FAULT indicates that the command was successfully received but that it could not beexecuted successfully.

• ERROR indicates that the command was not received successfully, or that the argumentswere out of range. The ERROR word should always be followed by an error code, andoptionally by a description.

Fault code Description0 Motor running1 Homing failed2 Clamp is open3 Move failed4 Focus is not home5 Clamp is home

Table B.1: Fault codes

Error code Description0 Undefined error1 Command syntax error2 Unknown command3 Invalid argument

Table B.2: Error codes

The following tables show a preliminary command set and various positions.

38

B.2. COMMANDS AND RESPONSES APPENDIX B. SERVER-MC PROTOCOL

Command Arguments Description Return valuesSET RESOLUTION FOCUS/ROTATION/CLAMP

0-3Setting resolution for a motor SET RESOLUTION [FOCUS/RO-

TATION/CLAMP] [0-3]<CR>SET SPEED FOCUS/ROTATION/CLAMP

10-32000Setting speed for a motor SET SPEED [FOCUS/ROTA-

TION/CLAMP] [10-32000]<CR>SET MODE FOCUS/ROTATION/CLAMP

0-3Setting mode for a motor SET MODE [FOCUS/ROTA-

TION/CLAMP] [0-3]<CR>GET SENSOR FOCUS/CD/CLAMP Getting value for a sensor GET SENSOR [FOCUS/CD/-

CLAMP] [0-1]<CR>MOVE FOCUS/ROTATION x Move x steeps with a motor MOVE [FOCUS/ROTATION]

x<CR>CHANGE CD POS - Move to change CD position CHANGE CD POS<CR>FOCUS POS - Move to focusing position FOCUS POS<CR>FFC - Find first channel on a new CD FFC<CR>FNC - Find next channel on a CD FNC<CR>HOME FOCUS - Return focus motor to a know posi-

tionHOME FOCUS<CR>

GET CURRENT CHANNEL - Get the current channel on a CD GET CURRENT CHANNELx<CR>

GET POSITION FOCUS/CLAMP Getting position for a motor GET POSITION [FOCUS/-CLAMP] x<CR>

SEMI OPEN CLAMP - Opens clamp enough to rotate a CD SEMI OPEN CLAMP<CR>OPEN CLAMP - Opens clamp enough to change a

CDOPEN CLAMP<CR>

CLOSE CLAMP - Close clamp against CD CLOSE CLAMP<CR>VERSION - Return the version of the device DCP Vx.xx<CR>

Table B.3: Command set

Position Description0 Home1 Focus2 Unknown

Table B.4: Focus positions

Position Description0 Home1 Semi open2 Open3 Unknown

Table B.5: Clamp positions

39

C Implementation Details

The following sections show a stripped-down version of the full implementation documenta-tion.

40

Contents1 Module Documentation 1

1.1 Hardware Communication . . . . . . . . . . . . . . . . . . . . . . . 11.2 DCP Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Common Application Framework . . . . . . . . . . . . . . . . . . . 21.4 Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Class Documentation 32.1 CameraCom Class Reference . . . . . . . . . . . . . . . . . . . . . . 32.2 DCPController::Command Class Reference . . . . . . . . . . . . . . 52.3 Configuration Class Reference . . . . . . . . . . . . . . . . . . . . . 102.4 DCPController Class Reference . . . . . . . . . . . . . . . . . . . . 122.5 DCPCS Class Reference . . . . . . . . . . . . . . . . . . . . . . . . 162.6 ErrorException Class Reference . . . . . . . . . . . . . . . . . . . . 172.7 Logger Class Reference . . . . . . . . . . . . . . . . . . . . . . . . . 182.8 Configuration::Parameter< T > Class Template Reference . . . . . . 212.9 SerialCom Class Reference . . . . . . . . . . . . . . . . . . . . . . . 242.10 Session Class Reference . . . . . . . . . . . . . . . . . . . . . . . . 272.11 SessionManager Class Reference . . . . . . . . . . . . . . . . . . . . 292.12 TCPServer Class Reference . . . . . . . . . . . . . . . . . . . . . . . 30

1 Module Documentation

1.1 Hardware Communication

Classes

• class CameraComManages communication with the camera.

• class SerialComProvides support for serial communication.

1.1.1 Detailed Description

The Hardware Communication module handles all communication with supported hard-ware. This currently involves camera and microcontroller.

1.2 DCP Controller

Classes

• class DCPControllerInterface between client and hardware.

1

1.3 Common Application Framework 2

• class DCPController::CommandRepresentation of command with command-specific arguments.


The DCP Controller module acts as an interface between client and hardware.

1.3 Common Application Framework

Classes

• class ConfigurationProvides parameters to the program.

• class Configuration::Parameter< T >

Represents a configurable parameter.

• class ErrorExceptionException representing communication error.

• class LoggerProvides logging functionality.


The Common Application Framework provide utility classes that are used across thesystem, spanning over several layers.

1.4 Server

Classes

• class DCPCSDefines the startup of the application.

• class SessionRepresents a client-server session, performing data exchange.

• class SessionManagerManages open sessions (connections).

• class TCPServer

2

2 Class Documentation 3

Defines a TCP socket-based server.


The Server module handles all communication with clients.

2 Class Documentation

2.1 CameraCom Class Reference

Manages communication with the camera.

Public Member Functions

• CameraCom ()• ∼CameraCom ()• CameraCom (std::string &cameraSettingsFile, int sizeX, int sizeY, int bitsPer-

Pixel)• bool CaptureImage (char ∗∗imgData)• int GetImageWidth ()• int GetImageHeight ()• int GetBitsPerPixel ()• const std::string & GetCameraError ()


The CameraCom class manages the communication with the camera.

2.1.2 Constructor & Destructor Documentation

2.1.2.1 CameraCom::CameraCom ( )

Default constructor. Reads camera and image settings from configuration.

2.1.2.2 CameraCom::∼CameraCom ( )

Destructor. Closes the camera driver.

2.1.2.3 CameraCom::CameraCom ( std::string & cameraSettingsFile, intsizeX, int sizeY, int bitsPerPixel )

Overloaded constructor. Initializes camera with given parameters.

3

2.1 CameraCom Class Reference 4

Parametersin cameraSet-

tingsFileAbsolute or relative path to file containing camera settings.

in sizeX Width of image.in sizeY Height of image.in bitsPerPixel Bits per pixel.

2.1.3 Member Function Documentation

2.1.3.1 bool CameraCom::CaptureImage ( char ∗∗ imgData )

Freezes the current image exposure and transfers data to memory.

Parametersout imgData Pointer to starting address of image buffer.

Returns

true on a successful operation.

Note

The image buffer must be deallocated manually after calling this function.

Here is the caller graph for this function:

��

2.1.3.2 bool CameraCom::GetBitmap ( char ∗∗ imgData, int & imgSize )

2.1.3.3 int CameraCom::GetBitsPerPixel ( )

Gets bits per pixels of images.

Returns

Bits per pixels.

4

2.2 DCPController::Command Class Reference 5

2.1.3.4 const string & CameraCom::GetCameraError ( )

Gets an error message describing the latest occured camera error.

Returns

Latest camera error message.

2.1.3.5 int CameraCom::GetImageHeight ( )

Gets the height of images.

Returns

Height.

2.1.3.6 int CameraCom::GetImageWidth ( )

s Gets the width of images.

Returns

Width.

2.2 DCPController::Command Class Reference

Representation of command with command-specific arguments.


• Command ()• Command (DCPController::Commands commandName, std::vector< std::string> &args)

• const DCPController::Commands & GetCommandName () const• void SetCommandName (DCPController::Commands commandName)• const std::vector< std::string > & GetArguments () const• void SetArguments (std::vector< std::string > &args)• const std::string & GetArgument (unsigned short i) const• const Status & GetResponseStatus () const• const std::string & GetResponsePayload () const• const unsigned int GetErrorCode () const• const std::string GetResponse ()• void SetResponse (Status status, std::string responsePayload)• void SetResponse (Status status, int errorCode, std::string responsePayload, bool

throwError=false)• void SetResponse (std::string response, bool throwError=false)

5



The Command class represents a valid command along with command-specific argu-ments.

Successfully parsed data from client are represented by a Command instance. This in-stance is updated with response and status information when an operation is completed.


2.2.2.1 DCPController::Command::Command ( ) [inline]

Default constructor.

2.2.2.2 DCPController::Command::Command ( DCPController::CommandscommandName, std::vector< std::string > & args ) [inline]

Overloaded constructor. Initializes a command representation.

Parametersin command-

NameName of command.

in args Vector of command-specific arguments.


2.2.3.1 const std::string& DCPController::Command::GetArgument (unsigned short i ) const [inline]

Gets an argument from the Command’s set of arguments.

Parametersin i Zero-based index of arguments list.

Returns

Argument specified by i.

2.2.3.2 const std::vector<std::string>& DCPCon-troller::Command::GetArguments ( ) const[inline]

Gets the arguments associated with this Command.

Returns

Vector of command-specific arguments.

6


Note

An empty vector is returned if no arguments are available.

2.2.3.3 const DCPController::Commands& DCPCon-troller::Command::GetCommandName ( ) const[inline]

Gets the command name for this Command.

Returns

Command name.


��

2.2.3.4 const unsigned int DCPController::Command::GetErrorCode ( )const [inline]

Gets the error code for this Command.

Returns

Error code.

Note

The error code is undefined if the operation is successful.

2.2.3.5 const std::string DCPController::Command::GetResponse ( )[inline]

Gets the final formatted response for this Command. On a successful operation, theresponse is composed of a response status and any payload. On failure, the response iscomposed of a response status, an error code and an optional error description.

Returns

Response

7



��

��

��

��

2.2.3.6 const std::string& DCPController::Command::GetResponsePayload () const [inline]

Gets the response payload for this Command.

Returns

Response payload.

2.2.3.7 const Status& DCPController::Command::GetResponseStatus ( )const [inline]

Gets the response status for this Command.

Returns

Response status.

Note

The response status is undefined until the operation has finished.

2.2.3.8 void DCPController::Command::SetArguments ( std::vector<std::string > & args ) [inline]

Sets the arguments for this Command.

Parametersin args Arguments for this command.

8


2.2.3.9 void DCPController::Command::SetCommandName (DCPController::Commands commandName ) [inline]

Sets the command name for this Command.

Parametersin command-

NameCommand name.

2.2.3.10 void DCPController::Command::SetResponse ( Status status,std::string responsePayload ) [inline]

Sets the response for this command.

Parametersin status Response status.in responsePa-

yloadResponse payload.


��

2.2.3.11 void DCPController::Command::SetResponse ( Status status, interrorCode, std::string responsePayload, bool throwError = false )[inline]

Sets the response for this command. Use this overload to set the response when anerror occured.

Parametersin status Response status.in errorCode Error Code.in responsePa-

yloadResponse payload.

in throwError If true, automatically throws an appropriate ErrorException.

9

2.3 Configuration Class Reference 10

Note

Error code should be properly set to a value defined by the error codes or faultcodes defined by the protocol.

2.2.3.12 void DCPController::Command::SetResponse ( std::string response,bool throwError = false ) [inline]

Explitcitly sets the final response for this Command.

Parametersin response Response.in throwError If true, automatically throws an appropriate ErrorException.

Note

The response should only be set explicitly when hardware returns an ERROR orFAULT message. This is done in order to preserve any error code and/or errordescription from hardware.

2.3 Configuration Class Reference

Provides parameters to the program.

Classes

• class ParameterRepresents a configurable parameter.

Public Types

• enum ParameterNames


• Configuration ()• void SaveConfiguration ()• void ResetConfiguration ()• template<typename T >

Parameter< T > & GetParameter (const ParameterNames &parameterName)• template<typename T >

const T GetParameterValue (const ParameterNames &parameterName)

10

2.3 Configuration Class Reference 11


This class provides parameters to the program. Default values can be overridden byusing a configuration file on the format: parameter1=value1 parameter2=value2 # Thisis a comment parameter3=value3

2.3.2 Member Enumeration Documentation

2.3.2.1 enum Configuration::ParameterNames

Supported configuration parameters.

Enumerator:

SERVERPORTLOGLEVELLOGOUTPUTCOMPORTBAUDRATEDATABITSSTOPBITSPARITYFLOWCONTROLSERIALTIMEOUT


2.3.3.1 Configuration::Configuration ( )

Default constructor. Initializes the Configuration by setting default values and readingvalues from an existing configuration file.


2.3.4.1 template<typename T > Parameter<T>&Configuration::GetParameter ( const ParameterNames &parameterName ) [inline]

Gets the Parameter representation the specified parameter name.

Parametersin parameter-

NameName of parameter.

11

2.4 DCPController Class Reference 12

Returns

Parameter object matching parameterName.

2.3.4.2 template<typename T > const T Configuration::GetParameterValue (const ParameterNames & parameterName ) [inline]

Gets the parameter value of the specified parameter name.

Parametersin parameter-

NameName of parameter.

Returns

Parameter value for matching Parameter.

2.3.4.3 void Configuration::ResetConfiguration ( )

Resets the current configuration with default values.


��

2.3.4.4 void Configuration::SaveConfiguration ( )

Saves the current configuration to file.

2.4 DCPController Class Reference

Interface between client and hardware.

12


Collaboration diagram for DCPController:

��

��

��

��

��

Classes

• class CommandRepresentation of command with command-specific arguments.

Public Types

• enum Commands• enum Status• enum ErrorCode• enum MotorType


• DCPController ()• ∼DCPController ()• std::string Operate (std::string &data)


The DCPController class acts as the true interface between client and hardware. Con-tains business logic for performing all supported commands.

13


2.4.2 Member Enumeration Documentation

2.4.2.1 enum DCPController::Commands

Supported incoming commands.

Enumerator:

SENSORCLAMPFFCFNCVERSIONWORKPOSCHANGEPOSGETPOSHOMEFOCUSMOVEXSETMODESETRESSETSPEEDSETALLAFSNAPIMAGELED

2.4.2.2 enum DCPController::ErrorCode

Error codes.

Enumerator:

UNDEFINED_ERRORSYNTAX_ERRORUNKNOWN_COMMANDINVALID_ARGUMENT

2.4.2.3 enum DCPController::MotorType

Motor types.

Enumerator:

M_FOCUSM_CLAMPM_ROTATION

14


2.4.2.4 enum DCPController::Status

Operation status.

Enumerator:

OKERRORFAULT


2.4.3.1 DCPController::DCPController ( )


2.4.3.2 DCPController::∼DCPController ( )

Destructor.


2.4.4.1 string DCPController::Operate ( std::string & data )

Performs a DCP operation based on incoming data.

Parametersin data Command data received from client.

Returns

Response of operation.


��

15

2.5 DCPCS Class Reference 16

2.5 DCPCS Class Reference

Defines the startup of the application.

Collaboration diagram for DCPCS:

��

��

��

��

��


• DCPCS ()• ∼DCPCS ()• void Stop ()


Defines the startup of the application. Contains the the entry point for the DCPCS.Initializes global functionality, and starts the server component to accept incomingclient connections.


2.5.2.1 DCPCS::DCPCS ( )

Standard constructor. Initializes global functionality.

16

2.6 ErrorException Class Reference 17

2.5.2.2 DCPCS::∼DCPCS ( )

Destructor.


2.5.3.1 int DCPCS::Run ( )

2.5.3.2 void DCPCS::Stop ( )

Stop the server.

2.6 ErrorException Class Reference

Exception representing communication error.


• ErrorException ()• ErrorException (const std::string &msg)• virtual ∼ErrorException () throw ()• virtual const char ∗ what () const throw ()


This exception class indicates a communication error or an invalid request message.


2.6.2.1 ErrorException::ErrorException ( ) [inline]


2.6.2.2 ErrorException::ErrorException ( const std::string & msg )[inline, explicit]

Overloaded constructor. Creates an ErrorException with specified message.

Parametersin msg Message describing the exception thrown.

17

2.7 Logger Class Reference 18

2.6.2.3 virtual ErrorException::∼ErrorException ( ) throw () [inline,virtual]

Destructor.


2.6.3.1 virtual const char∗ ErrorException::what ( ) const throw ()[inline, virtual]

Gets message associated with this exception.


��

��

��

��

2.7 Logger Class Reference

Provides logging functionality.


• Logger (boost::asio::io_service &ioService)• ∼Logger ()• void Log (LogLevels logLevel, const std::string &message)• void SetLogLevel ()• void SetOutput ()


This class provides logging functionality across the system. The logger supports dif-ferent severity levels and can output logdata to file, console, or both.

18



2.7.2.1 Logger::Logger ( boost::asio::io_service & ioService )

Constructor. Creates a thread to run a private io_service for logging.

Parametersin ioService Service to provide core I/O services.

2.7.2.2 Logger::∼Logger ( )

Destructor. Shuts down the private io_service.


2.7.3.1 void Logger::Log ( LogLevels logLevel, const std::string & message )

Logs a message.

Parametersin logLevel Severity log level.in message Message to log.

19



��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

2.7.3.2 void Logger::SetLogLevel ( )

Sets the severity log level to use by reading value from Configuration.

Note

: This call must be done after the Configuration is initiated.

20

2.8 Configuration::Parameter< T > Class Template Reference 21


��

2.7.3.3 void Logger::SetOutput ( )

Sets the log output to use by reading value from Configuration.

Note

: This call must be done after the Configuration is initiated.


��

2.8 Configuration::Parameter< T > Class Template Reference

Represents a configurable parameter.


• Parameter (const std::string &name, const T &value, const std::string &descrip-tion)

• Parameter (const std::string &name, const T &value)• const std::string & GetName () const• const std::string & GetDescription () const• const T & GetValue () const• void SetValue (const T value)

21



template<class T> class Configuration::Parameter< T >

Represents a configurable parameter of type T used in Configuration.

Template Parameters<T> {int, bool, string}.


2.8.2.1 template<class T> Configuration::Parameter< T >::Parameter (const std::string & name, const T & value, const std::string &description ) [inline]

Constructor. Creates a parameter with given name and value.

Parametersin name Name of parameter.in description Description of parameter.in value Value of parameter.

2.8.2.2 template<class T> Configuration::Parameter< T >::Parameter (const std::string & name, const T & value ) [inline]

Overloaded constructor. Creates a parameter with given name and value. The descrip-tion of the parameter is set to same as the name.

Parametersin name Name of parameter.in value Value of parameter.


2.8.3.1 template<class T> const std::string& Configuration::Parameter< T>::GetDescription ( ) const [inline]

Gets the description of the parameter.

Returns

Description of parameter.

22


2.8.3.2 template<class T> const std::string& Configuration::Parameter< T>::GetName ( ) const [inline]

Gets the name of the parameter.

Returns

Name of parameter.

2.8.3.3 template<class T> const T& Configuration::Parameter< T>::GetValue ( ) const [inline]

Gets the value of the parameter.

Returns

Value of parameter.


��

��

��

��

��

2.8.3.4 template<class T> void Configuration::Parameter< T >::SetValue (const T value ) [inline]

Sets the value of the parameter.

Parametersin value Value of parameter.

23

2.9 SerialCom Class Reference 24


��

��

��

��

2.9 SerialCom Class Reference

Provides support for serial communication.


• SerialCom ()• ∼SerialCom ()• void Open ()• void Close ()• std::string WriteRead (const std::string &request)• std::string Read ()• void Write (const std::string &request)• void SetTimeout (const boost::posix_time::time_duration &duration)


Provides support for serial communication. Currently only supports ASCII data trans-fers. Manages socket timeouts by applying the concept of deadlines.


2.9.2.1 SerialCom::SerialCom ( )

Constructor. Creates a serial port for the device name specified in Configuration, with-out opening it.

2.9.2.2 SerialCom::∼SerialCom ( )

Destructor. Closes the serial port.

24



2.9.3.1 void SerialCom::Close ( )

Closes the serial device.


��

��

��

2.9.3.2 void SerialCom::Open ( )

Opens the serial device and sets communication settings.


��

2.9.3.3 string SerialCom::Read ( )

Reads data from serial device.

Returns

Response.

25



��

2.9.3.4 void SerialCom::SetTimeout ( const boost::posix_time::time_duration& duration )

Sets the timeout for read and write operations. In order to disable the timeout, provideboost::posix_time::seconds(0) as parameter.

Parametersin duration Timeout value.

2.9.3.5 void SerialCom::Write ( const std::string & request )

Writes data to serial device.

Parametersin request Data to write.


��

26

2.10 Session Class Reference 27

2.9.3.6 string SerialCom::WriteRead ( const std::string & request )

Sends a request to serial device and returns the response. Opens and closes the connec-tion as needed.

Parametersin request Request to serial device.

Returns

Response from serial device.

2.10 Session Class Reference

Represents a client-server session, performing data exchange.

Collaboration diagram for Session:

��

��

� �

��

��

��

��

��

��


• Session (boost::asio::io_service &ioService, SessionManager &manager)• ∼Session ()• void Start ()• void Stop ()• tcp::socket & GetSocket ()

27

2.10 Session Class Reference 28


This class handles the actual socket and TCP connection, performing data exchangewith a client. This class inherits enable_shared_from_this and utilizes shared_ptr inorder to keep the Session object alive as long as there is an operation that refers to it.


2.10.2.1 Session::Session ( boost::asio::io_service & ioService, SessionManager& manager ) [explicit]

Construct a connection with the given io_service and SessionManager.

2.10.2.2 Session::∼Session ( )

Destructor.


2.10.3.1 tcp::socket & Session::GetSocket ( )

Gets the TCP socket for this Session.

Returns

TCP socket in use.

2.10.3.2 void Session::Start ( )

Starts the Session and data transfer. As long as the the Session is alive, it reads datafrom client, forwards request to DCPController and sends response back to client.

2.10.3.3 void Session::Stop ( )

Stops the session.


��

28

2.11 SessionManager Class Reference 29

2.11 SessionManager Class Reference

Manages open sessions (connections).


• void Start (session_ptr session)• void Stop (session_ptr session)• void StopAll ()


Manages open sessions (connections) so that they may be cleanly stopped when theserver needs to shut down.


2.11.2.1 void SessionManager::Start ( session_ptr session )

Adds the specified session to the manager and starts it.

Parametersin session Session to be started.

2.11.2.2 void SessionManager::Stop ( session_ptr session )

Stops the specified session.

Parametersin session Session to be stopped.


��

29

2.12 TCPServer Class Reference 30

2.11.2.3 void SessionManager::StopAll ( )

Stops all sessions.


��

2.12 TCPServer Class Reference

Defines a TCP socket-based server.

Collaboration diagram for TCPServer:

��

��

��


• TCPServer (boost::asio::io_service &ioService, unsigned short portNumber)• void Stop ()


Defines a TCP socket-based server, asynchronously accepting incoming client connec-tions and services client requests. The TCP server will run on the TCP port given tothe constructor.

30

2.12 TCPServer Class Reference 31


2.12.2.1 TCPServer::TCPServer ( boost::asio::io_service & ioService,unsigned short portNumber )

Standard constructor. Initializes an acceptor to listen on the port given by portNumber.

Parametersin ioService Service to provide core I/O services.in portNumber Port number to listen on.


2.12.3.1 void TCPServer::Stop ( )

Stops the server by cancelling all outstanding asynchronous operation.


��

31

Date post:	15-Mar-2020
Category:	Documents
Upload:	others
View:	14 times
Download:	0 times

Control software for positioning and autofocus in an ... · Control software for positioning and...

Documents