Scientific Fishery Systems, Inc
1
Observer’s Associate
A consistent, unbiased system using machine
vision and fish morphometrics to identify
species
From
Scientific Fishery Systems, Inc.P.O. Box 242065
Anchorage, AK 99524
907.563.3474
Dr. Eric O. Rogers
Scientific Fishery Systems, Inc
2
Observer’s Associate Team
• Principle Investigator - Pat Simpson - SciFish
• Lead Scientist Eric O. Rogers, PhD (Physics) - SciFish
• Luke Jadamec, Fisheries Observer Trainer
• Joe Imlach PE, PhD (ME) Imlach Consulting
• Chris Bublitz, UAF Fisheries Industrial Technology Center
Scientific Fishery Systems, Inc
3
Issues identified by SciFish
• Increasing pressure on resource
• Increasing complexity of new legislation
• Possible environmental changes affecting fishery in unknown ways
• Appropriately harvesting and managing the fishery are increasingly difficult tasks
=> Need the best data possible <=
Scientific Fishery Systems, Inc
4
Current Sources of Data
• AFSC Survey Trawls– Practical limits to time and scope
• Observer’s Reports– Most effective means of
monitoring CPUE– Statistically small sample– Potentially biased by factors
outside observer’s and vessel operators control
– Of questionable value in legal action due to statistical nature of data
Scientific Fishery Systems, Inc
5
SciFish’s Proposal• Using funding form the NSF build
and test an automated onboard fish cataloging system using COTS Hardware and Software that will:– Assist commercial fishery observers
with their monitoring and assessment tasks at sea
– Provide detailed unbiased species counts to manage the Community Development Quota (CDQ) program in Western Alaska
– Provide new detailed information on the ecological health of each species to assist in fisheries management
– Provide detailed information on fish morphometrics that will be of value to researchers in several academic areas, such as fish population studies and fish evolution
Scientific Fishery Systems, Inc
6
Key Concepts
• COTS hardware and software
• Candle the fish to separate from background
• Machine Vision and Morphometrics
• Neural Net
• Sample all the fish
• System scales - can add CPU’s for faster processing and add metrics and/or color for greater accuracy
Scientific Fishery Systems, Inc
7
Observer’s Associate Benefits
• More and better data means fewer surprises for managers and skippers
• A healthier fishery through management based upon more complete knowledge
• Sample entire catch, no extrapolation• Fair and impartial catch statistics - a
level playing field• Easy to identify and reward “clean”
Vs “dirty” boats• Brings in non-traditional funds for
fisheries research (NSF $)• Fringe Benefit => Provides length,
width, etc. for each fish in addition to species
Scientific Fishery Systems, Inc
8
Observer’s Associate Mechanical Design
Transparent conveyor
Light table
Light reflector
Camera(s)
Image processor in watertight compartment
Heat exchanger
Belt cleaner
Scientific Fishery Systems, Inc
9
Observer’s Associate Logic Flow
ImageCapture
BoundaryDetection
MeasureFish
IdentifySpecies
DataStorage
Fish
Imag
e
FishMetrics
Fish
Species
Fish O
utline
Fish Im
age
Fish M
etrics
Scientific Fishery Systems, Inc
10
Flatfish Features Used by People
Scientific Fishery Systems, Inc
11
Typical Flatfish Features Used by Machine Vision
Body Width Standard LengthTail Length Standard LengthTail Fork Length or Max width to tip for rounded tails Standard LengthBody Width Standard Length(Total Width Body Width) / Standard Length(Ellipse {standard length and body width} - body perimeter) Standard Length“Fin” Perimeter (Total Perimeter – Body Perimeter) Standard Length (Ellipse Area – Body Area) (Standard Length * Body Depth)Fin Area / (Standard Length)2
Scientific Fishery Systems, Inc
12
Concept Test
• Scan Pictures from Northeast Pacific Flatfishes Book– Scale to meter stick in picture
– Extract measurements
• Reduce measurements to independent metrics– Principle component analysis
• Train Neural Net• Create 100 fish / species by adding
various percentages of white noise• Test classifier with “white noise”
fish
Scientific Fishery Systems, Inc
13
Normalized Machine Vision Flatfish Metrics
W1 W2 W3 W4 W5 W6 W7 W8Arrowtooth Flounder -3.79 0.06 -0.28 0.33 -0.58 0.17 -0.33 -0.69Greenland Turbot -2.95 -0.06 -1.31 -0.22 0.47 -0.48 -0.33 0.48Pacific Halibut -1.89 -0.99 -1.37 0.00 -0.55 -0.39 0.52 0.76Rex Sole -1.75 0.26 1.06 -0.26 0.80 0.37 0.62 0.82Dover Sole -1.18 -1.98 0.70 0.44 0.16 0.56 -0.58 0.16Deepsea Sole -0.61 -0.49 1.70 -0.34 -0.17 -1.02 -0.45 -0.43Sand Sole -0.56 1.11 0.06 0.18 0.25 0.18 -0.44 0.67Petrale Sole -0.51 0.40 -0.10 0.14 0.43 -0.22 -0.16 -0.91Kamchatka Flounder -0.17 1.27 -0.34 0.10 -0.33 0.16 0.27 0.12English Sole 0.00 1.00 -0.87 -0.03 0.11 0.20 1.21 -0.65Bering Flounder 0.51 1.93 0.23 -0.49 -0.25 0.21 -0.67 -0.35Flathead Sole 0.93 0.87 0.51 -0.09 -0.18 0.73 -0.36 -0.40Rock Sole 1.15 -1.58 1.10 0.24 0.23 -0.25 0.59 -0.88Butter Sole 1.56 -0.55 1.46 -0.05 -0.58 0.08 0.39 1.27Yellowfin Sole 2.31 0.48 -0.20 0.06 -0.09 -0.21 0.74 -0.32Starry Flounder 2.74 -2.77 -1.58 -0.45 0.04 0.43 -0.39 -0.25Alaskan Plaice 4.19 1.05 -0.78 0.46 0.23 -0.51 -0.64 0.60
TailLength
Tail ForkLength
BodyDepth
FinDepth
El-bPerimeter
"Fin"Perimeter
El-bArea
FinArea
Pacific Halibut 0.9 0.7 0.8 0.9 1.0 1.1 0.5 0.5Alaskan Plaice 1.6 1.2 1.3 1.5 1.5 1.5 1.3 1.6Greenland Turbot 0.9 0.0 0.8 0.9 2.0 0.8 0.5 0.5Arrowtooth Flounder 0.7 0.0 0.8 0.4 1.2 0.7 0.8 0.4Dover Sole 1.0 1.2 0.9 0.8 -0.4 0.7 0.5 0.9Rex Sole 0.9 1.3 0.7 0.9 1.9 0.7 0.9 1.0Yellowfin Sole 1.1 1.3 1.2 1.3 1.0 1.3 1.4 1.3Flathead Sole 1.2 1.2 1.1 1.0 1.1 1.0 1.5 1.1Rock Sole 0.8 1.7 1.2 1.2 -0.4 1.0 1.0 1.4Butter Sole 1.2 2.4 1.0 1.0 0.9 1.1 1.0 1.3Starry Flounder 1.3 1.4 1.2 1.7 -1.6 1.2 0.7 0.9Kamchatka Flounder 1.1 0.8 0.9 0.9 1.9 1.1 1.3 0.9Deepsea Sole 0.8 1.5 1.2 0.7 1.3 0.8 0.8 1.1Petrale Sole 0.9 0.4 1.0 1.0 1.2 0.9 1.1 1.0English Sole 0.9 0.6 0.9 1.2 1.4 1.2 1.4 1.0Sand Sole 1.1 0.8 0.9 0.9 2.0 0.9 1.0 1.0Bering Flounder 1.2 0.8 1.1 0.9 2.2 1.0 1.5 1.0
Metrics after reduction to Principle Component Vectors
Scientific Fishery Systems, Inc
14
0
1 0
2 0
3 0
4 0
5 0
6 0
7 0
8 0
9 0
1 0 0
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 0
W h it e N o is e ( % )
Misc
lassif
ied (#
/1700
sam
ples
)
F a ls e P o s i t i v e F a ls e N e g a t i v e
F ig u r e 3 . Y e l lo w f in S o le M is c la s s i f ic a t io n P e r c e n t a g e V s . W h it e N o is e
0
1 0
2 0
3 0
4 0
5 0
6 0
7 0
8 0
9 0
1 0 0
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 0
W h it e N o is e ( % )
Mis
clas
sifie
d (#
/170
0 sa
mpl
es)
F a ls e P o s it iv e F a ls e N e g a t iv e
F ig u r e 4 . R o c k S o le M is c la s s i f ic a t io n P e r c e n t a g e V s . W h i t e N o is e
Neural Net Classification Results
Scientific Fishery Systems, Inc
15
Observer’s Tasks
• Identify Species that Observer’s Associate does not
• Quality Control
• Ensure Appropriate Sampling
• Operate the Observer’s Associate
• Ensure data integrity and file reports
Scientific Fishery Systems, Inc
16
Plan
• Assemble Advisory Panel
• Apply for ASTF Bridge Grant
• Build “Proof of Concept” Prototype
• Train and Test Prototype
• Apply for NSF Phase II Grant
• Build true prototype
• Test for volume onshore
• Test for suitability at sea
• Initial implementation in the Yellowfin Sole fishery
Scientific Fishery Systems, Inc
17
Advisors PanelComposition
• Regulators
• Conservationists
• Fisheries Scientists
• CDQ Groups
• Fishermen
• Owners
• Fisheries Consultants
Scientific Fishery Systems, Inc
18
Advisory Panel Questions
• Are the issues identified by SciFish of Concern to the industry?
• Is the technology presented a viable solution?
• Are the other, more appropriate solutions to the problems?
• What is the best way to implement this solution?
• Design Changes?• Are there other applications to add
value to the system?• Number of classes for fish Vs
accuracy of classification, Vs throughput of fish Vs cost