Date post: | 06-Jan-2017 |
Category: |
Data & Analytics |
Upload: | institute-of-contemporary-sciences |
View: | 23 times |
Download: | 0 times |
Data Science Conference11-12 October 2016Belgrade, Serbia
Aircraft challenge
Marko, don’t forget to show
the aircraft to the audience
Dr David Warren
3400 G shock for 6.5 ms
500 lb. Dropped from 10 ft with a ¼-inch-diameter contact point
1100 ºC flame for 30 minutes.
260 ºC for 10 hours
Immersion in aircraft fluids for 24 hours
Immersion in sea water for 30 days
5,000 pounds crush for 5 minutes on each axis
Pressure equivalent to depth of 20,000 ft.
Avoiding Black Boxes with Data ScienceRaffaele Rainone & Marko Vasiljevski
Data Science Conference11-12 October 2016Belgrade, Serbia
Hkjk;l
Dr Raffaele Rainone
Hkjk;l“I don’t know…, pure mathematician, Python developer, data scientist, pizza lover, feeling a bit home sick for Italy, …”
Chatting about…
Aviation Safety
A story about flight data monitoring (FDM)A story of a flight data analyst
A story of a flight safety statistician
Chatting about…
Data science in (a bit of) action
Jet-engine health - Working with Mr. BayesDetecting safety concerns – PCA, a friend
Finding cuckoo’s eggs – Mr. Markov’s chains
Flight Data Monitoring (FDM)
Flight Data Monitoring (FDM)
Flight Data Monitoring (FDM)
Flight Data Monitoring (FDM) what?!
Part of safety management system (SMS)Airlines worldwide obliged to do FDMSpotting deviations from safe operationNon-punitive – learn from mistakesConfidential
Quick Access Recorder
Flight Data Recorder
Data Acquisition Unit
002AF5C0 00 00 76 07 04 01 64 08 06 01 3C 0C F2 0E 3C 0C ..v...d...<.ò.<.002AF5D0 00 00 58 02 02 0C 00 0C 48 08 00 00 00 00 00 0C ..X.....H.......002AF5E0 00 0C 00 00 00 00 E0 0F 00 00 00 0C 40 0C 00 0C ......à[email protected] 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90 01 ...............�002AF600 47 02 77 07 03 00 2B 0E DF 0F 02 0C A5 0F 78 00 G.w...+.ß...¥.x.002AF610 55 04 D3 01 D8 0F EA 03 A4 0F EE 0F FD 0F 00 00 U.Ó.Ø.ê.¤.î.ý...002AF620 08 00 10 00 00 00 08 00 11 00 00 00 06 00 1E 00 ................002AF630 F8 0F 10 00 00 00 8C 0B 0C 00 0A 00 00 00 00 00 ø.....Œ.........002AF640 00 00 77 07 96 0F 64 08 06 01 3C 0C F2 0E 08 08 ..w.–.d...<.ò...002AF650 00 00 00 00 01 00 AC 00 4C 08 01 08 00 00 00 00 ......¬.L.......002AF660 00 00 04 02 D0 02 0D 02 12 02 00 0B 44 01 00 00 ....Ð.......D...002AF670 A4 06 40 01 03 00 00 00 40 02 00 00 00 00 00 00 ¤.@[email protected] 02 00 77 07 00 00 2B 0E DF 0F 04 0C A5 0F 00 08 ..w...+.ß...¥...002AF690 54 04 D4 01 DA 0F EA 03 A4 0F 73 0E 00 00 01 00 T.Ô.Ú.ê.¤.s.....002AF6A0 08 00 10 00 01 00 08 00 00 00 03 00 06 00 1D 00 ................002AF6B0 FB 0F 11 00 02 00 01 00 00 00 03 00 00 00 00 00 û...............002AF6C0 01 00 76 07 00 00 64 08 06 01 3C 0C F2 0E 3C 0C ..v...d...<.ò.<.002AF6D0 6A 02 21 03 03 00 00 00 48 08 00 00 00 00 00 00 j.!.....H.......002AF6E0 00 00 00 00 00 00 08 00 00 00 00 00 00 00 00 00 ................002AF6F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FC 0F ..............ü.002AF700 CC 08 77 07 03 00 2B 0E DF 0F 02 0C A5 0F 79 01 Ì.w...+.ß...¥.y.002AF710 54 04 D4 01 D8 0F EA 03 A7 0F EE 0F FE 0F 01 00 T.Ô.Ø.ê.§.î.þ...002AF720 08 00 00 00 00 00 00 00 11 00 02 00 06 00 00 00 ................002AF730 F8 0F 00 00 00 00 8C 0B 00 00 00 00 00 00 00 00 ø.....Œ.........002AF740 00 00 76 07 94 0F 64 08 06 01 3C 0C F2 0E 08 08 ..v.”.d...<.ò...002AF750 9A 0C 18 00 61 0C 68 01 48 08 F8 0F A4 0C 00 00 š...a.h.H.ø.¤...002AF760 00 00 00 00 00 00 00 00 00 00 00 00 00 00 5B 02 ..............[.002AF770 00 00 03 02 00 00 2B 0F F0 0F 2A 0F 00 00 00 00 ......+.ð.*.....002AF780 00 00 77 07 00 00 2B 0E DF 0F 04 0C A5 0F 2C 00 ..w...+.ß...¥.,.002AF790 54 04 D4 01 D8 0F EB 03 A6 0F 00 0D 00 00 01 00 T.Ô.Ø.ë.¦.......002AF7A0 08 00 17 00 03 00 0F 00 00 00 00 00 06 00 1C 00 ................002AF7B0 F8 0F 10 00 02 00 01 00 00 00 03 00 00 00 00 00 ø...............002AF7C0 00 00 76 07 11 00 64 08 06 01 3C 0C F2 0E 3C 0C ..v...d...<.ò.<.
What now?Data frames
Questions, suggestions, congestions?
The story of a flight data analyst
Questions, suggestions, congestions?
The story of a flight safety statistician
The story of a flight safety statistician
STATISTICS
How do we monitor safety in FDM
012345678
Event Count 3.141592653589793238462643383384
And…
Domain knowledgeExperience
Common sense
Uncertainty - good companion
Normal acceleration - Tdwn Normal acceleration – Lift-off
Statistic ValueTotal count 80,663
Average 1.31(Min, Max) (1.03,
2.40)Range 1.37
Standard deviation 0.10
Statistic ValueTotal count 80,663
Average 1.19(Min, Max) (0.66,
1.63)Range 0.97
Standard deviation 0.05
Wider histogram,
less confidence
in mean value
Narrowerhistogram,
more confidence
in mean value
How apples relate to flight safety
Micromanaging events
0369
Top 5 Events – January 2016
Micromanaging events
High sp
eed t
axiing
High la
teral
g tax
iing
Flap o
versp
eed
Airspe
ed hi
gh 10
000-5
000 f
t
Exces
sive b
reakin
gPu
ll up
Stick
shake
r0369
Top 7 Events – January 2016
Most severe safety event – a riddle
It doesn’t happen in the air
It doesn’t happen at the gate
It happens in the office
Micromanaging events
High sp
eed t
axiing
High la
teral
g tax
iing
Flap o
versp
eed
Airspe
ed hi
gh 10
000-5
000 f
t
Exces
sive b
reakin
gPu
ll up
Stick
shake
r0369
Top 7 Events – January 2016
Not looking at your data!
Trending?
Correlation?
Questions, suggestions, congestions?
Enough, it’s a data science conference!
Improving current state
Jet-engine health with Mr. Bayes
Engine performance monitoring
N1 N2
Fuel
Flow
Exhaust Gas
Temperature
Engine performance monitoring
Fuel
Flow
Exhaust Gas
Temperature
Engine performance monitoring
End of take-off flow
End of take-off flow
End of take-off flow
End of take-off flow
End of take-off flow
End of take-off flow
End of take-off flow
End of take-off – a problem
If not detected – problems with engine healthDifficult to do with classical signal processing
What if a parameter fails (not so rarely)
End of take-off – a solution
Time at vertical navigation mode selected (VNAV)Drop/rise in fuel flow (FF) and gas temperature (EGT)Search for changes around that point in timeGather the knowledgeCalculate probability for EOT for unseen flight
End of take-off – a solution
Time at vertical navigation mode selected (VNAV)Drop/rise in fuel flow (FF) and gas temperature (EGT)Search for changes around that point in timeGather the knowledgeCalculate probability for EOT for unseen flight
End of take-off – a solution
End of take-off – a solution
End of take-off – a solution
End of take-off – a solution
End of take-off – a solution
If crazy about this, search for PyData London 2016 videoM. Vasiljevski & R. Rainone
“Python flying at 40,000 feet”
Questions, suggestions, congestions?
Finding new concerns
Novelty detection with principal component analysis
Standardising
mean = 0standard deviation = 1
Formally…
Previous 700 x 123 matrix is A Multiply transpose of A with A to get covariance matrix, C.
It’s 123 x 123 Do singular value decomposition of C to find eigenvectors
and eigenvalues Eigen vectors coincide with directions of highest variance Keep just first couple of vectors to reduce dimensionality
Questions, suggestions, congestions?
Helping business
Flight data upload monitor based on Markov chains
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
State1 = M x State0State2 = M x State1 = M x (M x State0) = M2
x State0StateN = MN x State0
Don’t have to know states, just powers of M (probabilities)
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Flight data upload data data data
Questions, suggestions, congestions?
Credits
CHRIS JESSE
RAFFAELE (PIZZA) RAINONE
MARTA
VASI
LJEV
SKI
MILA
N
BOROTA
MILIC
A IVA
NIŠEV
IĆ+ foeniculumvulgare
ZIZI(paid for my tickets)
Main takeaways from this talk
LEARN
HAVE FUN
SHARE KNOWLEDGE
THINK
PLAY
THANK YOU & SAFE FLYING