Post on 18-Sep-2020
transcript
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Developing Computer-Vision Solutions forSensing Problems in Intelligent Systems
A research vision on its challenges and opportunities
Soumyabrata Devhttps://soumyabrata.github.io/
The ADAPT SFI Research CentreTrinity College Dublin, Ireland
University of DerbyApril 27, 2018
1 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Where have I been, and where am I now?
Postdoc at The ADAPT SFI Research Centre, Trinity CollegeDublin, Ireland (2017-present).
Ph.D. in Electrical Engineering, NTU Singapore (2012-2017).
Visiting Doctoral Assistant, EPFL Lausanne (Aug-Dec 2015).
Engineer at Ericsson (2010-2012).
Undergrad in electronics engineering, NIT Silchar, India(2006-2010).
2 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
My high-level research interests
Research Vision
Identifying specific computational problems in intelligent systems;and solve them in a collaborative manner with the domain experts.
My research interests are mainly in the field of imageprocessing, computer vision, machine learning and deeplearning.
My early faculty research plan -• Solar Energy Analytics [60%]• Smart Traffic Analytics [30%]• Neuroimaging Analytics [10%]
Firm believer of open science 1 and research reproducibility 2.
1Advisor role at Overleaf, 207 Regent Street, London.
2P. Vandewalle, J. Kovacevic, M. Vetterli, Reproducible Research in Signal Processing – What, why, and how,
IEEE Signal Processing Magazine, vol. 26, no. 3, pp. 3747, 2009.
3 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
1 Solar Energy Analytics
2 Smart Traffic Analytics
3 Neuroimaging Analytics
4 Conclusion
4 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
How to Solve Nature’s Energy Crisis Using Imaging Data?
Solar (and other renewable) energy is necessary to beharnessed to reduce our dependency on traditional fossil fuels.
Can imaging data assist in solar analytics?• Traditionally done via satellite images.• Can we build intelligent systems to perform better?
Spatial resolution
Tem
por
al r
eso
lutio
nS
hort
term
Long
term
5 m
inut
es1
hour
1 da
y1
wee
k
500 meters 50 kilometers 500 kilometers
Whole Sky Imagers(WSI)
Satellites
5 se
cond
s
50 meters
Machine learning techniques on earth’s imaging data canprovide us better insights in solar analytics.
5 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Intelligent System Design
WAHRSIS: Wide Angle High-Resolution Sky Imaging System
Easy-to-design model 3.Low-cost (2000USD) and high image-resolution (18MP).
(a) WAHRSIS (b) Image
Planning to further miniaturize these sky cameras 4 5foreffective deployment at multiple locations.
3S. Dev, F. M. Savoy, Y. H. Lee and S. Winkler, DIY Sky Imager For Weather Observation: A complete guide
to build a ground-based sky imager using off-the-shelf components with automatic cloud coverage computation,IEEE Signal Processing Society, 2016.
4S. Dev et al. WAHRSIS: A low-cost, high-resolution whole sky imager with near-infrared capabilities, Proc.
IS&T/SPIE Infrared Imaging Systems: Design, Analysis, Modeling, and Testing, May 2014.5
S. Dev et al. Design of low-cost, compact and weather-proof whole sky imagers for high-dynamic-rangecaptures, Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2015. 6 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Challenges & Opportunities
Imaging data obtained from the ground can help us inunderstanding the solar irradiance fluctuations. 6
08:0009:00
10:0011:00
12:0013:00
14:0015:00
16:0017:00
18:000
200
400
600
800
1000
Sola
r ra
dia
tion [
Watt
/m2]
2015-12-07
Weather station
Clear sky model
10:30 (758 W/m2) 10:32 (283 W/m2) 10:34 (714 W/m2)
Fig: Impact of clouds on direct solar irradiance.
6S. Manandhar and S. Dev et al. Analyzing Solar Irradiance Variation From GPS and Cameras, IEEE AP-S
Symposium on Antennas and Propagation and USNC-URSI Radio Science Meeting, 2018 7 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Challenges & Opportunities
In what ways can the rapid fluctuations of the solar irradiancebe best captured?
Estimating solar energy 7 from captured sky camera images.
Hemispheric sampling Projection on the image.
Techniques from computational geometry 8 + machinelearning 9 can assist in harnessing such insights from thesky/cloud images.
7S. Dev et al. Estimation of solar irradiance using ground-based whole sky imagers, Proc. IEEE International
Geoscience and Remote Sensing Symposium (IGARSS), July 2016.8
S. Dev, A. Ghasemi, M. Vetterli, A. Scholefield, Point localization in Multi-camera system, Doctoralinternship report, Ecole Polytechnique Federale de Lausanne, Switzerland, 2015.
9S. Dev et al., Machine Learning Techniques and Applications For Ground-based Image Analysis, IEEE
Geoscience and Remote Sensing Magazine, 2016.
8 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Challenges & Opportunities
How can we obtain a reliable short-term solar energygeneration forecasting 10?
Input at t − 2’
Input at t’
Horizontal translation
Vertical translation
Actual at t + 2’
Predicted at t + 2’
Initial contacts with: Center for Energy Research, UC SanDiego and Solar Energy Research Institute of Singapore.
10S. Dev et al. Short-term prediction of localized cloud motion using ground-based sky imagers, Proc.
TENCON 2016 - 2016 IEEE Region 10 Conference, November 2016.
9 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
1 Solar Energy Analytics
2 Smart Traffic Analytics
3 Neuroimaging Analytics
4 Conclusion
10 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
How Can Imaging Data Help in Assistive Driving?
Massive interest in the field of autonomous driving usingimage- and video- analytics.
Can we provide an end-to-end system in detecting the traffic-and road- signs from low-resolution car dashboard cameras?
Automatic detection of traffic- light and signage.
Possibility of a multi-modal approach with Near-Infra-Red(NIR) 11 + GPS 12, in addition to RGB camera images.
11S. Dev, F. M. Savoy, Y. H. Lee, S. Winkler, WAHRSIS: A low-cost, high-resolution whole sky imager with
near-infrared capabilities, Proc. IS&T/SPIE Infrared Imaging Systems: Design, Analysis, Modeling, and Testing,May 2014.
12S. Manandhar, Y. H. Lee, S. Dev, GPS Derived PWV for rainfall monitoring, Proc. IEEE International
Geoscience and Remote Sensing Symposium (IGARSS), July 2016.
11 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Challenges & Opportunities
My current project (at The ADAPT Centre, Dublin) involvesbillboard object detection and localization.
Several challenges in this task -• Deep learning models do not generalize across variouslocations.• Lack of publicly available datasets.• Based on only RGB camera images.
12 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Challenges & Opportunities
Color-based 13 object detection can help in identifying trafficsigns.
Techniques from set theory 14 + deep learning 15 are helpfulfor object detection.
Initial work with ai Robotics, an Anuva Ventures start-upcompany incubated in Singapore.
13S. Dev et al. Color-based segmentation of sky/cloud images from ground-based cameras, IEEE Journal of
Selected Topics in Applied Earth Observations and Remote Sensing, 2017.14
S. Dev et al., Rough Set Based Color Channel Selection, IEEE Geoscience and Remote Sensing Letters, 2017.15
S. Dev et al. CloudSegNet: A Deep Network for Nychthemeron Cloud Segmentation, Proc. IEEEInternational Conference on Image Processing (ICIP), 2018 (under review).
13 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
1 Solar Energy Analytics
2 Smart Traffic Analytics
3 Neuroimaging Analytics
4 Conclusion
14 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Can Medical Imaging Data Assist in UnderstandingChronic Schizophrenia?
Grey matter volumes in cortical and sub-cortical regions ofbrain 16 can provide insights in chronic schizophrenia. 17
How can we calculate the grey matter volumes in cortical andsub-cortical regions of brain using multimodal neuroimaging?
16From web.stanford.edu/group/hopes/cgi-bin/hopes_test/dementia-in-huntingtons-disease/
17Comparison of grey matter volume and thickness for analysing cortical changes in chronic schizophrenia: a
matter of surface area, grey/white matter intensity contrast, and curvature [Kong et al. Psychiatry Res. 2015].
15 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Challenges & Opportunities
Performed early experiments in this field and providedsuper-pixel imaging solutions for generation of Cortical andSub-cortical masks 18.
Slice 96 of .nii image Oversegmentation Three-level segmentation.
Initial work with National Neuroscience Institute, Singapore.
18S. Dev, Multimodal Neuroimaging: Analyzing MRI images using SPM, Technical report, 2017.
16 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Where am I now, and what’s next?
Finishing my first industry-affiliated project with the ADAPTSFI Research Centre.
Ongoing projects: deep-learning based pipeline for automaticadvert integration in consumer videos; multi-modal approachusing cameras and GPS for intelligent systems design.
What’s farther ahead?: Studying the broad spectrum ofintelligent systems, providing image-assisted solutions, andalso expanding to other research directions.
17 / 18
Solar Energy Analytics Smart Traffic Analytics Neuroimaging Analytics Conclusion
Times when we were not researching on the clouds, we clickedselfies!
Thank You!
18 / 18