Too big or not too

big…Big-data challenges

in Civil Engineering

applications

José Barateiro

Lisboa, 23rd November, 2015

Who am I?

LNEC | 2

Outline

• What is Big Data?

• Science paradigms

• Big data in Civil Engineering

LNEC | 3

What is Big Data?

LNEC | 4

Source: Gartner, 2015

The V’s

LNEC | 5

Source: Wired, 2013

Three Vs

LNEC | 6

4th Vs: Veracity,

Value…?

What is Big Data?

“Big data is high-volume, high-velocity and high-variety

information assets that demand cost-effective, innovative

forms of information processing for enhanced insight and

decision making.”

LNEC | 7

Source: Gartner, 2015

1st Paradigm

• Thousand years ago:

science was empirical

describing natural

phenomena

LNEC | 8

Source: Jim Gray, 2007

2nd Paradigm

• Last few hundred years:

theoretical branch using models, generalizations and theories

(E.g., Newton, Kepler, Maxwell’s laws and equations)

LNEC | 9

Source: Jim Gray, 2007

• Last few decades:

a computational branch simulating complex phenomena

LNEC | 10

Source: Jim Gray, 2007

3rd Paradigm

Finite element, finite difference,

spectral, boundary, Turbulence

models, Monte-Carlo

simulations, etc.

• Today: data exploration (eScience) unify theory,

experiment, and simulation

Data captured by instruments or generated by simulators

Processed by software

Information/knowledge stored in computer

Scientist analyzes database/files using data management and

statistics

LNEC | 11

Source: Jim Gray, 2007

4th Paradigm

Data-driven science

• Managing and extracting useful information from massive

data sets

Modeling

Data management

Analytics

Visualization

• Data Warehouse and Business Intelligence systems are

data-driven Decision Support Systems

LNEC | 12

Raw data to information

LNEC | 13

Big data challenges and opportunities

• Data quality and data validation?

• Processing

• Organizational culture Making decisions?

Is this our science?

• Actors Data scientists

Business scientists

IT specialists

LNEC | 14

Big data in Civil Engineering• Scenario 1: Static monitoring

LNEC | 15

000 E+0

5 E+6

10 E+6

15 E+6

20 E+6

25 E+6

1930 1940 1950 1960 1970 1980 1990 2000 2010 2020

# S

enso

r m

easu

rem

ents

Year

Total RMD RAD Total RMD RAD

Big data in Civil Engineering

• Scenario 2: Dynamic monitoring

(example: Ponte 25 de Abril)

87 (Sensors) x 500Hz (frequency) * 60 *

60 * 24 = 3.758.400.000 measurements

per day.

Number of sensors will be extended to

approx. 200

Computing correlations: 1.4 x 1019 space

(per day)

LNEC | 16

Raw data to information

LNEC | 17

LNEC | 18