Post on 08-Aug-2020
transcript
Table 1. Search strategy for MEDLINE, EMBASE, CINAHL, Cochrane library
Search strategy
1 (remote*).ab.ti.2 (measure*).ab.ti.3 (imag*).ab.ti.4 (monitor*).ab,ti.5 (camera* OR webcam* OR web-cam* OR video).ab,ti.
6(invasive* OR non-invasive* OR noninvasive* OR contact* OR non-contact* ORnoncontact* OR remote*).ab,ti.
7(respire* OR respiratory rate OR breathing rate OR pulse OR heart rate ORblood pressure OR SpO2).ab,ti.
8 1 AND 2 AND 59 2 AND 3 AND 6 AND 710 4 AND 6 AND 711 5 AND 612 “Monitoring, Physiologic/methods” [MAJR]13 5 AND 1214 6 AND 1215 8 OR 9 OR 10 OR 11 OR 13 OR 14
*Word truncation search
Table 2. Adapted GRRAS tool for quality assessment of measurement comparison studies
Section Original guidelines from GRRAS Scoring
Title and abstractIdentify in title or abstract that interrater/intrarater reliabilityor agreement was investigated
N/A
IntroductionName and describe the diagnostic or measurement deviceof interest explicitly
The image-based vital signs monitor is described +
No description −
Specify the subject population of interest Specifies the subject population of interest +No population specified −
Specify the rater population of interest (if applicable) N/A
Describe what is already known about reliability/agreementand provide a rationale for the study
Describes what is already known about the reliabilityof image-based monitoring method and provide arationale for the study
+
No statement on the current knowledge of the methodand no rationale stated
−
MethodsExplain how the sample size was chosen. State the determinednumber of raters/subjects/objects/replicate observations
Explains how the sample size was chosen and/or statethe determined number of subjects/replicate observations
+
No explanation of sample size. Number of subjects/replicateobservations not stated
−
Describe sampling method Describes sampling method +No description of sampling method −
Describe the measurement/rating process (e.g. time intervalbetween repeated measurements, availability of clinicalinformation, blinding)
Describes the measurement process +
No description of measurement process −
State whether measurements/ratings were conductedindependently
Two (or more) methods of measurements conducted independently +
Measurements not conducted independently −
Describe the statistical analysis Describes the statistical analysis planned +No description of statistical analysis planned −
ResultsState the actual number of raters and subjects/objectswhich were included and the number of replicateobservations which were conducted
States the actual number of subjects who were includedand number of observations (e.g. duration of recording)
+
No statement of the number of subjects/observations −
Describe the sample characteristics of raters/subjects N/A
Report estimates of reliability and agreement includingmeasure of statistical uncertainty
Reports estimates of reliability and agreement includingmeasure of statistical uncertainty using gold standardmeasures
+
Reports estimates of reliability and agreement includingmeasure of statistical uncertainty using acceptable standardmeasures
?
Reports estimates of reliability and agreement includingmeasure of statistical uncertainty but using inappropriatemeasures/no estimates of reliability/agreement given
−
Discussion Discuss the practical relevance of results Discusses the practical relevance of results +No discussion of practical relevance of results −
Auxillary material Provide detailed results if possible (e.g. online) N/A
Table 3. Classification of statistical methods used to compare image-based vital signs measurementagainst reference device
Gold standard Acceptable Inappropriate
• Bland-Altman plot/limits of agreement (LOA) analysis• Intraclass correlation coefficient• Lin’s concordance correlation coefficinet• British standards reproducibility/repeatability coefficientFor all methods in this category, if repeat measurementsper patient have not obviously been accounted for, thenclassify as acceptable
• Mean square error/deviation• Root mean square error/deviation• Mean absolute error/deviation• Accuracy• Categorization (e.g. proportion of time/measurementstest within a threshold of gold standard)• Descriptive only - plot of test against gold standard,or test and gold standard signals overlaid
• (Pearson’s) correlationcoefficient• T test
Table 4. Full extracted data set
Ref Author (year) Vital
sign Referen
ce
device
No.
channel
s
Wavele
ngth Camera detail Manufactu
rer
Resoluti
on
(MP)
Lighting
source
Setting Age
range Participant
type N Mal
e Fem
ale Ethnicity
reported Varied
ethnicity Interventio
n/change Dist
ance
(m)
ROI FR
(/s)
Duratio
n per
subject
(mins)
Stats
score
Population
/set-up
reporting
Study
reportin
g
quality
1 Aarts, L (2013)
HR ECG PPG
Multi Visible uEye IDS
imaging
0.9 Ambient Critical
Care Unit 3 days -
4 weeks
gestatio
n
NICU/PIC
U 19 Yes Varied 1 Whole
body
15 5 2 3 Good
2 Addison, P
(2017) HR PPG Multi Visible Zyla Andor Ambient Academic Healthy
volunteers 1 1 0 No Unknown Movement Face/neck <1 1 0 Weak
3 Addison, P (2018)
HR PPG Multi Visible acA1920-155uc
Basler 2.3 Fluorescent light 250lx
Academic 20 - 35 Healthy volunteers
10 5 5 Yes Varied Hypoxia 2 Face/neck 70 60 1 3 Fair
4 Alzahrani, A
(2015) HR Other ref Multi Visible iPhone 5 Apple 8 (max) Ambient Academic Healthy
volunteers 5 No Unknown Exercise 1 Face/neck 30 2 1 1 Weak
5 Amelard, R
(2016) HR PPG Mono NIR GS3-U3-
41C6NIR Point Grey 4.1
(max)
Halogen
illumination
Academic 9 - 60 Healthy
volunteers 24 13§ 11§ No Unknown 1.5 Face/neck 60 <1 1 2 Fair
6 Antink, C
(2015) HR ECG Multi Visible C260 Logitech Ambient Academic 21 - 27§ Healthy
volunteers 5 4 1 Yes Varied Screen
(reading) 0.5 Face/neck 30 2 2 3 Good
7 Bai, G (2018)
HR PPG Multi Visible C920 Logitech Fluorescent
light 30 to 250
lx
Academic Healthy
volunteers 6 4 2 No Unknown Movement
Lighting 0.5 Face/neck 30 1.5 3 2 Good
8 Balakrishnan, G
(2013) HR ECG Multi Visible Lumix GF2 Panasonic 0.9 Ambient Academic 23 - 32 Healthy
volunteers 18 11 7 Yes Varied Face/neck 30 1.5 1 2 Fair
9 Blackford, E
(2015) HR ECG
PPG Multi Visible scA640-120gc Basler 3.2 10 x SoLux
black back full
colour
spectrum
(5000K)
Academic 18 - 28 Healthy
volunteers 25* 17 8 No Unknown Movement 1.5 Face/neck 120 60 1 2 Fair
10 Blackford, E (2016)
HR ECG PPG
Multi Visible Lumix DMC-GH4
Panasonic 2.1 2 x 1000W, 3200K quartz-
halogen light
source
Academic 19 - 47 Healthy volunteers
23 11 12 No Unknown 25 Face/neck 29.9
7
21.3 1 2 Fair
11 Capdevila, L
(2012) HR Other ref Multi Visible Ixus 80is Canon 3 Controlled
illumination
(unspecified)
Academic 18 - 50 Healthy
volunteers 11 8 3 No Unknown 1.5§ Face/neck 30 5 1 1 Weak
12 Cheatham, S
(2015) HR PPG
Other ref Multi Visible iPod Touch (5th
gen)
Apple 5 Ambient Academic 21 - 31 Healthy
volunteers 30 18 12 No Unknown Position Face/neck 15 3 1 Fair
13 Chen, D (2015)
HR Sphyg Multi Visible DMC-FX07GT Panasonic Ambient Academic 22 - 31 Healthy
volunteers 8 7 1 No Unknown 0.07 Face/neck 30 3.75 1 2 Fair
14 Cheng, J (2017)
HR ECG Multi Visible C920 Logitech 0.3 LED lamp Academic 22 - 30 Healthy
volunteers 14 8 6 Yes No
variation Screen
(video) Lighting
0.5 Face/neck 30 38 1 3 Fair
15 Cho, D (2016)
HR PPG Multi Visible C615 Logitech 0.3 Ambient Academic Healthy
volunteers 13 11 2 Yes No
variation 0.5 Face/neck 15 3 3 3 Good
16 Coppetti, T
(2017) HR ECG
PPG Multi Visible iPhone 4/
iPhone 5
Apple 5-8
(max)
Ambient Hospital
(non CCU) 52 - 76 Adult
patients 108 73 35 Yes No
variation 0.3§ Face/neck 30 <1 1 2 Fair
17 Cui, Y (2015)
HR PPG Multi Visible Portable USB-camera
(unspecified)
0.3 Ambient + fluorescent
light
Academic 22 - 28 Healthy volunteers
15 10 5 Yes No variation
Exercise 0.5 Face/neck 30 2 1 3 Fair
18 Davila, M
(2017) HR ECG
PPG Multi Visible Grasshopper
03K2C IEEE-
1394b Point Grey 0.3 Ambient Academic 19 - 71 Mixed
population 20 10 10 Yes Varied Breathing
pattern Cold
pressor
3 Face/neck 60 20 1 3 Fair
19 de Hann, G
(2013) HR PPG Multi Visible USB UI-
2230SE-C
IDS 7.7 Ambient Academic Healthy
volunteers 117 Yes Varied Exercise Face/neck 20 1 1 1 Weak
20 Eaton, A (2018)
HR PPG Multi Visible VIXIA HF
R700
Canon 2.1 Fluorescent
light (640 to
900 lumens)
Academic 19 - 25§ Healthy
volunteers 16 4§ 12§ Yes Varied Screen
(video) Talking
1 –
1.5§ Face/neck 30 6 1 1 Weak
21 Estepp, J (2014)
HR ECG PPG
Multi Visible scA640-120gc Basler 3.2 10 x SoLux
black back full
colour
spectrum (5000K)
Academic 18 - 28 Healthy
volunteers 25* 17 8 No Unknown Movement 1.5 Face/neck 120 60 1 2 Fair
22 Fallet, S (2016)
HR ECG Multi Visible NIR
NIR + RGB
camera
(unspecified)
1.3 Infrared
lighting system
and artificial
lighting
(unspecified)
Academic Healthy
volunteers 12 No Unknown Exercise
Breathing
pattern Lighting
Face/neck 20 16 3 0 Fair
23 Fletcher, R
(2015) HR PPG Multi Visible Galaxy S5, S6,
Note 10.1
iPhone 6
DCR TRV103
HTC One
Samsung
Apple
Sony
HTC
0.9 Ambient and
fluorescent
light Academic Healthy
volunteers 6 No Unknown 0.5 Face/neck 15 5 1 1 Weak
24 Fouladi, S
(2015) HR PPG Multi Visible Laptop
webcam
(unspecified)
Ambient Academic Healthy
volunteers 8 No Unknown 0.5 Face/neck 30 2 1 1 Weak
25 Garbey, M (2007)
HR PPG Mono MWIR MWIR camera Indigo 0.3 Ambient Academic Healthy volunteers
34† 24 10 No Unknown 2 Face/neck 30 5 1 2 Fair
26 Gault, T (2013)
HR ECG Mono LWIR Infrared
camera
(unspecified)
Ambient Academic Healthy
volunteers 32 19 13 No Unknown Exercise
Cold
pressor
1 Face/neck 30 1 2 Fair
27 Gupta, O (2016)
HR PPG Multi Visible LWIR
ICX445
GridEYE
sensor
Sony
Panasonic
1.3 Illumination
change
(unspecified)
Academic 25 – 40 Healthy
volunteers 9 Yes Varied Cognitive
stress Face/neck 30 4 2 2 Fair
28 Gupta, P (2017)
HR PPG Multi Visible C270 Logitech Ambient Academic Healthy
volunteers 20 No Unknown Face/neck 28 <1 3 0 Fair
29 Gupta, P (2018)
HR PPG Multi Visible C270 Logitech Ambient Academic Healthy
volunteers 50 No Unknown 0.5 Face/neck 28 <1 1 1 Weak
30 Holton, B
(2013) HR PPG Multi Visible C910 Pro Logitech 0.3 Ambient Academic 22 - 62 Healthy
volunteers 18‡ 16 2 Yes Varied 0.6 Face/neck 15 1.5 1 3 Fair
31 Huang, R
(2016) HR ECG Multi Visible Xperia Z1 Sony 0.3 Ambient Academic 22 - 28 Healthy
volunteers 6 4 2 No Unknown Movement
Make up 0.3 Face/neck 30 2 1 2 Fair
32 Imaduddin, S
(2015) HR PPG Multi Visible Webcam
(unspecified) Logitech 0.3 Ambient Academic Healthy
volunteers 50 No Unknown Face/neck 30 2 3 0 Fair
33 Iozzia, L (2016)
HR ECG Multi Visible GigE vision
XCG-C30C Sony Ambient Academic 20 - 36 Healthy
volunteers 26 14 12 Yes Varied Position Face/neck 60 10 3 2 Good
34 Jeanne, V
(2013) HR PPG Mono NIR Antisleep Smart Eye 0.02 Infrared
lighting (700 to
1000nm)
Academic Healthy
volunteers 4 No Unknown Lighting 0.6 Face/neck 20 2 1 1 Weak
35 Koprowski, R
(2016) HR PPG Multi Visible C170
CAM69U Logitech
Gembird
0.3 Ambient (200
to 500 lx) +/- LED, bulb with
filament,
fluorescent
Academic 30 - 55 Healthy
volunteers 10 No Unknown Lighting Face/neck 30 4 1 1 Weak
36 Kumar, M
(2015) HR PPG Multi Green
Visible FL-U3-
13E4M-C,
FL-U3-13E4C-
C
Point Grey 1.3 Fluorescent
lighting (500
lx)
Academic 22 - 32§ Healthy
volunteers 12 7 5 Yes Varied Reading
Screen
(video) Talking Lighting
0.5 Face/neck 30 2 2 3 Good
37 Kuo, J (2015)
HR ECG Multi Visible ATC9K HD Oregon
Scientific
2.1 Ambient Academic 27 - 78 Healthy
volunteers 10 7 3 Yes No
variation Driving
scenario Face/neck 29 15 1 2 Fair
38 Kwon, S (2012)
HR ECG Multi Visible iPhone 4 Apple 0.3 Ambient Academic 23 - 30 Healthy
volunteers 10 8 2 Yes No
variation 0.3 Face/neck 29.9
9
1 1 3 Fair
39 Lee, D (2015)
HR PPG Multi Visible Canon 6D Canon 0.9 Ambient Academic Healthy
volunteers 10 8 2 Yes No
variation Lighting
(via screen) 1 Face/neck 60 2 1 3 Fair
40 Lee, J (2016)
HR Sphyg Multi Visible Galaxy S3
I9300 Samsung Ambient +/-
fluorescent
light
Academic Healthy
volunteers 6 Yes No
variation Lighting Props
(glasses)
0.3 Face/neck 10 2 2 Fair
41 Lee, K (2012)
HR PPG Multi Visible UVC2M
VUU22 V-UAV35
PWC-3800
P1311
1031W
Asus
Logitech Logitech
Samsung
Axis
Axis
0.3 Ambient Academic 26 - 35 Healthy
volunteers 5 5 0 No Unknown 0.6§ Face/neck
Upper
limb
Chest/thor
ax
30 3 1 1 Weak
42 Lewandowska,
M (2011)
HR ECG Multi Visible
LWIR 9000 Pro ThermaCam
SC3000
Logitech
FLIR
0.3 Ambient Academic 20 - 64 Healthy
volunteers 10 8 2 Yes No
variation 1 Face/neck 20 <1 0 3 Fair
43 Li, X (2014)
HR Other ref Multi Visible iPad Apple 0.3 Indoor light +
computer
screen light
Academic 19 - 40 Healthy
volunteers 38 21 17 No Unknown 0.35 Face/neck 30 10 1 2 Fair
44 Lin, J (2015)
HR PPG Multi Visible LifeCam HD-
5000 Microsoft 0.3 Luminescent
light
Academic 20 - 35 Healthy
volunteers 15 12 3 No Unknown Exercise 0.5 Face/neck 30 3 1 2 Fair
45 Lin, Y (2017a)
HR Other ref Multi Red Green
TRDB-D5M Terasic 0.3 NIR LEDs Academic 22 - 30 Healthy
volunteers 10 7 3 Yes No
variation Movement 0.5 Face/neck 22 1 3 3 Good
46 Lin, Y (2017b)
HR Other ref Multi Visible C920 Logitech 0.3 Philips Hu
smart light bulb
Academic 24 Healthy
volunteers 2 1 1 Yes No
variation Lighting 0.5 Face/neck 15 2 1 3 Fair
47 Liu, X (2018)
HR ECG Multi Visible 1080 Logitech 0.3 Fluorescent lamp
Academic 18 - 60 Healthy volunteers
40 24 16 No Unknown Movement 0.5 Face/neck 30 3 1 2 Fair
48 Madan, C
(2017) HR ECG
PPG Multi Visible C920 Logitech 0.3 Academic 18 - 25 Healthy
volunteers 24 10 14 No Unknown Position
Emotional
stress
0.57
§ Face/neck 12 9 1 1 Weak
49 Mannapperuma,
K (2015)
HR PPG Multi Visible C910 Logitech 0.3 Fluorescent
light
Academic 22 - 59 Healthy
volunteers 18‡ 16 2 Yes Varied Lighting 0.6 Face/neck 15 1.5 1 3 Fair
50 McDuff, D (2018)
HR ECG PPG
Multi Visible scA640-120gc Basler 0.3 10 x SoLux
bulbs (5000K)
Academic 18 - 28 Healthy
volunteers 25 17 8 Yes Varied Movement 1.5 Face/neck 120 60 1 3 Fair
51 Melchor
Rodriguez, A
(2015)
HR PPG Multi Visible C910 Logitech 2.1 Ambient Academic 23 - 35 Healthy
volunteers 10 8 2 No Unknown Movement 0.3 Face/neck 15 2 1 2 Fair
52 Mestha, L
(2014) HR PPG Multi Visible Commercial
HD webcam
(unspecified)
0.3 300 lx lighting Critical
Care Unit 37 - 40
weeks
gestatio
n
NICU/PIC
U 8 No Unknown 1 Face/neck
Chest/thor
ax
30 3 1 Fair
53 Monkaresi, H (2014)
HR ECG Multi Visible 9000 Pro Logitech 0.3 Fluorescent light
Academic 27 Healthy volunteers
10 8 2 Yes Varied Movement Exercise Cognitive
task
Face/neck 30 48 1 2 Fair
54 Moreno, J
(2015) HR Other ref Multi Visible Ixus 80is Canon 0.3 40W lamp Academic 19 - 32 Healthy
volunteers 20 15 5 No Unknown 0.8 Face/neck 30 5 3 2 Good
55 Poh, M (2010)
HR PPG Multi Visible MacBook Pro Apple 0.3 Ambient Academic 18 - 31 Healthy
volunteers 12 10 2 Yes Varied Movement 0.5 Face/neck 15 1 1 3 Fair
56 Prakash, S
(2018) HR Other ref Multi Visible Surface Pro Microsoft 2.1 D65 standard
illuminant
Academic 20 - 28 Healthy
volunteers 25 18 7 No Unknown Movement
Lighting 1.3 Face/neck 29.9
7
8 3 2 Good
57 Rahman, H
(2016) HR ECG Multi Visible Laptop
webcam HP 0.3 Ambient Academic 25 - 50 Healthy
volunteers 10 8 2 Yes Varied 0.5 Face/neck 30 5 0 3 Fair
58 Rasche, S
(2016) HR ECG
PPG Sphyg Other ref
Multi Visible NIR
UI-3370CP-C-
HQ
UI-3370CP-
NIR-GL
IDS 0.1 Ambient +
fluorescent
light
Critical
Care Unit 59 - 81 ICU 70 50 20 No Unknown 0.6 Face/neck
Upper
limb
Chest/thor
ax
100 28.6 1 2 Fair
59 Remer, I (2015)
HR PPG Multi Visible Camera phone Sony-Ericsson
8 (max) Red laser Academic 22 - 30§ Healthy volunteers
10 6§ 4§ No Unknown Other intervention
0.25§
Upper
limb
30 1.5 1 0 Weak
60 Roald, N (2013)
HR ECG PPG
Multi Visible PMW-EX1
LifeCam Sony
Microsoft
2.1 Ambient Academic Healthy
volunteers 1 1 0 Yes No
variation 1 Face/neck
Upper
limb
25 5 0 3 Fair
61 Rustand, A
(2012) HR PPG Multi Visible LifeCam Microsoft 0.2 Ambient Academic Healthy
volunteers 6 No Unknown 0.4 Face/neck
Upper
limb
30 3 2 1 Fair
62 Scalise, L
(2012) HR ECG Multi Visible LifeCam VX-
3000 Microsoft 0.3 Large band
light source
Critical
Care Unit 30 - 37
weeks
gestatio
n
NICU/PIC
U 7 3 4 No Unknown 0.2 Face/neck 15 4 1 2 Fair
63 Shan, L (2013)
HR Sphyg Multi Visible GT-I9108 Galaxy S2
Samsung 0.9 Ambient Academic Healthy volunteers
10 4 6 Yes No variation
0.4 Face/neck 30 <1 1 3 Fair
64 Shao, D (2017)
HR Other ref Multi Visible F-032C Pike Fluorescent
light
Academic 24 - 34 Healthy
volunteers 23 15 8 Yes Varied 0.5 Face/neck 60 <1 0 3 Fair
65 Stricker, R
(2014) HR PPG Multi Visible Eco274CVGE SVS-Vistek 0.3 Ambient Academic 28 - 38§ Healthy
volunteers 10 8 2 No Unknown Movement
Talking 1.1 Face/neck 30 6 1 2 Fair
66 Sun, N (2006)
HR Other ref Mono LWIR Thermal
camera
(unspecified)
0.3 Ambient Academic Healthy
volunteers 34† 24 10 Yes Varied 2 Face/neck 30 <1 1 3 Fair
67 Sun, Y (2012)
HR PPG Multi Visible MC1311
9000 Pro Mikrotron
Logitech
1.3
0.07
Ambient Academic 22 - 50 Healthy
volunteers 10 10 0 No Unknown Exercise 0.35 Face/neck
Upper
limb
50 46 1 2 Fair
68 Taralunga, D
(2016) HR ECG Multi Visible Smartphone
camera (unspecified)
0.4 Ambient Academic 31 Healthy
volunteers 1 1 0 Yes No
variation 0.5 Face/neck 29 1 1 3 Fair
69 Tasli, H (2014)
HR PPG Multi Visible Off-the-shelf
camera
(unspecified)
Ambient Academic 20 - 35 Healthy
volunteers 10 No Unknown Exercise Face/neck 30 3 1 1 Weak
70 Tran, D (2015)
HR PPG Multi Visible C920 Logitech 0.3 Ambient Academic 20 - 35 Healthy
volunteers 10 7 3 No Unknown Movement
Lighting Distance
0.5
to
1.2
Face/neck 10 4.5 3 2 Good
71 Trumpp, A
(2018) HR ECG
PPG Multi Visible
NIR UI-3370CP-C-
HQ
UI-3370CP-
NIR-GL
IDS 0.1 NIR
illumination
Hospital
(non CCU) 53 - 77 Adult
patients 41 24 17 No Unknown 0.5 Face/neck 100 30 1 2 Fair
72 Tsouri, G
(2012) HR PPG Multi Visible Webcam
(unspecified) Logitech 0.08 Ambient Academic 18 - 45 Healthy
volunteers 45 25 20 Yes Varied Face/neck 15 1 2 2 Fair
73 Tsouri, G
(2015) HR PPG Multi Visible Webcam
(unspecified) Logitech 0.08 Fluorescent
light
Academic 18 - 45 Healthy
volunteers 41 Yes Varied 0.65 Face/neck
Chest/thor
ax
15 1 2 3 Good
74 van Gastel, M
(2015) HR PPG Multi Visible F046B Mono
F046C RGB Marlin 0.3 800nm LED
illumination
Academic 22 - 30 Healthy
volunteers 6 4§ 2§ Yes Varied Movement 2.5 Face/neck
Chest/thor
ax
15 2 2 3 Good
75 Viejo, C (2018)
HR PPG Sphyg Other ref
Multi Visible Tablet camera
(unspecified) Google 2.1 Ambient Academic 20 - 38 Healthy
volunteers 15 3§ 12§ No Unknown Exercise
0.3 Face/neck 15 9 1 2 Fair
76 Wedekind, D
(2017) HR ECG
PPG Multi Visible UI-3370CP-C-
HQ IDS 0.1 Ambient Critical
Care Unit ICU 18 13 5 No Unknown 0.6 Face/neck 100 18 1 2 Fair
77 Wei, J (2018)
HR ECG Multi Visible VIXIA HF R62 Canon LED light source
Academic 18+ Healthy volunteers
136 57 79 No Unknown Screen (video)
0.6 Face/neck 60 2 1 2 Fair
78 Wei, L (2012)
HR PPG Multi Visible C170 Logitech 0.3 Ambient +
Fluorescent
lighting
Academic 20 - 80 Healthy
volunteers 20 11 9 No Unknown Lighting 0.5 Face/neck 30 1 2 2 Fair
79 Xu, S (2014)
HR PPG Multi Visible CoolPix L610
U8652
iPhone 4
Nikon
Huawei
Apple
0.9
0.4
0.9
Ambient Academic Healthy
volunteers Yes Varied 0.5 Face/neck
Upper
limb
15 1.5 2 2 Fair
80 Yan, B (2017)
HR ECG Multi Visible iPhone 6S Apple Ambient Academic 18 - 40 Healthy
volunteers 40 20 20 Yes Varied Exercise 0.3 Face/neck 6 1 2 Fair
81 Yan, Y (2015)
HR Sphyg Other ref
Multi Visible Colour camera
(unspecified) Canon 0.9 Fluorescent
light
Academic 20 - 35 Healthy
volunteers 15 7 8 Yes Varied 0.5 Face/neck 15 1 1 3 Fair
82 Yang, Y (2016)
HR ECG Multi Visible Pike F-032C Allied Vision
0.3 Ambient + white light
Academic 25 - 31 Healthy volunteers
17 11 6 Yes Varied 0.8 Face/neck 30 10 0 3 Fair
83 Yu, Y (2013)
HR Other ref Multi Visible Video camera
(unspecified) 0.4 Fluorescent
light
Academic Healthy
volunteers 1 1 0 Yes No
variation Exercise 0.6 Face/neck 25 3 1 2 Fair
84 Yu, Y (2015a)
HR ECG Multi Visible HDR-PJ580V Sony 1.6 Fluorescent
light
Academic Healthy
volunteers 7 No Unknown Exercise 0.6 Face/neck 25 <1 1 1 Weak
85 Yu, Y (2015b)
HR ECG Multi Visible HDR-PJ260VE Sony Fluorescent
light
Academic Healthy
volunteers 8 No Unknown Exercise 0.6 Face/neck 50 2 1 1 Weak
86 Abbas, A
(2011) RR ECG
PPG Clinical
Mono LWIR VarioCAM InfraTec Ambient Critical
Care Unit 24 - 34
weeks
gestatio
n
NICU/PIC
U 7 No Unknown 0.7 Face/neck 30 6.5 1 2 Fair
87 Al-Naji, A
(2016) RR Clinical Multi Visible D5300 DSLR Nikon 0.8 Ambient Academic 8
months Healthy
volunteers
(paediatric)
1 0 1 Yes No
variation 1 Chest/thor
ax
30 2.5 1 3 Fair
88 Bartula, M
(2013) RR Other ref Mono Visible
NIR Monochrome
camera
(unspecified)
0.4 Ambient
(varying
conditions) Academic Healthy
volunteers 5 4 1 No Unknown Breathing
pattern Chest/thor
ax
20 5 1 1 Weak
89 Benetazzo, F
(2014) RR Other ref Multi Visible Kinect RGB-D
SL Microsoft Infrared light
source
Academic 25 - 33 Healthy
volunteers 5 2 3 No Unknown Movement
Lighting 1.4 Chest/thor
ax
5 1 2 Fair
90 Bennett, S
(2015) RR Other ref Mono LWIR A-Series FLIR 0.3 Ambient Academic Healthy
volunteers 1 No Unknown Breathing
pattern Face/neck 60 2 1 0 Weak
91 Bennett, S (2017)
RR Other ref Mono LWIR T1030sc FLIR 0.8 Ambient Academic Healthy volunteers
1 1 0 No Unknown Breathing pattern
2 Face/neck 13.5 2.5 1 2 Fair
92 Cenci, A (2015)
RR PPG Multi Visible Xtion Pro live Asus Ambient Critical
Care Unit 32 – 24
weeks
gestatio
n
NICU/PIC
U 3 3 0 No Unknown 0.7 Chest/thor
ax
2.5 0 1 Weak
93 Cho, Y (2017)
RR Other ref Multi LWIR One for
Android FLIR 0.02 Ambient Academic 23 - 38 Healthy
volunteers 23 10 13 Yes Varied Temperatur
e Movement
0.35 Face/neck 9 6 1 3 Fair
94 Elphick, H
(2015) RR Other ref Mono LWIR A40 FLIR Ambient Academic 6 - 36 Healthy
volunteers 71 No Unknown Face/neck 0 1 -
95 Fei, J (2005)
RR Other ref Mono MWIR Phoenix Indigo 0.3 Ambient Academic Healthy
volunteers 9 No Unknown Movement
Exercise 2 Face/neck 55 5 0 1 Weak
96 Fei, J (2010)
RR Other ref Mono MWIR SC6000 FLIR 0.3 Ambient Academic Healthy
volunteers 20 No Unknown 2 Face/neck 55 3 1 1 Weak
97 Goldman, L (2012)
RR Other ref Mono LWIR A40 FLIR 0.08 Ambient Hospital
(non CCU) 0.33 -
13.75 Paediatric
patients 17 10 7 No Unknown 0.7 Face/neck
Chest/thor
ax
50 10 3 2 Good
98 Hu, M (2017)
RR Clinical Multi Visible LWIR
USBFHD01M MAG62
RER Vision Magnity
Electronics
0.3 0.3
Ambient Academic 23 - 28 Healthy volunteers
11 No Unknown Breathing pattern
1.5 Face/neck 30 7 1 2 Fair
99 Iles, R (2012)
RR Clinical Multi Visible Digital camera
(unspecified) Ambient Academic 0 - 1
day old Healthy
neonates No Unknown Movement Chest/thor
ax
0 1 -
100 Janssen, R
(2016) RR Other ref Multi Visible UI-2230SE-C IDS 0.4 Ambient Critical
Care Unit Healthy
volunteers NICU/PIC
U
4
(HV
) 2(N)
3 1 No Unknown Movement
(HV) Breathing
pattern
(HV) Lighting
(HV)
1 Face/neck
Chest/thor
ax
10 4.5 1 1 Weak
101 Jeong, H (2017)
RR PPG Other ref
Mono LWIR Ti450 Fluke 0.3 Ambient Academic 25 - 33 Healthy
volunteers 3 3§ 0§ No Unknown Breathing
pattern 1 Face/neck 60 7 1 2 Fair
102 Jorge, J (2017)
RR ECG PPG
Multi Visible ICX274AL Sony 2 Ambient +
fluorescent
Critical
Care Unit 37
weeks
gestation
NICU/PIC
U 30¶ No Unknown 0.3 Whole
body
20.3 900 1 2 Fair
103 Kwasniewska,
A (2017)
RR Other ref Mono LWIR Lepton FLIR 0.004 Ambient Academic 20 - 42 Healthy
volunteers 11 9 2 No Unknown Talking 0.7 Face/neck 13 2 1 2 Fair
104 Lewis, G (2011)
RR Other ref Mono LWIR TVS-700
SC-6000 Indigo
FLIR
0.08
0.3
Ambient Academic 18 - 30 Healthy
volunteers 19 Yes Varied Breathing
pattern Face/neck 29 6 1 2 Fair
105 Li, M (2014)
RR Other ref Mono Visible FMVU-
03MTM Point Grey 0.3 IR light Academic 19 - 42 Healthy
volunteers 5 2 3 No Unknown Movement 1.4 Face/neck
Chest/thor
ax
30 20 1 2 Fair
106 Loblaw, A
(2013) RR Clinical Mono Visible
NIR Kinect Microsoft Infrared laser
source
Academic Healthy
volunteers 2 1 1 No Unknown 0.5 Chest/thor
ax
2 1 1 Weak
107 Murthy, R
(2004) RR Other ref Mono MWIR Phoenix Indigo 0.3 Ambient Academic Healthy
volunteers 10 No Unknown Exercise 2 Face/neck 31 <1 1 1 Weak
108 Mutlu, K (2018)
RR Other ref Mono LWIR A325sc FLIR 0.08 Ambient Academic 20 - 30§ Healthy
volunteers 6 6§ 0§ No Unknown Movement 0.6 Face/neck 60 3 1 1 Weak
109 Nam, Y (2016)
RR Other ref Multi Visible One M8 HTC 0.03 Ambient Academic 20 - 40 Healthy
volunteers 11 9 2 No Unknown Breathing
pattern 0.5 Chest/thor
ax
25 2 1 2 Fair
110 Pereira, C
(2015a) RR Other ref Mono LWIR VarioCAM InfraTec 0.8 Ambient Academic 24 - 31 Healthy
volunteers 5 3 2 No Unknown 1.5 Face/neck
Chest/thor
ax
30 3 1 2 Fair
111 Pereira, C
(2015b) RR Other ref Mono LWIR VarioCAM InfraTec 0.8 Ambient Academic 21 - 31 Healthy
volunteers 11 7 4 No Unknown Breathing
pattern 2 Face/neck 30 10 1 2 Fair
112 Pereira, C
(2017) RR Other ref Mono LWIR VarioCAM InfraTec 0.8 Ambient Academic 21 - 31 Healthy
volunteers 12 7 5 No Unknown Breathing
pattern 2 Face/neck 30 19 1 2 Fair
113 Reyes, B (2017)
RR Other ref Multi Visible One M8 HTC 0.08 Fluorescent
light
Academic 19 - 52 Healthy
volunteers 15 14 1 No Unknown Breathing
pattern 0.6 Chest/thor
ax
30 2 1 2 Fair
114 Tan, K (2010)
RR Clinical Multi Visible Webcam
(unspecified) Ambient Academic 4 (P) Healthy
volunteers Paed patient
2 No Unknown Distance 0.5 -
1 Chest/thor
ax
10 10 0 2 weak
115 van Gastel, M
(2015) RR PPG Multi Visible
NIR USB UI-
2230SE-C F046B (x3)
IDS Allied
Vision 0.4
0.3
Incandescent
light bulbs
Critical
Care Unit Healthy
volunteers
and
NICU/PIC
U
4
(HV
) 2 (N)
4 (HV
)
0 (HV
)
No Unknown Breathing
pattern Movement
1 Face/neck
Chest/thor
ax
20 60 1 2 Fair
116 Zhu, Z (2005)
RR Clinical Mono MWIR Phoenix Indigo 0.3 Ambient Academic Healthy volunteers
1 No Unknown Movement Face/neck 50 0 0 Weak
117 Blanik, N
(2014b) SpO2 PPG Mono Visible
NIR
Pike 210B Allied
Vision
2.1 Red and IR
source
Academic Healthy
volunteers 13 No Unknown 0.3 Upper
limb
31 <1 0 1 Weak
118 Guazzi, A (2015)
SpO2 PPG Multi Visible AT-200CL Jai Two LED lighting panels
Academic Healthy volunteers
5 Yes Varied Hypoxia 1.5 Face/neck 16 40 2 2 Fair
119 Humphreys, K
(2005) SpO2 PPG Mono Visible
NIR
PL-A741 Pixelink NIR lights
(760nm,
880nm)
Academic 26§ Healthy
volunteers 1 1§ 0§ No Unknown 0.3 Upper
limb
60 1 1 Weak
120 Kong, L (2013)
SpO2 PPG Multi Visible WAT-535EX2
(x2) Watec 0.08 LED lights Academic 18 - 58 Healthy
volunteers 30 No Unknown Breathing
pattern 1.5 Face/neck 25 2 3 2 Good
121 Shao, D (2016)
SpO2 PPG Multi Visible
NIR
PL-B741EU Pixelink 0.02 LED arrays Academic 24 - 30 Healthy
volunteers 6 3 3 Yes Varied Breathing
pattern 0.3 Face/neck 10 5 1 3 Fair
122 Verkruysse, W
(2017) SpO2 PPG Multi Visible
NIR
Stingray
F046B Allied
Vision
IR light source Academic Healthy
volunteers 41 27 14 No Unknown Hypoxia
Temperatur
e
1.6 Face/neck 15 10 2 2 Fair
123 Jeong, I (2016)
BP ECG Sphyg
Multi Visible EX-FH20 Casio 0.04 LED lights Academic 20 - 50 Healthy
volunteers 7 4 3 No Unknown Exercise 0.3 Face/neck
Upper
limb
420 13 0 2 Weak
124 Murakami, K
(2015) BP PPG Multi Visible CEJH-15007 Sony 0.3 Fluorescent
lamp
Academic 25 - 60 Healthy
volunteers 10 10 0 No Unknown 0.9 Whole
body
60 1 0 2 Weak
125 Secerbegovic,
A (2016) BP ECG
Other ref Multi Visible Commercial
digital camera
(unspecified)
0.4 Ambient Hospital
(non CCU) 44 - 48 Healthy
volunteers 3 3 0 No Unknown Exercise 1.5 Face/neck
Upper
limb
25 9 1 2 Fair
126 Sugita, N
(2015) BP Sphyg Multi Visible TGX02c Baumer 0.2 White LED
(1000 lx)
Academic 21 - 24 Healthy
volunteers 20 20 0 No Unknown Valsalva 1 Upper
limb
140 5 0 2 Weak
127 Al-Naji, A
(2017b) HR RR
Other ref Multi Visible D5300 Nikon 0.8 Ambient Academic 18 - 40 Healthy
volunteers 10 8 2 Yes Varied Movement
Lighting 50 Face/neck 60 1 3 Fair
Distance 128 Al-Naji, A
(2017a) HR RR
PPG Other ref
Multi Visible Hero 4k GoPro 2.1 Ambient Academic 2 - 40 Healthy
volunteers 15 10 5 No Unknown Movement
Lighting Talking
3 Face/neck 60 4 2 2 Fair
129 Antognoli, L
(2018) HR RR
ECG Other ref
Multi Visible LifeCam
Cinema Microsoft 0.9 Light source
(unspecified)
Critical
Care Unit 26 – 40
weeks
gestatio
n
NICU/PIC
U 7 3 4 No Unknown 0.5 Chest/thor
ax
30 <1 3 2 Good
130 Aubakir, B
(2016) HR RR
Clinical Other ref
Multi Visible
LWIR Lepton Smartphone
camera
(unspecified)
FLIR 0.005
0.9
Ambient Academic Healthy
volunteers 11 6§ 5§ No Unknown 0.3 Face/neck 240 <1 1 1 Weak
131 Bernacchia, N (2014)
HR RR
ECG Other ref
Multi Visible NIR
Kinect Microsoft Infrared emitter Academic 21 - 29 Healthy volunteers
10 5 5 No Unknown 1.2 Chest/thor
ax
30 <1 1 2 Fair
132 Blanik, N (2014a)
HR RR
ECG PPG
Multi Visible LWIR
Pike 210B VarioCam hr
head
Allied Vision
InfraTec
2.1 0.1
Ambient Critical Care Unit
NICU/PICU
8 No Unknown Whole
body
31 10 1 0 Weak
133 Bousefsaf, F
(2013) HR RR
PPG Other ref
Multi Visible Lifecam
Cinema Microsoft 0.08 Ambient Academic 22 - 79 Healthy
volunteers 12 10 2 Yes Varied Movement 1 Face/neck 30 <1 1 3 Fair
134 Chekmenev, S
(2005) HR RR
Other ref Mono LWIR Phoenix FLIR 0.08 Ambient Academic 24 - 35 Healthy
volunteers 4 3 1 No Unknown 1 Face/neck 30 1.5 1 2 Fair
135 Cooke, W
(2011) HR RR
ECG PPG Sphyg Other ref
Mono LWIR SC-4000 FLIR Ambient Pseudo-
health 21 - 23 Healthy
volunteers 45 25 20 No Unknown Lower body
positive
pressure
1.8 Face/neck 30 32 1 2 Fair
136 He, X (2017)
HR RR
PPG Other ref
Mono NIR DNV16HDZ
camcorder Bell &
Howell
16
(max)
Infrared light
source (900nm)
Academic Healthy
volunteers 1 1 0 No Unknown Movement
Distance 1 to
5 Face/neck
Chest/thor
ax
30 1 1 2 Fair
137 Hu, M (2018)
HR RR
ECG Clinical
Multi LWIR NIR
MAG62 RGB-IR
camera
(unspecified)
Magnity 0.3 Infrared light
source (850nm)
Academic 21 - 38 Healthy
volunteers 12 10 2 Yes No
variation Sleep 2 Face/neck 24 1 2 Fair
138 Klaessens, J
(2014) HR RR
ECG PPG
Multi Visible LWIR
DSC-H9 Gobi-384
Sony
Xenics
0.3
0.1
NIR light
source
Critical care
unit 24 to 39
weeks
gestatio
n
NICU 7 2 3 No Unknown 0.5 Face/neck 25
(IR)
30
(V)
1 3 Fair
139 Li, M (2016)
HR RR
ECG Other ref
Multi Visible NIR
FMVU-
03MTM MT9V022177
ATM (sensor)
Point Grey Micron
0.3 Infrared light
source
Academic 19 - 41 Healthy
volunteers 17 7 10 No Unknown Movement
Position 1.4 Face/neck
Chest/thor
ax
30 25 1 2 Fair
140 Maclaren, J
(2015) HR RR
PPG Other ref
Mono Visible Spy camera
(CMOS) Swann
Communica
tions
0.2 White LED
light
Hospital
(non CCU) 33 - 36§ Healthy
volunteers 3 No Unknown 0.1 Face/neck 30 1 1 Weak
141 McDuff, D
(2014a) HR
RR PPG Multi Visible Novel RGBCO
camera 0.7 Ambient +
Indoor
illumination
Academic 18 - 35 Healthy
volunteers 14 6 8 Yes Varied Cognitive
task 3 Face/neck 30 4 1 3 Fair
142 McDuff, D
(2014b) HR RR
PPG Other ref
Multi Visible Novel RGBCO
camera 0.7 Ambient +
fluorescent
Academic 18 - 30 Healthy
volunteers 10 3 7 Yes Varied Cognitive
stress 3 Face/neck 30 4 0 3 Fair
143 Pereira, C (2018)
HR RR
PPG Other ref
Mono MWIR ImageIR 9300 InfraTec 0.8 Ambient Academic 18 - 50 Healthy volunteers
20 13 7 No Unknown Angle of camera (lat
view)
1.5 Face/neck 50 3 3 2 Good
144 Poh, M (2011)
HR RR
PPG Other ref
Multi Visible MacBook Pro Apple 0.3 Ambient Academic 18 - 31 Healthy
volunteers 12 8 4 Yes Varied 0.5 Face/neck 15 1 1 3 Fair
145 Prochazka, A
(2016) HR RR
Other ref Multi Visible NIR
Kinect with
infrared sensor Microsoft 2.1
0.2 (IR)
IR light source Academic Healthy
volunteer 1 No Unknown Face/neck 10 2 1 0 Weak
146 Sanyal, S
(2018) HR RR
PPG Clinical
Multi Visible G2 LG Ambient +
flash
illumination
Academic 20 - 30 Healthy
volunteers 25 15 20 Yes Varied 0.5 Face/neck 30 <1 1 3 Fair
147 Scalise, L
(2013) HR RR
ECG Other ref
Multi Visible Kinect CCD camera
(unspecified) Microsoft Ambient Academic Healthy
volunteers 18 No Unknown Face/neck 1 0 -
148 Shao, D (2014)
HR RR
ECG PPG Other ref
Multi
Visible Pike F-032-B
Pike F-032-C
Webcam
(unspecified)
Allied
Vision
Allied
Vision Logitech
Ambient Academic 22 - 32 Healthy
volunteers 10 6 4 Yes Varied 0.5 Face/neck
Upper
limb
Chest/thor
ax
<1 3 2 Good
149 Sun, Y (2011)
HR RR
PPG Mono Visible
NIR
EoSens
MC1360-63 Mikrotron 0.3 Infrared light
source
Academic 20 - 55 Healthy
volunteers 13 11 2 No Unknown Exercise 0.35 Upper
limb
50 12 1 2 Fair
150 Sun, Y (2013)
HR RR
PPG Mono Visible
NIR
EoSens MC
1363-63 Mikrotron 0.1 Infrared light
source
Academic 21 - 65 Healthy
volunteers 10 7 3 No Unknown 0.3 Upper
limb
200 40 1 2 Fair
151 Takano, C
(2007) HR RR
PPG Multi Visible Network
Handycam IP Sony 0.3 Ambient +
fluorescent
Academic 22 - 27 Healthy
volunteers 14 0 14 Yes No
variation Face/neck 5 <1 0 2 Weak
152 Verkruysse, W
(2008) HR RR
PPG Multi Visible Powershot
A560,A570,A6
40
Canon 0.3 or
0.08
Ambient +
fluorescent
Academic Healthy
volunteers 2 No Unknown Movement
Exercise 1 - 2 Face/neck 15 2 0 1 Weak
153 Wei, B (2017)
HR RR
PPG Other ref
Multi Visible NIR
iPhone 4 Apple 0.3 Ambient Academic 22 - 31 Healthy
volunteers 8 No Unknown Breathing
pattern 0.3 Face/neck 30 6 1 2 Fair
154 Zhao, F (2013)
HR RR
ECG Other ref
Multi Visible
NIR MT9V024 Aptina
Imaging
Corporation
0.4 Ambient +
LED @ 830nm
Academic 27 - 50 Healthy
volunteers 15 7 8 Yes No
variation 0.5 -
3 Face/neck
Upper
limb
Chest/thor
ax
15 2 1 3 Fair
155 Bal, U (2015)
HR SpO2
ECG PPG
Multi Visible Thinkpad T430 Lenovo 0.3 Ambient + fluorescent
Critical Care Unit
14 – 35 (Healthy
voluntee
rs) 2
months
– 14
Healthy volunteers NICU/PIC
U
2 (HV
) 7 (P)
7 (2 HV,
5 P)
2 No Unknown 0.5 Face/neck 30 1 1 2 Fair
years
(PICU) 156 Humphreys, K
(2007) HR SpO2
PPG Mono NIR PL-A741 Pixelink 1.3 LED array
lighting
Academic 21 - 45 Healthy
volunteers 10 9 1 Yes No
variation 0.3 Upper
limb
32 <1 3 3 Good
157 Jain, M (2017)
HR BP
Sphyg Multi Visible HDR-CX405 Sony 2.1 Ambient Academic 20 - 40 Healthy
volunteers 45 No Unknown 0.5 Face/neck 50 1 1 2 Fair
158 Jensen, J (2014) HR BP
PPG Sphyg
Multi Visible X220 Hero3+
Lenovo GoPro
0.3
0.9
Ambient Academic 21 - 27 Healthy
volunteers 16 11 5 Yes No
variation 0.5§ Face/neck
Upper
limb
30 1 1 2 Fair
159 Tarassenko, L
(2014) HR RR SpO2
PPG Other ref
Multi Visible Grasshopper2 Point Grey 5 (max) Ambient +
fluorescent
Hospital
(non CCU) 49 - 80 Adult
patients 46 36 10 No Unknown 1 Face/neck
Chest/thor
ax
12 240 1 2 Fair
160 Villarroel, M
(2014) HR RR SpO2
ECG PPG Other ref
Multi Visible AT-200CL Jai 2 Ambient +
fluorescent
Critical
Care Unit 37
weeks
gestatio
n
NICU/PIC
U 2¶ No Unknown 1 Whole
body
20 1200 1 2 Fair
161 Wieringa, F
(2005) HR RR SpO2
ECG PPG
Mono Visible Mono CMOS
Camera FillFactory
NV
1.3 LED ringlight Academic 23 - 39 Healthy
volunteers 7 6 1 Yes No
variation 0.72 Upper
limb
7 <1 0 3 Fair
*†‡¶ = indicates the same clinical dataset or subset
§ = information obtained via personal communication with authors
The studies have been arranged in groups of the vital signs of interest (heart rate, respiratory rate, oxygen saturation, blood pressure, followed by studies measuring more than
one vital signs). Within each group they are arranged alphabetically (first author).
Table 5. Bland-Altman statistics for studies measuring heart rate
First Author Mean difference (bpm) Upper limit of agreement Lower limit of agreement
Poh, MZ (2010) -0.05 4.44 -4.55Sun, Y (2011) 0.33 1.96 -1.29Rustand, A (2012) -0.77* 8.96* -10.50*Scalise, L (2012) -0.90 7.99 -9.79Sun, Y (2012) -0.33 3.93 -4.59Aarts, L (2013) 0.30 5.50 -5.00Bousefsaf, F (2013) -0.03 4.16 -4.22de Hann, G (2013) 0.13* 1.69* -1.44*Kong, L (2013) 0.62 4.25 -3.00Sun, Y (2013) -0.04 2.93 -3.00Wei, L (2013) -0.20 1.40 -1.90Zhao, F (2013) 0.68 5.54 -4.18Bal, U (2014) 0.00* 4.60* 3.20*Mestha, LK (2014) 2.52 8.00 -2.96Monkaresi, H (2014) 0.86 3.72 -2.00Shao, D (2014) 0.86 4.19 -2.47Xu, S (2014) -0.16 6.51 -6.82Cheatham, SW (2015) 0.59 4.05 -2.87Chen, DY (2015) 1.93 10.40 -6.54Imaduddin, SM (2015) -1.00 13.00 -15.00Kumar, M (2015) -0.02 0.72 -0.75Remer, I (2015) -0.36 3.75 -4.47Tran, DN (2015) -0.55 10.60 -11.69van Gastel, M (2015) 0.62 2.30 -1.00Yu, YP (2015) -0.39* 3.42* -4.23*Yu, YP (2015) 0.11* 5.08* -4.91*Amelard, R (2016) -1.00 0.37 -2.37Cho, D (2016) 0.70 10.03 -10.01Fallet, S (2016) 1.85 8.50 -6.59Gupta, O (2016) -1.9 2.9 -6.8Iozzia, L (2016) 0.04 6.58 -6.58Lee, JS (2016) -3.05* 3.54* -9.02*Rasche, S (2016) -0.10 0.70 -0.90Al-Naji, A (2017a) 0.07 0.66 -0.52Al-Naji, A (2017b) 1.70 2.60 -0.81Cheng, J (2017) 1.14 23.01 -20.73Gupta, P (2017) 2.00 8.00 -5.00Lin, YC (2017) 0.55 4.53 -3.44Madan, CR (2017) 0.63 12.00 -10.74Wei, B (2017) -0.20 4.30 -4.80Yan, BP (2017) 0.29 2.30 -1.72Addison, P (2018) 0.00* 3.98* -4.36*Antognoli, L (2018) -1.00 9.00 -11.00Bai, G (2018) -1.20 0.70 -3.20Eaton, A (2018) 1.80 8.00 -4.40Gupta, P (2018) 0.00 8.00 -7.00Hu, M (2018) -0.50 4.29 -5.29Liu, X (2018) 0.00 5.00 -5.00Pereira, CB (2018) 0.80 9.10 -7.50Sanyal, S (2018) -0.12 8.03 -8.27Wei, J (2018) -0.96 0.63 -2.55
Mean difference (ms)
Capdevila, L (2012) 0.00 32.03 -32.80Bernacchia, N (2014) -0.01 0.10 -0.12Antink, CH (2015) 4.37 87.94 -79.20
Melchor Rodriguez, A (2015) 0.00 37.20 -37.20Moreno, J (2015) -0.32 43.88 -44.90Blackford, EB (2016) 0.13 0.63 -0.36Davila, MI (2017) -0.02 73.98 -74.02
First Author Mean difference (Hz) Upper limit of agreement Lower limit of agreement
Humphreys, K (2007) -0.02 4.87 3.07Nam, Y (2016) 0.00 0.10 -0.09
* Numbers not reported within text; estimated from published figures using WebPlotDigitizer tool
Table 6. Pearson correlation coefficients for studies measuring heart rate
First Author R
Takano, C (2007) 0.90Poh, MZ (2010) 0.98Cooke, WH (2011) 0.99Poh, MZ (2011) 1.00Sun, Y (2011) 0.90Capdevila, L (2012) 1.00Scalise, L (2012) 0.94Bousefsaf, F (2013) 1.00de Hann, G (2013) 1.00Gault, TR (2013) 0.96Holton, BD (2013) 0.89Jeanne, V (2013) 1.00Roald, NG (2013) 1.00Sun, Y (2013) 0.95Zhao, F (2013) 0.95Bal, U (2014) 0.93Bernacchia, N (2014) 0.91Jensen, JN (2014) 1.00Li, X (2014) 0.99McDuff, D (2014a) 1.00McDuff, D (2014b) 0.94Monkaresi, H (2014) 0.99Shao, D (2014) 0.98Chen, DY (2015) 0.91Cui, Y (2015) 0.98Melchor Rodriguez, A (2015) 0.99Moreno, J (2015) 0.99Remer, I (2015) 0.79van Gastel, M (2015) 1.00Yan, Y (2015) 0.94Yu, YP (2015) 0.99Amelard, R (2016) 1.00Blackford, EB (2016) 0.97Rahman, H (2016) 0.99Yang, Y (2016) 0.97Cheng, J (2017) 0.91Coppetti, T (2017) 0.62Davila, MI (2017) 0.94Madan, CR (2017) 0.95Shao, D (2017) 0.82Wei, B (2017) 0.95Yan, BP (2017) 0.99Eaton, A (2018) 0.22Gupta, P (2018) 0.95Hu, M (2018) 0.93
Liu, X (2018) 0.98Sanyal, S (2018) 0.92Viejo, CG (2018) 0.85Wei, J (2018) 1.00
Table 7. Percentage accuracy for studies measuring heart rate
First Author Percentage Accuracy (%)
Chekmenev, SY (2006) 100.0Sun, N (2006) 92.1Garbey, M (2007) 90.3Bal, U (2014) 98.0Villarroel, M (2014) 94.0Xu, S (2014) 98.0Kuo, J (2015) 29.4Lee, D (2015) 99.6Mannapperuma, K (2015) 100.0van Gastel, M (2015) 99.8Aubakir, B (2016) 92.0Iozzia, L (2016) 98.4Rasche, S (2016) 83.0Lin, YC (2017) 99.2Prakash, SKA (2018) 81.0Trumpp, A (2018) 95.6
Table 8. Mean percentage error for studies measuring heart rate
First Author Mean percentage error (MPE) (%)
Sun, N (2006) 7.8Kwon, SJ (2012) 1.5Balakrishnan, G (2013) 1.5Shan, L (2013) 0.9Li, X (2014) 1.5McDuff, D (2014b) 3.1Cui, Y (2015) 2.3Tran, DN (2015) 1.6Gupta, O (2016) 4.6Li, MH (2016) 5.0Gupta, P (2017) 1.8Li, MH (2017) 5.0Eaton, A (2018) 14.0
Table 9. Mean absolute error for studies measuring heart rate
First Author Mean absolute error (MAE) (bpm)
Poh, MZ (2010) 2.29Poh, MZ (2011) 0.95Lee, KZ (2012) 3.22Rustand, A (2012) 0.60Jensen, JN (2014) 0.66Li, X (2014) 0.72McDuff, D (2014b) 0.01Tarassenko, L (2014) 3.00Tasli, HE (2014) 2.90
Villarroel, M (2014) 2.83Alzahrani, A (2015) 1.17Fletcher, RR (2015) -0.31Fouladi, SH (2015) 1.00Imaduddin, SM (2015) 4.64Lin, J (2015) 2.73Mannapperuma, K (2015) 4.50Tsouri, GR (2015) 4.31van Gastel, M (2015) 1.20Yu, YP (2015) 2.18Fallet, S (2016) 3.14Koprowski, R (2016) 0.00Lee, JS (2016) 2.28Prochazka, A (2016) 0.81Al-Naji, A (2017b) 0.99Coppetti, T (2017) 7.10Jain, M (2017) 1.73Lin, YC (2017) -2.96Gupta, P (2018) 1.63Liu, X (2018) 1.50McDuff, DJ (2018) 0.57Prakash, SKA (2018) -1.66
Table 10. Root mean square error for studies measuring heart rate
First Author Root mean square error (RMSE) (bpm)
Poh, MZ (2011) 1.24Rustand, A (2012) 4.95Tsouri, GR (2012) 3.50Bousefsaf, F (2013) 1.97de Hann, G (2013) 0.40Holton, BD (2013) 6.92Jeanne, V (2013) 0.53Yu, YP (2013) 2.19Zhao, F (2013) 3.10Bal, U (2014) 2.06Jensen, JN (2014) 1.53Li, X (2014) 1.27Monkaresi, H (2014) 1.69Shao, D (2014) 1.87Villarroel, M (2014) 3.95Cui, Y (2015) 0.03Lee, D (2015) 4.65Melchor Rodriguez, A (2015) 18.98 (ms)Tran, DN (2015) 1.53van Gastel, M (2015) 2.13Yan, Y (2015) 1.20Yu, YP (2015) 1.88Li, MH (2016) 5.87Addison, P (2017) 1.47Al-Naji, A (2017a) 0.31Al-Naji, A (2017b) 1.01Jain, M (2017) 2.33Li, MH (2017) 5.87Yan, BP (2017) 1.03Addison, P (2018) 2.15Antognoli, L (2018) 12.20Liu, X (2018) 1.42
Pereira, CB 2018 (2018) 3.43Prakash, SKA (2018) 5.56Sanyal, S (2018) 0.89Viejo, CG (2018) 0.06
Table 11. Bland-Altman statistics for studies measuring respiratory rate
First Author Mean difference (bpm) Upper limit of agreement Lower limit of agreement
Bartula, M (2013) 0.19 5.01 -4.63Bousefsaf, F (2013) -0.27 4.82 -5.36Sun, Y (2013) 0.03 1.35 -1.29Zhao, F (2013) -0.02 1.65 -1.69Shao, D 2014 (2014) 0.02 2.45 -2.40Pereira, CB (2015a) 0.03 1.40 -1.40Pereira, CB (2015b) 0.00 1.20 -1.20Al-Naji, A (2016) 0.21 0.88 -0.46Janssen, R (2016) 0.07 2.81 -2.67van Gastel, M (2016) 0.00 20.00 -20.00Al-Naji, A (2017a) 0.07 0.43 -0.3Al-Naji, A (2017b) 0.32 2.40 -1.70Cho, Y (2017) 0.08 2.02 -1.87Hu, M (2017) -0.30 2.39 -3.00Reyes, BA (2017) -0.02 0.80 -0.85Wei, B (2017) -0.70 3.30 -4.70Antognoli, L (2018) 0.30 6.60 -6.00Hu, M (2018) -0.64 1.87 -3.13Pereira, CB (2018) 0.08 2.30 -2.10Sanyal, S (2018) 0.80 8.25 -6.65
Mean difference (ms)
Bernacchia, N (2014) 0.00 0.96 -0.96
Mean difference (s)
Lewis, GF (2011) 0.00 0.40 -0.39
Table 12. Pearson correlation coefficients for studies measuring respiratory rate
First Author R
Fei, J (2005) 0.99Takano, C (2007) 0.93Lewis, GF (2011) 1.00Poh, MZ (2011) 0.94Bartula, M (2013) 0.98Bousefsaf, F (2013) 0.99Sun, Y (2013) 0.95Zhao, F (2013) 0.98Benetazzo, F (2014) 0.98Bernacchia, N (2014) 0.96McDuff, D (2014a) 0.93Shao, D (2014) 0.93Cenci, A (2015) 0.95Pereira, CB (2015a) 0.92Pereira, CB (2015b) 0.97Janssen, R (2016) 0.99van Gastel, M (2016) 0.98Cho, Y (2017) 1.00Hu, M (2017) 0.97Pereira, CB (2017) 0.98Reyes, BA (2017) 1.00Wei, B (2017) 0.97Hu, M (2018) 0.83Sanyal, S (2018) 0.66
Table 13. Percentage accuracy for studies measuring respiratory rate
First Author Percentage Accuracy (%)
Murthy, R (2004) 92.0Chekmenev, SY (2006) 100.0Fei, J (2010) 98.3Aubakir, B (2016) 94.0
Table 14. Mean percentage error for studies measuring respiratory rate
First Author Mean percentage error (MPE) (%)
Al-Naji, A (2016) 1.3Li, MH (2016) 3.4Nam, Y (2016) 1.4Li, MH (2017) 3.4Matlu, K (2018) 6.2
Table 15. Mean absolute error for studies measuring respiratory rate
First Author Mean absolute error (MAE) (bpm)
Poh, MZ (2011) 0.12Loblaw, A (2013) 1.50Benetazzo, F (2014) 0.40Pereira, CB (2015a) 1.39Pereira, CB (2015b) 0.71van Gastel, M (2016) 1.74Al-Naji, A (2017b) 0.28Kwasniewska, A (2017) 0.27
Table 16. Root mean square error for studies measuring respiratory rate
First Author Root mean square error (RMSE) (bpm)
Poh, MZ (2011) 1.28Zhao, F (2013) 0.09Li, MH (2014) 0.74Shao, D (2014) 1.20Li, MH (2016) 1.12van Gastel, M (2016) 2.57Al-Naji, A (2017a) 0.20Al-Naji, A (2017b) 0.37Cho, Y (2017) 0.46Li, MH (2017) 1.12Pereira, CB (2017) 0.28Reyes, BA (2017) 0.41Antognoli, L (2018) 7.60Pereira, CB (2018) 0.71Sanyal, S (2018) 3.89
Table 17. Bland-Altman statistics for studies measuring SpO2
First Author Mean difference (%) Upper limit of agreement Lower limit of agreement
Shao, D (2016) -0.07 2.51 -2.65
Table 18. Pearson correlation coefficient for studies measuring SpO2
First Author R
Blanik, N (2014b) 0.80Tarassenko, L (2014) 0.64Guazzi, A (2015) 0.89Shao, D (2016) 0.94
Table 19. Mean percentage error for studies measuring SpO2
First Author Mean percentage error (MPE) (%)
Verkruysse, W (2017) 0.1
Table 20. Mean absolute error for studies measuring SpO2
First Author Mean absolute error (MAE) (%)
Guazzi, A (2015) 1.85
Table 21. Root mean square error for studies measuring SpO2
First Author Root mean square error (RMSE) (%)
Shao, D (2016) 1.30Verkruysse, W (2017) 1.15
Table 22. Pearson correlation coefficient for studies measuring blood pressure (compared to PTT)
First Author R
Jensen, JN (2014) -0.80Murakami, K (2015) -0.89Jeong, IC (2016) -0.80Secerbegovic, A (2016) -0.72
Table 23. Mean absolute error for studies measuring blood pressure
First Author Mean absolute error (MAE) (mmHg)
Secerbegovic, A (2016) 9.48Jain, M (2017) 3.90
Table 24. Normalized mean square error for studies measuring blood pressure
First Author Normalized mean square error (RMSE) (%)
Jain, M (2017) 4.63
Table 25. Heart rate monitoring applications
Name Developer Non-contact?
Accurate Heart beat 100Solutionz NoHRV4Training A.S.M.A. B.V. NoHeart Rate Monitor: Heartbeat Monitor Aamir Ullah NoHeart Rate Monitor Aexol sp. Z o. o. sp k. Nomy pulse by I.C.E. Akers Media Group, Inc. NoHeart Rate Monitor AMTEE Apps NoHeart Rate Calculator Andrew Park NoPulsometr - Heart Rate Monitor Andrey Fetisov NoReal Heart Rate Monitor Androfreaks NoInstant Heart Rate Monitor AppAspect Technologies NoMy Heart Rate Measurement Apps Entertainment NoHeart Rate Monitor AppSourceHub NoHeart Beat Rate Observer Ashvin Patel NoResting Heart Rate ASSIC Health NoASUS Remote Heart Rate ASUSTeK COMPUTER INC. YesMy Heart Rate Monitor Atthaporn Chanprokon NoInstant Heart Rate Azumio Inc. NoPedometer 2018 BECOOL - Phone cooler NoiCare Health Monitor Beijing Jiajia kangkang Co. Ltd. NoiCare Health Monitor BigBIg Studio NoVitaPulse Black Dream NoHeart Rate Monitor blue brain NoHeart Rate Monitor BM innovations GmbH NoCardiio: Heart Rate Monitor Cardiio, Inc. YesHeartBeating Monitor Catrnja Dev NoFacial Heart Rate (Free) CCApp YesHeart Beat Rate Monitor ccloud apps NoHealth Heart Rate Da Costa NoPulse Oximeter - Heart Rate and Oximeter digiDoc Technologies AS NoHeart Rate Monitor Dirk Vranckaert NoPocket Oximeter & Heart Rate Monitor Dream studios, LLC NoHeart Rate Monitor DROID INFINITY NoHeart Rate Variability ECG for Everybody NoElite HRV Elite HRV NoUnique Heart Rate Eryumn NoCardiograph Heart Rate ExaMobile S.A. NoHealthy Beats - Heart Extentia NoiBiker Cycling Tracker Fitdigits NoHeart rate GR Giuseppe Romano NoInstant Heart Rate Monitor Gurupritsingh saini NoHeart Care Heart Care NoBLE Heart Rate Watch hekori NoMi Heart rate with Smart alarm Hubert Wilczynski NoInstant Heart Rate Igost Technologies Pvt NoSmart Heart Rate Monitor Janki Maniyar NoANT+ to BLE Heart jas0nchan NoBluetooth Heart Rate Jeremiah Huston NoSimple Heart Rate Monitor Jian Yih Lee NoHeart Rate Monitor & Instant Pulse Rate Joan Michael NoHeart rate monitor Julian Steinbock (Sibex) NoHeart Beat Expert KathyZhao NoMotiFIT-Heart Rate Kavanagh Software Inc NoHeart Rate OS - Android Watch LFApp NoQuick Heart Rate Monitor LUKKEN NoCardiograph MacroPinch NoHeart Trace magic09 applications No
Heart Rate Monitor marco45 NoHeart Rate Monitor: EKG Pulse Tracker Master App Solutions NoHeart Rate Plus Matteo Tagliafico NoHeart Rate Monitor MauzeSoft NoPulse checker Heart rate app Maxim Gudzik NoHeart Rate Monitor Meet Your Need Products NoHeart Rate Monitor Metari2+store NoHeart Rate Alarm Miguel Sanchez NoHeart Rate Monitor Mobile Essentials NoBLE Heart Rate & HRV MUC-InnOvation NoHeart Rate Monitor PRO Nattapon Kittsuphat NoReal Heart Rate Monitor Nermo NoHeart Rate Monitor: HR App Netviking AB NoHeart Rate Plus Ngo Na NoHeart Beats Counter nPSoftware NoPulsmesser - Herzfrequenz Philipp Blanke NoHeart Rate Monitor Plumbum Artz NoHeart Rate - Sport G PNN Software NoHeart Rate Measurement Real Preecha Subpasri NoBLE Heart Rate Monitor Pribble Software, LLC NoHeart Rate Zones PS NoHeart Rate Plus PVDApps NoCardio Monitor - Pulse Measure Qgs Inc NoHeart Rate Monitor - Pulse App Tracker Raman Kupriyianchyk NoHeart Rate Monitor REPS NoRthm Health Rthm Technologies Inc. NoRuntastic Heart Rate Runtastic NoQuick Heart Rate Monitor sanghoon Lee NoResting Heart Rate SensorBLE LLC NoHeart Rate Analyzer Sergio Gudkov NoHeart Rate Monitor SeverTech NoSoul Cycle Avoid Low Heart rate Sittiporn Nimmantewin NoVeryfit for heart rate Smart Wearable Device NoHeartRate+ Coherence SoftArea srl NoCardiograph Monitor BPM dete.. StartApps & Entrepreneurs NoCardiac diagnosis SUNG DO KIM NoHeart Rate Monitor Pro Sunny Studio NoInstant Heart Rate Monitor Tech Resort NoHeart Rate Monitor Theta Mobile NoInstant Heart Rate Time2Relax NoPulse - Heart Rate Monitor Vadim Akhmerov NoWhat’s My Heart Rate ViTrox Technologies YesmyWorkouts Heart Rate wellcrafted NoHeart Rate Monitor - Pulse BPM wenpeng zeng NoHeart BPM Monitor Winfy Software LLP NoHeart Rate Monitor + BPM detector Women for best free Apps NoHeart Beat Rate Xdroid NoHeart Rate Zombie Soup NoHeart Rate Monitor Zopogo No