Development of Interactive

NAO Robot Music Application

for Children

Fabrianne Effendi

EEE30

Why NAO?

◎ Interactive and engaging, thus children learn better

from NAO

◎ Caters to needs of various children

◉ Autism kids more receptive to NAO than humans

◎ Immense potential in the field of music education

Methodology

Overview of music application

3-step process to impart new music skills

Key interactivity

features

Speech Recognition FeatureAllows NAO to recognise predefined words or phrases,

subsequently responding to it.

Animated Say Box

Used to increase interactivity and make the robot more

human-like through carrying out human-like gestures

while it speaks

Python Script Box

Allows for more complex and precise behaviours to be

programmed

Timeline BoxContains a motion layer and multiple behaviour layers,

allowing NAO to do multiple things simultaneously

FlashcardsHelp children in their learning as they are visually appealing,

capturing children’s attention

Results & Discussion

Finding Optimal Speech Recognition

Confidence Threshold

Environment Accurate

detection

Confidence

level

Quiet ✓ 25-45%

Noisy ✗ <25%

Thus, NAO’s optimal speech recognition confidence threshold

was set to 25% for optimal detection accuracy.

Confidence level for speech recognition

Evaluation of methods to

integrate flashcards with NAO

robot music application

Object recognition

Mode of

detection

Able to

differentiate

flashcards

Accurate

detection

under various

lighting

conditions

Speed of

detection (1:

slowest; 3:

fastest)

Main disadvantage

Detects images

based on the

recognition of

key points

✗

(only able to

differentiate 3D

objects)

✓ 1 Unable to detect

external shape of

2D objects

ALColourBlob Detection

Mode of

detection

Able to

differentiate

flashcards

Accurate

detection

under various

lighting

conditions

Speed of

detection (1:

slowest; 3:

fastest)

Main disadvantage

Detects 2

dimensional

vision-based

colour blob.

✓ ✗

(since it is

dependent on

predefined RGB

colour)

2 May detect similar

colour from

surroundings before

flashcard is shown,

resulting in incorrect

detection of

flashcard

NAOmark

Mode of

detection

Able to

differentiate

flashcards

Accurate

detection

under various

lighting

conditions

Speed of

detection

(1: slowest;

3: fastest)

Detects unique

NAOmark.

✓ ✓ 3

Each unique NAOmark corresponds to a unique

integer output

NAO Video

Conclusion

Conclusion

◎ Successfully developed NAO robot

music application

◎ Demonstrates future potential for

humanoid robots to be used as an

educational learning tool in

classrooms

◎ Future work:

◎ To test the application on

children

◎ To customise music application

for children with autism

Thank you!