SkyEye Solution

Programming Guide

Dec. 2013

Nuvoton Technology Corp.

incmicro Only

2

Outline

Build Environment and Flow

Build Process for SPI Flash Booting

MS Framework

MS Plugin

Demo Web Site

Configuration File

Library and Utility

Note.

1. Source files path described here are based on

Nuvoton released SkyEye BSP and firmware source

2. Execution file path indicated path when SkyEye in

execution time

3

Build Environment Setup

To check build environment script for Ubuntu # ./CheckBuildEnv_ForUbuntu.sh

Make sure below utilities are installed before you start to build SkyEye project

automake

autoconf

libtool

genromfs

cmake

mtd-utils

gzip

For Ubuntu, users can install these utilities with apt-get install commands

# sudo apt-get install automake autoconf libtool genromfs cmake mtd-utils gzip

4

SD / NAND Booting Boot from SD card or NAND flash

Partitioned as 2 disks by default Both 2 disks can export to PC as external disks through USB mass storage

Only 1st disk appeared through card reader to read SD card

Props Support FAT32 file system compatible with PC

Suitable for development stage

Firmware could be updated through card reader or USB mass storage easily

Short booting and access time

Cons Cost concern

5

Build Kernel

Copy 'mkimage' to toolchain bin folder (default is

/usr/local/arm_linux_4.2/bin/)

Unpack zipped BSP source in your Linux host

and then switch to BSP folder

# tar zxvf SkyEye_*_BSP_SRC.tar.gz

# cd BSP

Interactive build script

# ./build_skyeye_bsp.sh sd/nand/spi

6

Build Kernel for SD/NAND Booting Unpack initial ram file system (use sudo to unpack if necessary)

# sudo tar zxvf initramfs.tar.gz

# sudo chown -R $username initramfs

# sudo chgrp -R $username initramfs

Copy mtd header files from kernel header folder to toolchain include folder

# sudo cp -af linux-2.6.35.4_fa9x/include/mtd/ /usr/local/arm_linux_4.2/include/

To build busybox-1.15.2 # cd applications/busybox-1.15.2/

# cp ../../../SkyEye_*_busybox.config .config

# make

# make install

To build Linux kernel # cd ../../linux-2.6.35.4_fa9x

# cp ../../SkyEye_*_kernel_config_sd_* .config

# ./build sd

7

SPI Booting

Boot from SPI flash

Divided into 4 MTD partitions by default

Pros

JFFS2 file system with MTD partition support for flash

Low cost and suitable for production stage

Cons

Can not export to PC through USB mass storage

Spent more booting and access time

Need TurboWriter to update firmware

8

SPI Flash Layout

Minimum requirement of SPI flash size is 8MB

for N3290x (16MB for N3292x)

Layout information is defined as

w55fa9x_spi_flash_partitions in kernel source

path - arch/arm/mach-w55fa9x/dev.c

Default partition size

1st (256KB): SPI loader

2nd (2.8MB): Linux kernel binary

Specify in kernel configuration

3rd (5MB): SkyEye binary and network driver (13MB for

N3292x)

Specify in kernel configuration

4nd (8MB): Optional reserved area for firmware update

(16MB for N3292x)

9

SPI Flash Layout (Cont.) Kernel configuration (Only available in SkyEye BSP)

Device Drivers -> SPI support -> Nuvoton W55FA9x series SPI

To specify [The execution partition size] for 2nd partition size

Defined as CONFIG_SPIFLASH_PARTITION_EXECUTE_SIZE

To specify [The data partition size] for 3rd partition size

Defined as CONFIG_SPIFLASH_PARTITION_DATA_SIZE

Loader & Logo

Conprog

UserData_1

Reserved

0x0

0x4000

0x4000+CONFIG_SPIFLASH_PARTITION_EXECUTE_SIZE

0x40000+CONFIG_SPIFLASH_PARTITION_EXECUTE_SIZE+CONFIG_SPIFLASH_PARTITION_DATA_SIZE

0x4000

CONFIG_SPIFLASH_PARTITION_EXECUTE_SIZE

CONFIG_SPIFLASH_PARTITION_DATA_SIZE

MTDPART_SIZ_FULL

10

Build Kernel for SPI Booting

The same steps as Build Kernel for SD/NAND Booting

except for last step

To build Linux kernel

# cd cd linux-2.6.35.4_fa9x

# cp ../../SkyEye_*_kernel_config_spi_* .config

# ./build spi

11

Build SkyEye Project Unpack released zipped SkyEye source in your Linux host and then switch to source folder

# tar zxvf SkyEye_XXXXX_src.tar.gz (XXXXX is specified for one target board)

# cd source

To build SkyEye for specified target board

# ./build_skyeye.sh (if build_skyeye.sh existed) or

# ./build.sh XXXXX

After successful build, output firmware path in folder

output/build/XXXXX/nand/nand1-1/

Copy all files under output/build/XXXXX/nand/nand1-1/ to SD1-1 partition on bootable SD card

Insert SD card to target board and reboot to execute SkyEye

12

Build SkyEye JFFS2 Image for SPI Flash

Please Build SkyEye Project first

Use mkfs.jffs2 utility to create jffs2 image # mkfs.jffs2 -r $SRC_PATH -o $DST_PATH/skyeye.jffs2 -e 0x10000 --pad=$DataSize

ex. # mkfs.jffs2 –r ./nand/nand1-2 –o ./skyeye.jffs2 –e 0x10000 –pad=0x500000

$SRC_PATH contains firmware, network drivers and all other stuff After build successfully from released source, it means

output/build/XXXX/nand/nand1-2 and copy desired network driver manually

$DST_PATH is folder to place output jffs2 image files

$DataSize is CONFIG_SPIFLASH_PARTITION_DATA_SIZE metioned in SPI Flash Layout (Cont.)

skyeye.jffs2 file size must = $DataSize

Use sumtool to create summarized jffs2 image # sumtool -l -e 0x10000 -p -i $DST_PATH/skyeye.jffs2 -o $DST_PATH/skyeye.jffs2.summary

ex. # sumtool –l –e 0x10000 –p –i ./skyeye.jffs2 –o ./skyeye.jffs2.summary

skyeye.jffs2.summary file size must <= ($DataSize – 512KB) or remove some unnecessary files to rebuild jffs2 image again

Follow user guide to write SPI flash images with TurboWriter

13

MS Framework

Inherited from MJPG-streamer and designed for

Nuvoton Wi-Fi IP camera or remote control toy

with N329 series

SkyEye are divided into sub-tasks and each one

has dedicated plugin to implement

Communication, data passing and process

between plugins

Basic producer and consumer solution

Producer occupies dual buffer and outputs resources

according to its own configuration

Consumer could be informed to check updated

resource automatically or check updated resources

periodically

14

MS Framework (Cont.)

MSF for short

Composed of loader, plugins and configuration

files

Latest MSF version is 2.0 and there are some

difference between different versions

Descriptions prefixed with Ver. 1.0 / Ver. 2.0 are version

dependent and version information is defined as

DEF_MSF_VERSION in src/ms_framework.h

15

Block Diagram

Linux BSP

Nuvoton Library 3rd Party Library

SkyEye

MS Framework

Nuvoton Utility MS Framework

Plugin

N3290x/N3292x Chip

Wi-Fi Network Sensor Microphone Speaker

16

System Working Flow

Network Setup

(setup_network.sh)

Puncher / DDNS

Kernel Startup

(boot_script)

Wait Client

Connection

SkyEye Env Setup

(ramfsboot_go.sh)

NTP MS-Loader

(msloader_go.sh)

Plugin

17

SkyEye Working Flow

HTTP Request

MS Framework

HTTP Plugin

HTTP Response Streaming

Data

JPEG

Frame

MS Framework

JPEG Encoder Plugin

MS Framework

Video-In Plugin

Video Data

MP3 Frame

MS Framework

MP3 Encoder Plugin

MS Framework

Audio-In Plugin

Audio Data Client

export

export

export

export

import

import

import

18

MS Loader

Load plugins to execute according to predefined

options

ex. ./msloader -p "plugin_audio_in.so -s 16000“ -p

"plugin_alaw_encoder.so"

Provide resource management functions

For plugins to export its own resource data to others

plugin

For plugins to import other plugins’ resource data

For plugins to allocate and free resource by itself

19

Dual buffer of resource data

One for active import(read) and one for active

export(write)

One exports for multiple import through rwlock

Resource Passing

Plugin A

Res-2

Res-1

Plugin B Export Import

Plugin C

Plugin D

Write

Read

20

Resource Data Structure Defined in inc/msf.h

m_pBuf Data type: unsigned char *

Contain exported data

Must be allocated and freed by plugin itself

Must set it to NULL if it is freed

m_uiBufByteSize Data type: uint32_t

Actual byte size of m_pBuf

Must set it to 0 if m_pBuf is freed

m_uiBufUpdateTime Data type: uint64_t

Updated time in millisecond of m_pBuf

m_uiBufSeqNumber Data type: uint64_t

Resource output sequence

21

Framework Interface

Defined in inc/msf.h

Function List

m_pfnAllocateRes

m_pfnFreeRes

m_pfnResizeRes

m_pfnImportUnlockRes

m_pfnImport(Lock)

m_pfnImportDirty(Lock)

m_pfnImportWaitDirty(Lock)

m_pfnImportTimedwaitDirty(Lock)

m_pfnExportGrant(Lock)

m_pfnExportCommit(Unlock)

m_pfnSendCmd

22

MS Framework – Resource Allocation

m_pfnAllocateRes

Suggest to invoke this function to allocate memory

space for resource data before using or exporting it to

public

m_pfnFreeRes

Suggest to invoke this function to free memory space of

allocated resource when exit or no need any more

m_pfnResizeRes

Suggest to invoke this function to resize a specified

resource data

23

m_pfnExportGrant

Grant access to active write resource data

m_pfnExportCommit

Commit current access resource data to an active read

one

eMSF_COMMIT_DO_COPY

Data is copied from Res-1 to Res-2

eMSF_COMMIT_DO_SWAP

MS Framework – Export Resource

Res-1

Read Write

Res-2 Res-1

Write Read

Res-2 Commit

Res-1

Write Read

Res-2 Res-1

Write Read

Res-2 Commit

24

Export Resource Data

Grant only once for further if commit with

eMSF_COMMIT_DO_COPY

Grant first to get an active resource data pointer

Grant befoe each commit if commit with

eMSF_COMMIT_DO_SWAP

Each grant would return active write resource

data pointer

Data Process ExportGrant ExportCommit eMSF_COMMIT_DO_COPY

Data Process ExportGrant ExportCommit eMSF_COMMIT_DO_SWAP

25

MS Framework – Import Resource m_pfnImport(Lock)

Import active read resource data

m_pfnImportDirty(Lock) Import active read resource data that update time is newer than current

m_pfnImportWaitDirty(Lock) Wait to import newest active read resource data

m_pfnImportTimedwaitDirty(Lock) Wait specified time to import newest active read resource data

m_pfnImportUnlockResff Work with lock functions to unlock resource

(Lock) functions Lock resource to prevent other plugins to modify active read data if any concern

26

Plugin Command m_pfnSendCmd

Command need to be defined with

MACRO_MSF_MAKE_CMD(ui8CmdHandlePluginVersi

on, eCmdHandlePluginID, ui8CmdID)

This function would be called by other thread therefore

it has to be thread safe

27

MS Plugin

Interface header file

inc/plugin_if.h

Each plugin has a unique ID declared in

enumeration E_MSF_PLUGIN_ID

Each plugin must implement interface declared

as structure S_MSF_PLUGIN_IF Ver. 2.0 Each plugin has its own configuration file

under source/res/conf/factory(_ChipID)

Plugin configuration file name prefixed with _ will be

treat as disabled plugin and will not load by MS-Loader

28

Plugin Interface - Variable m_uiID

This ID should be defined in plugin_if.h first

m_uiVersion Version information

m_uiResCount MS-Loader would help to prepare resource according to this value

Remember to use m_pfnAllocateRes to allocate all resources

Remember to use m_pfnFreeRes to free all resources

m_bInitialized Ver. 2.0 Set TRUE after plugin initialization is done

m_strName Ver. 2.0 has to be exact the same as plugin output name (ex. plugin_timer.so’s m_strName must be assigned as timer) and Ver. 1.0 does not have this limitation

m_ControlMutex Plugins could use this mutex to lock critical operation if necessary and loader would help to initialize this mutex

Others are handled by MS-Loader

29

Plugin Interface - Function m_pfnMain

MS loader will create a new thread to invoke this function at startup

time

Parameter

strCmdArg: Option argument string for this plugin

m_pfnCommand

Command handle function

This function must be thread-safe because it could be invoked from

other plugin threads

Parameter

uCmd: Command defined in plugin header file or broadcast command

pValue: command value

pRetValue: return current command value if required or set NULL to ignore

Return value

DEF_MSF_SUCCESS: Command is sent successfully

Others: Failed to send command

30

Execution Script misc/msloader_go.sh Ver. 1.0 Each plugin can specify its own parameters to execute

Ver. 2.0 Plugin loading depends on its configuration file name within SkyEye execution script

Filename prefix with “_” means this plugin is disabled

msloader -p "plugin_video_in.so -s $MSF_CONFIG_VIDEO_IN_RESOLUTION -r $MSF_CONFIG_VIDEO_IN_FPS" \

-p "plugin_jpeg_encoder.so -q $MSF_CONFIG_JPEG_ENCODER_QUALITY" \

-p "plugin_audio_in.so" \

-p "plugin_mp3_encoder.so" \

-p "plugin_http.so -w /mnt/skyeye/htdocs -p 80 $MSG_CONFIG_JPEG_BITRATE" \

-p "plugin_recorder.so -a mp3 -p /mnt/rec_folder" \

-p "plugin_timer.so -i 1 -w 1" \

-p "plugin_alarm_in.so -f /mnt/skyeye/etc/alarm.conf" \

-p "plugin_alarm_config.so" \

-p "plugin_alarm_out.so -f /mnt/skyeye/etc/alarm.conf" \

-p "plugin_avi_reader.so" \

-p "plugin_alaw_encoder.so" \

-p "plugin_rtsp.so -p /cam1/mpeg4" \

&

31

To Add New Plugin

Below steps are based on Nuvoton released

SkyEye firmware source

Add new plugin ID enumeration to

E_MSF_PLUGIN_ID in inc/plugin_if.h

Create new plugin folder (ex. plugin_new) under

plugins/ and add new plugin source files

Assign plugin variables and implement plugin

interface

Add new configuration file under

res/conf/factory(_ChipID)

32

To Build New Plugin

automake and autoconf tool are used to simplify

build process

Copy one Makefile.am from exist plugin folder

and replace plugin name and source file name

with new ones in Makefile.am

ex. If you copied Makefile.am from timer plugin, replace

all patterns “plugin_timer” with “plugin_new”

Add new plugin name to SUBDIRS variable in

plugins/Makefile.am

Add new plugin makefile path to

AC_CONFIG_FILES variable in configure.ac

Rebuild project

33

Plugin Ver. 1.0 Only some plugin options are saved in res/conf/factory/msloader.conf Ver. 2.0 Each plugin has its own configuration file to save options under res/conf/factory/plugin/

Plugin List

plugin_audio_in / plugin_video_in

plugin_mp3_encoder / plugin_alaw_encoder / plugin_ulaw_encoder / plugin_aac_encoder

plugin_jpeg_encoder / plugin_h264_encoder

plugin_http / plguin_multicaster / plugin_rtsp / plugin_ts_writer / plugin_flv_streamer

plugin_recorder / plugin_avi_reader

plugin_cmd_server

plugin_alarm_config / plugin_alarm_in / plugin_alarm_out

plugin_timer

Plugins marked with green are only available in N3292x series

34

Plugin Configuration File

Format

param_name(No space)=param value#param

comment#option(No space)

Ex. Param_AAA=xxx#xxx or yyy#-a

Limitation

No space in param name and option column

No # and = in all columns

Accept characters in all columns

.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQ

RSTUVWXYZ_-+&1234567890@[]?%/

35

Video-In Plugin

Work with V4L v1 compatible driver

Export raw image data captured from sensor

Export frame difference for motion detection

HW motion detection works on packet pipe

SW motion detection works on planar pipe

Support at most 2 ports with 4 pipes to output

different resolutions and color type at the same

time

N3292x has 2 H/W video capture ports support to

connect 2 pysical sensors called dual sensor or 1

physical sensor called share port

Each port can output 2 pipes raw data composed of

planar and packet format

Dimension of port 1 must be less than or equal to port 0

36

Video-In Port/Pipe Illustration

Port 0 Pipe 0 (Planar)

Pipe 1 (Packet)

Single-Sensor

Share-Port

Port 0 Pipe 0 (Planar)

Pipe 1 (Packet)

Port 1 Pipe 2 (Planar)

Pipe 3 (Packet)

Port 0 Pipe 0 (Planar)

Pipe 1 (Packet)

Port 1 Pipe 2 (Planar)

Pipe 3 (Packet)

Dual-Sensor

37

Video-In Plugin For H264 encode purpose, width and height must be multiple of 16

For JPEG encode purpose, width and height must be multiple of 8

Brightness/Contrast controls are sensor dependent and need customization for non-support sensor

Factory configuration file path – res/conf/factory/plugin/video_in.conf

Option Description

PipeX Resolution Resolution of each videoin pipe

1280x720 / 64x480 / 320x240

PipeX Color Type Color type of each videoin pipe

0: None, 1: YUV422, 2: YUV422P, 3: YUV420P_MB

PortX Output FPS FPS of each videoin port, 1 ~ 30

PortX Sensor Clock Sensor clock of each videoin port, 12 ~ 24

PortX Motion Detection 0: Auto(HW first), 1: HW, 2: SW

38

Motion Detection – H/W or S/W blockize frame

Whole frame is divided into 8x8 blocks

Only central pixels (4,4) of each block is used to

calculate difference

N329 H/W motion detection only works on packet pipe.

·

39

Motion Detection – H/W or S/W compare frame

H/W or S/W compares each central pixel with previous frame to check difference.

DiffBuf must be word alignment and result is stored in following format

1 bit DIFF(MSB) + 7 bits Y Differential for one block

MSB = 1 if Y difference > BlockDiff (32 levels, default level is 20)

BlockDiff in res/conf/alarm.conf (/mnt/skyeye/etc/alarm.conf - Path for execution time)

B3 B2 B1 B0

B7 B6 B5 B4

…

0 0 B21 B20

Low Word

High Word

Padding 0 if block count is not word alignment

40

Motion Detection – F/W detect motion

Compare frame per 100ms

Count number of bytes with MSB == 1 in DiffBuf

A motion is detected if the count >= FrameDiff

(Default 3%)

FrameDiff in res/conf/alarm.conf

(/mnt/skyeye/etc/alarm.conf - Path for execution time)

Flag g_uiNotify[eALARM_SRC_IDX_MD] and

g_uiNotify[eALARM_SRC_IDX_MD1] in alarm

plugins

1: If a motion is detected

0: If no motion is detected during ReprieveTime

(Default 10s)

ReprieveTime in res/conf/alarm.conf

(/mnt/skyeye/etc/alarm.conf - Path for execution time)

41

Audio-In Plugin

Work with OSS compatible (ALSA) driver

Export raw PCM data captured from built-in ADC

Software voice detection

Factory configuration file path –

res/conf/factory/plugin/audio_in.conf

Option Description

Device Name Device (Default: /dev/dsp1)

Sample Rate Sample rate (Default: 8000)

Channel Channel count (Default: 1)

Threshold Voice detection threshold

42

Voice Detection

Function CalAudioPower in plugin_audio_in.c

PCM buffer pointer

PCM buffer byte size

Apply low pass filter to whole PCM buffer

Smpl[N]’PCM = (Smpl[N]’PCM + Smpl[N+1]’PCM) / 2

Compare summation of each 10ms duration(slot)

of PCM absolute value with defined threshold

option and count slots that exceeded threshold

Export percentage of exceeded slot information -

S_AUDIO_IN_VOICEDET_DATA

(Exceeded slot count / total slot count in buffer) * 100%

PCM Buffer

10ms 10ms 10ms 10ms 10ms 10ms 10ms 10ms …

N exceeded slots / M ) * 100% (

43

JPEG Encoder Plugin Need to import packet type or planar type (not marco block planar) raw data from video-in plugin

Therefore encode pipe is not one-to-one map to original video-in pipe

Export encoded jpeg data Hardware IP

Support max to 4 pipes encoding

Nuvoton media library involved

Factory configuration file path – res/conf/factory/plugin/jpeg_encoder.conf

Option Description

PipeX Quality Quality (Default: 8)

PipeX Bitrate Output bitrate (Default: 0)

44

H264 Encoder Plugin

Only available in N3292x series

Need to import marco block planar type raw data

from video-in plugin

Export H264 bit stream (baseline profile)

Support max to 2 pipes encoding

Nuvoton media library involved

Factory configuration file path –

res/conf/factory/plugin/h264_encoder.conf

Option Description

PipeX Bitrate Control output bitrate in Kbps

PipeX Quality Encode quality range from 0(best) to 51(worst)

PipeX GOP Control GOP, 0 means auto

45

Encoder Pipe Mapping

Encoder pipe would re-order according to

available video-in pipe that supports to encode

JPEG Encode Pipe 1 Port 0

Pipe 0 (YUV420 Planar)

Pipe 1 (YUV422)

Port 1 Pipe 2 (YUV422Planar)

Pipe 3 (Packet)

JPEG Encode Pipe 2

JPEG Encode Pipe 3

Example 1.

JPEG Encode Pipe 0

JPEG Encode Pipe 0 Port 0

Pipe 0 (MB Planar)

Pipe 1 (YUV422)

Port 1 Pipe 2 (YUV422Planar)

Pipe 3 (Packet) JPEG Encode Pipe 2

Example 2.

JPEG Encode Pipe 1

H264 Encode Pipe 0

46

MP3 Encoder Plugin

Import raw PCM data from audio-in plugin

Export encoded MP3 data

Software encoder

Nuvoton media library involved

Factory configuration file path –

res/conf/factory/plugin/mp3_encoder.conf

Option Description

Bitrate Bitrate (Default: 128)

Frame Count Export MP3 frame count

47

AAC Encoder Plugin

Import raw PCM data from audio-in plugin

Export encoded AAC bit stream

Support hardware (Only available in N3292x series)

and software encoder

Nuvoton media library involved

Factory configuration file path –

res/conf/factory/plugin/aac_encoder.conf

Option Description

Bitrate Encode bitrate

HW Encoer 0: Disable, 1: Enable

48

ALAW/ULAW/ADPCM Encoder Plugin

Import raw PCM data from audio-in plugin

Export encoded alaw, ulaw or adpcm data

Software encoder

Nuvoton media library involved

Factory configuration file path –

res/conf/factory/plugin/alaw_encoder.conf,

res/conf/factory/plugin/ulaw_encoder.conf,

res/conf/factory/plugin/adpcm_encoder.conf

49

Option Description

WWW folder Root folder of web pages (Default: /mnt/skyeye/htdocs)

Port HTTP server port (Default: 80)

Timeout In second and 0 means no timeout

HLS_XXXX HLS related options

HTTP Plugin HTTP server to parse HTTP request and execute simple CGI script

Audio/Video streaming server to import encoded data from JPEG encoder and MP3 encoder plugin

Parse HTTP command request and bypass predefined command to other plugins

Send response through connected socket Video

Audio

Command

Factory configuration file path – res/conf/factory/plugin/http.conf

50

HTTP Plugin (Cont.)

HTTP server creates a new thread to serve each

client request

New Thread

Send Video

Do Command

Receive Audio

Request

String

…

Server

Thread

New Thread

Request

String

New Thread

Request

String

51

Recorder Plugin Import encoded data from video encoder and audio encoder plugin

Snapshot as JPEG or record as AVI or MP4

Continuous record with specified file duration

Nuvoton media library involved

Factory configuration file path – res/conf/factory/plugin/recorder.conf

Option Description

Record Path Record file save path

Snapshot Path Snapshot file save path

File Duration File duration in millisecond (Default: 300000)

Audio Format mp3/alaw/ulaw/pcm/aac

File Format avi/mp4

Resolution Record resolution (Default: 640x480)

Video Format mjpg/h264

52

AVI Reader Plugin

Decode audio and video data from recorded AVI

file and export decoded data

Need to specify file name through command

Nuvoton media library involved

Factory configuration file path –

res/conf/factory/plugin/avi_reader.conf

53

Timer Plugin

Export time data read from system timer

periodically

Responsible to reset watch dog timer

Factory configuration file path –

res/conf/factory/plugin/timer.conf

Option Description

Interval Time interval (Default: 1 second)

WDT 1 to enable, 0 to disable watch dog function (Default: 1)

54

Alarm-In Plugin

Import motion detection/voice detection/time data

from video-in/audio-in/timer plugin

Export alarm triggered information

Monitor alarm source status to trigger alarm

Parameters are saved in specified configuration

file path

Factory configuration file path –

res/conf/factory/plugin/alarm_in.conf

Option Description

Config Path Configuration file path (Default: /mnt/skyeye/etc/alarm.conf)

55

Alarm-In Plugin Export Data

S_ALARM_IN_RES defined in plugin_alarm_in.h

m_eAlarmSrcIdx - Alarm in source index changed at

present

m_uiAlarmInRes - All alarm in source status in bit fields

to check m_eAlarmSrcIdx is rising or falling

m_paAlarmInSrcName - Alarm in source name

m_sNotifyTimeStamp - Notification timestamp

56

Alarm-Config Plugin

Import from alarm-in and recorder plugin

Parameters are saved in specified configuration

file path

Factory configuration file path –

res/conf/factory/plugin/alarm_config.conf

Option Description

Config Path Configuration file path (Default: /mnt/skyeye/etc/alarm.conf)

57

Alarm-Config Export Data

S_ALARM_CONFIG_DO_ACTION

m_au32DoAction: Indicate to snapshot or record

S_ALARM_CONFIG_OUT_ACTION

m_sAlarmInRes: alarm_in plugin exported data

m_uiOutAction: GPIO/email/FTP/Dropbox

m_eAttachType: txt/image/video

m_astrAttachFileName: Attached file name if

m_eAttachType is image or video

58

Alarm-Out Plugin

Import alarm output information

(S_ALARM_CONFIG_OUT_ACTION) from

alarm-config plugin

Trigger GPIO action

Send email with text or image attached

Upload image or video file to FTP or Dropbox

Parameters are saved in specified configuration

file path

Factory configuration file path –

res/conf/factory/plugin/alarm_out.conf

Option Description

Config Path Configuration file path (Default: /mnt/skyeye/etc/alarm.conf)

59

Multicast Plugin

Import encoded data from JPEG encoder and

MP3 encoder plugin

UDP multicasting in local network

Video would be send through specified port - 1

Audio would be send through specified port

Option Description

Role 0: sender/1: receiver

Interface Network interface name (Default: ra0)

Group IP Group IP address (Default: 225.225.225.225)

Port Port (Default: 9527)

60

RTSP Plugin Migrated from Spook Live Video Streamer

Not support flow control

Default connection URL rtsp://xxx.xxx.xxx.xxx/cam1/mpeg4 for MJPG pipe 0

rtsp://xxx.xxx.xxx.xxx/cam1/mpeg4-1 for MJPG pipe 1 is exist

rtsp://xxx.xxx.xxx.xxx/cam1/h264 for H.264 pipe 0

rtsp://xxx.xxx.xxx.xxx/cam1/h264-1 for H.264 pipe 1 if exist

Factory configuration file path – res/conf/factory/plugin/rtsp.conf

Option Description

MJPG URL Connection path to view MJPG

MJPG Bitrate Bitrate control for MJPG in bps

MJPG Audio Type alaw/ulaw/dvi/aac

H264 URL Connection path to view H264

H264 Audio Type alaw/ulaw/dvi/aac

Frame Count Frame buffer count (Allocated stream buffer size equals Frame_Count

* Frame_Size)

Frame Size Frame buffer size in byte (Allocated stream buffer size equals

Frame_Count * Frame_Size)

61

FLV Streamer Plugin

To support Real Time Messaging Protocol

(RTMP)

Need crtmpserver support or other Flash media

server

Bypass H.264 and AAC merged stream to

crtmpserver

Factory configuration file path –

res/conf/factory/plugin/flv_streamer.conf

Option Description

Keep Streaming Keep streaming or not after plugin loaded

0: Disabled, 1: Enabled

62

TS Writer Plugin

To support Apple HTTP Live Streaming (HLS)

Record H.264 and AAC merged stream as TS

files in ram disk

Factory configuration file path –

res/conf/factory/plugin/ts_writer.conf

Option Description

Record Path Path to save recorded TS files

Duration Record duration in millisecond per TS file

63

Demo Web Site Web pages located in res/htdocs/SkyEye

HTML

CGI

CSS

Javascript

jQuery

jPlayer

Java Applet

Cambozola

Stream viewer

http://www.charliemouse.com:8080/code/cambozola/

utility/java/cambozola-0.92

JLayer

MP3 decode library

http://www.javazoom.net/javalayer/javalayer.html

Recorder

To send client audio to SkyEye

utility/java/Recorder

64

HTTP Stream/Command Request

Video Stream

/video.cgi?identify_key=xxx

Audio Stream

/server.audio?kbsize=xxx

A/V Stream

/server.stream

Audio Upstream

/audio.input

Command

/server.command?command=xxx&value=yyy

plugin_http helps to bypass command request sent

from client browser to other plugin

Document

Nuvoton SkyEye Solution Streaming Protocol

65

Development Mode

Default released firmware is packed into single

binary file to reduce copy time

During development, users can follow below

steps to modify separate file more easily

In console window

Copy all files under /mnt/skyeye/ to /mnt/nand1-2

(Ignore warning messages “cp: cannot create

symlink…: Operation not permitted”)

Rename skyeye_romfs.bin under /mnt/skyeye/ to other

name

# cp -af /mnt/skyeye/* /mnt/nand1-2/

# cp -af /mnt/nand1-1/etc/ /mnt/nand1-2

# mv /mnt/nand1-1/skyeye_romfs.bin /mnt/nand1-1/_skyeye_romfs.bin

# sync

66

Configuration File

All configuration files are text files and placed

under /mnt/skyeye/etc/ when execution

network_config

msloader.conf

puncher.conf

alarm.conf

network_config and puncher.conf support to

read/write through web pages and SkyEyeToolkit

tool

msloader.conf and alarm.conf support to

read/write through web pages

67

network_config

Wi-Fi network configuration for station

Content format

Key and value pair separated by space

Option Description

BOOTPROTO DHCP/STATIC

IPADDR xxx.xxx.xxx.xxx

GATEWAY xxx.xxx.xxx.xxx

SSID Specify SSID

AUTH_MODE OPEN/SHARED/WEPAUTO/WPAPSK/WPA2PSK/WPANONE

ENCRYPT_TYPE NONE/WEP/TKIP/AES

AUTH_KEY Specify authentication key

WPS_TRIG_KEY UP/DOWN/RIGHT/HOME

68

network_config (Cont.)

Wi-Fi network configuration for soft AP

Single mode

Network script will try station mode first or enter soft AP

mode if failed to set station mode

Concurrent mode

Some Wi-Fi drivers support run station and soft AP

mode simultaneously

Option Description

AP_IPADDR xxx.xxx.xxx.xxx

AP_SSID Specify SSID

AP_AUTH_MODE OPEN/SHARED/WEPAUTO/WPAPSK/WPA2PSK/WPANONE

AP_ENCRYPT_TYPE NONE/WEP/TKIP/AES

AP_AUTH_KEY Specify authentication key

AP_CHANNEL 1~13 (most of world)

69

msloader.conf

MS-Loader configuration file

Content format is the same as plugin

configuration

Option Description

Time_Zone GMT-xxxx

Phyical_Record_Path This path will be linked as /mnt/rec_folder by default for recorder

Check_Duration Specify duration in seconds to check if plugin is occupied or not to

unload idle plugin

TS_Output_Path This path will be mount as ramdisk by default for ts_writer

70

puncher.conf

libconfig format

Holes specifies UPnP parameters to map internal

port and external port

DDNS stores registered information

DDNS = (

{

/* NO-IP */

Server = "dynupdate.no-ip.com";

Username = "g9413737@yuntech.edu.tw";

Password = "1qaz2wsx";

Hostname = "nuvoton.no-ip.org";

}

);

Holes = (

{

/* fill?, [TRUE, FALSE] */

isFill = FALSE;

/* Internal port, String, [1-65535] */

InternalPort = "80";

/* External port, String, [1-65535] */

ExternalPort = "8080";

/* Protocol, String, [TCP, UDP] */

Protocol = "TCP";

/* Duration, String, [0-2^32] */

Duration = "0";

}

);

71

alarm.conf – Alarm-In

libconfig format

GPIO, MD(Motion Detection), VD(Voice

Detection) and SCHEDULE GPIO = {

CONFIG = {

# second, 1 ~ 60

PollingTime = "5" ;

PinMask = "15";

};

};

MD = {

CONFIG = {

BlockDiff = "1"; # Block diff, 1 ~ 31

FrameDiff = "5"; # percentage, 1 ~ 100

ReprieveTime = "10"; # second, 5 ~ 60

};

};

VD = {

CONFIG = {

SlotDiff = "15"; # percentage, 1 ~ 100 %

FrameDiff = "25"; # percentage, 1 ~ 100 %

ReprieveTime = "10"; # second, 5 ~ 60

};

};

72

alarm.conf – Alarm-In (Cont.)

SCHEDULE allows to configure a period of time

(Begin_Time to End_Time) weekly during

specified duration (Begin_Data to End_Date)

SCHEDULE = {

CONFIG = (

{

# [0]

# Enable, boolean: 0,1

Enable = "1";

Begin_Date = "2012/09/18";# Beginning date

End_Date = "2032/09/18"; # Ending date

# Ex:Sun~Sat=127, 0: Sun, 1: Mon, 2: Tue, 3: Wed, 4: Thu, 5: Fri, 6: Sat

Weekly = "127";

Begin_Time = "21:00:00"; # Beginning time

End_Time = "8:00:00"; # Ending Time

},

…

);

};

73

alarm.conf – Alarm-Config

libconfig format

SMTP, FTP and DROPBOX for network notifications

Mission is used to assign server actions like

GPIO/Snapshot/Record when alarmed triggered

Matrix = {

GPIO =

{

Mission = "7"; # 0: None, 1: GPIO, 2: Snapshot, 4: Record

SMTP = "1"; # 0: OFF, 1: Default, 2: JPG, 4: AVI

FTP = "1"; # 0: OFF, 1: Default, 2: JPG, 4: AVI

DROPBOX = "1"; # 0: OFF, 1: Default, 2: JPG, 4: AVI

};

…

…

SCHEDULE = (

{

Mission = "7"; # 0: None, 1: GPIO, 2: Snapshot, 4: Record

},

{

Mission = "7"; # 0: None, 1: GPIO, 2: Snapshot, 4: Record

}

);

};

74

alarm.conf – Alarm-Config (Cont.)

Attached file option – Not implemented

TXT = (

{

SUBJECT = "SkyEye Alarm";

CONTENT = "Alarm!";

}

);

JPG = (

{

COUNT = "1";

}

);

AVI = (

{

DURATION = "5";

}

);

75

alarm.conf – Alarm-Out

libconfig format

GPIO supports pin mask

PinMask to specify which GPIO pin to react – Just

demo and not actual implement

SMTP supports general parameters to send

Email

Host/Port/Username/Password are parameters used

to login mail server site to send mail

MailTo to specify receiver

Subject to specify mail subject

Body to specify mail content

76

alarm.conf – Alarm-Out (Cont.)

FTP supports general parameters to connect and

upload data

Host/Port/Username/Password are parameters used

to login FTP site to upload data

RemotePath to save data

Dropbox supports APP authentication access

Each application like SkyEye needs to register to

Dropbox to get a unique AppKey and AppSecret

After user’s authentication with Dropbox site,

application can get OAuthAccessTokenSecret and

OAuthAccessToken for further access

RemotePath to save data

77

Library

Nuvoton

libnmedia – recorder, avi reader, audio/video encoder

libnuio – access hardware register

3rd party

libconfig-1.4.7 – access config file

http://www.hyperrealm.com/libconfig/

miniupnpc – UPnP

http://miniupnp.free.fr/

qrencode 3.3.1 – generate QR code image

http://fukuchi.org/works/qrencode/index.html.en

dnsmasq (BSP included) – DHCP server

http://www.thekelleys.org.uk/dnsmasq/doc.html

78

Utility

Nuvoton Owned or Modified

png2fb – copy PNG file (32-bit ARGB) to frame buffer

and display

# ./png2fb –f /tmp/qr.png (To show qr.png on frame buffer)

kpdin – detect keypad input

# ./kpdin –t 60 (To scan one keypad input in 60 seconds)

puncher – punch UPnP and DDNS setting with

configuration file (puncher.conf)

# ./puncher –f /mnt/skyeye/etc/puncher.conf (Do puncher

job according to puncher.conf)

mtdtool – firmware verify, block erase, flash program for

online/offline firmware update

http://www.linux-mtd.infradead.org/

79

Utility (Cont.)

3rd party

curl-7.26.0 – URL command line tool

http://curl.haxx.se/

# ./curl --url smtps://smtp.gmail.com:465 --ssl-reqd --mail-

from xxxx@gmail.com --mail-rcpt yyyy@gmail.com --user

xxxx:xxxx_password –anyauth (To send mail)

ntpclient-2010 – network time protocol

http://doolittle.icarus.com/ntpclient/

# ./ntpclient –s –h tock.stdtime.gov.tw –i 3 (To get

network time info from tock.stdtime.gov.tw)

qrencode-3.3.1 – generate QR code image

http://fukuchi.org/works/qrencode/

# ./arm-none-linux-gnueabi-qrencode -s 4 -m 1 -o

/tmp/qr.png http://192.168.100.1 (To create QR code

image contains string “http://192.168.100.1”)

80

To Add and Build New Utility Below steps are based on Nuvoton released SkyEye firmware source

Create new utility folder (ex. utility_new) under utility/ and add your own files

Copy one Makefile.am from exist utility folder and replace utility name and source file name with new ones in Makefile.am

ex. If you copied Makefile.am from puncher utility, replace all patterns “puncher” with “utility_new”

Add new utility name to SUBDIRS column in utility/Makefile.am

Add new utility makefile path to AC_CONFIG_FILES column in configure.ac

Rebuild project

81

To Add and Build New 3rd Lib/Utility Build 3rd library or utility from its source according to its guideline before build SkyEye project

Supposed SkyEye project has been build once and folder 3rdlib/ is already created

Comment below two lines in build.sh # rm -rf $PRJ_PATH/3rdlib

# mkdir $PRJ_PATH/3rdlib

For library, please copy necessary header files to 3rdlib/include/ and new library so files to 3rdlib/lib/

For utility, please copy new utility binary under 3rdlib/bin/

Rebuild project

82

Nuvoton Media Library

Support to encode/decode audio/video data and

record/play file with multiple format

Support audio codec

MP3 / uLaw / aLaw / AAC / ADPCM

Support video codec

JPEG / H264

Support file format

AVI / MP4

Private library

83

Reference

Puncher

MTD Tool

84

Puncher

Source path is utility/puncher

Use miniupnpc library to do UPnP

Use DDNS server (ex. No-IP) predefined web

APIs - http://www.noip.com/integrate/request

Puncher

miniupnpc libconfig libb64

DDNS

85

MTD Tool Source is SkyEye_HQDVB_2.6.35.4_mtdtool_src.tar.gz

To update and backup firmware of 4 partitions on SPI flash

Use CRC32 algorithm to calculate checksum

86

MTD Tool Command

Usage example:

To verify image correcting. (0: success, other: fail)

mtdtool chksum <image path> <-b> <-b>: If the image have HTTP boundary part, to indicate the parameter to skip the part.

To backup image. (0: success, other: fail)

mtdtool backup <image path> –m <1st partition size>

–m <2nd partition size>

–m <3rd partition size>

–m <4th partition size>

To program image. (0: success, other: fail)

mtdtool flash <image path> <-b>

87

MTD Tool Command

Usage example:

To verify image correcting. (0: success, other: fail)

mtdtool chksum <image path> <-b>

<-b>: If image do not have HTTP boundary, specify -b to skip parsing HTTP boundary.

To backup image. (0: success, other: fail)

mtdtool backup <image path> –m <1st partition size>

–m <2nd partition size>

–m <3rd partition size>

–m <4th partition size>

To program image. (0: success, other: fail)

mtdtool flash <image path> <-b>

88

MTD Image Header

Header information

Magic number(m_magic): 0x19820621

Length(m_len): Image byte size

Checksum(m_crc): Checksum of image.

Partition number(m_parts): Partition number in image.

Partition information(m_imginfos[4])

Structure of partition information

m_magic m_len m_crc m_parts m_imginfos[4]

m_valid m_imagesize m_offset