Post on 21-Jan-2016
transcript
1
Multimedia Information and Mobile-Learning
Duc Phuong NguyenMartin GuggisbergHelmar Burkhart
University of BaselSwitzerland
2
Challenge 1: Content adaptation
Jobs done: 40Queues : 30Status: OKElapsed time: 1 hourRemain time: 2 hours
Short text can be sent via SMS
Client-side globus-hostname command is not returning a fully qualified domain name (FQN). The submission scripts use globus-hostname to send information to the server about how to contact the client. The fix is to make globus-hostname return the FQDN.There are firewalls and ports (GRAM: 2119, LDAP: 2135) which are not open. Check that GLOBUS_TCP_PORT_RANGE is set to 3000 to 3090 on the client (or another range acceptable for the server to contact the client on), and that all intermediate firewalls are open for this range in both directions. GRAM Job submission failed because the executable file permissions do not allow execution".Probable cause: You submitted a script, e.g., globus-job-run symphony:2119/jobmanager-loadleveler $PWD/simple.ll but simple.ll does not have the execute permission bit set.The error msg "Authentication Failed remote certificate signed by unknown CA" means the client cannot find the CA for the cert presented by the server.Make sure the CA cert is present in the trusted cert dir set with setup-gsi. Also, check the time is consistent across the machines.The error msg "GRAM Job submission failed because the connection to the server failed (check host and port) Make sure that the gatekeeper service is enabled.The error msg "GRAM Job submission failed because data transfer to the server failed" means the job manager has problems opening some file. Make sure $GLOBUS_LOCATION is readable by all the users (mode 755)
What about this text?
How can we sent this information to mobile users?
3
The second message started on 10.12.2006 at 17h …
…………………………………………………
Challenge 2: Content overview
Difficult to get the most important message first:„You have 10 voice messages in your mail box…The first message
started on 10.12.2006 at 14h ………………………………………………………………………………………………………
The ninth message started on 10.12.2006 at 17h …
…………………………………………………The last message is actually what important to me in this example“
4
Challenge 3: Integration
Suppose that we have a system that solves
Challenge 1 (content adaptation)Challenge 2 (content overview)
How can we integrate it into our E-Learning system?
5
Solution 1: Content adaptation
CoMobile
Speech1.wav
Speech2.wavLong texts
Audio
Voice call
• Quite difficult to convert mathematic formulae to speech
6
Solution 2: Content overview
CoMobile Keywords extraction(Post processing)
Audio
1. Lunch2. Food3. Project meeting4. Concert……………..
Speech1.wav
Speech2.wav
Voice call Recorded lectures
Text
List of keyword can be used to identify the important voice messages
SMS
7
CoMobile framework- Uni Basel
- Access learning materials - Search, notification- Collaboration (sharing, discussion) - Self-assessment, voting, quiz
8
CoMobile framework- Uni Basel
- Access learning materials - Search, notification- Collaboration (sharing, discussion) - Review, self-assessment, voting,
quiz
9
CoMobile-Example
CoMobile
Voice message
Alice
Voice message
Bob
Text message
Christian
Voice and Text
Pengi
10
CoMobile-Example
CoMobile
Voice messageVoice message
Voice and Text
Text message
12
4
3
11
CoMobile framework-Overview
1. Content adaptation: long texts are converted into speech and then can be accessed by mobile phones
2. Search on audio data
3. Platforms and languages independence
4. Flexible: easy to integrate via CoMobile-services
12
CoMobile framework-Components (I)
All components are open source software:- CoMobile main component
- Plug-in components: - Text to Speech: Festival- Speech recognition: Sphinx- Optical character recognition: GOCR
Plug-in components can be replaced without many modifications.
13
CoMobile framework-Components (II)
Service-oriented: works on document‘s level (e.g: input=audio files, output=text files)
CoMobile calls the component‘s public service
The component invokes CoMobile callback service when it finish the task
Communication between CoMobile and the plug-in components are done via HTTP
14
CoMobile framework- Easy to integrate
Service-oriented simplifies the intergration process:
PHP-based system with CoMobile Java-based system with CoMobile
15
CoMobile framework-Components (III)
Festival-Text to Speech: support many languages (English, German, French, Spanish, etc.)
Sphinx-Speech recognition: free acoustic model for English (64.000 words; WER = 18%)
Optical Character Recognition: GOCR, tesseract (supported by Google). Recognition rate is still quite low for handwritten notes (accuracy rate= appr. 20% with empirical tests).
16
Open issue: Recognition of notes taken by mobile phone camera
Handwritten notes and the low resolution of mobile phone‘s camera decrease the accuracy of OCR systems significantly
17
CoMobile example: Existing Web-based applications can use mobile phones to access and submit information by integrating with CoMobile services
http://informatik.unibas.ch/courses/multimedia/lecture1.mpg
Text retrieval, features extraction, vector space retrieval, inverted list, ranking
18
Uni Basel-CoMobile features
Content adaptation and content overview for mobile devices.
Data submission: send information (text, images, audio, video) via SMS, MMS, voice recording.
SMS notification service (replies, reminds, alerts).
Access information via various channels: Voice, SMS, MMS, WAP, Bluetooth.
19
Transformation issue
Information loss in transformation chain Recognition accuracy Quality of search: keywords, semantic
Text Audio
Graphic
20
Conclusion
Content transformation is an important issues in mobile applications.
Uni Basel-CoMobile: an easy to integrate and flexible framework that support collaborative learning with mobile devices.
The CoMobile framework is open source and look forward for contributions.
21
Uni Basel-CoMobile-Demo
CoMobile demonstration
22
Discussion
Cost: Users have to pay for SMS, MMS, as well as voice call. „Flat rate“ package will be common in the near future.
Storage: 1 minute video = 1 MB (approximately). There exists 2GB flash card for mobile phone
Bandwidth: 14.4 kbps (GSM), 140 kbps (GPRS), 1920 kbps (UMTS)
23
SMS gateway - Kannel
RS-232AT Commands
AT+CNMI=1,2,0,1,1SMS Text
HTTP request to forward SMS Text
24
MMS gateway - Mbuni
RS-232AT Commands
AT+CNMI=1,2,0,1,1MMS
Text, Picture, Audio
HTTP request to forward MMS
Mbuni
25
PBX Asterisk
PCI socketVoice
AGI: Perl, Java, PHP scripts are called
Telephone line