Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | wilfrid-owens |
View: | 221 times |
Download: | 0 times |
Multimedia ResearchMultimedia Researchinin
E-EducationE-Education
Veljko Milutinović, Fellow of the IEEE
An Overview of the Ongoing Projects
http://galeb.etf.bg.ac.yu/~vm
e-mail: [email protected]
Page 2 of 79
Major R&D BottlenecksMajor R&D Bottlenecks
• Integrated Educational SystemsIntegrated Educational Systems
• Concept Understanding and Interactive CooperationConcept Understanding and Interactive Cooperation
• Intelligent Search and Retrieval for Educational PurposesIntelligent Search and Retrieval for Educational Purposes
• Innovative ApplicationsInnovative Applications
Page 3 of 79
U. of California at BerkeleyU. of California at Berkeley
• Integrated Systems: Intel MES GrantIntegrated Systems: Intel MES Grant• Concept Understanding: WISEConcept Understanding: WISE• Intelligent Search: GISTIntelligent Search: GIST• Innovative Applications: BMRCInnovative Applications: BMRC
Page 4 of 79
MITMIT
• Integrated Systems: Microsoft I-CampusIntegrated Systems: Microsoft I-Campus
• Concept Understanding: OXYGENConcept Understanding: OXYGEN
• Intelligent Search: NCAMIntelligent Search: NCAM
• Innovative Applications: NRENInnovative Applications: NREN
Page 5 of 79
Stanford UniversityStanford University
• Integrated Systems: Challenge 2000 MultimediaIntegrated Systems: Challenge 2000 Multimedia
• Concept Understanding: Media XConcept Understanding: Media X
• Intelligent Search: CBIRIntelligent Search: CBIR
• Innovative Applications: SUMMIT, CERAS, MM Collab, ShakespeareInnovative Applications: SUMMIT, CERAS, MM Collab, Shakespeare
Page 6 of 79
Project Name (University)Project Name (University)
• Project Leader(s)Project Leader(s)• Project URLProject URL• Project Essence in ASCIIProject Essence in ASCII• Project Essence in JPEG/MPEGProject Essence in JPEG/MPEG
Page 7 of 79
Intel MES Grant (Berkeley)Intel MES Grant (Berkeley)
• James Demmel
• http://www.intel.com/pressroom/archive/releases/CO081897.HTMAA
• RISC approach to MESRISC approach to MES
Page 8 of 79
WISE (Berkeley)WISE (Berkeley)
• Eric BaumgartnerEric Baumgartner
• http://wise.berkeley.edu/pages/intro/wiseIntro01.html
• Risk approach to K12Risk approach to K12
Page 9 of 79
GIST (Berkeley)GIST (Berkeley)
• Joe HellersteinJoe Hellerstein
• http://now.cs.berkeley.edu/AMhttp://now.cs.berkeley.edu/AM
• Risk approach to generalized search tree for secondary and Risk approach to generalized search tree for secondary and multimedia storage; supports any lookup over that data multimedia storage; supports any lookup over that data
Page 10 of
79
BMRC (Berkeley)BMRC (Berkeley)
• Larry RoweLarry Rowe
• http://www-plateau.cs.berkeley.edu/http://www-plateau.cs.berkeley.edu/
• Risk approach to contents and technology managementRisk approach to contents and technology management
Page 11 of
79
Microsoft I-Campus (MIT)Microsoft I-Campus (MIT)
• Thomas L. Magnanti
• http://web.mit.edu/newsoffice/tt/1999/oct06/microsoft.htmlhttp://web.mit.edu/newsoffice/tt/1999/oct06/microsoft.html
• Risk approach to $25 millionRisk approach to $25 million
Page 12 of
79
OXYGEN (MIT)OXYGEN (MIT)
• Anant Agarval, John Ancorn, Krste Asanovic, Rodney Brooks, …Anant Agarval, John Ancorn, Krste Asanovic, Rodney Brooks, …• http://oxygen.lcs.mit.edu/http://oxygen.lcs.mit.edu/• Risk approach to bringing abundant computation, multimedia, Risk approach to bringing abundant computation, multimedia,
and communication naturally into people's lives, through an and communication naturally into people's lives, through an infrastructure of mobile and stationary devices connected by a infrastructure of mobile and stationary devices connected by a self-configuring networkself-configuring network
Page 13 of
79
NCAM (MIT)NCAM (MIT)
• Geoff FreedGeoff Freed
• http://www.ncddr.org/du/researchexchange/v06n03/multi.htmlhttp://www.ncddr.org/du/researchexchange/v06n03/multi.html
• Risk approach to accessible multimedia and distant educationRisk approach to accessible multimedia and distant education
Page 14 of
79
NREN (MIT)NREN (MIT)
• MIT, ARPA, DOE, NASA, NSFMIT, ARPA, DOE, NASA, NSF
• http://www.ccic.gov/pubs/blue94/section.3.2.htmlhttp://www.ccic.gov/pubs/blue94/section.3.2.html
• Risk approach to gigabit communications infrastructure for Risk approach to gigabit communications infrastructure for e-research and e-educatione-research and e-education
Page 15 of
79
Challenge 2000 MM (Stanford)Challenge 2000 MM (Stanford)
• Shari Golan, Barbara Means, Bill PenuelShari Golan, Barbara Means, Bill Penuel
• http://ctl.sri.com/projects/displayProject.jsp?Nick=ch2000mmhttp://ctl.sri.com/projects/displayProject.jsp?Nick=ch2000mm
• Risk approach to the next decade challenges in MESRisk approach to the next decade challenges in MES
Page 16 of
79
Media X (Stanford)Media X (Stanford)
• John PerryJohn Perry
• http://mediax.stanford.edu/about/education.htmlhttp://mediax.stanford.edu/about/education.html
• Risk approach to multimedia courses in interdisciplinary major Risk approach to multimedia courses in interdisciplinary major undergraduate and graduate programsundergraduate and graduate programs
Page 17 of
79
CBIR (Stanford)CBIR (Stanford)
• Jia Li, James Z. Vang, Gio WiederholdJia Li, James Z. Vang, Gio Wiederhold
• http://www-db.stanford.edu/IMAGE/http://www-db.stanford.edu/IMAGE/
• Risk approach to contents based image retrieval: semantics-sensitive integrated matching for picture libraries, wavelet-based image indexing and searching
Page 18 of
79
SUMMIT (Stanford)SUMMIT (Stanford)
• Parvati DevParvati Dev
• http://www-smi.stanford.edu/projects/summit.htmlhttp://www-smi.stanford.edu/projects/summit.html
• Risk approach to MES in medicineRisk approach to MES in medicine
Potentials of R&D in MESPotentials of R&D in MESthroughthrough
Italy/Serbia Cooperation Italy/Serbia Cooperation
An Overview of Belgrade University Projects
for High-Tech Computer Industry
in the USA and EU (IPSI)
Page 20 of
79
Some of the Recent ProjectsSome of the Recent Projects
NCR, Compaq, SUN, Intel,
Comshare, Zycad, QSI, Virtual,
TechnologyConnect, BioPop, eT, MainStreetNetworks,
Salerno, Pisa, Siena, L’Aquila, ...
Page 22 of
79
Response:Response: Academia Academia
Flynn, M. J., Computer Architecture, Jones and Bartlett, USA (96)position 1 (12 citations)
Bartee, T. C., Computer Architecture and Logic Design, McGraw-Hill, USA (91)position 1 (2 citations)
Tabak, D., RISC Systems (RISC Processor Architecture), Wiley, USA (91)position 1s (6 citations)
Stallings, W., Reduced Instruction Set Computers (RISC Architecture), IEEE CS Press, Los Alamitos, California, USA (90)position 1s (3 citations)
Heudin, J. C., Panetto, C., RISC Architectures, Chapman-Hall, London, England (92)position 3s (2 citations)
van de Goor, A. J., Computer Architecture and Design, Addison Wesley, Reading, Massachusetts, USA (2nd printing, 91)position 4s (3 citations)
Tannenbaum, A., Structured Computer Organization (Advanced Computer Architecures), Prentice-Hall, USA (90)position 5s (4 citations)
Feldman, J. M., Retter, C. T., Computer Architecture, McGraw-Hill, USA (94)position 7s (2 citations)
Stallings, W., Computer Organization and Architecture, Prentice-Hall, USA (96)position 9s (3 citations)
Murray, W., Computer and Digital System Architecture, Prentice-Hall, USA (90)position >10s (2 citations)
Wilkinson, B., Computer Architecture, Prentice-Hall, USA (91)position >10 (2 citations)
Decegama, A., The Technology of Parallel Processing (Parallel Processing Architectures), Prentice-Hall, USA (90)position >10s (2 citations)
Baron, R. J., Higbie, L., Computer Architecture, Addison-Wesley, USA (92)position >10s (1 citation)
Tabak, D., Advanced Microprocessors (Microcomputer Architecture), McGraw-Hill, USA (95)position >10s (1 citation)
Zargham, M. R., Computer Architecture, Prentice-Hall, USA (96)position >10s (1 citation)
Hennessy, J. L., Patterson, D. A., Computer Architecture: A Quantitative Approach, Morgan-Kaufmann, USA (96)na (0 citations)
Hwang, K., Advanced Computer Architecture, McGraw-Hill, USA (93)na (0 citations)
Kain, K., Computer Architecture, Addison-Wesley, USA (95)na (0 citations)
Page 27 of
79
SummarySummary
The world’s best journals – IEEE (50):The world’s best journals – IEEE (50): A European record in ICTA European record in ICT
Books with 7 Nobel Laureates:Books with 7 Nobel Laureates:
Kenneth Wilson, Ohio (North-Holland)Kenneth Wilson, Ohio (North-Holland) Leon Cooper, Brown (Prentice-Hall)Leon Cooper, Brown (Prentice-Hall) Robert Richardson, Cornell (Kluwer-Academics)Robert Richardson, Cornell (Kluwer-Academics) Jerome Friedman, MIT (Kluwer-Academics)Jerome Friedman, MIT (Kluwer-Academics) Herb Simon, CMU (IOS)Herb Simon, CMU (IOS) Arno Penzias, AT&T (IOS)Arno Penzias, AT&T (IOS) Harold Kroto, England (IOS)Harold Kroto, England (IOS)
Page 28 of
79
Multimedia Internet GalleryMultimedia Internet Gallery
Funding:Funding: Fraunhofer IPSI, Darmstadt, GermanyFraunhofer IPSI, Darmstadt, Germany
Implementation:Implementation: IPSI, Belgrade, SerbiaIPSI, Belgrade, Serbia
Project Termination Date:Project Termination Date: August 31, 2003.August 31, 2003.
Users:Users: Germany, Serbia, USA, Canada,…Germany, Serbia, USA, Canada,…
Demo:Demo: www.ipsi.co.yuwww.ipsi.co.yu
Page 29 of
79
IPSI Belgrade, Ltd.IPSI Belgrade, Ltd.
3D@3D MULTIMEDIA 3D@3D MULTIMEDIA ARTSHOP GALLERYARTSHOP GALLERY
[email protected], [email protected]://galeb.etf.bg.ac.yu/vm, http://www.ipsi.co.yu
Page 30 of
79
AuthorsAuthors
Milutinovic Veljko
Toskov IvanVujovic Ivana
Skundric Nikola Milutinovic Darko
Stojanovski Aleksandar Nikezic Gavro
Anucojic Goran
Radakovic MiroslavMarinkovic Ivan
Page 31 of
79
Introduction – IPSI BelgradeIntroduction – IPSI Belgrade
IPSI BelgradeIPSI Belgrade is a company jointly founded by German and Serbian capital
Partners:
• IPSI Fraunhofer, Darmstadt, Germany
• Telecom Italia Learning Services, Italy
• NYU, School of Continuous Professional Studies, USA
• Purdue University, School of Technology, USA
Page 32 of
79
Introduction – IPSI BelgradeIntroduction – IPSI Belgrade
- Multimedia Workspaces of the Future- Multimedia Applications for the Web- Environments for Cooperative Working and Learning- Virtual Information and Knowledge Environments- Mobile Interactive Media- Open Adaptive Information Management Systems- Publication Engineering and Technology- Hardware Design and Operating Systems- Networks and WWW- Semantic Web and Datamining
Page 33 of
79
Introduction – IPSI BelgradeIntroduction – IPSI Belgrade
Products:
• Advanced Multimedia Virtual Gallery• Tools for B2B Matchmaking on the Web• Web security for P2P, The injection cache, The STS cache, Genetic Search with Spatial/Temporal Mutations, Customer Browser Satisfaction Web Search, Browser Acceleration, Technology Transfer, Testing Infrastructure for EBI, Distant Web Educating Machine, e-Tourism, …
Page 34 of
79
MMAG: Problem StatementMMAG: Problem Statement
- Creating Web based art gallery with “look and feel” of the real world exhibitions
- Visitor moves through the gallery by “walking with options”
- 2D on 3D and 3D on 3D, with multimedia options
Page 35 of
79
Existing SolutionsExisting Solutions
- Musee national des Arts asiatiques http://www.museeguimet.fr/tour-guimet/index.html
- Web Server of the Galleria degli Uffizi in Florence http://www.uffizi.firenze.it
- The Distributed Interactive Virtual Environment (DIVE) http://www.sics.se/dive/
- The Web3D Repository http://www.web3d.org/vrml/artgal.htm
Page 36 of
79
Proposed SolutionProposed Solution
- Virtual reality gallery: Multimedia In Action
- Advanced search capabilities: Dream Search
- Artist’s criteria room generation: Do It By Yourself
Page 37 of
79
Why is it better?Why is it better?
- Dynamically generated/exploitable gallery- Dynamically generated/exploitable gallery
- Content based search engine- Content based search engine
- User satisfaction - User satisfaction
Page 38 of
79
Conditions and AssumptionsConditions and Assumptions
- PC- Internet connection- Internet Explorer 5.0 or higher - Netscape 7.0 - Cortona VRML plug-in for IE- Basic multimedia tools and standards
Page 39 of
79
Analysis and ImplementationAnalysis and Implementation
- Application is written in ASP.NET using C# as code-behind, and ADO.NET for database access.- Database server is SQL Server 2000.- Communication with the database is entirely made through XML (using SQLXML3.0 framework).
- Queries are made in XPath, while adding, changing and deleting of the records is done through UpdateGrams.
- Application is optimized for Internet Explorer 5.0 or higher, at the 1024x768 screen resolution. Netscape 7.0 or higher is also supported.
- 3D gallery is completely generated on the server side (dynamically) using VRML.
Page 40 of
79
Track 1Track 1
Track Requirements:Track Requirements:• To stand as the integrative part for the other two tracksTo stand as the integrative part for the other two tracks• To provide:To provide:
– User interactionUser interaction– Database connectivity (database independent)Database connectivity (database independent)– Search functions Search functions
(simple and advanced using Track3 output)(simple and advanced using Track3 output)– Information brokering between artists and buyersInformation brokering between artists and buyers– Administration toolsAdministration tools– Artworks management tools, etc.Artworks management tools, etc.– Thin client (3D scene generation on server side)Thin client (3D scene generation on server side)
Page 41 of
79
Track 1Track 1
Development Tools:Development Tools:• Application server platform:Application server platform:
– Windows XP ProfessionalWindows XP Professional– IIS 5.1IIS 5.1– MS SQL Server 2000MS SQL Server 2000
• Development platform:Development platform:– ASP.NETASP.NET– C# as code-behind.C# as code-behind.
• Communication with the underlying database:Communication with the underlying database:– XML & XSD using XPath queries (DB independent)XML & XSD using XPath queries (DB independent)– Currently using SQLXML3.0 add-on for ADO.NETCurrently using SQLXML3.0 add-on for ADO.NET
Page 43 of
79
Track 1Track 1
Administrator Tools:Administrator Tools:
• Separate entry point:Separate entry point:
http://<server_address>/artshop/adminhttp://<server_address>/artshop/admin
Page 44 of
79
Track 1Track 1
Users & Exhibitors:Users & Exhibitors:• Entry point:Entry point:
http://<server_address>/artshop/index.htmhttp://<server_address>/artshop/index.htm
Page 45 of
79
Track 1Track 1
Interesting Details:Interesting Details:
• Native XML DBMS under development at IPSI Fraunhofer• Practical testing of the XML/XPath database access• Dynamic addition (to the system) of new multimedia types• 3D view of search results
Page 46 of
79
Track 1Track 1
Interesting Details:Interesting Details:• Application that can connect on the fly to any DBMS which supports XML/XPath is an interesting Application that can connect on the fly to any DBMS which supports XML/XPath is an interesting
and possibly useful idea and possibly useful idea (user just has to set one XML file containing local field mapping, and one XSD to map the (user just has to set one XML file containing local field mapping, and one XSD to map the database fields to the pre-defined scheme)database fields to the pre-defined scheme)
• Cons:Cons:– XPath queries are lot less powerful then standard SQL queriesXPath queries are lot less powerful then standard SQL queries– Inherently, loss of speed Inherently, loss of speed
(one complex SQL query had to be simulated with couple of XPath queries and additional (one complex SQL query had to be simulated with couple of XPath queries and additional processing in the code).processing in the code).
– For now, SQLXML3.0 does not support complete For now, SQLXML3.0 does not support complete XPath standardXPath standard..
Page 47 of
79
Track 1Track 1
Errors Made:Errors Made:
• Initially, content analysis, picture processing, and adding data to database were completely Initially, content analysis, picture processing, and adding data to database were completely separated (as specified in the contract), separated (as specified in the contract), with the idea of later (partial) integration.with the idea of later (partial) integration.
• Turned out to be a bad ideaTurned out to be a bad idea(required a lot of intervention(required a lot of interventionfrom the ArtShop system administrator when adding artworks).from the ArtShop system administrator when adding artworks).
Page 48 of
79
Track 1Track 1
Lessons Learned:Lessons Learned:• Problem solved by complete integration of forementioned tasks Problem solved by complete integration of forementioned tasks
into the one system process which monitors input directory, automatically schedules picture into the one system process which monitors input directory, automatically schedules picture processing and content analysis, and takes care of updating of all necessary fields in all processing and content analysis, and takes care of updating of all necessary fields in all required databases.required databases.
• With that, we achieved maximum automation, With that, we achieved maximum automation, reduced time needed for artwork addition, reduced time needed for artwork addition, and reduced amount of data transferred through the Internet and reduced amount of data transferred through the Internet (between the administrator’s machine and the application host).(between the administrator’s machine and the application host).
Page 49 of
79
Track 2Track 2
Image-Content-Oriented SearchImage-Content-Oriented SearchTrack requirements:Track requirements:
• Images used for extracting objects are artistic paintings• Image analyses• Extraction of the features• Create XML file for each image• Fetch the database with the features
Page 50 of
79
Track 2Track 2
Algorithm:Algorithm:Open the picture
Load parameters, input and output directory
Determine the filter value
Put the picture into the reduced matrix
Determine histogram
Create objects
Merge objects into bigger objects
Create sorted array of objects
Create database objects, prepare them, and put them in XML file and tables in database
Page 51 of
79
Track 2Track 2
Determine histogram:Determine histogram: Create general histogram
Remove zero values
Sort histogram
Remove redundancies
Sort histogram
Refresh matrix
Page 52 of
79
Track 2Track 2
Regions forRegions for processing matrix: processing matrix:
P8
P4P5
P5P6P3
P3
P4P5
P3
P8
P8P8
P8
the first colon the last row
the last colon
the rest of the matrix
the last element in the last row
Page 53 of
79
Creating Creating objects:objects:
Any pixel left in the current region?
p8 = the current pixel
call matrix.unite_pixels method
p8.oi == 0 (it doesn’t belong to any object)
create new object
Any neighbour pixel left?
p8.oi != px.oi (don’t belong to the same object)
take the next pixel
take the next neighbour pixel
px = the neighbour pixel
true
true
true
false
true
false
false
false
Page 54 of
79
Merge Merge objects objects into into bigger bigger objects:objects:
Is Picture_Objects list empty?
q==true (are there any objects inside Border which colors after translation are the same as the color of Core object)
q = true; Edge[0] = Picture_Objects[0]
i < number of objects inside Edge list
Find all neighbour objects, which colors after translation are the same as the color of Core object, and put them into Border
Move all pixels from objects inside Edge to Core object
Remove objects that are inside Edge from Picture_Objects list
Is Border empty
q=false; add Core object to big list; Remove pixels belonging to Core object from Picture_Objects
false
true
true
Take ith object
i = i+1
Move all objects inside Border to Edge
false
true
false
true
false
Page 55 of
79
Track 2Track 2
Tools used in development:Tools used in development:• C# programming language – the chosen tool
– Advantage: Includes the best properties from other programming languages (C++, Java, Visual Basic)
– Disadvantage: slower processing speed than C++, which is not necessary in this application
• C++ - the best alternative tool
– Advantage: faster processing speed (unnecessary)
– Disadvantage: more complicated code, 50% of all bugs due to use of pointers
Page 57 of
79
Track 2Track 2
Picture after applyingPicture after applyinghistogram values:histogram values:
Page 58 of
79
Track 2Track 2
Picture representedPicture representedthrough extracted objects:through extracted objects:
Page 59 of
79
Track 2Track 23D HSL space => 1D histogram3D HSL space => 1D histogram
Index of histogram array Hue Saturation Luminance Description
0 any any <=30 black
1 any any >=lummax white
2 any <20 >30<30+lum
the darkest grey
... … … … …
ilmax-1 any <20 >=30+(ilmax-3)*lum<30+(ilmax-2)*lum
the lightest grey
ilmax <=hue/2>=240-hue/2
>20<20+1*sat
>30<30+lum
the darkest red with the smallest saturation
ilmax+1 <=hue/2>=240-hue/2
>=20+1*sat< 20+2*sat
>30<30+lum
the darkest red with smaller saturation
… … … … …
ilmax+isat-2 <=hue/2>=240-hue/2
>=20+(isat-3)*sat< 20 +(isat-2)*sat
>30<30+lum
the darkest red with the biggest saturation
ilmax+1*(isat-1) <=hue/2>=240-hue/2
>20<20+1*sat
>=30+lum< 30+2*lum
darker red with the smallest saturation
ilmax+1*(isat-1)+1 <=hue/2>=240-hue/2
>=20+1*sat< 20+2*sat
>=30+lum< 30+2*lum
darker red with smaller saturation
… … … … …
ilmax+1*(isat-1)+isat-2 <=hue/2>=240-hue/2
>=20+(isat-3)*sat< 20 +(isat-2)*sat
>=30+lum< 30+2*lum
darker red with the biggest saturation
ilmax+2*(isat-1) <=hue/2>=240-hue/2
>20<20+1*sat
>=30+2*lum< 30+3*lum
dark red with the smallest saturation
Page 60 of
79
Track 2Track 23D HSL space => 1D histogram3D HSL space => 1D histogram
ilmax+2*(isat-1)+1 <=hue/2>=240-hue/2
>=20+1*sat< 20+2*sat
>=30+2*lum< 30+3*lum
dark red with smaller saturation
… … … … …
ilmax+(ilmax-3)*(isat-1)+1 <=hue/2>=240-hue/2
>=20+1*sat< 20+2*sat
>=30+(ilmax-3)*lum< 30+(ilmax-2)*lum
the brightest red with smaller saturation
… … … … …
ilmax+(ilmax-3)*(isat-1)+(isat-2) <=hue/2>=240-hue/2
>=20+(isat-3)*sat< 20 +(isat-2)*sat
>=30+(ilmax-3)*lum< 30+(ilmax-2)*lum
the brightest red with the biggest saturation
ilmax+1*(ilmax-2)*(isat-1) > hue/2< 3*hue/2
>20<20+1*sat
>30<30+lum
the darkest orange-red with the smallest saturation
ilmax+1*(ilmax-2)*(isat-1)+1 > hue/2< 3*hue/2
>=20+1*sat< 20+2*sat
>30<30+lum
the darkest orange-red with smaller saturation
… … … … …
ilmax+1*(ilmax-2)*(isat-1)+(ilmax-3)*(isat-1)+(isat-2)
> hue/2< 3*hue/2
>=20+(isat-3)*sat< 20 +(isat-2)*sat
>=30+(ilmax-3)*lum< 30+(ilmax-2)*lum
the brightest orange-red with the biggest saturation
ilmax+2*(ilmax-2)*(isat-1) >=3*hue/2< 5*hue/2
>20<20+1*sat
>30<30+lum
the darkest red-orange with the smallest saturation
ilmax+2*(ilmax-2)*(isat-1)+1 >=3*hue/2< 5*hue/2
>=20+1*sat< 20+2*sat
>30<30+lum
the darkest red-orange with smaller saturation
… … … … …
ilmax+(ihmax-1)*(ilmax-2)*(isat-1)+1 >=(2*ihmax-1)*hue/2<(2*ihmax+1)*hue/2
>=20+1*sat< 20+2*sat
>30<30+lum
the darkest magenta-red with smaller saturation
… … … … …
ilmax+(ihmax-1)*(ilmax-2)*(isat-1)+(ilmax-3)*(isat-1)+(isat-2)
>=(2*ihmax-1)*hue/2<(2*ihmax+1)*hue/2
>=20+(isat-3)*sat< 20 +(isat-2)*sat
>=30+(ilmax-3)*lum< 30+(ilmax-2)*lum
the brightest magenta-red with the biggest saturation
Page 61 of
79
Track 2Track 2
Lessons learned:Lessons learned:
• It is impossible to extract objects using only colors as a criterion• It is impossible to extract objects,
even using textures, edges, different transformations as criteria• Semantics should be used in segmentation• Colors are the most important features in artistic paintings
Page 62 of
79
Track 3Track 3
Track requirements:Track requirements:• Possibility of moving through 3D galleries
• Automatic generation of 3D galleries based on user’s query
• Manual generation of 3D galleries
• User interface for image zooming
• Application for image processing
Page 63 of
79
Track 3Track 3
Underlying algorithms:Underlying algorithms:
• Dynamic creation of gallery
• Creation of static galleries
• Algorithm for picture zooming
• Algorithm for picture processing
Page 64 of
79
Track 3Track 3
Creation of galleries:Creation of galleries:
• Validation of the created gallery
• Forming VRML files depending on users query
• Determining the number of pictures in the gallery
• Drawing a 2D floorplan based on the 3D gallery
• “Forest fire” algorithm for filling the floorplan with color
Page 65 of
79
Track 3Track 3
Picture processing:Picture processing:
• Loading image into memory
• Clone image into different-size copies
• Filtering of copies
• Parting of copies
Page 66 of
79
Track 3Track 3
Development tools:Development tools:
• C# in .NET Framework for programming image processing
• Macromedia Dreamweaver for programming zoom tool
• VRML Pad v2.0
Page 68 of
79
Track 3Track 3
Creation of galleries:Creation of galleries:
• Making files based on users data
• Putting data on serverso it can be available for artists
• Artist chooses which gallery he/she will be using for exhibition
• User can move through 3D world
• Selecting the textures for gallery
• Selecting the starting position of the user
Page 69 of
79
Track 3Track 3
NF filter details:NF filter details:
• If the new picture is smaller, every pixel is one pixel of the old picture.
• If the new picture is bigger, pixels are calculated based on the pixels surrounding the current.
Page 70 of
79
Track 3Track 3
Errors made:Errors made:
• Requests were not precise, so there was a gap at the end of the project between wanted and done
• Better results could be done with better using of ASP and XML
Page 71 of
79
Track 3Track 3
Lessons learned:Lessons learned:
• Every member of the team gets a part where his experience is dominating
• More planning at the start reduces a lot of work later
• Good communication between programmers can save a lot of time
Page 73 of
79
Future PlansFuture Plans
• Improving the existing 3D dynamic gallery
• Improving search engine capabilities
• Improving feature extraction algorithms and objects recognition
Page 74 of
79
Future Track 1Future Track 1
3D Multimedia Showroom Environment:3D Multimedia Showroom Environment:
• Implementing a generalized Web based 3D Multimedia Showroom Environment
• Exhibiting various MM data types: images, 3D objects, videos, audio, etc.
• Set of MM data types should be extendable
Page 75 of
79
Future Track 2Future Track 2
MM Object Feature Extraction:MM Object Feature Extraction:
• Implementing algorithms and software components for extracting features from MM data types (images, videos, 3D objects), in order to enable content based search
• System should be extendable (“plug-in”)
Page 76 of
79
Future Track 3Future Track 3
Semantic Abstraction of MM Feature Spaces:Semantic Abstraction of MM Feature Spaces:
• Developing methods and SW components which derive mapping from extracted features of MM objects to semantic concepts
• Using intelligent classification algorithms (Neural Networks, Fuzzy Classifier)
• Developing semantic query engine (answering questions, which could previously only be answered by humans)
Page 77 of
79
UsabilityUsability
• Art GalleriesArt Galleries
• MuseumsMuseums
• Exhibition FairsExhibition Fairs
Page 78 of
79
Instead of ConclusionInstead of Conclusion
IPSI Belgrade, [email protected]://www.ipsi.co.yu