Date post: | 19-Dec-2015 |
Category: |
Documents |
View: | 231 times |
Download: | 2 times |
Automatic Facets:Faceted Navigation and Entity Extraction
Tom ReamyChief Knowledge Architect
KAPS Group
Knowledge Architecture Professional Services
http://www.kapsgroup.com
2
Agenda
Introduction: Elements– Facets, Taxonomies, Software, People
3 Environments– E-Commerce, Enterprise, Internet
Design Issues – Facets and Entities
Conclusion – Integrated Solution
3
KAPS Group: General
Knowledge Architecture Professional Services Virtual Company: Network of consultants – 12-15 Partners – Inxight, FAST, etc. Consulting, Strategy, Knowledge architecture audit Taxonomies: Enterprise, Marketing, Insurance, etc. Services:
– Taxonomy development, consulting, customization– Technology Consulting – Search, CMS, Portals, etc.– Metadata standards and implementation– Knowledge Management: Collaboration, Expertise, e-learning– Applied Theory – Faceted taxonomies, complexity theory, natural
categories
4
Elements
Facet – orthogonal dimension of metadata Entity / Noun Phrase – metadata value of a facet Entity extraction – feeds facets, signature, ontologies Taxonomy and categorization rules Auto-categorization – aboutness, subject facets People – tagging, evaluating tags, fine tune rules and
taxonomy
5
Essentials of Facets
Facets are not categories– Categories are what a document is about – limited number– Entities are contained within a document – any number
Facets are orthogonal – mutually exclusive – dimensions– An event is not a person is not a document is not a place.
Facets – variety – of units, of structure– Numerical range (price), Location – big to small– Alphabetical, Hierarchical – taxonomic
Facets are designed to be used in combination• Wine where color = red, price = excessive, location = Calirfornia,• And sentiment = snotty
6
Advantages of Faceted Navigation
More intuitive – easy to guess what is behind each door• Simplicity of internal organization• 20 questions – we know and use
Dynamic selection of categories• Allow multiple perspectives• Ability to Handle Compound Subjects
Systematic Advantages – fewer elements– 4 facets of 10 nodes = 10,000 node taxonomy– Ability to Handle Compound Subjects
Flexible – can be combined with other navigation elements
7
Essentials of TaxonomiesInternal Organization Formal Taxonomy – parent – child relationship
– Is-A-Kind-Of ---- Animal – Mammal – Zebra – Partonomy – Is-A-Part-Of ---- US-California-Oakland
Browse Classification – cluster of related concepts– Food and Dining – Catering – Restaurants
Taxonomies deal with complex, not compound– Conceptual relationships – category membership– Contextual relationships – Computers & Software
Taxonomies deal with semantics & documents– Multiple meanings and purposes– Essential attributes of documents are not single value
8
Developing Facets: Tools and TechniquesSoftware Tools
Text Analytics – Taxonomy management, entity extraction, categorization, sentiment
Search – Integrated features, at index, Internet sources CM – Enterprise environment, taggers and policy Programmable Rules
– Business and Subject matter expertise– Auto-populate variety of metadata – author, title, date, etc.– Relevance – best bets to weights and classes of documents
People – refine, monitor – it’s not automatic
9
Developing Facets: Tools and TechniquesSoftware Tools – Auto-categorization Auto-categorization
– Training sets – Bayesian, Vector Machine– Terms – literal strings, stemming, dictionary of related terms– Rules – simple – position in text (Title, body, url)– Advanced – saved search queries (full search syntax)– NEAR, SENTENCE, PARAGRAPH– Boolean – X NEAR Y and Not-Z
Advanced Features– Facts / ontologies /Semantic Web – RDF +– Sentiment Analysis – positive, negative, neutral
10
Developing Facets: Tools and TechniquesSoftware Tools – Entity Extraction Dictionaries – variety of entities, coverage, specialty
– Cost of update – service or in-house– Inxight – 50+ predefined entity types– Nstein – 800,000 people, 700,000 locations, 400,000 organizations
Rules– Capitalization, text – Mr., Inc.– Advanced – proximity and frequency of actions, associations– Need people to continually refine the rules
Entities and Categorization– Total number and pattern of entities = a type of aboutness of
the document – Bar Code, Fingerprint
11
Elements: People
Programmers, Librarians, Taxonomists, Metadata specialist– Integrate, design, develop rules, monitor activity & quality
Authors, Subject Matter Experts– Input into design (important facets), rules, activity meaning
Users – Web 2.0– Feedback – quality and usability– Suggestions – missing terms, bad categorization & entity– Tags Clouds & folksonomy – for social networking features,
not for information retrieval
12
Three Environments
E-Commerce– Catalogs, small uniform collections of entities– Uniform behavior – buy this
Enterprise– More content, more types of content– Enterprise Tools – Search, ECM– Publishing Process – tagging, metadata standards
Internet– Wildly different amount and type of content, no taggers– General Purpose – Flickr, Yahoo– Vertical Portal – selected content, no taggers
15
Enterprise Environment – When and how add metadata
Enterprise Content – different world than eCommerce– More Content, more kinds, more unstructured– Not a catalog to start – less metadata and structured content – Complexity -- not just content but variety of users and activities
Combination of human and automatic metadata – ECM– Software aided - suggestions, entities, ontologies
Enterprise – Question of Balance / strategy– More facets = more findability (up to a point)– Fewer facets = lower cost to tag documents
Issues– Not enough facets– Wrong set of facets – business not information– Ill-defined facets – too complex internal structure
16
Facets and Taxonomies Enterprise Environment – Case One – Taxonomy, 7 facets
Taxonomy of Subjects / Disciplines:– Science > Marine Science > Marine microbiology > Marine toxins
Facets:– Organization > Division > Group– Clients > Federal > EPA– Instruments > Environmental Testing > Ocean Analysis > Vehicle– Facilities > Division > Location > Building X– Methods > Social > Population Study– Materials > Compounds > Chemicals– Content Type – Knowledge Asset > Proposals
17
External Environment – Text Mining, Vertical Portals
Internet Content – Scale – impacts design and technology – speed of indexing– Limited control – Association of publishers to selection of content to none– Major subtypes – different rules – metadata and results
Complex queries and alerts– Terrorism taxonomy + geography + people + organizations
Text Mining – General or specific content and facets and categories– Dedicated tools or component of Portal – internal or external
Vertical Portal – Relatively homogenous content and users– General range of questions
18
Internet Design
Subject Matter taxonomy – Business Topics– Finance > Currency > Exchange Rates
Facets – Location > Western World > United States– People – Alphabetical and/or Topical - Organization– Organization > Corporation > Car Manufacturing > Ford– Date – Absolute or range (1-1-01 to 1-1-08, last 30 days)– Publisher – Alphabetical and/or Topical – Organization– Content Type – list – newspapers, financial reports, etc.
22
Integrated Facet ApplicationDesign Issues - General
What is the right combination of elements?– Faceted navigation, metadata, browse, search, categorized
search results, file plan
What is the right balance of elements?– Dominant dimension or equal facets– Browse topics and filter by facet
When to combine search, topics, and facets?– Search first and then filter by topics / facet– Browse/facet front end with a search box
23
Integrated Facet ApplicationDesign Issues - General Homogeneity of Audience and Content Model of the Domain – broad
– How many facets do you need?– More facets and let users decide– Allow for customization – can’t define a single set
User Analysis – tasks, labeling, communities• Issue – labels that people use to describe their
business and label that they use to find information Match the structure to domain and task
– Users can understand different structures
24
Automatic Facets – Special Issues
Scale requires more automated solutions– More sophisticated rules
Rules to find and populate existing metadata– Variety of types of existing metadata – Publisher, title, date– Multiple implementation Standards – Last Name, First / First Name, Last
Issue of disambiguation:– Same person, different name – Henry Ford, Mr. Ford, Henry X. Ford– Same word, different entity – Ford and Ford
Number of entities and thresholds per results set / document– Usability, audience needs
Relevance Ranking – number of entities, rank of facets
25
Putting it all together – Infrastructure Solution
Facets, Taxonomies, Software, People Combine formal power with ability to support multiple
user perspectives Facet System – interdependent, map of domain Entity extraction – feeds facets, signatures, ontologies Taxonomy & Auto-categorization – aboutness, subject People – tagging, evaluating tags, fine tune rules and
taxonomy The future is the combination of simple facets with rich
taxonomies with complex semantics / ontologies
Questions?
KAPS Group
Knowledge Architecture Professional Services
http://www.kapsgroup.com