Post on 14-Dec-2015
transcript
1© disclosure or duplication without consent
prohibited
COMET K2 Competence Center - Initiated by the Federal Ministry of Transport, Innovation & Technology (BMVIT) and theFederal Ministry of Economics & Labour (BMWFI). Funded by FFG, Land Steiermark and Steirische Wirtschaftsförderung (SFG)
Is Enterprise Search useful at all? Lessons Learned from Studying Usage behavior.
I-KNOW 2014Graz, Austria
Dr. Alexander Stocker
2© disclosure or duplication without consent
prohibited
Intro User Satisfaction User Evaluation
Enterprise SearchAgenda
3© disclosure or duplication without consent
prohibited
Enterprise SearchBusiness Case
Findability Survey 2013 http://de.slideshare.net/findwise/enterprise-search-and-findability-survey-2013
4© disclosure or duplication without consent
prohibited
Huge demand from industry Enterprise Search is becoming more and more popular EU-Market for EU Enterprise Saerch vendors is between 100 and 200 Mio. €
Lack of application oriented research Lack of academic case studies Lack of academic best practices Lack of academic user evaluations
Is the topic Enterprise Search in the hand of practicioners, only?
Enterprise SearchIntroduction
5© disclosure or duplication without consent
prohibited
Enterprise search includes any organization with text content in electronic form, search in an organization’s external website, search in the organization’s internal websites (i.e. its intranet), and search in other electronic text held by the organizations in the form of email,
database records, documents on fileshares, etc.
(„Challenges in Enterprise Search“, Hawking, 2004)
Enterprise SearchDefinition
6© disclosure or duplication without consent
prohibited
Enterprise SearchSatisfaction
Mindmetre: Report „Minding the Search Gap“
7© disclosure or duplication without consent
prohibited
Enterprise SearchSatisfaction
Findability Survey 2013 http://de.slideshare.net/findwise/enterprise-search-and-findability-survey-2013
8© disclosure or duplication without consent
prohibited
Web Seach Interlinked Web pages.
Standard formats
Relevance of a web page is (initially) calculated by assessing, how many (good) links point to it
Information on the Web wants to be found. Hence content is optimized for search.
Web search has indexed open content, access rights are not relevant
Searcher is satisfied with a good answer (out of many possible good answers). Search queries are of a more common nature.
…
Search on the Web vs. Search in the Enterprise
Enterprise Search
Heterogeneous content (structured and unstructured)
No links between the documents. Different ways of relevance assessment needed
Users are not motivated to optimize their documents for search
Complex group & roles structures. Access rights first, information access second
In most cases, there is only one document relevant for a user. This has to be „found“ as quick as possible.
..
9© disclosure or duplication without consent
prohibited
Case Study Research organization in the automotive domain, employing about 200 people Enterprise Search Pilot to find information in project-relevant documents on a
fileshare (Microsoft Sharepoint Foundation 2013) Pilot study with 10 participants to identify user-centric aspects of Enterprise Search
implementation
Evaluation Short structured interviews
(tasks, documents and tools, information demand, search-scope and time) Definition of possible & realistic search tasks Participants performed these search tasks afterwards Participants had to think aloud what they did and why they did it Transcription and analysis of interviews Identification of commonalities and differences
Enterprise SearchUser Evaluation
10© disclosure or duplication without consent
prohibited
Enterprise Search(Microsoft Sharepoint Foundation 2013)
11© disclosure or duplication without consent
prohibited
Finding the right keywords is crucial, but very challenging Better keywords lead to more relevant search results Thinking process for finding the best keywords is very challenging Explication of a personal information demand for a search engine is a challenge
Evaluation results Keyword selection and query formulation
12© disclosure or duplication without consent
prohibited
The benefit of metadata for search gets lost, if metadata is not administered in organizational practice Metadata is used in search to filter search results („facets“) Missing or/and false metadata reduces search quality (e.g. the shown author is not the
correct author of a document, but the creator of its template) Users lack motivation to provide correct & useful metadata, as they do not perceive
any benefit
Evaluation results Usage of metadata
13© disclosure or duplication without consent
prohibited
Consistent document content (formulations) would increase perceived usefulness of search The availability of essential terms in documents increases search success
(e.g. authors, location, date, consistent project titles, MOM vs. meeting minutes, ..) Knowledge about document content and writing practices is crucial
(terms, syntax, semantics, …) Common writing practices of essential project-information in documents have to be
found, first, otherwise searchers may always have to guess(“..what terms could be used in the document containing the information I seek..”)
Evaluation results Document Content
14© disclosure or duplication without consent
prohibited
The relevance assessment of a search engine does not necessarily meet the user needs The search engine calculates relevance displayed in the hit list based on the
appearance of query terms within documents The ranking is crucial for perceived usefulness of search. Employees requested
other mechanisms, e.g. ranking in term of up-to-dateness
Evaluation results Relevance
15© disclosure or duplication without consent
prohibited
Enterprise Search has to compete against existing individual information seeking practices (search alternatives). So far, anything had to be „found“ without enterprise search. Employees (may) have
created individual search strategies. Face-2-Face conversations between colleagues are the number one tool for
information acquisition (e.g. the location of a document is gained by asking a colleague who knows it)
Users know the structure of project-folders, they browse instead of search Users take advantage of links to projects, project documents and others via their
desktop or via messenger/skype etc.
Evaluation results Search Alternatives
16© disclosure or duplication without consent
prohibited
Enterprise Search has to immediately provide an added value from a user‘s perspective for daily work. Enterprise Search can require more time to find information compared to browsing
through (known) folder structures Finding the right office-document was challenging, because of the high number of
similar documents in different versions (e.g. V21_meeting_minute_xy) User satisfaction was especially very low, when keywords were not available in
documents
Evaluation results Overall usefulness and satisfaction
17© disclosure or duplication without consent
prohibited
1. Enterprise Search Introduction
2. Enterprise Search Satisfation
3. Enterprise Search Potentials & Pitfalls
Pioneering work to motivate other researches studying user-aspects of enterprise search (adoption)
Call for Research Cases: Enterprises having a successful enterprise search implementation with a very high user satisfaction are warmly welcome
Enterprise SearchConclusion
18© disclosure or duplication without consent
prohibited
Kontakt
Dr. Alexander Stocker
VIRTUAL VEHICLE
alexander.stocker@v2c2.at
www.alexanderstocker.at