A Comparative Usability Evaluation of User Interfaces for Online Product

Catalogsby

Ewa Callahan & Jurgen Koenemann

presented byAndrew Trieu

ICS 280 - Information VisualizationDepartment ICS at UCI

June 6, 2002

Introduction

A comparative study of the usability of hierarchically structured and zoomable Electronic Product Catalogs (EPC), available online,

VERSUS the InfoZoom, a novel interface

utilizing a compression technique, direct manipulation, and zooming.

Introduction (con’t)

The authors had designed a test search interface and used it for comparison between EPC and InfoZoom. Performance on both types of interfaces was evaluated in terms of search speed, accuracy, efficiency, and user satisfaction.

How EPC comes about

Due to the increased in the WWW, many companies have used it as a medium to promote and sell their products and services. The increase in electronic commerce creates a need for Electronic Product Catalogs.

Customers

The customer enjoys a higher level of interactivity such as a variety of searching possibilities, comparison features to get access to more information about the different product, etc. However, navigation problems are the second most cited reason for not shopping online.

Introduction (con’t)

The most common catalog interfaces on the Web are hierarchically organized catalogs, where the consumer sees the list of the products and can follow the links to their more detailed description.

Problems & SolutionProblems of the hierarchically organized

catalogs are:Confusing,Items in dropdown menus are not

visible,Lengthy result pages, andLack of filtering options of results.The solutions are:Recommendation approachInteraction approach

Recommendation Approach

The first approach includes the recommendation systems which suggest to the user what the user should buy (recommendation approach). This will require consumers to enter personal data which many consumers will not do.

Interaction Approach

The second approach concentrates on improving and facilitating the consumers’ interaction with the interface, allowing the user to select products more easily.

The authors were interested in this direction and wanted to investigate whether or not highly interactive interfaces that support the user in the selection of products can improve the finding of products.

Evaluation of Existing Online Catalogs

For the evaluation, the authors had selected 14 German and English automobile EPCs available on the Web in the Fall of 1999.

A Summary of the Results

Analysis of the Result

Database sizeSearching modeAttributesSpecifying RangesSearch HistoryResult PagesSorting Search ResultsItem Comparison

InfoZoom

Based on Ahlberg and Schneiderman’s dynamic queries and Furnas’ fisheye techniques

Displays database relations as tables with attributes as rows and objects as columns

Overview mode and Compressed modeOffers many different ways of searchingCan easily compare, sorting, and re-

sorting the results.

Overview Mode

Compressed Mode

Usability Study

Based on the feature comparison of InfoZoom and the hierarchical interfaces, InfoZoom should be a more efficient tool that facilitates searches for products.

The authors conducted a comparative usability study to confirm their analysis. Also, they performed a controlled experiment that resembled online counterpart.

Materials

The database used for this study (AutoDB) contains 1690 cars with over 20 attributes.

The hierarchical interface was implement with forms in MS Access in a way to resemble the “Look & Feel” of the typical online EPC catalog features.

Hierarchical Interface (1)

Hierarchical Interface (2)

Hierarchical Interface (3)

Participants

26 participants (randomly assigned to either InfoZoom or hierarchical interface)

15 males and 11 females13 ages between 21-30, 9 ages between

31-40, one teenager, and one over age 51

16 use computer above 10hr/wk, 6 use computer between 3-9hr/wk, and 4 use computer less than 2hr/wk.

Tasks

Find Attribute value for given ObjectT1: Does the Jaguar XJR have side

airbags?Find Database Subset Attribute ValueT2: How many different models does the

Peugeot class 106 have?T4: How many cars have the following

characteristics: convertible, 100-150 HP, front drive, and cost 40.000 DM or less?

Tasks (con’t)

For given Attribute(s) value(s) Find Object(s)

T3: Which middle class car is the cheapest one in the database?

T6: Which Funcar has the quickest acceleration?

Compare Attribute Values for 2 ObjectsT5: Which car has better fuel consumption:

the Audi A6 1.8 or the Volvo S70 2.0?

Tasks (con’t)

Find Database Attribute ValueT7: How many cars are in this catalog?Compare Attribute Values for SubsetT8: What is the price range (from the

cheapest to the most expensive model) for the Alfa Romeo 145 series?

T9: Is the acceleration of the Mercedes-Benz S430 Automatic slow, normal, or quick for a car of its class?

Procedure

Participants were briefly introduced to the purpose of the study and upcoming tasks.

InfoZoom participants got a 5 minutes demo about InfoZoom.

Participants were introduces to 9 tasks. After each task, they answer 3 questions about satisfaction, perception of the time needed, & perceived efficiency.

After last task, summarize their experience

Results

Results were analyzed for each category of data:

Accuracy - equal between both interfacesCompletion time

AVGIZ = 19.4 min.; AVGHI = 27.7 min.Navigation (out of 7 scale)

Ease of use: IZ = 5.5; HI = 4.9Efficiency: IZ = 5.0; HI = 4.8Overall, both were valued higher for IZ.

Results (con’t)

Results were analyzed for each category of data:

Satisfaction (7-point Likert scale)Comfortable: IZ = 5.15; HI = 5.23Fun: IZ = 6.3; HI = 5.0Overall satisfaction with both

system was high

Completion rate & accuracy

Hierarchical Interface

InfoZoom

Observations

The authors observed the participants throughout the experiment. Throughout the experiment, the authors noted:

Users’ strategiesCritical incidents - breakdowns and

things that went especially wellUsers’ comments

Hierarchical Interface

Dislike the ways of looking for the desired information from the main menu.

Seem lost on pages that looked alikeAnnoyed by the long pull-down menus and

lengthy result pages without sortingConfused about default valuesCriticized at the result pagesStopped the search after getting message

“no car matched” the second time

InfoZoom InterfaceFeel overload of information upon their

first glance at overview screen.Difficulty in choosing the appropriate

search strategy at the beginning of each task.

Participants who preferred selecting objects on the screen have difficulty to double-click on small sectors.

Difficulty with tasks that required comparison or gathering more than one different piece of information.

Conclusion

EPC’s interfaces were rated high, however InfoZoom was rated higher in overall performance.

InfoZoom supports Choice by Attribute searching strategy better than hierarchical interface.

InfoZoom interface is easy to learn.InfoZoom is a promising solution for

an interactive online catalog interface used in e-commerce.