Evaluation of Lecturer Web-Based Application SIPADU-STIS

Using Heuristic and Observation Method

Lutfi Rahmatuti Maghfiroh

Politeknik Statistika STIS

SIPADU-STISDepartment….

Adminis-tration …

Lecturer

Management of class schedules (submitting changes in schedule or room)

Test score entry

Distribution of semester results (PA lecturer)

Guidance for the final project (supervisor)

Confirm the seminar schedule and final exam for the final project

~200

Student

…..

Previous System Evaluation

BlackboxTesting

SUS

Aisha Adetia, Sistem Informasi Key Performance Indicator (KPI) Pegawai STIS. Skripsi. 2011. Sekolah Tinggil Imu Statistik.Alrasyid, Harun, Pengembangan Sistem Informasi Perpustakaan Sekolah Tinggi Ilmu Statistik (Aplikasi Desktop Admin, Web Admin dan Pemberitahuan Keterlambatan dengan SMS Gateway). Skripsi. 2011. Sekolah Tinggil Imu Statistik.Hardjantho, Abialam Koesnandy, Analisis User Acceptance Terhadap SIPADU-STIS Menggunakan Teori Technology Acceptance Model (TAM) dan Metode Analisis Structural Equation Modeling (SEM). Skripsi. 2014. Sekolah Tinggi Imu Statistik.

Previous System

AdministrasionService via Web App

Interaction via UI

Better UI

Better Service

Yustianto, Purnomo, Suhardi, Robin Doss, Consolidating service engineering perpective. 2015 International conference on information technology system and innovation (ICITSI), Bandung-Bali, November 16-19, ISBN 978-1-4673-6664-9.Ben Shneiderman, Designing the User Interface (3rd Ed.), Addison-Wesley, Jeff Johnson, GUI Bloopers 2.0 - Common User Interface Design Don’ts and Dos, Morgan Kaufmann, 2008Wilbert O. Galitz, The Essential Guide to User Interface Design: An Introduction to GUI Design Principles and Techniques (3rd Ed.), Wiley Publishing, 2007.

Methods

Descriptive analysis

Respondents determine the

usability problem

Respondents fill out the

questionnaire

User use system for

daily activity

6 novice and 5 expert

Jakob Nielsen on April 24, 1994. https://www.nngroup.com/articles/ten-usability-heuristics/. Accesed on October 15, 2019Yusoh, Suweena, Sureena Matayong. Heuristic Evaluation of Online Satisfaction Survey System for Public Healthcare Service: Applying Analytical Hierarchical Process. 2017 2nd International Conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE).Salvado, Valeria Farinazzo Martins, Lincoln de Assis Moura Jr. Heuristic Evaluation For Automatic Radiology Reporting Transcription Systems. 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010)Almarashdeh, Ibrahim, Mutasem, AIsmadi. Heuristic Evaluation of Mobile Government Portal Services: An Experts' Review. The 11th International Conference for Internet Technology and Secured Transactions (ICITST-2016). 978-1-908320/73/5/$31.00 © 2016 IEEEPaz, Freddy, Freddy A. Paz, Daniela Villanueva, Jos´e Antonio Pow-Sang. Heuristic Evaluation as a Complement to Usability Testing: A Case Study in Web Domain. 2015 12th International Conference on Information Technology - New GenerationsBorovina Nihad. Heuristic based evaluation of mobile services web portal usability. 22nd telecommunications forum TELFOR 2014Botella, Federico, Jose A. Gallud, and Ricardo Tesoreiro. Using Interaction Patterns in Heuristic Evaluation. A. Marcus (Ed.): Design, User Experience, and Usability, Pt I, HCII 2011, LNCS 6769, pp. 23–32, 2011. © Springer-Verlag Berlin Heidelberg 2011Rais, Nendy Akbar Rozaq, Hari Agustiyo, Moch. Arfian Ardiansyah. Evaluasi Heuristic Study Kasus Tiket.com. Seminar Nasional Teknologi Informasi dan Multimedia 2018. ISSN : 2302-3805.Aziza, Rifda Faticha Alfa, Yahya Taufiq Hidayat. 2019. Analisa Usability Desain User Interface Pada Website Tokopedia Menggunakan Metode Heuristics Evaluation. Jurnal TEKNOKOMPAK, Vol. 13, No. 1, 2019, 7-11. ISSN 1412-9663Salgado, Andr´e de Lima, Renata Pontin de Mattos Fortes. Heuristic Evaluation for Novice Evaluators. Springer International Publishing Switzerland 2016. A. Marcus (Ed.): DUXU 2016, Part I, LNCS 9746, pp. 387–398, 2016. DOI: 10.1007/978-3-319-40409-7 37Subbian, Vignesh, Philip A. Wilsey, and Fred R. Beyette, Jr., Heuristic Evaluation of User Interface for Point-of-Care Diagnosis and Rehabilitation of Mild Traumatic Brain Injury. 6th Annual International IEEE EMBS Conference on Neural Engineering San Diego, California, 6

Questionnaire

Heuristic Aspect of Nielsen

Visibility of System Status

Match between the system and the real

world

User control and freedom

Consistency and standards

Recognition rather than recall

Error prevention

Flexibility and efficiency of use

Aesthetic and minimalist design

Help users recognize, diagnose, and recover

from errors

Help and documentation

10 Aspect

assessment

Usability

problemlevel

not a usability problems,

cosmetic problems,

minor usability problems,

major usability problems,

usability catastrophes

Visibility of System Status

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Is the status of

the system always visible?

Is the main menu

easy to understand?

1 2 3 4

9.1% difficult

9.1% rarely

18.2% never

63.6%

18.2%

18.2%

cosmetic problem minor usability problem

Not a usability problem

9.1%

Match between the system and the real world

0% 10%20%30%40%50%60%70%80%90%100%

How is the text used on the page?

Are the terms used as they should be?

Are the steps in accordance with the business

processes that should be?

1 2 3 4

45.5%

27.3%

18.2%

9.1%

cosmetic problem Not a usability problem

minor usability problem major usability problem

All match

User control and freedom

0% 10%20%30%40%50%60%70%80%90%100%

Can users easily cancel activities just

done in the system?

Does the user can easily control the

activities carried out in the system?

1 2 3 4

45.5%

27.3%

27.3%

cosmetic problem minor usability problem Not a usability problem

18.2% difficult

9.1% difficult

Consistency and standards

0% 20% 40% 60% 80% 100%

Are there inconsistencies in the

system?

Is the layout of the system part

consistent?

Is the color consistent?

Is the language consistent?

Is the image / logo consistent?

Is the layout of the posts consistent?

1 2 3 4

54.5%

27.3%

18.2%

cosmetic problem minor usability problem Not a usability problem

9.1% not at all

9.1% not at all

9.1% not at all

9.1% not at all

9.1% many

inconsistency

36.4% not at all

9.1% all consistent

Error prevention

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Does the system provide a warning

before you take a risk action?

1 2 3 4

45.5%

27.3%

27.3%

cosmetic problem minor usability problem Not a usability problem

36.4% always

Recognition rather than recall

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Are the steps easy to understand?

Are the steps easy to learn?

Are the steps easy to remember?

1 2 3 4

54.5%36.4%

9.1%

cosmetic problem Not a usability problem major usability problem

9.1% difficult

Flexibility and efficiency of use

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

What is the speed of the application

when first opened?

What is the speed of the menu

response when clicked?

What is the speed when moving

pages?

Is the system flexible?

1 2 3 4

36.4%

18.2%

36.4%

9.1%

cosmetic problem Not a usability problem

minor usability problem major usability problem

9.1% very slow

18.2% some inflexible

82.2% fast

82.2% fast

Aesthetic and minimalist design

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Is the system design good?

1 2 3 4

54.5%

18.2%

9.1%

9.1%

9.1%

cosmetic problem minor usability problem

Not a usability problem major usability problem

usability catastrophe

Help users recognize, diagnose, and recover from errors

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Are there any solutions displayed to resolve

the error?

Is the error message easy to understand?

Can the user recognize the error message

that occurs?

Does the error message use good and polite

sentences?

Did the error message appear helpful?

Can users diagnose errors that occur?

What is the level of ease in getting

assistance information?

Can the user return to the state before the

error occurred if an error occurred?

1 2 3 4

54.5%

9.1%

27.3%

9.1%

cosmetic problem major usability problem

minor usability problem Not a usability problem

9.1% never

9.1% never

9.1% never

Help and documentation

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%

Is the help menu and documentation

easy to find?

Are the help and documentation

menus helpful enough?

1 2 3 4

45.5%

27.3%

18.2%

9.1%

cosmetic problem minor usability problem

Not a usability problem major usability problem

27.3% unhelpful

45.4% difficult

Conclusion

From this research it was found that all aspects still have shortcomings or

problems

even though only 9.1% stated it.

From the 10 aspects evaluated, there are

4 aspects that have Minor usability problems

(Visibility of system status / feedback, Use control and freedom, Consistency and

Standards and Error Prevention),

5 aspects have major usability problems (Match between system and the real

world , Recognition Rather than Recall, flexibility and

efficiency of Use help users to recognize, dialogue, and

recovers from errors and also Help and documentation)

1 aspect that has Usability catastrophe

(Aesthetic and Minimalist Design).

Suggestion

developers should redesign aspects of

Aesthetic and Minimalist Design

•Match between system and the real world ,

•Recognition Rather than Recall,

•Flexibility and efficiency of Use

•Help users to recognize, dialogue, and recovers from errors

•Help and documentation

Improvements to improve services to lecturer,

especially in the aspects of

Thank you