Database Web Application Usability Testing
1 like4,393 views
The document provides an overview of usability testing conducted on the newly designed WVU Libraries database web application using TechSmith Morae. It focuses on the primary target audience, which consists mainly of undergraduate students, and highlights various tasks, user feedback, and improvements identified during testing. Significant insights reveal common user challenges, completion rates, and error rates, suggesting that further enhancements to the interface and navigation could improve overall user experience.
Database Web Application Usability Testing
- 1. USABILITY TESTING Database Web Application
- 2. Test Information OVERVIEW TechSmith Morae was used on a laptop computer to conduct usability testing of the newly designed WVU Libraries Database web application. This round of usability testing was internal and focused on WVU Libraries primary target audience.
- 3. Test Information TARGET AUDIENCE 73%* of WVU Libraries users are undergraduate students who are 48% female or 52% male, are primarily Caucasian, African American, Hispanic, or Asian in ethnicity, are United States citizens or are international students, are 21 years in their average age, and are 49% from the state of West Virginia or 51% from other United States, territories, or countries. Target audience numbers are populated from both survey polls and server statistics.*
- 4. Test Information DATABASE USE 96%* of the WVU Libraries Database web application usage is from non-Downtown Campus Library WVU employee computer users ‒ 64% of which are returning users, and 61% of which are from ages 18 to 34 ‒ on and off Morgantown campus, whereas 4% of Database web application usage is from WVU employee at the Downtown Campus Library. The Geographic Breakdown of these 96% of users is as follows: 99% from the United States; 86% from the state of West Virginia; 77% from Morgantown, West Virginia; 1% outside the United States. Statistics are calculated by Google Analytics.*
- 5. Test Information TASKS 1. Go to WVU Libraries’ Databases. 2. Find the database PubMed. 3. What is the earliest coverage in the database SportDiscus? 4. What is a database for the subject Marketing, that is also an encyclopedia? 5. Your assignment is to listen to an aria from an opera. What database will you use? 6. How many databases cover the subject of psychology? 7. Find a subject guide for engineering. 8. Go to the tutorial for the database Ad$pender. 9. What are the best databases for finding sources in biology? 10. What is the starting year of coverage for the database Web of Science? 11. How can you get help with database selection or searching? 12. You need a brief video clip on chemistry lab safety. Where would you find it? 13. Find 2 databases that contain statistics for market research. 14. What is a database for Theater that is contains Images? 15. How many primary sources can you find for West Virginia databases?
- 6. Test Information *(SUS) QUESTIONS 1. I think that I would like to use this system frequently 2. I found the system unnecessarily complex 3. I thought that the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system * System Usability Scale (SUS)
- 7. Erica Erick Kayla Rebecca Shawn Alex Brandon Chris Christian Dave Test Information PARTICIPANTS
- 8. Test Results: All Tasks AVERAGE TIME* Average time includes delivery of tasks.
- 9. Test Results: All Tasks SUCCESS RATES* Percentages based on a 0 to 3 scoring system.
- 10. Test Results: All Tasks ERROR RATE* Error rate is a standard deviation based direct path to information as it is known to test administrator, and the rate of the user encountering a functionality or system error.
- 11. Test Results: All Tasks SUS SURVEY* Results are based on survey results taken after usability testing, and are concerned with the Database web application interface and functionality.
- 12. Test Results TASK 1Requested Task: Go to WVU Libraries’ Databases. Average Time: Completion Rate: Error Rate: Failure Rate: 0.74 min. 90% 0.00 10% Participant Observations • 10% of users couldn’t find the Databases page • 40% of users eventually located the Databases page by utilizing the homepage search or ‘Databases by Subject’ link • 50% of users utilized a search engine for ‘WVU Libraries Databases’ Suggestions • Better incorporate Databases on WVU Libraries’ homepage Videos • https://testing.lib.wvu.edu/usability4/task1a.mp4 • https://testing.lib.wvu.edu/usability4/task1b.mp4
- 13. Test Results TASK 2Requested Task: Find the database PubMed. Average Time: Completion Rate: Error Rate: Failure Rate: 1.17 min. 90% 0.00 10% Participant Observations • 30% of users couldn’t find the ‘Database Titles’ link • 20% of users aren’t seeing the A-Z section • 40% of users expressed interest in / wanted to search so they either went back to the libraries homepage, or used the browser’s default <Ctrl> + <F> to search. • 50% of users utilized a search engine for ‘WVU Libraries Databases’ Suggestions • Incorporate predictive search for Databases on the homepage and on the Databases web application page • Move the subject/title/help navigation to the left, pluralize ‘Databases’, add the word ‘by’, and make it look more like navigation options (instead of being grayed-out) • Add ‘by Subject’ as a facet • Add padding and emphasis to A-Z section to make it stand out more • Eliminate “by title ‘Letter’” from results, it’s confusing to read Videos • https://testing.lib.wvu.edu/usability4/task2a.mp4 • https://testing.lib.wvu.edu/usability4/task2b.mp4
- 14. Test Results TASK 3Requested Task: What is the earliest coverage in the database SportDiscus? Average Time: Completion Rate: Error Rate: Failure Rate: 1.59 min. 60% 0.20 40% Participant Observations • 100% of users don’t click or see ‘More Information’ in the databases results, 80% never go back to this, and only 20% eventually clicked on ‘More Information’ just because the task required them to do so • 30% are still not seeing ‘Database Titles’ Errors • Empty Help with a bullet under ‘More Information’ • Facets appear as a gray box for search results • New/Trial databases don’t show in breadcrumbs Suggestions • Possibly employ an accordion as a button instead of a blue ‘More Information’ link • Fix information in databases ‘More Information’ • Add facets for search, new, and trial Videos • https://testing.lib.wvu.edu/usability4/task3a.mp4 • https://testing.lib.wvu.edu/usability4/task3b.mp4
- 15. Test Results TASK 4Requested Task: What is a database for the subject Marketing, that is also an encyclopedia? Average Time: Completion Rate: Error Rate: Failure Rate: 1.01 min. 10% 0.00 90% Participant Observations • 40% of users aren’t seeing / understanding resource types as faceting • 20% of users are not understanding the difference between Databases by subject and titles • 20% of students are still going back to the homepage to search • As users spend more time with the system they are learning how to use the system as a successful NUI (natural user interface) Suggestions • Facets should only populate under resource types dynamically as they are applicable and remain to what’s pre and post breadcrumb faceting • Add hinting for important functionality on web applications Videos • https://testing.lib.wvu.edu/usability4/task4a.mp4 • https://testing.lib.wvu.edu/usability4/task4b.mp4
- 16. Test Results TASK 5Requested Task: Your assignment is to listen to an aria from an opera. What database will you use? Average Time: Completion Rate: Error Rate: Failure Rate: 1.07 min. 100% 0.00 0% Participant Observations • 20% of users still aren’t seeing / understanding resource types as faceting • 10% of users are still not understanding Databases by subject versus titles • 20% went to resource type ‘Video and Sound’ first, and 30% went to the subject of Art while 70% went to the subject of Music • 20% of users are still going back to the homepage to search • 20% of users used the Music tag Suggestions • Create visual understanding with color for facets in the left-hand, and color code (and possibly use different icons) for tags based on resource type versus subject • Reduce the amount of tags as is appropriate Videos • https://testing.lib.wvu.edu/usability4/task5a.mp4 • https://testing.lib.wvu.edu/usability4/task5b.mp4
- 17. Test Results TASK 6Requested Task: How many databases cover the subject of psychology? Average Time: Completion Rate: Error Rate: Failure Rate: 0.46 min. 90% 0.00 10% Participant Observations • At this point all users were familiar with the faceting, subject, and title filtering of the database web application • 10% of users misread the pagination indicator, the other 90% performed the task with ease Suggestions • Possibly add wording to pagination such as ‘Viewing results 1-10 out of 25 total’ or incorporate how many pages / results on pages Videos • https://testing.lib.wvu.edu/usability4/task6a.mp4 • https://testing.lib.wvu.edu/usability4/task6b.mp4
- 18. Test Results TASK 7Requested Task: Find a subject guide for engineering. Average Time: Completion Rate: Error Rate: Failure Rate: 0.66 min. 80% 0.00 20% Participant Observations • 40% of users don’t know what a subject guide was, or why it was needed • 10% of users used the homepage to try and find a subject guide Suggestions • On the macro level the WVU Libraries website could better explain the functionality of Research Guides living at libguides.wvu.edu (which is confusing), and that underneath Research Guides there are Subject Guides, Course Guides, Style Guides, Finding Guides, etc. Videos • https://testing.lib.wvu.edu/usability4/task7a.mp4 • https://testing.lib.wvu.edu/usability4/task7b.mp4
- 19. Test Results TASK 8Requested Task: Go to the tutorial for the database Ad$pender. Average Time: Completion Rate: Error Rate: Failure Rate: 1.38 min. 60% 0.00 40% Participant Observations • 10% looked in ‘Help’ • 20% of users went back to the homepage to search • 80% went straight to databases instead of ‘More Information’ • Users feel there are some databases that have too many tags Suggestions • Database title search states: By title ‘A’ Database Results: By title ‘A’, which is confusing and should read instead just as ‘Database by Title: A’ Videos • https://testing.lib.wvu.edu/usability4/task8a.mp4 • https://testing.lib.wvu.edu/usability4/task8b.mp4
- 20. Test Results TASK 9Requested Task: What are the best databases for finding sources in biology? Average Time: Completion Rate: Error Rate: Failure Rate: 0.99 min. 100% 0.10 0% Participant Observations • 100% of users didn’t see or utilize “Librarian Top Picks’ • 40% of user utilized the ‘Primary Sources’ facet • 20% of users went to Subject Guide for Biology • 20% of users utilized the ‘Statistics’ facet Error • When users click on the ‘Statistics’ resource type facet for the database subject of Biology there are no results Suggestions • ‘Librarian Top Picks’ need to be used across subject, or defaults should stick • ‘Librarian Top Picks’ should have a different aesthetic treatment to make them visible when they appear Videos • https://testing.lib.wvu.edu/usability4/task9a.mp4 • https://testing.lib.wvu.edu/usability4/task9b.mp4
- 21. Test Results TASK 10Requested Task: What is the starting year of coverage for the database Web of Science? Average Time: Completion Rate: Error Rate: Failure Rate: 0.80 min. 100% 0.00 0% Participant Observations • 30% of users said 1955 based on ‘More Information’ • 70% of users went straight to database and overlooked ‘More Information’, and 71% of those users said 1864 based on information in Web of Science • 10% of users searched on the homepage Videos • https://testing.lib.wvu.edu/usability4/task10a.mp4 • https://testing.lib.wvu.edu/usability4/task10b.mp4
- 22. Test Results TASK 11Requested Task: How can you get help with database selection or searching? Average Time: Completion Rate: Error Rate: Failure Rate: 0.84 min. 100% 0.10 0% Participant Observations • 10% of users would go to ‘Ask A Librarian’ green tab • 20% of users would go ask a librarian in person • 20% of users would go to ‘Contact Us’ • 30% of users would go to ‘Help’ Videos • https://testing.lib.wvu.edu/usability4/task11a.mp4 • https://testing.lib.wvu.edu/usability4/task11b.mp4 • https://testing.lib.wvu.edu/usability4/task11c.mp4
- 23. Test Results TASK 12Requested Task: You need a brief video clip on chemistry lab safety. Where would you find it? Average Time: Completion Rate: Error Rate: Failure Rate: 1.72 min. 20% 0.67 80% Participant Observations • 80% of users tried to use the ‘Video and Sound’ resource type facet • 30% of those users eventually went out to the homepage to search, 20% stated they would search in the subject guide, and 50% said they would go to YouTube • 20% of users used the ‘Handbook’ tag to determine a database Error • 80% of users clicked on the ‘Video and Sound’ resource type facet for the database subject of Chemistry, and there were no results • Users can compound resource type facets and get no results Videos • https://testing.lib.wvu.edu/usability4/task12a.mp4 • https://testing.lib.wvu.edu/usability4/task12b.mp4 • https://testing.lib.wvu.edu/usability4/task12c.mp4
- 24. Test Results TASK 13Requested Task: Find 2 databases that contain statistics for market research. Average Time: Completion Rate: Error Rate: Failure Rate: 0.95 min. 90% 0.00 10% Participant Observations • 20% of users read database description to resolve task • 20% of user went to homepage to search Videos • https://testing.lib.wvu.edu/usability4/task13a.mp4 • https://testing.lib.wvu.edu/usability4/task13b.mp4
- 25. Test Results TASK 14Requested Task: What is a database for Theater that is contains Images? Average Time: Completion Rate: Error Rate: Failure Rate: 0.60 min. 90% 0.00 10% Participant Observation • 10% of users didn’t understand faceting and filtering Suggestions • Possibly develop facets to work in reverse order, so by resource type then by subject. Videos • https://testing.lib.wvu.edu/usability4/task14a.mp4 • https://testing.lib.wvu.edu/usability4/task14b.mp4
- 26. Test Results TASK 15Requested Task: How many primary sources can you find for West Virginia databases? Average Time: Completion Rate: Error Rate: Failure Rate: 0.84 min. 80% 0.10 20% Participant Observations • 20% of users used tags to find primary resources or West Virginia databases • 20% of users went to the homepage to search Error • Homepage search results didn’t yield database subject search results Videos • https://testing.lib.wvu.edu/usability4/task15a.mp4 • https://testing.lib.wvu.edu/usability4/task15b.mp4
- 27. The WVU Libraries’ database web application was redesigned as a NUI, or Natural User Interface. In computing a NUI is the common parlance used by designers and developers of human-machine interfaces to refer to a user interface that is effectively invisible, and remains invisible as the user continuously learns increasingly complex interactions. These usability test results demonstrates that as a user interacts with the WVU Libraries’ database web application, the user continuously learns how to use the interface. More so, one can see the decrease in percentages from one task to another in regards to users not seeing or knowing how the database web application interface functions. Indeed, a NUI is ideal since our current analytics show that 96% of the WVU Libraries database web application usage is from non-Downtown Campus Library librarian/staff computer users on and off Morgantown campus. Ergo, the database web application’s return users (ages 18 to 34) are majorly using this web application without assistance of any kind. Furthermore, according to this usability test 11 out of 15 tasks had a failure rate ranging from 10% to 80%, and only 10 out of 15 tasks had an easy completion rate ranging from 10% to 50%; these rates can be dramatically improved. Conclusion SUMMARY
- 28. The following changes to analytics, content, development, and user interface should dramatically eliminate errors, reduce the percentage of failure rates, and decrease the amount of time for task completion Analytics - Install filters for librarian / staff traffic at the Evansdale and HSC libraries User Interface - Better incorporate Databases on WVU Libraries’ homepage - Move the subject/title/help navigation to the left, and possibly change color - Add padding and emphasis to A-Z section to make it stand out more - Possibly employ an accordion as a button instead of a blue ‘More Information’ link, and remove link from database title - Create visual understanding with color for facets in the left-hand, and color code (and possibly use different icons) for tags based on resource type versus subject - ‘Librarian Top Picks’ should have a different aesthetic treatment to make them visible when they appear - Possibly add # to simulate pagination reload Content - Pluralize ‘Databases’, add the word ‘by’, and make it look more like navigation options (instead of being grayed-out) - Fix information in databases ‘More Information’ - Add hinting for important functionality on web applications - Reduce the amount of tags as is appropriate - Possibly add wording to pagination such as ‘Viewing results 1-10 out of 25 total’ or incorporate how many pages / results on pages - Database title search states: By title ‘A’ Database Results: By title ‘A’, which is confusing and should read instead just as ‘Database by Title: A’ Development - Incorporate predictive search for databases on the homepage and on the Databases web application page - Add ‘by Subject’ as a facet - Add facets for search, new, and trial - Facets should only populate under resource types dynamically as they are applicable and remain to what’s pre and post breadcrumb faceting - ‘Librarian Top Picks’ need to be used across subject, or defaults should stick - Possibly develop facets to work in reverse order, so by resource type then by subject. Conclusion ENHANCEMENTS
- 29. “Pay attention to what users do, not what they say.” - Jakob Nielsen THANK YOU