WorldCat Usability Study
UX Research
R O L E
UX Research Intern
D U R A T I O N
Jan 2025 - May 2025
S K I L L S
Usability Testing
Research design
Qualitative analysis
T O O L S
Microsoft suite
T E A M
2 UX Research Interns
AUXS UX Researcher
AUXS Department Head
PROBLEM STATEMENT
What happens when the very system students depend on to borrow materials across libraries suddenly becomes confusing and unreliable?
For students and faculty at Duke, borrowing materials from other libraries is often essential to their research and coursework. But when a new management system disrupted the Interlibrary Loan (ILL) process, it quickly became harder for users to access the resources they needed. To address this, the Assessment and User Experience Strategy (AUXS) department at Duke University Libraries (DUL) set out to test WorldCat Discovery (WCD), a cataloging tool already used at many nearby universities, to see if it could provide a smoother, more reliable experience for the Duke community.
Understanding this led us to our two main research questions:
How straightforward is it for users to complete basic tasks on this platform?
Are there any challenges with WCD that makes the interface less usable?
1. HOW WE SET THE STAGE
Each participant spent about 20 minutes walking through the site, giving us a focused window into how real users navigated its strengths and stumbling blocks.
TESTING SET-UP
Borrowing our set-up from NC State’s Tiny Café, we set the stage with fresh pastries and coffee, turning our usability study into a relaxed, café-like experience that made participation feel natural and inviting.
Image of Tiny Cafe, what our set-up was inspired by.
2. WHAT WE ASKED, WHAT WE LEARNED
To ground our study, we started by asking participants about their experiences and familiarity with WorldCat, giving us context before watching them navigate the system.
TASKS
Task 1a: “Imagine you’re a researcher looking for a book called ‘Washington at Valley Forge’, written by Russell Freedman. How would you search for this book?”
Follow up question: On a scale of 1-5, how would you rank the difficulty of completing this task? With 1 being extremely easy and 5 being extremely difficult. Please explain your answer.
Task Type 1: Known Item Search
Task 1b: “You want to read this book, but Duke does not have it. Show us how you would request this book from another library.”
Follow up question: On a scale of 1-5, how would you rank the difficulty of completing this task? With 1 being extremely easy and 5 being extremely difficult. Please explain your answer.
Task 2a: “For a class, your professor is asking you to research the Black Death. Perform a search for this subject and look over the first page of results.”
Follow up questions:
Do the items on the first page of results give you enough information to determine what sources you want to use or request?
On a scale of 1-5, how would you rank the relevancy of the items on the first page of results? With 1 being not relevant at all and 5 being extremely relevant. Please explain your answer.
Task Type 2: Subject Search
Task 2b: “You decide you want to see the items that are the closest match to your topic at the top of the results list. Sort the results to put the most similar items to your search at the top.”
Follow up questions: On a scale of 1-5, how would you rank the difficulty of completing this task? With 1 being extremely easy and 5 being extremely difficult. Please explain your answer.
Task 2c: “After reading your assignment description more closely, you notice that your professor wants you to find an article. Limit the results to look at articles only.”
Follow up questions: On a scale of 1-5, how would you rank the difficulty of completing this task? With 1 being extremely easy and 5 being extremely difficult. Please explain your answer.
Once testing wrapped up, we turned to thematic analysis to make sense of the data. This approach allowed us to sift through individual comments and task observations, organize them into patterns, and surface the recurring usability issues that would guide our recommendations.
THEMATIC ANALYSIS
Key Theme 1: Search Result Irrelevancy
For both main task topics (known item search & subject search), participants had a tough time finding what they wanted to find
A couple participants mentioned they would stop using WorldCat Discovery, and move to a system they were more familiar with (like Google Scholar)
KEY THEMES & FINDINGS
What users saw upon searching for the item:What users should have ideally seen:Key Theme 2: Visibility (or lack of) filters
All participants found the “article” filters very easily in the final Subject Search task
Participants were trying to use the filters throughout the test to improve relevancy of their searches
Participants had an extremely tough time finding the “sort by” filters
Key Theme 3: General knowledge of library search strategies
Participants that would use the advanced search or put quotes around search phrases had an easier time finding relevant searches
Key Theme 4: Finding the Request/Interlibrary Loan button
Nearly all participants were able to find the button without any issue
Participants appreciated the button being high contrast, and being placed in an area that made it easy to find
3. TURNING INSIGHTS INTO OPPORTUNITIES
Each theme pointed us toward the changes that mattered, shaping the foundation of our recommendations.
RECOMMENDATION 1
Pain point/Theme
Search Result Irrelevancy: This theme showed participants’ frustration with multiple irrelevant results that came up on their initial searches.
Recommendation
Change the system’s default sort option to “best match”.
Defaulting to ‘Best Match’ ensured that results closely aligned with what users searched for.
RECOMMENDATION 2
Pain point/Theme
General Knowledge/Usage of Library Search Strategies to Increase Search Relevancy: Participants who had prior knowledge of search strategies had an easier time finding the most relevant search items.
Recommendation
Make the advanced search option more obvious, both on the search page and the results page
‘Advanced Search’ option can be difficult for participants to see
RECOMMENDATION 3
Pain point/Theme
Visability (or lack of) of Filters: Some negative aspects included participants mentioning the filter section being overwhelming.
Recommendation
Increase color contrast of filter labels
Collapse filter menus on left to dropdown menus
Menu/label color doesn’t pass accessibility contrast.
Filter menu on the left is set to be open on default.
4. LOOKING AHEAD (AND LOOKING BACK)
NEXT STEPS
The recommendations provide actionable changes that should be implemented fairly easily. Once implemented, it would be best to test again, and see if results are more positive.
Implement the changes & test again
This system is already available to the public, but at what point would it be best to push/advertise this system out even more? Designating a threshold if doing iterative testing would help determine this.
Decide on a threshold
WHAT I LEARNED
Designing the tasks took up much of our time. It took us 2 pilot tests to come up with tasks that make sense to the participant, but also help us achieve our evaluation goals.
Task design is a balance
Anything can change at any given time, so it’s important to stay flexible and remember to keep an open mind on where things may go. We had to change our test design and purpose, due to many confounding factors. Being able to pivot greatly allowed us to stay on track and get things done before the deadline.
Be ready to be flexible
SOMETHING I WOULD HAVE DONE DIFFERENTLY…
Though I researched other institutions that use WorldCat Discovery, I didn’t research how other non-WCD search systems function. Researching this would have helped me distinguish whether the relevancy issue was a WCD-only issue, or if this is common across other platforms as well. Though the tasks worked, if the lack of relevancy in searches is common across multiple platforms, the scope of the issue would be larger than just WorldCat.