Accademia di San Luca Usability Study

a usability project analyzing one of the National Gallery of Art’s archival databases

Cover Image: Orazio Borgianni, Christ among the Doctors, c. 1609, oil on canvas. Rijksmuseum, Amsterdam, On loan from the Broere Charitable Foundation, SK-C-1709.

A screenshot of the Accademia di San Luca’s current homepage.

Overview

This project focused on improving the usability of the National Gallery of Arts’ The History of the Accademia di San Luca, c. 1590–1635: Documents from the Archivio di Stato di Roma, a research database focused on archival documents related to the Accademia di San Luca in Rome, Italy.

Orazio Gentileschi, The Lute Player, c. 1612/1620, oil on canvas. National Gallery of Art, Washington.

Process

The Client

The National Gallery of Art’s Digital Experiences Team

The project

Analyzing the usability of the Accademia di San Luca archival database.

the toolbox

  • Google Workspace

    • Docs, Calendar, Sheets, Slides, & Meet

  • Microsoft Word & Excel

    • I finalized the written report, including creating a coherent theme and importing our finalized data tables into the document.

  • Calendly

    • Used to schedule user interview sessions

  • Figma

the team

Rachel, Arthi, Sapphire, and Daniel.

Four UX Consultants from The Center of Digital Experiences @ Pratt Institute.

my role

I led two of our user interviews, recorded notes for and designed the finished report.

Getting Started

At our kick-off meeting, we prepared a set of questions for our clients, such as:

  • What are the top priorities for the website’s usability?

  • What defines success for this project?

  • Are there any particular points of frustration with the current design you have been able to identify?

  • Do you have any current power users who would be willing to participate in our research?

Methodology

Following the meeting, our team began developing task questions, and recruiting potential participants simultaneously.

the Participants

number of user types

The National Gallery of Art’s target audience for this project was historians/researchers, teachers, and students. Our clients were able to provide us with four users that possessed previous knowledge of the ASL website, the remaining three participants were recruited through my team members and I reaching out to history/art history degree holders. User occupations included teachers, museum professionals, and students. Interviews were conducted over the course of two weeks using Google Meet.

  • 1. What is your current profession?

    2. How frequently do you visit other museum websites?

    ● Less than 5 times a year

    ● Between 5-10 times a year

    ● More than 10 times a year

    3. What is your familiarity with the Accademia di San Luca?

    ● Not familiar at all

    ● Somewhat familiar

    ● Very familiar

    4. How often do you visit the Accademia website and how does it impact your research?

    ● Less than 5 times a year

    ● Between 5-10 times a year

    ● More than 10 times a year

    5. What is your primary purpose for visiting the Accademia website?

    6. How frequently do you visit other academic research databases or digital archives?

    ● Less than 5 times a year

    ● Between 5-10 times a year

    ● More than 10 times a year

  • Questions were crafted to be very specific to certain areas of the website, such as directing users to potentially interact with the search bar or map tools:

    1. Based on what you already know about the website, find something you are interested in.

    2. Find out what year the Rest on the Flight into Egypt etching by Carlo Saraceni was created.

    3. Locate the Federico Zuccaro drawing Disputation of Saint Catherine of Alexandria and find out the approximate year it was created.

    4. Where would you find archival documents associated with the church of San Luigi Dei Francesi?

    5. Find the map by Joachim von Sandrart and Giovanni Battista Falda from 1677 and locate the Farnese Palace.

    6. Find the transcription for the Die 30 mensis Martii 1593 document.

    7. You need to reference the Roma con li forti map. Find where you can see this cartouche.

    8. Please identify the difference in medium between the Der Statt Rom, from Cosmographei and Disegno nuovo di Roma moderna maps.

    9. You're interested in writing a research paper on Alessandro Agazzini, find out how many places he is associated with in his artist profile.

  • 1. On a scale from 1 to 10 (1=very unlikely, 10=very likely), how likely are you to recommend the Accademia project site to a colleague with similar interests?

    2. On a scale from 1 to 5 (1=very dissatisfied, 5=very satisfied), how would you rate your overall experience with the site?

    3. What did you like the most about using this site?

    4. What were some frustrations you experienced in regards to this site?

    5. How frequently would you use this site?

    ● Never

    ● Very Rarely (once per month)

    ● Rarely (2-3 times/month)

    ● Occasionally (2-3 times/week)

    ● Frequently (1-2 times/day)

    ● Very Frequently (3+ times/day)

results

likelihood of recommending the ASL to others (score out of 10)

The Post-Test Questionnaire proved to be useful for collecting important data on how users felt about the overall experience of the ASL website.

Participants were divided into three groups:

  • Novice Users

  • Intermediate Users

  • Advanced Users.

Novice Users were most likely to potentially recommend the site to colleagues with a similar interest (7.7/10), while Advanced Users who had prior experience with the ASL were least likely to recommend the site (6.5/10).

Related to this, Advanced Users gave the overall experience of the ASL website an average of 3/5, compared to Novice Users whose average experience rating was 4.3/5.

Average Overall Task Scores (score out of 100)

Users’ performance was measured based on if the task was completed, and how long it took. From this, we calculated each user’s average overall task score. Interestingly, Intermediate Users score average was the lowest at 61%. Advanced Users average overall score was 75%.

Overall, it was surprising to see that Advanced Users who were more familiar with the ASL or similar archival databases, gave the website the lowest scores out of all three user types.

information architecture

On the original homepage, the ASL’s navigation bar was seated on the left-hand side of the page. Sections can be navigated to from the sidebar, however the bar does not follow the user as they scroll down the page, nor is there a ‘scroll to top’ option available. While the information provided on the page is important, its layout is not ideal.

The site’s “breadcrumbs“ are also unfriendly to user navigation. They currently sit above the site’s title, and on the homepage it reads “Accademia > English > Intro,” which is not an actual path.

A screenshot of the ASL’s current breadcrumbs, below sits the title heading for the archive.

We recommended that the NGA restructure the homepage’s navigation layout by replacing the sidebar with a header navigation bar, and rework how the breadcrumbs operate throughout the website pages.

Our proposed redesign for the navigation. This mock up replaces the sidebar with a navigation bar that follows users to every page.

Proposed redesign of the updated search, including a filter.

Navigation

The archive’s search bar sits at the top right of the webpage rather than as the focal point. This is counter-intuitive to the ASL’s purpose as a research database. Adjustments needed to be made to the homepage that would make the search tool the main focus of that page, and continuously present on other pages.

We developed a set of mock ups that portray the search tool as the center of the homepage, including adding in search refinement options for visitors to filter results without having to use a separate page.

Our re-imagined homepage, with the search tool center stage.

Maps

The ASL has a webpage dedicated to maps. Each map’s individual page also contains tools for individual maps, along with compare options that users can use to analyze multiple maps at once. Users were generally satisfied with their Maps page interactions.

However, multiple users experienced difficulty in finding certain marker points for Task 5 since there were no map keys/legends available on the map interface, and no users selected the compare map option. Users also had to scroll up and down the page in search of maps when using the Maps page in particular, because there is no available search tool specifically for the webpage.

These usability issues stem from multiple areas. Users had trouble with interacting with the maps in part because the map key is located under the map’s description in a drop down list, and the compare map tool has no indications for its use unless the user hovers over the icon.

A screenshot of the interactive map. The red box highlights some of the map markers.

A screenshot highlighting the current compare map tool.

Our proposed redesigns include:

  1. Create a search and filter function for the Maps main page, like the one we’ve proposed for the other ASL document pages.

  2. Adding a key/legend to the interactive map interface, represented by a key icon, which would tell users what the color-coded map markers represent.

  3. Improve upon the discoverability of the maps’ comparison tool by adding a hover tooltip that would tell users what it does.

A redesigned individual map page featuring the proposed search bar.

The interactive map featuring a key tool.

This improved comparison tool highlights its uses.

Closing Thoughts

Our team having a bit of a laugh during presentation rehearsal.

After our team finished compiling our research and analysis, I finalized the written report - including creating a coherent design theme, refining some of the findings and recommendations, and importing our completed data tables into the document.

Our results were presented to the client through a 10-minute presentation, with a 30-minute conversation afterwards. The NGA’s Digital Experiences team responded positively to our key findings and recommendations. I believe my team’s overall findings, recommendations and design proposals can greatly improve the usability of the Accademia di San Luca website. Our clients have already shared that they plan to implement our recommendations, and they plan to continue to reach out to us for feedback on the process. Given the chance to continue working on this project, I would definitely be interested in seeing how the the data we collected could shift by increasing the number of participants.