Ressources numériques en sciences humaines et sociales OpenEdition Nos plateformes OpenEdition Books OpenEdition Journals Hypothèses Calenda Bibliothèques OpenEdition Freemium Suivez-nous

Comparative Analysis of the User Experience offered by International Art and Cultural Heritage Content Providers

Analyse comparative de l’expérience utilisateur proposée par les fournisseurs internationaux de contenu de l’art et du patrimoine culturel

Article issu de la 18e conférence EUTIC: Humanisme numérique et durabilité sociale, MSH Bordeaux, 11-13 octobre 2023 organisée dans le cadre de l’Appel à projets MSHBx 2023 EUTIC2023.


Minas PERGANTIS
Department of Audio and Visual arts, Ionian University, Greece
mperg[at]ionio.gr

Aristeidis LAMPROGEORGOS
Department of Audio and Visual arts, Ionian University, Greece
a18labr[at]ionio.gr

Laida LIMNIATI
BrilliantPR Digital Agency, Athens, Greece
laida.limniati[at]brilliantpr.gr

Andreas GIANNAKOULOPOULOS
Department of Audio and Visual arts, Ionian University, Greece
agiannak[at]ionio.gr


Abstract

Since the dawn of the digital era, art and cultural heritage-related content has made the transition from physical collections in galleries and museums to hybrid or digital-only collections, which are readily available for all through the world wide web. Presenting such content in a way that better suits the needs and habits of people searching the web for art and culture is a complicated challenge, which involves among other aspects, the development of a streamlined user experience and appropriate user interfaces. Although general best practices for UX/UI exists, art and culture content providers must cater to the special requirements that the content and their visitors demand.
In this study, a comprehensive comparative analysis of the user experience offered by a series of different international institutions providing content related to art and cultural heritage, is performed. This analysis aims to not only create a better understanding regarding common practices followed by such institutions, but also to investigate how and if the UX provided caters to the needs of searchers of art and culture as indicated by other relevant research.
In order to accomplish its research goals, the study presents a detailed heuristic evaluation framework which aims to evaluate the user experience and user interface of art and culture content providers based on various aspects deriving from the principles of usability, accessibility and aesthetic value, while at the same time taking into account recent findings concerning the peculiarities of the specific fields. Using this framework, a series of international art and cultural heritage content provider websites are analyzed and a comprehensive comparison between them is performed. This comparison’s intention is to emphasize good practices and common designs as they are observed through the contrasts that emerge from the evaluation process.
The study’s contribution is twofold. The results of the comparison are used to provide a better understanding of the landscape of UX/UI implementation in the fields of art and culture, emphasizing the peculiarities that arise from the nature of cultural content and the specific needs and habits of users relating to the fields of art and culture. In addition to that, a rigid evaluation framework is provided that can be used for self-assessment by art and culture content providers of any scale and type, including digital galleries, museums, archives and libraries. This process may eventually lead to an improved approach to UX and UI for the field of art and culture on the web and through this contribute to the satisfaction of users who are interested in all such content.

Keywords: web design, user experience, art and culture, usability, evaluation, heuristics.

Résumé

Depuis l’avènement de l’ère numérique, les contenus liés à l’art et au patrimoine culturel sont passés de collections physiques dans des galeries et des musées à des collections hybrides ou uniquement numériques, facilement accessibles à tous sur le web. Présenter ce contenu d’une manière qui corresponde mieux aux besoins et aux habitudes des personnes qui recherchent de l’art et de la culture sur le web est un défi complexe qui implique, entre autres, le développement d’une expérience utilisateur rationalisée et d’interfaces utilisateur appropriées. Bien qu’il existe des meilleures pratiques générales pour l’UX/UI, les fournisseurs de contenu artistique et culturel doivent répondre aux exigences particulières du contenu et de leurs visiteurs.
Dans cette étude, une analyse comparative complète de l’expérience utilisateur offerte par une série d’institutions internationales différentes fournissant un contenu lié à l’art et au patrimoine culturel est réalisée. Cette analyse vise non seulement à mieux comprendre les pratiques communes suivies par ces institutions, mais aussi à déterminer si et comment l’expérience utilisateur répond aux besoins des chercheurs d’art et de culture, comme l’indiquent d’autres recherches pertinentes.
Afin d’atteindre ses objectifs de recherche, l’étude présente un cadre d’évaluation heuristique détaillé qui vise à évaluer l’expérience utilisateur et l’interface utilisateur des fournisseurs de contenu artistique et culturel sur la base de divers aspects découlant des principes de convivialité, d’accessibilité et de valeur esthétique, tout en tenant compte des découvertes récentes concernant les particularités de ces domaines spécifiques. Dans ce cadre, une série de sites web internationaux de fournisseurs de contenu sur l’art et le patrimoine culturel sont analysés et font l’objet d’une comparaison détaillée. L’objectif de cette comparaison est de mettre en évidence les bonnes pratiques et les conceptions communes telles qu’elles sont observées à travers les contrastes qui émergent du processus d’évaluation.
La contribution de l’étude est double. Les résultats de la comparaison sont utilisés pour mieux comprendre le paysage de la mise en œuvre de l’UX/UI dans les domaines de l’art et de la culture, en mettant l’accent sur les particularités qui découlent de la nature du contenu culturel et des besoins et habitudes spécifiques des utilisateurs dans les domaines de l’art et de la culture. En outre, un cadre d’évaluation rigide est fourni, qui peut être utilisé pour l’auto-évaluation par les fournisseurs de contenu artistique et culturel de toute échelle et de tout type, y compris les galeries numériques, les musées, les archives et les bibliothèques. Ce processus peut éventuellement conduire à une approche améliorée de l’UX et de l’UI pour le domaine de l’art et de la culture sur le web et contribuer ainsi à la satisfaction des utilisateurs qui s’intéressent à ce type de contenu.

Mots-clés : conception de sites web, expérience utilisateur, art et culture, utilisabilité, évaluation, heuristique.


Introduction

With the advancements in modern digital technology and the world wide web, works of art and objects of cultural heritage have transitioned from their traditional physical presence in galleries or museums to a hybrid presence, through digital repositories and the web. Moreover, newer forms of digital art and cultural content only exist in the digital world. In order to properly present such digital manifestations of art and culture to potential viewers, many challenges need to be overcome. These include the development of an optimal user experience, aided by a streamlined user interface (Bafriah et al., 2020). This development must not only adhere to the principles and best practices of general purpose UX/UI development, but also cater to the needs of users looking specifically for content related to art and culture (Ceccarelli et al., 2011). Recent studies have shown that user behavior, during the process of discovering content related to art and culture, displays noticeable differences in behavioral patterns, as opposed to seeking general purpose content (Pergantis et al., 2023).

Based on these findings, this study proposes a detailed heuristic evaluation framework aimed at assessing websites that present works of art and objects of cultural heritage, encompassing not only the principles of usability, accessibility, and aesthetic value, but also the unique characteristics and needs of users consuming online content relevant to the fields of art and culture. The detailed evaluation process that is presented, is applied to a series of online repositories from international institutions prevalent in the fields of art and culture. This comprehensive comparative analysis of the cultural user experience offered by these acclaimed content providers, delivers ample insight on the basic principles and common practices of publishing works of art and objects of cultural heritage online and may also be used as a means of self-assessment for other individuals or institutions, that offer similar content and are aimed at the same target group of users.

Moreover, this work acts as another step towards streamlining the art and culture user experience, based on a more detailed understanding of art user’s habits and needs. Proper evaluation of the various unique aspects of this user experience may lead to an overall upgrade of the offered UX/UIs, which in turn will be beneficial for both consumers and providers.

1. Literature Review

Art has always been an essential part of the human experience since the very early stages of its development, even during prehistoric times, where it first appeared in the form of ritual items, ornaments, and decorations (Robb, 2015). This material culture evolved throughout the ages giving birth to the earliest collections of objects of art and culture belonging to nobles or religious leaders (Brittanica, 2014). These collections became the precursors of modern-day public museums or art galleries, which in turn, with the arrival of the digital era transitioned into a hybrid state that involves not only the physical manifestation of the venue with its exhibited items, but also a virtual manifestation in the form of a digital repository (Lynch, 2005) with the objects of art and culture being represented through digital resources. This digital transition was empowered further by the diffusion of the Internet and the world wide web which became an integral tool for preserving existing art and promoting new artists (Manjoo, 2017).

The presence of content related to art and cultural heritage on the web gave rise to a series of challenges that had to do with the optimal way to present this content to interested parties. These challenges are ultimately issues concerning user experience (UX). UX can be defined as the effect on a person caused by their interaction with a system or a service over the course of time and is essentially a complex construct consisting of multiple parameters (Robert & Lesage, 2017). An inherent part of UX evaluation is usability testing, which often involves the objective measurement of user related activities through metrics such as task execution time, number of clicks, errors, and others (Vermeeren et al., 2010). The concept of usability, although essential, is not enough to encompass the full extent of UX which involves a high amount of subjective qualities. UX works on multiple levels that are related to both aesthetics and functionality and as such can be strongly dependent on the specific context of the interaction between a user and a system (Robert & Lesage, 2017).

This aforementioned context is to an extent dependent on the thematic fields related to the system with which the user interacts. As mentioned, in the case of this study focus is given on the fields of art and culture and this requires an expanded understanding of UX and UX evaluation based on the peculiarities of these fields. cultural User Experience (CUX) refers to the use of UX design and evaluation methods in the field of cultural heritage and is a field that is gaining ground and expanding throughout the cultural industry and academia (Konstantakis et al., 2017). This may refer to not only fully digital experiences such as interacting with an online repository, but also hybrid experiences such as visiting a physical venue that makes use of extended reality technologies to enhance the visitor experience (Li et al., 2015). CUX evaluation requires additional emphasis on the habits and needs of users interested in art and culture which have been shown to differ significantly from user behavior during general purpose use of the web (Pergantis et al., 2023).

General purpose UX evaluation involves a series of techniques that aim to assess the effect that an interaction with a system has on the user over time and in a specific context (Robert & Lesage, 2017). The evaluation may be based on formal analysis techniques, automated data collection techniques, empirical experiments with users and heuristic approaches by expert evaluators (Nielsen & Molich, 1990). The techniques used in each case may vary greatly and include interviews, focus groups, log analysis, prototyping, field, and lab studies and more. (Balasubramoniam & Tungatkar, 2013; Rajeshkumar, 2013; Vermeeren et al., 2010). They often focus on qualitative or quantitative metrics, but can oftentimes make use of both and depending on the technique the type of evaluators may be the users themselves or experts from the field (Rajeshkumar, 2013; Vermeeren et al., 2010). Moreover, techniques also vary based on the type of the study, the type of system or service, or even the stage of development that the evaluation takes place in (Rajeshkumar, 2013). For the purposes of this study, the proposed evaluation framework is based on the principles of heuristic evaluation.

The concept of heuristic evaluation was introduced by Nielsen & Molich (1990) and is a method of informal analysis of UX/UI that is based on experts identifying potential issues on the systems, services or products that are part of the UX. The evaluation process is based on the system’s compliance with a set of best practices which are referred to as heuristics. When conducted by three to five experts heuristics evaluation has been shown to provide a good overview of usability problems (Nielsen & Molich, 1990). Specifically for the evaluation of the UX/UI of a website or online application, the International Organization for Standardization (ISO) introduced standard 9241-151:2008 (Ergonomics of human-system interaction — Part 151: Guidance on world wide web user interfaces), which was later withdrawn. In specific fields heuristic evaluations have proven to provide useful feedback in issue identification even compared to user experience testing (Sivaji et al., 2011). Moreover, the combined use of traditional heuristics alongside newly developed or modified heuristics based on the field of the system being evaluated is also essential (ibid.).

In numerous cases of heuristic evaluation, a variation of the usability heuristics introduced by Nielsen (1994) is usually the basis of the evaluation framework, with the ISO 9241-151 standard also being popular (Al-Khalifa, 2010). Specifically on the field of cultural heritage, heuristics can be enhanced by taking into consideration the cultural dimensions in UI/UX design as well as the technologies prevalent on this specific field (Lam & Sajjanhar, 2018). Extended research on the user behavior of people searching the web for art and culture related content has provided new insights on the way these interactions play out (Pergantis et al., 2022; Pergantis et al., 2023a; Pergantis et al., 2023b). In this study traditional heuristics are combined with modified or new heuristics that derive from the findings concerning art and culture user behavior, in order to introduce a new heuristic evaluation framework intended for the evaluation of online repositories of content related to art and cultural heritage.

2. Methodology

2.1. Research Design

The research methodology was carried out in three stages:

Stage 1 involved the definition of the evaluation parameters based on a combination of traditional UX heuristics and new or modified UX heuristics that addressed the peculiarities of art and culture.

Stage 2 involved the application of the proposed heuristic evaluation framework to five digital repositories presenting works of art or objects of cultural heritage by three experts.

Finally, stage 3 involved the aggregation of evaluations from the experts and the drawing of conclusions regarding the landscape of art and cultural heritage repositories from a UX perspective.

2.2. Defining Heuristics

The proposed framework uses Nielsen’s (1994; 2005) usability heuristics research as a basis. Using the same general notions, a series of specific heuristics were defined, which included heuristics that were relevant to both web digital repository UX design and to art and cultural heritage content publishing. The total number of devised heuristics was 24, half of which evaluated traditional UX concepts and half of which evaluated modified or new concepts that were relevant to the fields of art and cultural heritage.

Table 1 presents these heuristics and includes their names, the general heuristic concept on which they were based, a short description of what is being evaluated and whether this heuristic is part of traditional UX evaluation or more closely related specifically to art and culture.

Table 1 – List of all heuristics involved in the proposed evaluation framework

NameEvaluated ConceptDescriptionFocus
Interaction feedbackVisibility of system statusGeneral UI indicators of user actions and visual or other feedback to related interactionsTraditional UX
Collection metricsVisibility of system statusIndicators concerning the quantity and availability of items or other published contentArt & culture
Use of user languageMatch between system and the real worldUsage of understandable and non-ambiguous language and termsTraditional UX
UI semiology conventionsMatch between system and the real worldUsage of commonplace icons, colors and imagery that conveys conventional understandingTraditional UX
GLAM AestheticsMatch between system and the real worldUsage of colors, shapes and design patterns encountered in real world galleries, libraries, archives, or museums (GLAM)Art & culture
Real world experience designMatch between system and the real worldUsage of functionality and elements that are derived from the real-world experience of visiting galleries or museumsArt & culture
Free UX flowUser control and freedomGeneral ability to move backwards and forwards during a task through actions like “back”, “undo” or “cancel”Traditional UX
Branching UX flowUser control and freedomAbility to traverse back and forth between different flows through links to relevant or derivative contentArt & culture
Accessibility and responsivenessConsistency and standardsUsage of best practices to make content available to people with disabilities and through multiple screen sizesTraditional UX
Conventional repository functionsConsistency and standardsAvailability of staple features of online repositories such as simple and advanced search, item lists and filtersArt & culture
Content ValidityConsistency and standardsClear and explicit mention of the content’s source, provenance, and usage rightsArt & culture
Connectivity StandardsConsistency and standardsUsage of linked open data or semantic web technologies to connect the content internally and externally.Art & culture
Warnings and constraintsError preventionUsage of warnings and UI constraints to avoid user errorsTraditional UX
Visible functionalityRecognition rather than recallExplicitly visible functionality through menus or other meansTraditional UX
Content related suggestionsRecognition rather than recallSuggestions of themes, categories or eras that are available and might be of interest to the usersArt & culture
Web performanceFlexibility and efficiency of useGeneral website performance as perceived by the usersTraditional UX
Functional simplicityFlexibility and efficiency of useEase of use and immediacy when performing basic tasksTraditional UX
Streamlined for screen sizeFlexibility and efficiency of useUI design that makes optimal use of screen space offering different features or content info based on screen sizeArt & culture
Customization and preferencesFlexibility and efficiency of useAvailability of UX related customization based on the users’ preferencesArt & culture
Signal to noise ratioAesthetic and minimalist designLow usage of aesthetic noise such as decorations or animation that offer no benefitTraditional UX
Highlight of artwork aestheticsAesthetic and minimalist designUI aesthetic design that highlights the aesthetic properties of the items themselvesArt & culture
Content richnessAesthetic and minimalist designUI design that allows rich textual or visual content to be conveyedArt & culture
Solutions or alternativesHelp users recognize, diagnose, and recover from errorsPrompts and links that offer solutions or alternative task flows when users are unable to reach the content they are interested inTraditional UX
High quality assistanceHelp and documentationUsage of contextual help and/or a help system that is searchable and offers concrete solutionsTraditional UX

Most heuristics that focus on traditional UX principles are streamlined versions of staple UX heuristics (Robert & Lesage, 2017; Vermeeren et al., 2010) and focus on elements of the user experience that are inherent in most web-based applications regardless of thematic context. These heuristics were selected based on their relevance to the basic experience offered by online repositories of art and culture, which involves the discovery of works of art or objects of cultural heritage through the processes of searching or browsing. At the same time heuristics that would make more sense in a more complicated web application environment, or in a media outlet or e-commerce context were omitted.

Heuristics based on the specific nature of art and cultural heritage are worth discussing a bit further. The importance for collection related metrics during the search process was established in research relating to art searcher behavior (Pergantis et al., 2022). Through these metrics users may gain a glimpse on the volume of information that might be available, thus allowing them to adjust their behavior.

The importance of matching the real world in order to create an equivalent experience is prevalent in the fields of art and culture, especially in relation to the usage of extended reality technologies such as VR (Ivancic et al., 2016) or XR (Terpstra & Huisinga, 2020), but familiarity can be an important factor for any web related experience, and this includes both aesthetic and functional aspects. Two heuristics were established, one evaluating the online repositories’ usage of conventional gallery or museum aesthetic elements and one evaluating the implementation of structures or functions that are part of the real-world visitor experience, such as thematic spaces and sequential exhibition routes.

The ability to transition between different but related items, themes or visual styles has been proven an important part of the art browsing experience online, through both qualitative (Pergantis et al., 2022) and quantitative means (Pergantis et al., 2023b) and as such a heuristic that evaluates this capability is an important addition to evaluating user control and freedom of a UX. Similarly, following the conventional standards for content validity, which involve the provenance and usage rights of an item, as well as connectivity, which involve the adoption of Semantic web and linked open data principles, both lead to new evaluation heuristics which are based on the nature of art and culture content and the needs of the users interested in it (Pergantis et al., 2023b). Additionally, in order to enable recognition rather than recall, it is useful to provide suggestions concerning categories, themes etc in a manner similar to providing information about the system’s functionality through a menu.

Another important distinction of users searching online for art and culture related content is their preference for desktop or laptop computers over mobile devices (Pergantis et al., 2023a; Pergantis et al., 2023b). This aspect is oftentimes ignored by designers due to the importance of accommodating the vast dominance of smart devices for general purpose web access. The heuristic “Streamlined for screen size” evaluates if the repositories take full advantage of the capabilities of modern computers while at the same time adhering to the principles of proper mobile design. Furthermore, in order to achieve flexibility and efficiency it is important to allow for customization of the UX through user preferences at least to some extent and this also calls for an additional evaluation heuristic which focuses on these capabilities. Finally in terms of evaluating aesthetic design, it is important to focus on the richness of the provided content as well as the UI’s ability to highlight and enhance the aesthetic identity of the artworks themselves.

Overall, the heuristics presented in Table 1 form a solid framework on performing heuristic evaluation on the UX provided by online repositories for content related to art and cultural heritage. The combinational approach focusing on both traditional UX heuristics and new heuristics based on the nature of the thematic context ensures a rigid heuristic evaluation system.

2.3. Evaluation Using the Framework

The heuristic evaluation framework that was detailed above was applied to five different online repositories which provide works of art and objects of cultural heritage. These online repositories are:

  • Artsy

Artsy (https://www.artsy.net/) is the most popular art marketplace, comprised of more than 1 million artworks from a network of over 4000 partners.

  • The Metropolitan Museum of art

The Metropolitan Museum of art (https://www.metmuseum.org/) is one of the most popular museums around the world. Its online version provides access to six of the museum’s historic American period rooms via virtual reality technology that enables internet users to “navigate” the rooms via all-encompassing, three-dimensional perspectives.

  • Harvard art museums

The Harvard art museums (https://harvardartmuseums.org/) are the online version of three museums that are part of the Harvard University. They are comprised of the Fogg Museum, the Busch-Reisinger Museum, and the Arthur M. Sackler Museum.

  • Europeana Collections

Europeana (https://www.europeana.eu/) is a digital library that grants digital access to European cultural heritage information in order to inspire and inform enthusiasts, professionals, educators and researchers.

  • The National Gallery of Denmark

The National Gallery of Denmark (https://open.smk.dk/en/) is the biggest art museum in the country, featuring more than 260,000 pieces.

Table 2 briefly presents the three experts that carried out the evaluation. A short description of their qualifications is provided alongside their current academic attribute.

Table 2 – List of the experts that carried out the evaluation

#DescriptionAttribute
E1Studies in Internet Communication and New Media. Professional experience in computer mediated communication and web development. Research work in the fields of web usability and accessibility and Human – computer interaction.Professor
E2Studies in Computer Engineering and Audio & Visual arts. Professional experience in web and mobile application development and UX/UI design. Research work in the fields of user behavior, cultural informatics, and search engines.Postdoctoral Researcher
E3Studies in Computer Engineering and Audio & Visual arts. Professional experience in UI development and website and graphic design. Research work in the fields of digital aesthetics, web archeology and information extraction.Doctoral Candidate

The experts were provided with the detailed list of heuristics of table 1 and were asked to grade each heuristic with a value from 1 to 5, with 1 indicating the lowest performance in this specific heuristic and 5 indicating the highest. The experts were not presented with specific tasks as it is commonplace in usability testing but were allowed to freely interact with the online repositories until they formed an educated opinion for each heuristic. The minimum number of potential points per repository was 24 and the maximum was 120. Finally, after every evaluation process was completed, the average score for each repository for each heuristic was calculated and rounded to an integer value. The sum of these scores resulted to the total evaluation score for each repository.

3. Results

Table 3 presents the average number of points awarded by each expert for every heuristic per repository, as well as the average total score for every online repository that was evaluated on the basis of the proposed framework for heuristic evaluation of the user experience offered. Moreover, the table presents the average score of all repositories and the total scores per repository for art & culture heuristics and for traditional UX heuristics.

Table 3 – List of all heuristics involved in the proposed evaluation framework

NameEvaluated ConceptFocusArtsyMETHarvardEuropeanaSMKAVG
Highlight of artwork aestheticsAesthetic and minimalist designArt & culture445344
Content richnessAesthetic and minimalist designArt & culture234454
Conventional repository functionsConsistency and standardsArt & culture443444
Content ValidityConsistency and standardsArt & culture454544
Connectivity StandardsConsistency and standardsArt & culture342212
Streamlined for screen sizeFlexibility and efficiency of useArt & culture422433
Customization and preferencesFlexibility and efficiency of useArt & culture412322
GLAM AestheticsMatch between system and the real worldart & culture423353
Real world experience designMatch between system and the real worldArt & culture523223
Content related suggestionsRecognition rather than recallArt & culture442544
Branching UX flowUser control and freedomArt & culture342443
Collection metricsVisibility of system statusArt & culture332453
Signal to noise ratioAesthetic and minimalist designTraditional UX545434
Accessibility and responsivenessConsistency and standardsTraditional UX223243
Warnings and constraintsError preventionTraditional UX332233
Web performanceFlexibility and efficiency of useTraditional UX143243
Functional simplicityFlexibility and efficiency of useTraditional UX344344
High quality assistanceHelp and documentationTraditional UX522323
Solutions or alternativesHelp users recognize, diagnose, and recover from errorsTraditional UX211222
Use of user languageMatch between system and the real worldTraditional UX443223
UI semiology conventionsMatch between system and the real worldTraditional UX343444
Visible functionalityRecognition rather than recallTraditional UX433444
Free UX flowUser control and freedomTraditional UX234223
Interaction feedbackVisibility of system statusTraditional UX232423
Art & culture443834434340
Traditional UX363735343636
Total807569777976

The following chart presents the total scores for each repository. With an average potential score of 72, it is clear that most repositories perform above average, with only one repository dropping below average. Although the overall picture is a slightly positive one, there is much room for improvement, with none of the repositories being close to the perfect score of 120.  Artsy is leading with a score of 80 points. This success might have partly to do with the repositories nature which lies between art and culture and e-commerce. The worst scored repository was the Harvard art museums which performed to a level not fitting to the universally acclaimed name.

Figure 1 – Total score per repository and average

The following chart presents the total scores for each repository per heuristic category. It becomes clear from the chart that overall, most repositories do better on the thematic specific heuristics. This is to be expected to some extent since all four international content providers are high quality institutions and organizations with numerous years of expertise on the field and as such, they are expected to understand the needs of their intended audience. Even so, some repositories (namely MET and Harvard) do not follow this trend and display average scores in both the art & culture related heuristics and the traditional ones.

Figure 2 – Score by type of heuristics per repository and average

Figure 3 presents the values of all twenty-four heuristics for every repository and the average value for all repositories as presented in table 3.

Figure 3 – Score per heuristic for every repository and average

Taking a closer look at Table 3 some specific heuristics stand out. The “Content Validity” heuristic which measures the explicit mention of sources, provenance and usage rights is scored high for every repository. The validity of information is an essential part of the fields of art and culture and as such special care is given to it by international institutions. On the other hand, the “Customization and preferences” heuristic is scored low more often than not. Most providers have not established a personalization system that will allow users to tailor their experience. Similarly, the “Connectivity Standards” heuristic is also scored low more often than not. Some usage of semantic web technologies was noted but it was limited. Most of the evaluated repositories are known to use LOD and the semantic web in their content diffusion initiatives but they have yet to incorporate them in to the websites themselves either as machine-readable elements or as part of the user experience.

Regarding the more traditional UX heuristics, the “Signal to noise ratio” detected on most repositories was high. The repositories offered their content with little to no aesthetic noise, focusing on streamlining the experience with the works of art at its center. The traditional heuristic which was scored the lowest was “Solutions or alternatives” which focused on the effort a UI makes to offer users alternative paths to potentially interesting content when search results were limited or non-existent. Most repositories didn’t even offer basic typo correction functionality on searches, which is a commonplace feature in similar apps from the field of e-commerce.

Conclusions

The heuristic evaluation framework that is proposed in this paper was based on a combination of well-established aspects of traditional UX evaluation metrics specifically in the fields of art and culture as expressed by previous research in this area. This framework may be utilized for an initial assessment of the UX/UI offered by a repository providing content of art and cultural heritage through the world wide web. The methodology was presented in a manner that allows interested parties to enrich or modify the framework to their specific needs, while at the same time establishing a detailed process of evaluation.

With regards to the results of the evaluation of the current landscape, each repository displayed a variation of strengths and weaknesses. This is an indication that for every repository and for the majority of heuristics, there is ample room for improvement through feasible means that are already employed by their indirect competitors in the field. Additionally content providers that are not as established may aspire to imitating or surpassing the experience offered by these institutions, in an effort to achieve more user engagement both in terms of quantity of visitors and in terms of user satisfaction.

Both the evaluation framework itself and the results regarding existing international repositories of art and culture may be used by parties interested in presenting similar content online, as a means to improve their services through self-assessment and through direct or indirect comparison with some of the most important institutions on the globe. The application of the framework established that there is ample room for improvement and this improvement may be to the benefit of both the users themselves and the providers who are aiming for maximum diffusion of their offered content.


Bibliographic references

Al-Khalifa H. S. (2010, October). « Heuristic evaluation of the usability of e-government websites: a case from Saudi Arabia ». In Proceedings of the 4th International Conference on Theory and Practice of Electronic Governance. 238-242.

Balasubramoniam V. & Tungatkar N. (2013). « Study of user experience (UX) and UX evaluation methods ». International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) 2(3), 1214-1219.

Barifah M., Landoni M. & Eddakrouri A. (2020). « Evaluating the user experience in a digital library ». Proceedings of the Association for Information Science and Technology. 57(1), e280.

Britannica Encyclopaedia, The Editors of. (2014). art Collection. Encyclopaedia Britannica. Available online https://www.britannica.com/art/art-collection (accessed September 2023).

Ceccarelli D., Gordea S., Lucchese C., Nardini F. M. & Tolomei G. (2011). « Improving europeana search experience using query logs ». In Research and Advanced Technology for Digital Libraries: International Conference on Theory and Practice of Digital Libraries, TPDL 2011, Berlin, Germany, September 26-28, 2011. Proceedings 1 (pp. 384-395). Springer Berlin Heidelberg.

Ivancic D., Schofield D. & Dethridge L. (2016). « A virtual perspective: measuring engagement and perspective in virtual art galleries ». International Journal of arts and Technology 9(3), 273-298.

Manjoo F. (2017). « How the internet is saving culture, not killing it ». The New York Times, 15.

Konstantakis M., Michalakis K., Aliprantis J., Kalatha E. & Caridakis G. (2017, July). « Formalising and evaluating cultural user experience », In 2017 12th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP) (90-94). IEEE.

Lam D. & Sajjanhar A. (2018, December). « Heuristic evaluations of cultural heritage websites ». In 2018 Digital Image Computing: Techniques and Applications (DICTA) (pp. 1-6). IEEE.

Li R. Y. C. & Liew A. W. C. (2015). « An interactive user interface prototype design for enhancing on-site museum and art gallery experience through digital technology ». Museum Management and Curatorship, 30(3), 208-229.

Lynch C. (2005). « Where Do We Go From Here? The Next Decade for Digital Libraries ». D-Lib Magazine 11 (7/8).

Nielsen J. & Molich R. (1990, March). Heuristic evaluation of user interfaces ». In Proceedings of the SIGCHI conference on Human factors in computing systems (249-256).

Nielsen J. (1994, April). « Enhancing the explanatory power of usability heuristics ». In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (152-158).

Pergantis M., Varlamis I. & Giannakoulopoulos A. (2022). « User Evaluation and Metrics Analysis of a Prototype web-Based Federated Search Engine for art and cultural heritage ». Information 13(6), 285.

Pergantis M., Kouretsis A. & Giannakoulopoulos A. (2023a). « Investigating Online art Search through Quantitative Behavioral Data and Machine Learning Techniques ». Analytics 2(2), 359-392.

Pergantis M., Varlamis I., Kanellopoulos N. G. & Giannakoulopoulos A. (2023b). « Searching Online for art and culture: User Behavior Analysis », Future Internet 15(6), 211.

Rajeshkumar S., Omar R. & Mahmud M. (2013, November). « Taxonomies of user experience (UX) evaluation methods ». In 2013 international conference on research and innovation in information systems (icriis) (533-538). IEEE.

Robb J. (2015). « Prehistoric art in Europe: a deep-time social history ». American Antiquity 80(4), 635-654.

Robert J. M. & Lesage A. (2017). « Designing and evaluating user experience ». In The handbook of human-machine interaction (321-338). CRC Press.

Sivaji A., Abdullah A. & Downe A. G. (2011, May). « Usability testing methodology: Effectiveness of heuristic evaluation in E-government website development ». In 2011 fifth Asia modelling symposium (68-72). IEEE.

Terpstra G. A. & Huisinga L. A. (2020). « Smart spaces: Restructuring art galleries as interactive portals ». In HCI International 2020–Late Breaking Posters: 22nd International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22 (411-418). Springer International Publishing.

Vermeeren A. P., Law E. L. C., Roto V., Obrist M., Hoonhout J. & Väänänen-Vainio-Mattila K. (2010, October). « User experience evaluation methods: current state and development needs ». In Proceedings of the 6th Nordic conference on human-computer interaction: Extending boundaries (521-530).


OpenEdition vous propose de citer ce billet de la manière suivante :
mshbordeaux (14 octobre 2024). Comparative Analysis of the User Experience offered by International Art and Cultural Heritage Content Providers. Maison des Sciences Humaines Bordeaux. Consulté le 8 février 2026 à l’adresse https://doi.org/10.58079/12hl9


Vous aimerez aussi...

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

This site uses Akismet to reduce spam. Learn how your comment data is processed.