Published: Dec 13, 2023

generative AI guide creates truly immersive, inclusive and hyper-personalised museum experiences

Like its counterparts around the world, National Gallery Singapore (NGS) has been exploring technology-enabled innovations to engage new and existing audiences. Specifically, the Gallery sought out digital transformation to overcome long-standing challenges like non-interactive audio tours, availability of docent-led experiences, and making art accessible to all. With the common goal of using technology to “make tomorrow inclusive”, NCS and NGS jointly created a novel AI guide that can communicate with visitors intelligently. The proof of concept (POC) leverages NEXT gpt, a flexible and open orchestration platform that allows developers to accelerate the development of Generative AI applications based on Large Language Models (LLM). This successful POC phase unveils the potential of Generative AI to revolutionise how people—from connoisseurs to general public— engage with art and history in museums.

There are gaps to bridge in a typical museum visit. Traditional audio guides lack engagement as they only provide one-way communication. While human guides (or docents) are highly valuable, their availability can be constrained by fixed schedules and language barriers. Also, audiences may not connect well with an audio guide or docent due to differing communication styles, or they may be uncomfortable asking questions in a group setting. Thus, there was a need for the Gallery to develop a solution that adapts to a broad spectrum of visitors and their communication preferences, while providing interactive tours that make art and history accessible and relatable to even those who are not familiar with art. 

With a holistic approach encompassing Customer Experience Strategy, Experience Design Behavioural Science, and on-site User Testing, NCS developed an initial prototype of a novel AI Guide (or AI Docent) in the form of a mobile application. This intelligent guide provides a hyper-personalised, interactive and inclusive experience to each and every visitor, with the ability to engage in conversation and provide nuanced responses to questions.
The solution, which leverages NEXTgpt, comprises of:

• Generative AI/LLM: Together with the Gallery, we developed a vast repository of artistic content and personas for the chatbot. Our multidisciplinary team expertly refined the model’s responses through effective prompt engineering. Generative AI is also applied in the generation of suggested questions, and the key challenge lies in ensuring that these questions do not inherently lack answers. To mitigate the occurrence of misleading or irrelevant questions, the team adopts a hybrid approach which involves assessing the suggested questions by comparing them against scholastically generated questions. This evaluation process serves to maintain the relevance of the questions within the context of the conversation and ensures their answerability.
• Automated speech recognition and generation: The docent understands the user’s input or questions and generates coherent and contextualised responses in simple, natural language.

With NCS’s deep expertise in custom hyper-accurate automatic speech recognition (ASR) systems such as ins8.ai, it is possible for an LLM as such to master multiple languages, accents and colloquial expressions.

Snapshot of capabilities:

•    Hyper-personalised
Visitors can decide the delivery of the content in terms of depth and style, with suggestions provided based on the user’s profile and preferences.  

•    Interactive and informative
The AI guide provides information about exhibits and answers questions from visitors in a conversational manner. By understanding contextual data, the guide provides accurate, new and non-repeated information to visitors every time. 

•    Inclusive
Visitors can also choose text-based interactions if audio is not a viable modality. 

The AI Guide is a revolutionary departure from conventional visitor engagement style and serves as a beacon for future museum experiences. 

In a comprehensive two-week visitor testing period, participants demonstrated great receptiveness, embracing the personal AI guide for its personalised interactions. “Very intuitive”, “strong richness of content” and “helps me spend more time at a painting and appreciate it more” were some of the positive comments gathered after the trial. 

The guide’s ability to engage in conversations helped maintain users’ curiosity and sparked interests in art and history. By having a distinct persona, the guide evoked a sense of presence and personability expected of a museum tour experience. 

By making appreciation of art and history accessible and relatable to one and all, NCS and NGS are a step closer to “making tomorrow inclusive and inspiring”, our shared objective for the AI Guide and a productivity-boosting content creation solution to come.

“With National Gallery Singapore’s mission to be a progressive museum that creates dialogues between the art of Singapore, Southeast Asia and the world, NCS’ AI Guide becomes an exciting yet natural extension for the Gallery to engage present and future generations around modern art.” 
– Kevin Lim, Director for Innovation & Technology, National Gallery Singapore