Digitization of Indigenous Knowledge for Extended Reality and Culture

As part of this project, Malaysian and German students developed a platform to gain insight into Borneo's cultural diversity and rich intangible cultural heritage. The AR-based »Museum Time Machine« presents the architectural evolution of Sarawak's ancient museum and its use. The team worked for a week to research the seven different construction phases of the museum building and create accurate 3D models. The students integrated the 3D models into the associated augmented reality application, which additionally depicted the history and architectural transformation of the museum from 1891 to 2022. A special focus was also placed on the historic building materials and the climatic environmental conditions of the structure. The team also produced individual audiovisual components for special events to further deepen the interaction with visitors through insights into the history of the building and its cultural use. In addition, a brochure with supplementary information about the museum was created, the pages of which serve as markers for enrichment with digital information. This makes the brochure the basis of an augmented reality application, but it can also stand on its own without augmented reality. The overall application allows users to interactively immerse themselves in the rich history of the museum on the one hand, and explore the cultures of Borneo themselves in the mirror of their museum representation on the other.

The subject matter of this sub-project was largely initiated by Gamia Dewanggamanik, a PhD student at the Glasgow School of Art, and Jembelia Ania, an experienced weaver from Kalimantan. Ania teaches children and young people from her village in the Indonesian part of the island to use these traditional techniques and make their own craft objects. With her, the team was able to evaluate where and how extended reality technologies could be usefully applied in their everyday work and teaching. One of her key concerns turned out to be that weaving motifs in particular was difficult for her to teach, as Ania's embodied knowledge means that the necessary hand movements come off very quickly and are not easily explicated.
In the end, three main components were created during the Summer School: a brochure written in Indonesian and English with over 40 pages that gives an insight into the art of weaving and its background. This can stand alone or be used with the second component, an AR application developed during the Summer School. The native AR application can digitally extend the pages of the brochure and overlay additional media content. This includes schematic step-by-step instructions on how to weave the various motifs, as well as videos recorded and edited by cultural scientists Alwin and Eileen in close collaboration with Ania and Gamia, in which Ania shows how to weave these motifs. A digital "basket creator" allows users to get creative and design individual baskets with selected patterns. A small marker-based AR game playfully conveys the different types of baskets and what they are used for.

It should not be forgotten that two baskets with different patterns were also created as part of this sub-project, the production of which can be traced with the help of the brochure, but also with the augmented reality app.
Both baskets can now also be admired in Berlin.

The theme of this sub-project was the development of an app to preserve the weaving art behind the production of Pua Kumbu and to bring museum visitors closer to the material use, the colors and the motifs of the Pua Kumbu. Pua Kumbu are traditional cotton textiles used for sacred ceremonies and healing rituals by the indigenous Iban communities of Sarawak. Traditionally, the dyeing and weaving processes are subject to certain rules and commandments but also taboos that are closely linked to Iban beliefs. The women of the village communities organize the production process together, while only master weavers carry out the weaving process themselves. The patterns produced represent the dreams of the master weavers. The main concern of the Iban is that today the Pua Kumbu patterns are commercialized and widely disseminated beyond the boundaries of the local village communities, while the traditions, narratives and meanings behind the patterns are forgotten. The student group therefore developed an AR app during the summer school to communicate and preserve the traditions associated with this weaving art beyond village borders.
In the app, a 3D model of a group of islands can be positioned as a digital world in real space at the beginning. Once a user has selected one of the islands after placement, which demonstrates, for example, the production of dyes for Pua Kumbu textiles, the app transports him or her into a lush 3D environment with the Pua Kumbu objects on display, whose color compositions and motifs the app explains. Selecting one of the other islands allows, for example, digital immersion in the model of a longhouse to learn more about the social coexistence of the Iban in the longhouse.

This sub-project of the Summer School dealt with the development of another AR application to explain the martial art Seni Silat. Seni Silat is practiced in many parts of Southeast Asia, particularly Indonesia, Malaysia, Singapore, and Brunei. Seni Silat is a rich and complex part of Southeast Asia's intangible cultural heritage. It is a collective term used to describe various martial arts styles that share common techniques, principles, and concepts. The word "Seni" refers to the artistic and cultural aspects. The practice of Seni Silat therefore particularly emphasizes the cultural and historical significance of martial arts through music, dance and historical costumes. Seni Silat is accordingly practiced not only as a competitive sport, but also as an artistic performance form that is performed at public events and ceremonies such as weddings and festivals. During the Summer School, we had the special opportunity to become spectators of the Sarawak Premier International Silat Championship, which was held simultaneously in Kuching.
Users of the AR app developed by the students first open the application on their smartphones and select a language. While reading the brochure, users can have additional content appear on the pages marked by a special AR icon with their smartphone. First, users are shown the special Silat clothing, with individual parts highlighted and explained by additional information. The application allows users to have additional interactions. For example, by tapping, the user can change the clothing of the figure shown and learn about the different possible garments for Silat. The app also explains some of the central movements of Seni Silat step by step and encourages users to copy them.