Sean Kheraj
Last year, I wrote about my early impressions of the possible uses of virtual reality technology for public history and history education. I also led a session in my fourth-year digital history class on virtual reality and its potential for generating a sense of historical presence, an ability to simulate the sensation of standing in past places. I have been somewhat enthusiastic about what this technology can add to museums, classrooms, and other settings for public history and history education.
My focus last year was on smartphone-based VR with stereoscopic viewers (Google Cardboard, Daydream View, Gear VR). This type of VR technology can generate a powerful sense of presence, but the user is limited to rotational movement along three perpendicular axes (pitch, roll, yaw). This is like being a camera fixed in space that can spin around, but cannot move within that space. Tethered VR headsets that use PCs and spatial tracking systems add translational movement (heave, sway, surge) to VR experiences creating six degrees of freedom of movement. These headsets also include tracked motion controllers that can reveal the user’s hand movements in VR environments and enable interaction with 3D objects. Altogether, this is sometimes called “room-scale VR.” The experience is incredibly immersive.
Recently, I put this kind of immersive VR experience to the test by reviewing three examples of public history VR projects that use room-scale technologies. I used an Acer Mixed Reality Headset, part of Microsoft’s line of virtual reality headsets that use “inside-out tracking” in order to achieve room-scale experiences. Two cameras on the front of the headset map and track the environment around me and the motion controllers generate allow me to interact with objects in a 3D space.
What did this add to VR experiences for public history and history education? How best could it be used? What are its limitations? Let’s find out:
Sketchfab
I mentioned Sketchfab in my post last year on this topic. Sketchfab is a web platform for presenting and sharing 3D models. This platform is capable of presenting these models in room-scale VR via the Microsoft Edge browser’s immersive view in Windows Mixed Reality. Several museums and other cultural heritage institutions have made use of Sketchfab to share 3D scans of artifacts in their collections.
For instance, the British Museum scanned a sculpture of the Pacific god, A’a, an artifact the museum acquired in 1890. The video below shows my view in VR as I examined this fascinating sculpture:
As you can see, the 3D model was covered in numbered icons, each of which I was able to point to and activate with the motion controller. Clicking on the icons brought up information panels that highlighted different aspects of the artifact and its history. Beyond that, I was able to use the motion controller to point to the floor and teleport to different spots around the artifact to get new viewing angles. The most immersive feeling, however, came from being able to walk and lean toward the artifact to get a closer view and to look inside its back panel. I was even able to float below the sculpture to see its underside.
The sense of immersion was even more powerful in a fully-rendered 3D environment. Using photogrammetry, museums have been able to generate highly accurate 3D scans of historical buildings and rooms. One of the best examples of this comes from Hallwyl Museum, a preserved nineteenth-century house in Sweden. Several rooms of this house have been scanned with remarkable detail using photogrammetry. In Sketchfab, I was able to view a model of the billiard room and the experience was astounding:
The museum added narration to the experience (which you can’t hear in the above video) to guide users through the space. Again, I was able to use the motion controller to teleport throughout the room and the inside-out tracking allowed me to simply walk around in the space, and crouch and lean to get different views of the environment. Scattered throughout the room were numbered icons which brought up information panels, same as the sculpture of A’a. Overall, it felt like I was actually standing in that room. The tracked movement of room-scale VR simulated the sense of being in that space so convincingly that I occasionally tried to reach out and lean on virtual objects as I moved around. Unfortunately, the video is unable to capture the stereoscopic 3D effect of the experience. The VR headset projects two separate images, one to each eye, creating the illusion of 3D space and tricking my brain into thinking I can reach out and touch the objects in the room.
Remembering Pearl Harbour
While Sketchfab serves as a platform for sharing 3D scans of cultural heritage objects and environments, there are other projects that attempt to convey a guided historical narrative in room-scale VR. “Remembering Pearl Harbour” is a collaborative project of Time and Life VR, the National WWII Museum, and Library of Congress. In three separate acts, users are guided through the events of the Japanese attack on Pearl Habor in 1941 through the eyes of one of the oldest American witnesses, Lt. James Downing. He was postmaster for USS West Virginia, one of the naval vessels attacked at Pearl Harbor.
The experience opens with historical photographs of Downing and scenes from the attack flying past your view as Downing narrates in his own words. You then emerge in an American home on December 8, 1941 (presumably Downing’s home as photos of him can be found throughout the house). On the nearby radio you hear President Franklin Roosevelt’s address declaring war (“a date which will live in infamy”). You are free to walk around the living room either by merely walking within the room-scale space or teleporting. There are a handful of interactive objects scattered about the room. Some are more interesting than others. For instance, you can pick up the Christmas stockings hanging above the fireplace and examine them (not much there). You can also pick up newspapers and copies of Life magazine, some of which you can actually flip through. After a time, you can touch the calendar on the wall and watch the days flip by until it lands on December 24 and you are transported to the kitchen. This time, the radio plays an address by Winston Churchill. The scenes in the house provide some sense of ambiance to the period. The furniture and appliances appear to be appropriate to 1941 and the ambient radio sounds convey a sense of the period.
Following the scenes in the house, you are suddenly dropped into the aftermath of the attack on the American naval base. You look out onto the burning wreck of several naval vessels, including the West Virginia, Arizona, and others. The experience here is also in full room-scale, allowing users to walk around, crouch, and interact with small objects on the ground. The re-creation of the scene is considerably less effective that the house. There are a couple of information panels and voiceover tracks to contextualize the events. Again, you hear from Downing himself. The scene, however, felt static. I kept wondering about the accuracy of the geography and the scale of the objects in my view. Was this how it might have appeared? The soundscape was quite limited. You could hear fires burning and the sound of waves on the beach, but there were no sounds of explosions or the human voices of the people who were there during the attack. In fact, there are no people represented in this VR experience. The scene is odd and empty. In an attack that killed more than 2,000 sailors and wounded hundreds of others, the absence of any people was the strangest and most limited part of the reconstruction.
The final scene takes place in the present. You are transported back to Downing’s house. Again, there are more photographs and magazines with which you can interact. Downing’s voice comes back in a couple more segments of narration before the credits roll.
Overall, “Remembering Pearl Harbor” is a high-quality VR experience. The house scenes in the 1940s are effective at creating a sense of immersion. The sounds and objects fit the scene and adequately establish the period. Walking through the house feels natural. The reconstruction of the attack itself felt lacking. While the views of burning ships could be staggering, the lack of people made the scene feel surreal. The strongest elements of the experience were the narrations featuring Downing. These were based on interviews with Downing himself. Standing in a reconstruction of his house while hearing his voice explain what he witnessed was a powerful combination.
Historium VR: Relive the History of Bruges
Educational VR experiences can be prone to exaggeration and occasionally oversell the technology.
“Experience the immersive story of 1530 Bruges trading routes.”
“Walk around famous locations in medieval Bruges.”
“Go back in time and step into medieval Bruges…”
These are just some of the ways Historium promotes its VR experience about sixteenth-century Bruges. Developed by Interactive Media studio Sevenedge, “Historium VR: Relive the History of Bruges” is part of Historium, a technology-enhanced historical tourist attraction in Bruges. The VR experience is one of many attractions at the Historium that attempt to reconstruct the medieval city as it was in the 1500s.
The VR experience is high quality and impressive. The developers drew from archival materials to build 3D models of a handful of sites and buildings in Bruges as they may have appeared in 1530. Users can select a guided journey through the five recreated sites with narration. This journey begins on a Portuguese caravel headed to the port at Bruges. You are thrust into the crow’s nest of the ship on the open seas joined by seagulls and a crew down below as you learn about trade in medieval Bruges. You then travel by a small boat along the shipping canal into the market with further narration before being teleported to the top of a scaffold high above the Waterhalle, the market. This is one of the more impressive reconstructions of historical buildings in this app. You then lift off the scaffold and fly over to the square with the town hall and the Chapel of the Holy Blood where the journey concludes with a representation of Jan van Eyck’s The Virgin and Child with Canon van der Paele.
Altogether the guided journey is a little under ten minutes long. Users can also choose from a map of the city to independently explore the five different sites in the Bruges. Here, users are restricted to standing in their room-scale space. Unfortunately, you cannot actually walk around the city itself outside of your virtual box. There is a short narration for each site and users can then look around and interact with a few nearby objects, including an archery mini-game. The narration adds some context and the independence to view each site is good. However, there isn’t much more to it. Each site feels hollow. There are ambient sounds of people moving about the city, but there are all but a few actual people represented in these VR reconstructions. I ended up standing around for a while tossing apples at any person I could see in the vain hope of engendering some kind of meaningful interaction. I also threw some cheese into the canal.
Conclusion
Each of these experiences was immersive in different ways. On the level of engagement, there is certainly some novelty to all of them and they would easily grab the attention of students, tourists, and other public audiences.
Historium’s VR experience was designed for a tourist audience and, as such, the experience was short and a little goofy. It is supposed to inform and entertain. The archery mini-game and the ability to toss apples at will are fun, but they added little to my understanding of medieval Bruges.
“Remembership Pearl Harbor” is a more somber experience. The voiceover from Lt. Downing was powerful as it merged with the immersion of the 3D environment. I think there is much more that can be done by blending room-scale VR experiences with oral history recordings. The reconstruction of the attack site at Pearl Harbor showed the limits of this technology for creating an immersive sense of the past. The absence of human bodies or any human activity in the scene created a surreal experience rather than a reconstruction of what the scene might have looked like in 1941. This choice is understandable from both a technological perspective and from the perspective of remaining sensitive to the subject matter.
Sketchfab offered an entirely different kind of VR experience from the other two. Instead of a guided narrative, Sketchfab is an open, exploratory and experimental experience. The nature of the platform allows great flexibility and multiple uses. The Hallwyl Museum’s photogrammetry re-creations were the most realistic and immersive experiences in this entire review. They conveyed the most powerful illusion of traveling back in time. Because the textures within the 3D environment were based on photographs of the space, they felt more authentic. The British Museum’s 3D scans of artifacts showed the greatest potential for teaching and research. Being able to walk around, over, and through the artifacts gave me a tactile sense that could only be achieved otherwise by holding the actual object in my hand. Teachers in archaeology, classics, and other fields that deal with artifacts could have a powerful teaching tool with Sketchfab as a platform for interacting with 3D scans.
To be honest, the use of photogrammetry to capture 3D representations of buildings, rooms, and other environments seems to be the most exciting technology at play in the use of VR for public history and history education. The ability to capture and recreate a real environment in a VR space holds the best potential for future VR users to walk in past places. Room-scale VR can create powerful illusions of immersion in virtual environments. The brain can be tricked into reaching for objects that aren’t there and feeling vertigo from heights while standing on the ground. It can also convey a simulated sense of the embodied experience of standing in past environments. What would it have felt like to stand in a kitchen in an American home in the 1940s? How did it feel to walk around the billiards room of a wealthy Swedish family in the late nineteenth century? How did the town square of a medieval city appear to a viewer standing in the middle of it? The best historical room-scale VR experiences should strive to create that sense of immersion in a past environment. Each of these experiences missed key pieces of the sense of being in a place (smells, sounds, other people, etc…), but where they got it right, it felt like time travel.
Sean Kheraj is an associate professor of Canadian and environmental history at York University. He blogs at http://seankheraj.com.
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License. Blog posts published before October 28, 2018 are licensed with a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License.