Designing for UX in the Virtual City: a Case Study
You’ve all heard of ”Virtual Worlds” in the past, I’m sure. Most of you have probably spent at least some time in one or another, be it Second Life, Azeroth, or San Andreas. Maybe, if you’re old school like me, you’ve even immersed yourself in one of the text-based open worlds of MUDs to slay dragons, orcs, and hobgoblins with your fellow adventurers. But did you know that there is a lot of interest these days in building one-to-one virtual copies of actual physical cities, with a lot less orc-slaying and a lot more ...what? Let's find out.
These digital carbon copies, or mirror worlds as Wade Roush has called them (https://www.technologyreview.com/s/408074/second-earth/), attempt to model buildings, roads, and the topography of the physical world in intricate detail, with the goal of providing urban planners, architects, engineers and city officials with an accurate playground to try out various things virtually before investing money and resources in actually building them. These models are often based on a standard called CityGML (https://www.citygml.org/), which is an open standardized data model and exchange format to store digital 3D models of cities and landscapes. The city models serve their purpose as tools for city planners, but aren't exactly pretty when seen up close. They are not intended as social platforms for people to spend time in, but have a much more utilitarian purpose. (screen capture from Helsinki3D+, https://kartta.hel.fi/3d/)
The Future of City Models
Today, there is growing interest in utilizing modern game engines and automated techniques to build open world city models where citizens, though avatars that they control, can inhabit their city both virtually as well as physically. In the not-too-distant future, people planning to visit a new city might first take a stroll through the virtual downtown of, say, Paris or London, say 'hi' to the locals and familiarize themselves with the place before actually going. And these virtual cities will become a social network in and of themselves, with citizens being able to socialize and interact with each other, as well as businesses who have a virtual presence in the virtual city.
At the University of Oulu, we have been busy building a prototype of such an open and collaborative virtual city, accessible through the computer screen or in an immersive manner using a head-mounted display such as Oculus Rift. But even though the detail of the city model is already pretty ok, nice visuals aren’t enough to convince people to visit and spend time inside the virtual world. Content, as they say, is still King. (Screen captures from the Virtual Oulu project in Unreal Engine 4)
Speaking of content, the sheer amount people produce in various social networking services daily really is mind boggling. Smartinsights.com does a yearly infographic called "What Happens Online in 60 Seconds", where they show just how much content is produced every minute. Trust me, it's a lot. Over 3 million Facebook posts. Half a million tweets. 66 thousand Instagram photos. And so forth.
Every. Minute.
A lot of this content is geo-tagged, too, and relevant to a certain location. All those Instagram photos of food taken at restaurants. All those tweets about a concert in town. All those Tinder matches. This led us to a minor of an epiphany. We have a virtual city - pretty, but devoid of interesting content. At the same time, we have people - both individual citizens and businesses - producing oogabytes (yes, that's a scientific term) of contextually relevant content across social networking sites.
Lightbulb.
In virtual cities, people have avatars. Those avatars represent people, real people, who operate in the virtual city as themselves. That is to say, I'm still me in the virtual city, not an undead-slaying, +10-Hammer-of-Smiting-wielding level 58 Elf Cleric. Therefore, wouldn't it make sense for me to walk around the virtual city and carry my social media feeds with me? Likewise, the virtual city has buildings, and those buildings have facades, and shops, and restaurants, just like they do in the physical world. If I were a store owner, wouldn't it make sense for me to showcase my social media feeds in a very tangible manner on the outside of my shop in the virtual city? Would I even pay good money for the advertisement opportunity?
In other words, could we go from this:
To this:
However, as these early concept illustrations show, adding all those feeds in willy-nilly might make the virtual city feel like that scene in Minority Report where Tom Cruise is walking through the shopping mall and all the targeted ads jump at him:
What, then, would be an acceptable and user friendly way to embed social media content inside the virtual city, both on a personal (avatar) level, as well as environmental (building) level? How should such content be presented to the user so that it doesn't become overly obtrusive or distracting, and make the virtual city seem cluttered? To find out, we designed a study* to understand the anticipated user experience of users in a virtual city that is augmented with various social media feeds and content types.
Designing the Social City: A Case Study
What follows is a summary of the study and main results. A research article with full details was published in the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. Link to the paper is at the bottom of the article.
We recruited 14 participants of varying ages and backgrounds, and invited them into our lab in focus groups of 3-4 people to co-design and co-create a prototype with us. Each focus group session consisted of: 1) familiarization with the 3D City, 2) co-design activity, and 3) concept design walkthrough. All sessions were videotaped for future analysis.
After allowing our participants ample time to familiarize themselves with Virtual Oulu, we began a co-design activity where we gave each participant an A3-sized print-out of an existing 3D scene from the virtual city together with various materials such as post-it notes, glue, colored pens, and cut-outs of social media services’ logos and avatar figures to construct their idea for presenting social media in the Virtual City. Once participants had created their paper sketches, each one was asked to describe and explain their design to the group, and finally participants were asked to discuss and give feedback to each other and highlight the most liked ideas in each design.
As Virtual Oulu does not yet contain functionalities for integrating social media services and content, we created two Visual Design Examples (ViDEs) with illustrative content from various social networking sites (Facebook, Twitter, Instagram, Tinder, etc.). The ViDEs visualized content both on a personal level (user’s own social media and notifications), and on a public level (content from e.g. local businesses, illustrated in the environment). As an overarching principle, both ViDEs combine various social media icons and feeds within one 3D VE, with the goal of allowing users to stay within one environment and not have to shift their attention to another site/device when e.g. a notification requires their attention.
The two ViDEs differ both in terms of information visualization and interaction design:
Concept A: social media and other information such as contact details and opening hours of businesses are presented on 2D billboards, which are always visible and oriented towards the user. Different social media feeds are presented as tabs, which can be changed by tapping. User’s own social media services are presented on his avatar, arranged in a 2D line. When the user opens his/her own feeds, different services are presented on tabs in the lower 3rd of the screen. Other user’s social media feeds are presented as 2D billboards attached to their avatar, and also spread across multiple tabs.
Concept B: social media content is presented to the user in three visualization steps: i) the user sees the brand or logo of various businesses in the environment, illustrated inside white circles and attached to the actual location of the business with a white line; ii) when the user approaches, icons of all social media services the particular business has linked to the 3D City become visible on the rim of the circle; iii) when user is near enough, the circle is replaced with a feed showing a combination of the newest posts from all linked social media sites. The user’s personal social media services are presented on his avatar as a 3D carousel. When the user opens his/her own feeds, they are presented also in a 3D carousel alignment, with all linked services visible simultaneously. Other user’s public social media feeds are presented similarly as other service providers' social media.
We walked the participants through both concepts, and asked them to think out loud and give feedback. We also utilized the laddering technique to prompt participants to provide more in-depth feedback and to to elaborate on their initial answers.
Results and Design Guidelines
So, what did we find? Here is a summary of some of the main findings; in the interest of not making this article overly long, I direct the interested reader to the full research paper (link at the bottom) for a comprehensive list of results and analysis.
Co-design Activity
Participants created rich and detailed designs integrating a heterogeneous set of data sources and features, including public and personal social media, links to websites, general public information, and promotional material from local businesses.
Overall, they appreciated the idea of integrating various social media services and content into the 3D City. The utility value of having all relevant content available through one highly visual service was seen as appealing, and was brought up multiple times during both the co-design task and concept evaluation.
“I think it is not a bad idea to have social media services jointly here within one service, as there are normally billions of places to check [before finding what I’m searching for]” (P7/M).
However, while participants agreed that having such information of local businesses and municipal organizations embedded into the virtual environment was useful, their opinions were divided on integrating their personal social media accounts into the service. As expected, more active social media users felt more positive about also having their personal content shown in the 3D City than people more reserved about sharing their profile in general. In general, however, participants expressed their desire to see their friends’ avatars, posts, and recommendations.
”For example if it is difficult to meet friends due to time constraints for example, then the person could be in two places simultaneously and be able to meet friends” (P7/M).
As the 3D scene on which the participants did their designs showed a main public square that hosts many events around the year in Oulu, several designs included information related to hyperlocal content such as event information:
“…there could be advertising for upcoming event ‘X’ through a video clip or a slideshow of photos pulled from Instagram projected to the facade of a building... and there could be a direct link to Facebook where you could participate or be interested in the event“ (P14/F)
Participants also created two-way interactions between their avatars and local businesses, so that they could for example book restaurant tables and hotel rooms directly, again illustrating the business potential of Virtual Cities.
Visual Design Examples
Our participants liked and disliked certain aspects of both Concept A and Concept B. The tabbed view and 2D alignment of personal social media icons of Concept A was appreciated more than the 3D representation of Concept B. However, the proximity-based 3-step visualization of Concept B was considered better than the always-visible 2D billboard visualization of Concept A. The billboards were seen as cluttering the environment, and participants raised issues related to visual occlusion, especially in crowded streets. An important design guideline for such systems, therefore, is to ensure that embedded content does not occlude the environment in a way that will become disturbing for the users.
“The problem might occur when there is a street with tens of shops, so how would it look when there are ten billboards on top of each other… especially narrower streets would feel very cluttered” (P7/M).
Participants also stressed the importance of being able to filter content, instead of having everything visible all the time, and suggested possible filtering on many levels including spatial, temporal, and contextual. This goes for both public information visualized in the environment, as well as personal feeds or even avatars of friends and strangers. Providing relevant filtering functionalities is another important design guideline derived from our study.
“If you want to just look around without a pop-up jungle, for example to see the city more clearly, there could be some [buttons] which allow filtering what is show” (P7/M).
The possibilities of two-way interaction were also brought up often in the focus groups. Participants expressed their desire to not only passively view information embedded in the environment, but also participate by adding comments and likes, or creating their own content within the virtual world. Further, participants wanted to be able to interact with other users by viewing, commenting, and posting to the user’s linked social media channels directly. These functionalities create their own challenges from a visual design perspective, as navigation and object manipulation in 3D environments is different from traditional 2D environments and the interface should intuitively highlight these affordances in a usable and non-intrusive manner. Specifically, it becomes important not to clutter the view with too many actuators, or provide the possibility to hide them when necessary. This design guideline links directly back to the previously discussed issues of occlusion and content filtering.
“There could be a menu where you could choose certain applications, then it would show, for example, only Instagram content of places” (P6/F)
Who is the Virtual City for?
The question of ‘who’s this for’ was brought up by our participants in the initial discussions, as well as when discussing the two ViDEs. While they often saw the 3D City as a kind of interactive map and valuable to tourists and people new to the city, they also noted the potential for long-time residents to better connect with their city and serendipitously discover places they might not be aware of.
“Not all people are familiar with the city, so markers are needed to display locations and sights” (P4/M).
Further, the convenience of having a comprehensive overview of local events and services in a geographically accurate form was highly appreciated. This goes to show that virtual cities, augmented with social networking services, have the potential to benefit residents and tourists alike. While it is too early to discuss potential ‘killer’ applications, the heterogeneous user base imagined by our study participants shows that virtual cities augmented with social media services have the potential for broad-based adoption.
In Conclusion
Through the user study discussed here, we have learned that integrating social networking services into Virtual Cities is perceived to provide added value, as both residents and tourists can gain in-depth and up-to-date information about various events, businesses, and other topics through one highly visual service. As virtual worlds and virtual reality keep developing further and further, it becomes redundant to maintain existing social networking services as separate from these worlds. This work has been a first step in understanding how this integration might happen. As a next step, we have begun work implementing a functional prototype that incorporates findings from this study. Screenshots from an early demo are shown below:
We have also built a tool that allows us to easily integrate social media content into a virtual city**:
And that, as they say, is that. Thank you for taking the time to read this rather lengthy piece, and I look forward to seeing you, dear reader, in some virtual city, hopefully in the not-too-distant future!
-------------------------------------------
*Kukka, H., Pakanen, M., Badri, M., & Ojala, T. (2017). Immersive Street-level Social Media in the 3D Virtual City: Anticipated User Experience and Conceptual Development. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 2422-2435). ACM. The full paper is available here.
**Badri, M., Pakanen, M., Alavesa, P., Kukka, H., & Ojala, T. (2017). Design, development, and usability evaluation of a system for adding and editing social media banners in the immersive street-level 3D virtual city. In Virtual Worlds and Games for Serious Applications (VS-Games), 2017 9th International Conference on (pp. 102-108). IEEE. The full paper is available here.