Muse
Creative storytelling platform enabling writers to explore inspirational worlds through mixed reality.
Submitted to Microsoft Research’s 2017 Design Expo
Overview
Microsoft Design Expo is an annual challenge among leading design schools to create a user experience prototype that solves a real-world problem. The 2017 theme was "Intentional Design for Positive Cultural Impact in Mixed Reality." I participated as an MHCI+D grad student at UW. The parts I owned are highlighted below.
My teammates:
Allison Chan, Olivia Thom, Margaret Tung, Maggie Wang
Opportunity
Design a product, service or solution that demonstrates the value and differentiation of Mixed Reality. We focused on writers as an audience, but with the understanding and hope that our concept might benefit other artists or creators as well.
How might we use mixed reality to help writers find inspiration?
Research
We read articles on the elements of storytelling, reflected on examples of storytelling done well in various forms (e.g., film, narrative, screenwriting), and interviewed artists and writers to gain insight on the creative process.
Elements of Storytelling
“It takes time for an image to ferment and see it line up with other images that seem like they could fit together as a story.”
Insights & Decisions
Successful stories are made up of five key elements:
Character, Setting, Genre, Plot, Conflict
Provide some structure, allowing writers to fill in the rest.
"And then..." is boring. “But therefore...” is interesting.
Introduce randomness and spontaneity.
Write what you know.
Take advantage of familiar environments as a backdrop for inspiration.
The Creative Process
“I’d like to be in more environments that let me sit down and write something down and send it to myself to continue later.”
Insights & Decisions
There is beauty in the seemingly mundane.
Manipulate familiar surroundings in novel ways to provide fresh perspective.
Moving around can stimulate ideas.
Encourage physically interacting with scenarios.
Writing is a process, and requires breaks and reflection.
Enable a way to “save” the virtual world to revisit it.
Ideation
We brainstormed various solutions and defined the concept for Muse, using storyboards and narrative to communicate our ideas and gain feedback. We then iterated and fleshed out key interaction moments.
DEFINING THE INTERACTIONS
Activating the system through voice command.
A mixed reality scene in someone's living room.
Concept ideas "scaled-down" holographic worlds
We chose to give writers autonomy through the use of voice commands, allowing them to activate and manipulate the holographic environment. We decided on a set of three physical orbs as gateways into the system (see below).
We wanted to highlight the benefits of mixed reality (versus virtual reality) by keeping the familiar home setting as a base, and by enabling the holographic world to be scaled down for easy observation.
Anchors
Users assign tags to physical orbs representing the character, genre, and setting elements of a story. The two remaining storytelling elements, plot and conflict, are left to the writers’ imagination.
Commands
Verbal commands can alter and influence the holographic elements evoked by the anchors, allowing users to shape, interact with, and dynamically change the mixed reality scenarios.
Mixed Reality
The imaginative world that unfolds integrates with users’ surroundings. Users stay grounded in a familiar setting while holograms bring fresh perspectives and inspiration.
Scalability
Through a simple hand gesture (drawing a bounding box on a surface), users can shrink their virtual environment and position it next to them as a reference while they write. This flexibility allows writers to take their holographic world with them to serve as their muse.
Video Prototyping
To showcase our vision and the nature of the 3D hologram design, we produced a video prototype. It features a writer experiencing a creative block who activates the Muse system in order to gain inspiration. In his mixed reality, a story is born.
My contributions:
Cal loved climbing trees so that he could look down at the world below him. But one day, he decided to look up. He climbed the trees up and up until he reached outer space. He had traveled so far and so long, only to realize the world above was much like the one below, the glow of the sun mirroring that of the campfire, the glistening stars reflecting the sparkles in the lake. Cal had been looking up for such a long time that he forgot how long he had been climbing. Finally, he looked down, and what he saw was like nothing he had seen before...
I wrote the video's narrative story, serving as one example of what Muse can help inspire:
Takeaways
WHAT I WOULD DO DIFFERENTLY
Allot more time for learning the tools: Cinema4D (for 3D modeling) and Adobe After Effects (for video animation and compositing) are complex software, and in hindsight, I would have started my exploration with them in Week 1.
More primary research: I would interview a wider range of writers (creative, non-fiction) to gain a broader perspective on the writing process.
IF I HAD MORE TIME
Build out the UI. This is tricky since the MR landscape is still developing and requires a lot of imagination. I would want to create more visuals (still and/or animated) showcasing user interactions with the system, as our video mainly focuses on the overall aesthetic concept.
Scale to other creative users (e.g., visual artists, musicians).