Augmented Reality Platform

At USA TODAY, I led the 0-1 creation of an Augmented Reality (AR) platform. This platform was designed to quickly plan, design, and implement spatial experiences. These experiences featured Investigative, historical, entertainment, and breaking news.

I was responsible for the creative design and translating the news team's stories and goals into interactive experiences that users could access through the USA TODAY app.

I produced over 30 experiences for the platform, collaborating closely and guiding a diverse and passionate group of journalists and newsrooms. From 2018-2022, the platform saw high user engagement, increased brand recognition, and was nominated by multiple news and design organizations. My work received 19 industry awards in storytelling, tech, and design.

Planning AR Experiences

I worked with journalists to research their story, breaking their reporting down into digestible chapters with opportunities for user exploration and participation. From there I constructed an interactive script, walking through the entire experience, annotating how we needed to direct the user, and what they would be doing at every point of the experience.
These Interactions used a prefabricated component UI/UX suite that I designed and created when developing the initial platform. These components were a series of user facing interactions and prompts to help them navigate their space and the experience.
These interactions used a mixture of screen and world space interactions depending on the action needed. The majority of controls and text were assigned to screen space accessible for the user, as we did not want to further stress any users with barriers to mobility, dexterity, and vision.

This interactive narrative script would then be storyboarded using my design system and UI wireframe components within Sketch/Figma. Both the storyboard and sketches were reviewed by the engineering team and the stakeholders for any adjustments required.

 Building Assets

Once I had stakeholder approval, I would research the unique assets required for the project. These included 3D models, textures, thematic UI components, illustrations, and audio. 

Art assets were constructed from schematics, historical documents, and photographs that I had to research and compile. In some cases, photogrammetry was used for more accurate representations when we had access to the subjects.

More immersive experiences accompanied audio for narration, perspective, and guidance. I ran interviews with subject matter experts, sourced historical recordings, and worked with the journalists themselves for narration that enhanced the story needs of the project.
For the project, "1619", I used historical documents and references to build a scale representation of the hull from the type of ship that trafficked enslaved African people to the Americas. This representation was set to be a full room scale experience, where the user would stand inside of and walk around the space.

The experience also features authors that I collaborated with who provided audio narration for the guidance of the story. Users stood within the ship and explored the hull as they heard the stories about these people, providing further perspective to this harrowing point in history.

When designing this large space experience, I organized the space to have room to look around, but everything interactable for the experience was within a few feet of the user. Not everyone has the same size of space to access, so I needed to make sure that the experience was usable in a range of small and large spaces so the user wasn't blocked by objects in their real space when trying to move closer to examine objects within the  AR scene.

This was additionally useful when considering users with mobility and fatigue issues.
I also created custom 3D/2D animations on these projects, so we could have more control over walking through the experiences and providing engaging moments for the user to participate with.

For SpaceX's Falcon 9, and NASA's Apollo 11, I leveraged references and schematics to accurately create rockets that could operate and separate as they did on their launches.

These assets had to be extremely performant, as these experiences were streamed into the users devices, and AR can be hardware intensive for mobile devices. This meant that 3D assets and graphics had to be optimized and simplified to reduce the device struggling to process, draining battery power, and heating up the users hand.

Experimental Media

Seven Days of 1961 is an intense historical retelling about the Freedom Riders. The brand for the project was an illustrative format, so I created a 2D graphic novel within 3D space, where the user is within a 3D representation of the bus that the Freedom Riders rode within, and the 2D illustrations are staged 3 dimensionally. This use of mixed media earned the "Best use of visual journalism and storytelling tools" award from the International News Media Association (INMA).
Toward the end of my time at USA TODAY, I worked on demonstrating interactive data visualization experiences, with a series of projects focused on the impact of climate change and rising sea levels. 

This leveraged transitional environment models to show the impact and damage of rising seas, as well as data graphs and slides the user could interact with to  show the change in conditions.

If you wish to learn more about these projects, take a look below!

Example AR Projects