Star Wars: Jedi Challenges
A smartphone-powered augmented reality experience by Disney and Lenovo. The product hardware and gameplay both were originally intended to be developed in-house as the next phase of Disney Playmation.
Project Type Wearable AR toy with companion app
Timeline Dec 2015 - July 2016
Users Star Wars fans, ages 13+
Involvement R&D, fit testing, playtesting, competitive analysis, game design, control schemes, AR menu design
Star Wars Competitive Analysis
Our lofty goal was to make something totally new in a world already filled with decades of Star Wars games. I researched the Star Wars digital landscape, analyzing everything from browser games to Lego Star Wars to VR experiences. I presented my findings to all of the Jedi Challenges teams from product to engineering, including:
- An overview of what games and experiences were already out there as a reference for our teams.
- Identification of key commonalities and design patterns that could be leveraged while iterating on ideas for future phases of development
- Interesting gaps in the current landscape that we could potentially turn into opportunities.
Fit Testing
The product was originally supposed to include the Force Band: a wristband which would enable the player to carry out gesture-based moves. During fit tests, I would walk kids through trying on a few versions of a wristband. I was looking for comfort, quality of fit, and ease of use in putting it on and taking it off. I quickly realized that you can’t just ask a child which option was their favorite, because nine times out of ten, the politic answer you'd get was “all of them.” Instead, I found that asking the kids to rank the options from best to worst led to more critical thinking on their part and more honest, helpful answers for our team.
Augmented Reality Explorations
Early on, there was a desire for the hardware engineering design to be carried out in-house, but unfamiliar augmented reality hardware would bring with it a number of new challenges for the players of our games.
Due to differences in people's interpupillary distance (IPD), a user might not be able to see anything at all when they first don the headset. Measuring the distance between the wearer’s pupils is what head-mounted VR/AR displays use to position camera output clearly, reducing eye strain. What audio instruction and control input was most effective in leading a user through the calibration process? I conducted a series of internal tests for our team to gain a better understanding of these needs around IPD calibration.
Using the learnings from these tests, I proposed control schemes and button placement for the AR headset and controller, accounting for the unique challenges of AR technology.
Headset Proposal
Headset will act as the console, used to access system controls and settings. The headset will mostly be used at start/end of gameplay session. Mid-play usage should primarily be only if something has gone wrong, or if the player is ready to end their session.
| Control | STATE: in-game | STATE: system menu |
|---|---|---|
| L1 | opens system menu; pauses gameplay | select/confirm |
| R1 | gameplay function | back/cancel |
| L2 | wheel or toggle to adjust volume | scroll; directional controls for IPD calibration |
Requirements
- Must be able to function without any peripherals connected
- Minimize how much the player needs to use the headset controls, since overuse could lead to risk of (a) arm fatigue and (b) knocking tracking out of alignment
- Needs to be usable even with a saber in one hand
- Pause and IPD calibration are critical functions that must be easy to access (and must be able to access IPD calibration even if visuals are misaligned)
- May need physical volume controls depending on how audio is implemented
- A wheel or similar bidirectional control (e.g. touchpad) with two directions enabled (i.e. back/forward or left/right) will let user scroll through options or calibrate IPD with one hand.
- With L1 as both open menu and select, if IPD calibration is the first option in the menu, then the user can just double-click L1 to go directly to calibration.
- With the more commonly used controls all on one side (with R1 being only back/cancel) the player will not have to reach across themselves/switch hands for saber as often.
- Having the buttons located on top may mean the user is more likely to brace the entire headset with their thumb, which could lead to a higher likelihood of stability. However, if the player uses (for example) their left hand to press the right button, that could result in increased instability. The solution proposed here is to have buttons that curve along the edges so the player can press on the bottom or side.
Lightsaber Proposal
The lightsaber will be the controller, used to access in-game settings. It cannot break with the overall look of a lightsaber, due to restrictions from Lucas. Controls must be accessible without too much maneuvering (ideally with one hand), while still not easy to hit accidentally during combat.
| Control | STATE: in-game | STATE: system menu |
|---|---|---|
| A | open in-game menu/pause | gameplay function |
| B | activate Force powers | back/cancel |
| C | select/confirm | select/confirm menu item |
| D | activate saber (toggle/hold) | gameplay function |
Version 1: Saber with Physical UI (see above)
- Saber will act as a controller for gameplay
- Physical controls can be broadly applicable to different types of gameplay, so game designers have more flexibility as play types are determined
Version 2: Saber as motion controller (not shown)
- Saber will act as a controller specifically for gameplay (all system controls in headset)
- Gives player six degrees of motion, hypothetically more intuitive for interaction with AR elements
- Optimally, most controls in this concept would be displayed visually in game (such as tapping on AR elements with your saber for selection, physically moving/walking, etc.)
Results
Star Wars: Jedi Challenges was released to positive acclaim and great sales, and even won a Webby Award for Best Use of Augmented Reality.