Star Wars: Jedi Challenges is a smartphone-powered augmented reality experience by Disney and Lenovo. The product hardware and gameplay both were originally intended to be developed in-house as the next phase of Disney Playmation. During this R&D phase, I led fit testing and playtesting efforts, explored the competitive Star Wars game space, and worked on game design, control schemes, and AR menu design.

jedi-challenges-kylo-ren.jpg

Star Wars Competitive Analysis

As part of our iterative R&D efforts, I researched and analyzed the Star Wars digital landscape, everything from browser games to Lego Star Wars to VR experiences. This accomplished the following:

  • Provided an overview of what games and experiences were already out there as a reference for our teams.

  • Identified key commonalities and design patterns that could be leveraged while iterating on ideas for future phases of development

  • Unearthed gaps in the current landscape that we could potentially turn into opportunities.

^ Back to top

FIT TESTING

Early on, we had planned for the product to include a wristband which would enable the player to carry out gesture-based moves. During fit tests, I would walk kids through trying on a few versions of a wristband. I was looking for comfort, quality of fit, and ease of use in putting it on and taking it off. I quickly realized that you can’t just ask a child which was their favorite, because the answer you’ll usually get is “all of them.” Instead, I found that asking the kids to rank the options from best to worst led to more critical thinking on their part and more honest, helpful answers for our team.

I also conducted a series of internal tests for our team to gain a better understanding of the needs around IPD (interpupillary distance) calibration. Measuring the distance between the wearer’s pupils is what head-mounted VR/AR displays use to position camera output clearly, reducing eye strain. Seeing as a user may not be able to see anything when they first don the headset, I wanted to make sure we were able to lead a user through the calibration process without visual instruction.

Back to top

Augmented reality UI & Control Schemes

Headset Proposal

  • Considerations and requirements:

    • Must be able to function without any peripherals connected

    • Minimize how much the player needs to use it, since overuse could lead to risk of (a) arm fatigue and (b) knocking tracking out of alignment

    • Needs to be usable with a saber in one hand

    • Pause and IPD calibration are critical functions that must be easy to access (and must be able to access IPD calibration even if visuals are misaligned)

    • May need physical volume controls depending on how audio is implemented

  • Headset will act as a console, used to access system controls and settings > separation of system menu and game menu, as in any modern console

  • Will mostly be used at start/end of gameplay session. Mid-play usage should primarily be only if something has gone wrong. This will minimize necessary usage.

  • Having the buttons located on top may mean the user is more likely to brace the entire headset with their thumb, which could lead to a higher likelihood of stability.

    • However, if the player uses (for example) their left hand to press the right button, that could result in increased instability.

    • One possible solution is to have buttons that curve along the edges so the player can press on the bottom or side.

  • A wheel or similar bidirectional control (e.g. touchpad) with two directions enabled (i.e. back/forward or left/right) will let user scroll through options or calibrate IPD with one hand

    • Risk: depending on how we implement IPD calibration, this may not be as intuitive as left and right flippers.

  • With L1 as both open menu and select, if IPD calibration is the first option in the menu, then the user can just double-click L1 to go directly to calibration

  • If the more commonly used controls are all on one side (with R1 being only back/cancel) the player will not have to reach across themselves/switch hands for saber as often

 

Control Scheme

STATE: in-game, no menus open

L1: opens system menu (also pauses gameplay)
R1: TBD (gameplay-specific?)
L2: wheel or toggle to adjust volume

STATE: system menu is open

L1: Select/Confirm
R1: Back/Cancel
L2: Scroll (and left/right controls for IPD calibration)

System Menu INFORMATION ARCHITECTURE AND DESIGN

System menu wireframe (v1)

System menu wireframe (v1)

System Menu Hierarchy

  • IPD Calibration

  • Visual Settings

    • Brightness

    • Tint

  • Audio Settings

    • Overall volume

    • Volume of music/dialogue/etc.

  • Devices

    • Set up connections (saber, marker)

    • Battery levels (phone, saber)

    • Do not disturb toggle (phone)

  • Help

    • Tutorials

    • Tips

  • Session

    • Timer (tracks how long you’ve been playing, could prompt user to take a break for eye strain)

    • Change user profile

    • Quit

System menu wireframe (v2)

System menu wireframe (v2)

System Menu + HEADSET Use Cases

  • New user onboarding:

    • Assume peripherals aren’t set up yet

    • Start with IPD calibration

    • Teach gaze selection

    • Marker and saber setup

    • Saber/in-game tutorials

  • Player is starting to feel eyestrain/visuals are looking blurry while playing: double-click L1 on headset to go directly to IPD calibration

  • Saber stops working: headset menu > saber settings to troubleshoot connection

  • Music is too loud and player can’t hear dialogue: headset menu > audio settings

  • Parents are yelling at player to turn it down right now: volume wheel (or audio settings)

  • Sibling’s turn to play: headset menu > change user profile

  • Friend comes over and wants to try it before they get their own:

    • If player 1 doesn’t want friend to affect their progress: headset menu > change user profile > guest

    • If friend has played before and knows how it works: headset menu > IPD

    • If friend has never played before: headset menu > help > reset FTUE

AUGMENTED REALITY LIGHTSABER PROPOSAL

  • Considerations and requirements

    • Must not break with the look of the actual lightsaber (Lucas requirement)

    • Controls must be accessible without too much maneuvering (ideally with one hand), while still not easy to hit accidentally during combat

  • Proposal 1: Saber as Motion Controller

    • Saber will act as a controller specifically for gameplay (all system controls in headset)

    • Gives player six degrees of motion, more intuitive for interaction with AR elements

    • Current concept has a saber activate/deactivate button, as well as a grip squeeze control (possibly for use with the Force).

    • Optimally, most controls in this concept would be displayed visually in game (such as tapping on AR elements with your saber for selection, physically moving/walking, etc.)

  • Proposal 2: Saber with Physical UI (see mockups)

    • Saber will act as a controller specifically for the gameplay

    • Physical controls can be broadly applicable to different types of gameplay (since we don’t have that defined yet)

    • Does not have to be consistent for future Playmation franchises – can be specific to Star Wars

 

STATE: in-game, no menus open
A: open in-game menu (also pauses gameplay)
B: TBD (could be for activating force powers)
C: select/confirm
D: activate saber (toggle or hold)

STATE: system menu open
A: TBD (game design needed)
B: Back/Cancel
C: Select/Confirm
D: TBD (game design needed)