Nickelodeon had an idea for a game, one that featured Dora the Explorer. An iPhone game where computer vision and a smartphone camera could let preschoolers scavenger hunt for everyday objects in the real world. Children could even snap a picture of the things they found and learn the name of it Spanish.
Nickelodeon had such confidence in this idea, that they built a prototype in-house and tested it with preschoolers.
"...are kids this age old enough to understand a tensorflow powered scavenger hunt?"
To find out, Nickelodeon came to Two Bulls, home of Two Moos, a studio known for thinking outside the box with kids media. We were asked to take a stab at validating Nickelodeon's concept with a low fidelity iOS prototype.
And it went over kids heads.
Nick was finding that most kids didn’t know what the camera had to do with the scavenger hunt, or how to use it. A real question came to the table...
← Original prototype from Nick
Sorry J, this was the only picture we had!
Leading the charge in design, I was responsible for researching our audience and identifying design constraints. I assessed Nickelodeon's prototype, deciding where to take cues from the previous solution and where to innovate. I worked with developers and producers to bring a new experience through two rounds of user testing.
← Y'all have got to quit putting these stickies up so high
Limited control and coordination are typical at this stage, with most kids only able to tap and drag. This meant keeping our interactions limited and simple.
Children in this age range fall in the preoperational stage of cognitive development. It's a pretty esoteric topic, but just know we had to be incredibly clear, deliberate, and literal in our script.
Yep, most of these kiddos can’t read yet. Most reading ability starts at age 6. This meant we had to rely heavily on voice-over to communicate and instruct users.
The inital flow I created was the focal point of our first face-to-face meeting with stakeholders at Nickelodeon. It demonstrated to Nick that we understood the past implementation. Giving us a space to introduce new features we were proposing and the hypotheses behind them.
Having laid out the full experience, we realized the pivotal moments all fell in the tutorial. Aligning on this gave us a clear priority for where our focuses would be for this short engagement. With the date of our first testing session set, we started to imagine how Dora might teach a preschooler to use a camera.
In the previous prototype, the tutorial was an animated demo video. By using a static video, Nickelodeon had no way of checking comprehension. Kids either got it, or they didn’t.
I introduced an interactive walkthrough for the tutorial. Using the tutorial to incrementally teach users how to use the camera and model gameplay behavior at the same time.
"...incrementally teach users how to use the camera and model gameplay behavior at the same time"
I also ditched the shutter button. User testing results from Nick showed the button doing more harm than good, being more of a usability issue than functional action.
In practice, Dora began a countdown when kids had found the item and were holding the camera still. This made for a real heartwarming moment when kids counted down in unison with Dora to take a picture.
← Taking a practice picture
← Finding a target item
← A very fancy testing
facility at Nickelodeon
In the first session, right off the bat, we identified some gaps in our script. These were moments when kids weren’t getting any feedback for their actions. We were able to handle that by recording more voice over for edge cases.
An unexpected hurdle came in the very beginning of testing when the facilitator handed the phone to our participants sitting on a couch. We had failed to mention anything around “getting up and moving around” in the tutorial. When it came time to get hunting, kids would point the camera at the teddy bear sitting half-way across the room. Sigh.
Again, some clever script writing and nudging got kids off the couch and moving.
In our final session of testing, we had almost perfect comprehension from all age groups. Completion times averaged much faster than previous sessions. The only obstacle that persisted was the unpredictably of our Tensorflow model. The current Tensorflow Object Recognition API wasn’t accurate enough for our needs. Any improvement in reliability would have to come from training a model ourselves (out of budget, sadly).
The success of this project spurred more concept work with Nickelodeon, having demonstrated our ability to think differently and build quickly.
This one was full of firsts for me. I had never designed for this demographic before, it made me put my assumptions and typical sense of empathy to the side.
It made for an exciting opportunity to teach and shape new behaviors at a very low level, paying close attention to feedback and cues.
This was also my first time working on a game, where I learned how to carry a narrative with voice over and interactive gameplay.
But by far the most valuable and applicable lesson I’ll take from this project is the importance of feedback. Thanks to physics, we expect feedback; for every action, a reaction. But in software, that reaction is must be considered and designed.
"for every action, a reaction...but in software, that reaction is must be considered and designed.''
Good feedback is clear, timely, and intentional, and when done correctly, can get a smartphone-equipped 3 year old tearing around a living room, helping Dora find a teddy bear.
Executive Producer - J Milligan
Lead Designer - Cole Johnson
Senior Engineer - Vitaly Sender
Product Owner - Rani Onyango
Production Assistant - Kelsey Coughlin
Voiceover - Britt Skyers-White