This is a project where a team of we as CDM students are helping YDX Innovation, an industry partner who came to us with a problem for children with autism. We are challenged to provide our client with a solution where essentially kids can learn emotions through this robot interacts with YouTube videos as well as have a companion.
Similar projects had been done to different degrees. And the user group is unique, the condition varies from kid to kid, and also out of research ethics, we are not allowed to test on our real users. All our research is going to be based on existing ones, and our deep conversation and learnings from parents, and therapists. Moreover, even though we narrow the users to children with autism aged from 3-5, it is the nature of autism that makes us hard to get a solution, as most of them (80%) is non-verbal, and some of them are having sensory disorders, so having a toy that has a digital screen and robot-like features, creates more problems than it solves.
Meanwhile, it is the our mission to combine the digital and physical experiences together, so that a toy can both be huggable and intelligent.
Miigo is not a therapist, but Miigo can be a facilitator. Miigo creates an experience where digital and physical solutions are combined, helping children with autism and their families to simplify, digitize and eventually transform their day-to-day lives. That is why Miigo has a toy-looking exterior, fluffy and a bit strange, in the shape that many children like. Also, Miigo is designed to bring the most commonly used methods, visual scheduling and expression learning to families in need. So those children can access similar support provided by professionals. With a familiar look, plus all the digital transformation Miigo has, we are confident that not only does Miigo reflect on real-world interactions, it also takes them to a more engaging and meaningful level.
It is not a surprise that whenever caregivers need to build or track a plan for their kids, the card-based traditional visual scheduler is often hard to use and not customized for their needs. But Miigo gets this solved, using a digital visual scheduling system. So caregivers use the parental terminal devices to make plans with their kids, then put a second phone inside Miigo, carrying out those daily tasks. In this way, those children will have the same experience to check what’s next and what’s finished. Other than visual scheduling, Miigo TV and Learn are also intelligent, which connect with a YouTube extension and detect facial expressions in videos, and essentially, help children understand emotions. Research suggests that visual scheduler plays an important role in therapy sessions and their daily lives for children with autism and their caregivers. And emotion training is an essential struggle in many families. Hence Miigo makes the experience frictionless and delightful for both kids and their caregivers.
There have been many innovations that try to tackle the life-saving matter, to support children with autism by new technologies. But it is also true that many of them are not accessible, either because of the technology used, or the price tag attached. So it is our intention to make Miigo a helpful facilitator, and also cheap to get. Miigo uses washable and children-friendly material, with easy-to-make and easy-to-get 3D printing components, such as phone and LED holder. And families can just put their own cell phones inside Miigo, using Miigo App to get the full experience we created for them. Moreover, when diving in details, the digitized visual scheduling system on Miigo, allows parents to create their own plans; the emotion training extension, helps children understand facial expressions better. And these are the exciting new pieces that will only be more powerful when technology progresses. Hence, Miigo is innovative in terms of pricing, digital experience, and also future-update-friendly.
Children are our future. And fighting autism is a life-saving matter. As mentioned, the unique price point and digital experience would help Miigo go to the market with a hit. As long as the technology behind the digital experience is much improved and implemented in Miigo, so it can understand more expressions, and talk like a human. So ultimately, Miigo can help so many families on the waitlists, and change their lives. With that in mind, after our first prototype, our client YDX Innovation decided to produce a hundred Miigos to test its manufacturing possibilities. They also reached out to health professionals to do more user tests with the children with autism network. What’s more exciting, is that YDX is also working with Google Aisa on this particular project. It is safe to say that space for Miigo to advance is unlimited.
Project Manager - Fan Wu
Product Manager / UX Research - Abe Nito
UX/UI Designer - Shawn Deng
Industrial Designer - Ghazal Janeb
Developers - Ryan Knauer / Jater Xu
Music Producer - Jater Xu
Project Supervisor - Rachel Ralph, PhD
The target audience is divided into three groups:
First Group: Therapists
This group of audience is always looking for taylor-made products to facilitate training sessions for children with autism. Miigo, with the features of the visual scheduler, watching Youtube video and emotion recognition training, is designed to facilitate therapy sessions and has the potential to be the best fit for therapists in the market.
Second Group: Parents
This group has a high demand for specially designed products like toys and robots for children with autism. And this group is also the possible buyers of this product once it is on the market. Miigo and its features would help parents to work on daily activities with their kids, and also facilitate training sessions with the potential assistance of premade videos.
Third Group: Children with Autism
The third group of our target audience is whom we designed Miigo for. This is specifically intended for those who have high-functioning autism (HFA) that would serve as an interactive buddy that could aid them in some of their daily activities and emotional needs.
Conducted user interviews with parents, robotics experts and therapists who have been working with children with autism for years. Here are some of the discoveries and research takeaways:
Focus: To understand general behaviours and patterns of kids and the perception of parents
Parent: Mom - with 2 boys with no autism, ages 3 and 6
Parent: Dad - with 1 boy with no autism, age 5
Focus: To understand their design process and to gain useful tips
Associate Professor - worked on Calmer, an effective pain management tool for premature babies
PhD Student - worked on CuddleBits, furry handheld affective robot pals
Focus: To understand the behavior of children with autism, therapy activities and learning tools used such as visual schedulers and zones of regulation
Developmental Assistant - works closely with children with autism and has a 5 year old daughter of her own with no autism
Zones of Regulation: Teaching emotions and regulation
Social Stories: A way of teaching kiddos rules, expectations and what to expect
Visual Supports for Autism: https://autismawarenesscentre.com/visual-supports-best-way-use/
Provincial Psychologist - for 3-5 year old kids with autism (early intervention), specializes in engagement behaviors and emotional development
Online academic sources with key takeaways particularly on emotion recognition, use of robots for autism therapy, and machine learning and customization of toys for this use.
Essentially, there are three groups of audiences that are divided into two terminals and the user experiences are different under each scenario. These are parents and therapists who ideally monitor Miigo through a terminal app. Kids can play with Miigo with and without the digital face as a toy.
For parents and therapists, their job is to make sure Miigo is set up properly, and ready to use. There is a separate parental dashboard app that allows supervisors to check their existing plans or build new ones. By default, there are system events that people can choose from, or even make their own by uploading/taking photos or videos to best represent what is in their households, and in such a way, kids are going to understand the daily plans even better.
It is recommended that supervisors should work on schedule planning with kids.
There is no way for now to cancel the plans that have already been set, it has to be modified through the parental app. This is open for future updates, and we would like to see parents and kids working on their plans together and making changes is part of this process.
For children, Miigo is supposed to be used under the supervision and assistance of a parent or therapist. There are two ways to interact with Miigo. Play as a toy without a phone, or use it under supervision from a parent or therapist to set up the digital visual scheduler.
Kids can browse through and check their plans, for both todo and done. And by clicking on each event, they can open it and start doing to activity. They can finish or ask for help when needed. Visual plans are set in order, according to research, children with autism are unlikely to skip an event or events, so we acknowledge that part and make it part of the design flow.
In this case study, user interface, or UI, is referred to digital screen designs, from the phone being inserted as Miigo’s face, the LED screen, and to the parental app that works separately on other phones.
For Miigo’s face, we use Samsung S7 as the default size for the prototype. But it is changeable later for future edits. Unlike the other robot toys that have black or dark backgrounds with simple facial expressions, we particularly designed Miigo’s face in a human-like way, and to match with the physical part, we expect children to treat Miigo as a cute, huggable, relatable companion, not only a machine-like robot.
Research shows that by showcasing vivid faces, kids with autism would have a better idea when it comes to expressing feelings and emotions.
The visual scheduler is where we digitalize normal card-based visual plans onto a screen. We want kids to have a place to check their todos, and what they have finished during their sessions with Miigo.
There is a fixed menu on the left and is used to switch from todo to done. For all finished events, the icon opacity will drop and there will be green checkmarks.
It is also clear to see that we designed our plans with minimalistic icons, to avoid overstimulation.
There is a separate screen for ongoing event, which also features the actions for going back to plans, moving to the next one, finishing and asking for help. However, as stated in the UX section, it is by default that children would follow the sequence, but in case they want to skip one or more events, the reminder page with an exclamation mark will show up, then lead the kids back to the schedule.
LED is another essential part to add extra interaction for better product experience. As for now, we followed the zone of regulation that is used by professionals to guide and teach children with autism. We incorporate these colours into our design as well.
White is the breathing light when Miigo is on, which indicates a neutral state. For emotions and feelings like sad, sick, tired, bored and others, LED turns blue. The green light is for happy, calm, feeling okay, and is a sign of ready to learn. And for feelings like frustrated, worried, losing control and more, we use the yellow light. Moreover, LED changes to red when indicating mad, angry, terrified feelings.
LED can also display simple icons to provide additional information, like time to sleep, a reward, a surprise, and many more.
Parental App, or Miigo Parents, is a terminal for supervisors to monitor Miigo’s plans and status. It is a conceptual design at this moment, it should be implemented further as needed.
Miigo Parents has a dashboard where key metrics are displayed, like how long Miigo has been turned on, how many tasks achieved with Miigo, so on so forth.
There is a switch that controls the connection between Miigo Parents and Miigo digital face. Whenever Miigo is set up with the other terminal ready, parents can turn it on and start Miigo Plan, the visual scheduler.
As of this moment, visual scheduler is the most important part to be included in this app. There are two ways of planning daily activities. First, this app provides icons by default, that basically represent some life events for parents to choose from. Those icons should be simple, and easy to understand. The other way is more unique, which allows parents to take pictures or videos based on real-life situations to make a plan of their own. It is ideal for kids to look at a plan that has the same thing in their household. They would feel much more relatable. Also if they may have a problem understanding the plan, the video is way more helpful as their parents can demonstrate what the activity means and how the kids should follow.
By uploading pictures and videos to make unique visual schedules, parents will have more control over the customization, and after consulting with experts, this is indeed a special approach that is worth exploring in the future.
For all icons that are used in this project, they are open-sourced and powered by Alibaba. For more information about this icon font, please visit https://www.iconfont.cn/
Roboto is the default font that we use on digital designs. It is free and can be found here on Google Fonts. https://fonts.google.com/specimen/Roboto
There are three major colours we use for our digital designs, matched with other light colours that may occur on visual schedules.
Voice plays an important role in Miigo. It has voice over and must convey clear and caring instructions. For now, we use IBM’s Watson Text to Speech service (https://www.ibm.com/watson/services/text-to-speech/) to generate synthesized audio output complete with appropriate cadence and intonation.
We created all our audio files here at https://text-to-speech-demo.ng.bluemix.net/, with the American English (en-US): AllisonV3 (female, enhanced dnn).
As of now, these kids only use the touch screen to interact with the prototype. Ultimately, it is great to have natural language processing so that kids and their caregivers can use voice as feedback with advanced AI-enabled voice recognition, just like Google Assistant and other equivalent services.
Based on the concept of the project, from the children’s point of view, it is important that Miigo looks like a toy. Although if we want to place Miigo in a category it might fall under the robot’s category for parents. The crucial aspect of Miigo is the look and feel of the toy and how children can relate to it as a friend and companion.
On the sketches developed in the first part of the project Designer’s attempts was to create a character for children with autism that make them feel that the toy is a huggable and kind creature.
All of the characters in the following sketches are creatures that have small elements of animals as a reminder.
The idea behind the chosen sketch is that the base of the body is a very simple shape and huggable and with the change of simple elements like ears and tail and feet it can be changed into a new character and it is simple to reproduce.
The main focus of choosing materials for this project was to make the toy more huggable and to give it a better look and feel. Another perspective is to make the toy durable and easy to wash for parents, therefore in the process of deciding on materials, the chosen ones were washed to check the shrinkage of them and whether if they pass color to other materials or lose colour in the process of washing.
After that we compared the before and after looks and feels and we narrowed it down to three main materials.
For choosing the stuffing inside and outside material of the toy were tested together to have the most huggable result. For testing this the team made four balls from each of the chosen outside materials and filled them with four different stuffings and asked users to vote for the best one on each category. The most voted ones are shown in the pictures below.
There is a foam layer between the outside fabric and the stuffing of the toy which acts as a structure and is made of a thin layer of foam.
After the foam layer there is a plastic layer which holds the stuffing inside in case the child spittes something on the toy.
For the parts of the toy that are supposed to handle weight the chosen material was the PLA. all these parts are 3D printed.
There are two different patterns presented for making the Miigo’s body there is an inner pattern which acts like a structure and is for a layer of foam inside Miigo’s body. there is an outer pattern that is for the furry fabric of miigo’s body in each of the pictures below the parts that need to be stitched are shown by ... lines and the parts that are supposed to be cut are shown by black lines.
One of the questions of this project is what happens when the phone is no longer attached to toys face? In this case we use printed cards with different Miigo’s facial expressions inside the phone holder. These cards are printed on a thick paper sheet and then laminated.
A build-a-bot (similar to build-a-bear) concept can be explored further where there is an online e-commerce platform where parents and kids can customize their Miigo the way they want and have it shipped to them. Miigo could come in different shapes, sizes and colors.
Another possibility is to include a feature on the parent mobile app where they are able to take photos of actual objects or areas they could use to enhance the visual scheduler for their children. This could make it easier for kids to recognize activities and objects and not to mention, fun.