The third day of AWE, which took place on Friday, June 3, had a more subdued tone. There was still excitement on the AWE expo floor, and a burst of celebration when the last of the Auggies were announced, but many of the sessions focused more on topics like healthcare, helping people with disabilities, and promoting ethical design and implementation.
Health and Wellness Applications of XR
Healthcare and Wellness was its own track at AWE. Covering this track exclusively would have been a worthwhile effort. But, with so much to see at AWE, only a few talks received the attention that they deserved.
Using XR to Improve Combat Care
The AWE Healthcare and Wellness track kicked off with “VADER: Using AR and VR Technology to Create a Physical-to-Virtual Collaboration Space for Remote Critical Care Assistance and Training.” If you think that’s a mouthful, “VADER” is an acronym that stands for “Virtual Augmented Distributed Engagement Reality.”
The talk was given by James Pinpin and Vivian Hsu, both software engineers at SOLUTE, a tech company that works with the Department of Defense. The project is an effort to enhance care for soldiers wounded in combat that can not immediately be seen by specialists.
“A remote expert surgeon with a VR perspective can lend remote assistance to a novice surgeon with an AR perspective,” Pinpin explained. “VADER will be relying on the most advanced network technology. … We are also designing VADER to work in disadvantaged situations.”
The medic or surgeon in the field uses equipment to generate a point-cloud hologram of the patient that includes biometric data gathered through a smartwatch. This model and information is then available to a remotely located expert, who can make annotations on the hologram visible to the medic with the patient.
“We want to make sure that whatever is done in VR is shown in reality,” said Hsu. “One of our proudest features is our avatar interaction, which does support fully articulated hand gestures.”
Earlier this year, SOLUTE was purchased by Sigma Defense, a systems integrator that Pinpin hopes will make SOLUTE even more available in environments with limited connectivity.
VR to Improve Child Cancer Treatment
Following on the AWE Healthcare and Wellness stage, a project that hit closer to home. “VR Therapy and Family Intervention: Research, Findings, and Practical Recommendations” was presented by Greg Tarnacki and Heather Bucalos, CEO and Board Director respectively of 18Loop, a non-profit organization using VR in pediatric cancer treatment.
“We’re hoping VR will help them recover in a way that will actually impact survival rates,” said Tarnacki. Tarnacki also pointed out that the cure rate for childhood cancer is higher than it once was at around 87%. However, unless that number hits 100, it isn’t high enough.
Around the globe at any given time, 400,000 children are undergoing cancer treatment. In the last 18 months, 46 of those children are using 18Loop. While that number is rapidly increasing, the main goals for the company right now are understanding and optimizing the intervention, as well as raising funds to improve and expand the initiative.
The first person in the program was Bucalos. She discovered VR while exploring pain and stress management solutions in 2018 after surviving cancer herself. In the process of becoming a registered nurse, she began incorporating breathing exercises used by women in labor into her developing VR care routine, which she used when her own cancer came back.
“The hospital room was no longer in my peripheral vision and that really allowed me to escape the stress,” said Bucalos. “[It was] giving me a tool to almost heal myself, in a way.”
Trials are still in the early stages but showing similar results. Of the users so far, 50% report better pain management, 84% report improved mood, and 96% feel better overall using VR during and after their in-hospital treatments.
Using VR to Understand Health Conditions
18Loop uses VR to get patients’ minds off of their condition. VR can also be used to allow someone without a medical condition to experience it for themselves. This was the view presented by Tipping Point Media Director of Experiential Strategy, Brittany Gilbert-DeMarco, and AbbVie Associate Scientific Director, Weston Pack, in “Disease Awareness Through Experiential Virtual Reality.”
“VR’s visualization capabilities are really beneficial because they can demonstrate or simulate conditions that are very difficult to replicate in real life,” said Gilbert-DeMarco. “[You can] literally see through the eyes of the patient.”
The pair worked together on a suite of VR experiences that recreate the effects of presbyopia, a vision problem that eventually affects just about everyone – but only after their mid-forties or so. That means younger doctors can understand what presbyopia is, but not what it feels like.
“We included some educational models … but we also included a lot of engaging activities,” said Pack. “We can also do a simulation of what’s happening in terms of visual acuity.”
Users can adjust the severity of the condition with a sliding scale to see what happens to vision as well as what happens to a model eyeball itself as the condition progresses. They can also do activities like mini-golf, archery, or just a simulated work day with the condition as well as common corrections.
Inclusion in the AWE Developer Track
A number of sessions in the AWE Developer Track also focused on vision, as well as hearing impairment. That includes how to use software developed for XR to help those with vision impairments and how to build XR for users with hearing and vision impairments.
Disability Inclusive Design
XR is inherently visual and definitely involves audio. This is unfair for some users when it comes to entertainment, but as XR is increasingly used in workforce enablement through use cases like remote collaboration, this can be a serious problem.
This was the topic of “Disability-Inclusive Design of XR Collaboration Tools”, presented by PEAT Emerging Tech Lead, Ashley Coffey, and XR Access Accessibility and Marketing Consultant, Meryl Evans. Their talk didn’t only propose building accessible XR collaboration tools for people with disabilities, they propose building these tools for the benefit of the whole community.
This comes about in two main ways. The first is that many of us, who don’t currently have vision or hearing complications now, probably will as we get older. So, in leaving others behind, we may well be leaving our future selves behind. The second is that accessibility tools don’t only benefit the people that they are designed for.
Evans calls this the “curb-cut effect” named for the depressed sections of street curbs. These were originally designed for people in wheelchairs but are also used by people pushing strollers or dollies or doing any number of things. The technological example of this that she gives is that 80% of people who use closed captions aren’t hearing impaired.
The pair presented a number of suggestions to make XR collaboration tools more disability-inclusive for people with vision and hearing impairments, mobility issues, and different neurodivergent conditions. While I would love to include the whole session here, it is more useful to link PEAT’s recent free online toolkit for inclusive XR and hybrid work.
Using XR to Solve Problems From Disability
Back on the AWE Healthcare and Wellness stage, Zappar co-founder and CTO, Connell Gauld, presented “Tackling Visual Impairment With XR.” And, if you’re still waiting on ARPost’s Auggie coverage, sorry for the spoiler in the photo below, but that’s right – Zappar won two awards.
One of Zappar’s main lines is connected packaging. But, it turns out that the system of codes that Zappar has developed to launch their experiences can do a whole lot more. These codes can also hold product information that an app called Zapvision can read aloud.
“Packaging represents this huge source of visually identifiable information that we all take for granted but is partially or entirely inaccessible for people who are blind or partially sighted,” said Gauld. “We’re working with local organizations around the world to test and validate our solution.”
If you’re worried about people with vision impairments finding the Zapcodes, don’t. A demo shown during the talk displayed a phone detecting one such code from so far away that the human eye couldn’t identify it on the packaging. The solution can even detect multiple codes at once and direct a user to a particular item if several items are detected in the same area.
While he can’t give out names just yet, Gauld also said that the company will soon announce a global partner that will be releasing the first Zapvision-enabled product packaging early next year.
Everything Awesome…
“I personally had so much fun thanks to you guys,” AWE co-founder and CEO, Ori Inbar, said in his closing remarks on the main stage. “This is it, folks. Everything Awesome has to end at some point.”
But it’s not really over. We still have our Auggies coverage to write. And, a lot of the demos, interviews, and insights experienced at AWE that didn’t make it into our day-by-day coverage will crop up in our articles throughout the year. Finally, while we don’t have a date yet for AWE 2023, we’re sure we’ll see you there. And, there’s always the AWE.LIVE app in between.
