Using Robotics And Immersive Technologies To Support WFH Employees

Based on observations, even when the pandemic wanes, the percentage of personnel working remotely will continue to grow. According to a recent enterprise survey by 451 Research, the emerging technology research unit of S&P Global Market Intelligence, 80 per cent have implemented or expanded universal work-from-home policies, and 67 per cent plan to keep at least some remote work policies in place long term or permanently.

However, remote employees may find their organisations’ administrative and technical support capabilities unsatisfactory. Remote support tends to lack the hands-on, interactive, and immersive services that we’ve come to expect from on-site technical and administrative support personnel. As enterprises retool their internal processes to support a post-pandemic workforce, they should explore a growing range of immersive and robotics capabilities – the kind that can deliver high-quality support to remote, mobile, and other nontraditional worksites.

Using robotics to automate remote support

Chief among these immersive technologies are RPA (robotic process automation), EC (embodied cognition), AR (augmented reality), VR (virtual reality), and MR (mixed reality).

RPA has already come to remote work environments. However, this software-centric approach is not the same as having a traditional human administrative assistant close at hand.

In this regard, EC tools, which use sensor-driven artificial intelligence to anchor robotics in physical environments, may bridge the gap by powering traditional hardware-based robots as multifunctional robotic digital assistants. As discussed in this recent article, Facebook researchers are developing EC technology to power a “truly embodied robot” that, among other capabilities, “could check to see whether a door is locked…or retrieve a smartphone that’s ringing in an upstairs bedroom.” Automating physical tasks such as these might boost worker productivity by fending off distractions and interruptions in the remote worker’s day.

Of course, smart robots would need to automatically sense the physical parameters of diverse home and other remote work environments. In that regard, Facebook’s R&D points to technological advances that will make that possible. Specifically, Facebook Research has just open-sourced a new audio simulation platform called “SoundSpaces,” which trains robotic agents to navigate 3-D environments through AI-based processing of visual and acoustic sensor data.

This technology, which the company describes as the “first audio-visual platform for embodied AI,” automatically creates high-fidelity, semantics-infused renderings of virtually any sound source from real-world sensed environments. It enables embodied AI agents to acquire multimodal sensory understandings and develop complex reasoning about objects and places. It runs on Facebook AI’s AI Habitat simulation platform and includes audio renderings from two sets of publicly available 3-D environments: Matterport3D and Replica Dataset.

Using dynamic captioning to augment remote support

AR can support remote work environments in which elements of the physical setting are unfamiliar to the worker. It displays “augmented” descriptive captions in a 3-D format over an employee’s direct camera view of his or her physical environment.

For support applications, AR might dynamically supplement the information in a worker’s remote physical environment with a dynamic display of descriptive labels and guidance. Wearable AR technology could automatically present remote workers with the information they need at the exact moment they need it.

These captions may be prepackaged with the AR-based support application, or they may be automatically inferred from the physical environments using AI-based technologies such as Facebook’s SoundSpaces. For example, some AR-based training applications could display step-by-step instructions for complicated assembly work. This eliminates the need for workers to retrieve that information manually from a desktop computer or paper manuals.

If the captions need to help, remote workers understand human behaviours and intentions (such as in caregiving contexts), technology such as this recent MIT project, may come in handy. Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory have developed a system that can detect and caption the behaviours of people within a room from Wi-Fi and other RF signals. Researchers claim that their approach can observe people through walls and other obstructions, even in complete darkness, and it can learn to detect those people’s interactions with objects, such as a cup of water.

Though this may give people the jitters on privacy grounds, it can be advantageous in support contexts such as remotely detecting when disabled personnel need assistance or when a fully-abled worker is still struggling to master a new physical work skill. The technology, called RF-Diary, can be set up to notify support personnel when the detected behaviours cross some threshold requiring remote intervention.

Using virtualisation to simulate remote support

Remote employees typically lack direct physical access to all or most of the physical business process in which they’re participating.

Simulating the parts of the process that the remote employee can’t interact with directly is where VR comes in handy. It is well suited to support scenarios where important participants, tools, and activities are entirely missing from the remote employee’s immediate physical environment. It may involve avatars to represent unseen personnel and computer-generated imagery to stand in for end-to-end workflows or specific projects, tasks, or outcomes. VR has long been a staple of remote training in medical, engineering, aerospace, military, and other fields. It can provide simulated learning and support capabilities in situations in which remote personnel can cultivate skills within their regular work routines without needing to visit in-person training sites.

Another promising use of VR is to enable head-office support staff to visualise the physical environment of the remote worker interactively. One emerging technology that might be useful in this regard is this Stanford-developed “neural holography” system for 3-D rendering. It incorporates AI that is trained with a camera-in-the-loop simulator to generate high-quality, immersive, 3-D visualisations in real-time.

Then there’s MR, which hybridises AR and VR by seamlessly blending real and simulated work environments. It is well suited to supporting remote environments where some, but not all, essential participants and activities are not always within a worker’s immediate field of vision. This may call for dynamic labelling of those aspects of the physical environment whose meaning is not entirely evident alongside simulation of those aspects that are missing from the environment.

Military and police agencies are using MR training tools in a variety of simulated scenarios, including active shooter, domestic violence, and traffic stops. The blend of physical and virtual environments creates realistic training without endangering the lives of soldiers or police officers.

What’s next

In the coming three to five years, immersive technologies will become fundamental to delivering high-quality administrative and technical support to remote workers. The “bring your own device” revolution of the past 20 years will ensure that many future workers will be very comfortable with immersive technologies such as AR, VR, and MR in their professional lives.

Using these immersive tools in conjunction with at-home robotics devices will enable enterprises to deliver 24×7 support to remote workers that’s almost as hands-on as what they might have received if they’d remained at their company’s traditional offices.