IBM’s ground-breaking AI astronaut assistant can detect emotions

The latest version of the artificial intelligence-powered astronaut assistant, CIMON-2, powered by IBM Watson technologies, was both successfully launched (December 5, 2019) and deployed (February 2020), signifying CIMON-2’s completion of the next step in its research on the effects of stress and isolation during long-term missions.  

CIMON-2 highlights

• It can detect emotions and language tones, and can react empathetically to its conversational partners (and does so by using Watson Tone Analyzer, AI provided by IBM).

• It can reach a specific posit by verbal commands.

• It can take and share photos and videos and show results to an astronaut.

SEE: Cheat sheet: Artificial intelligence (free PDF) (TechRepublic)

The test runs were all demonstrated on board the ISS and CIMON-2, the “ball-shaped and free-flying voice-controlled assistant”–described in an IBM press release–interacted with European Space Agency (ESA) astronaut Luca Parmitano. 

After the Germany-based team reviewed footage of the experiments, “the team was very satisfied with the performance of CIMON-2 thus far, and noted that the new, improved hardware and complex software–compared to CIMON-1–worked very well, a huge achievement for the application of AI in space,” said Matthias Biniok, space tech division leader DACH (Germany/Austria/Switzerland), IBM.

Launched to the ISS in early December from the Kennedy Space Center in Cape Canaveral on supply flight CRS-19, it is scheduled to remain up there for three years. 

Now, post-deployment this past February, the project team completed its initial analysis. The original CIMON, a joint project by IBM, the German Aerospace Center (DLR) and Airbus, went to space in June 2018 and returned to Earth in August 2019, after 14 months on the ISS.

AI responds to voice commands

CIMON-2 was, for the first time, navigated by verbal commands within the Columbus module, regardless of location. It was tested on its autonomous flight capabilities, voice control of navigation, and other tasks. New hardware and software were commissioned, and Parmitano asked CIMON-2 to fly to the Biological Experimental Laboratory (BioLab), located within the Columbus module.  

CIMON-2 “is a pioneering achievement in the use of AI in space,” Biniok said, and added that IBM’s Watson Tone Analyzer, which “allows the robot to detect emotions in conversations with the astronauts, which allows it to react empathically to the astronauts in real time,” represents a major update.

CIMON-2 took photos and videos that it shared with Parmitano. The newest version of the astronaut assistant includes more sensitive microphones and a more developed sense of orientation than the CIMON. Autonomy has increased 30%.

The companionable space robot

A goal of the project is to look into the potential of how an intelligent assistant like CIMON-2 can reduce an astronaut’s stress, as it behaves like a partner and companion–and work associate, as it supports in the extensive work on experiments, maintenance, and repairs. In addition to the stress of work duties, stress could potentially result from long-term isolation or group-dynamics during long-term missions. This benefit could be applied on Earth, too. 

Biniok added, “This [benefit to Earth, too] is especially important right now as the world is experiencing and learning more about isolation amid the COVID-19 pandemic.”   

Also see