ABB robot Yumi debuts as orchestra conductor in Italy

ABB robot Yumi debuts as orchestra conductor in Italy

ABB robot Yumi debuts as orchestra conductor
Andrea Bocelli and ABB CEO Ulrich Spiesshofer with Yumi. (Credit: ABB)

ABB’s Yumi robot leads Andrea Bocelli and orchestra in bravura  finale to robotics festival in Pisa. 

A robot has been trained to conduct Italian tenor Andrea Bocelli and the Lucca Philharmonic Orchestra in a performance of the aria ‘La Donna è Mobile’, from the Verdi opera Rigoletto.

Dubbed Yumi, the robot was designed by Swiss robotics and power company ABB. Yumi was trained for over 17 hours ahead of its moment in the spotlight, in a performance at the grand finale of the International Festival of Robotics held this week in Pisa, Italy.

Following Bocelli’s performance, Yumi continued conducting as soloist Maria Luigia Borsi sang the classic soprano aria, ‘O mio babbino caro’ from Gianni Schicchi by Puccini. To conclude, YuMi also conducted a passage from Mascagni’s intermezzo from the opera Cavalleria Rusticana.

Andrea Colombini, director of the Lucca Philharmonic Orchestra, who helped prepare Yumi for the event, said he was excited about the technology.

“Setting up the interaction between the elbow, forearm and wrist of the robot, making use of its versatility in repeated and demanding attempts to break down the upbeats and downbeats, was very successful,” he said, adding that the gestural nuances of a conductor have been fully reproduced to a level he previously thought unimaginable.

Read more: Flippy robot to automate jobs in CaliBurger restaurants

Teaching a robot to conduct

Yumi’s performance was developed in two stages. In rehearsals, Colombini’s movement were captured via a process called ‘lead-through programming’, where the robot’s two arms were guided to follow the human conductor’s motions with great attention to detail; these movements were then recorded.

The second step involved fine-tuning the movements in ABB’s RobotStudio software, and synchronising motions to music. With ABB’s technical expertise, the lead-through programming let Colombini focus on doing what he does best, bringing the music to life.

The opera singer Andrea Bocelli, who has been blind since the age of 12 and thus could not see the robot conducting, but could hear the orchestra’s response to its movements, described the experience as “so much fun.”

“It showed that a robot could really conduct an orchestra – but only with the excellent work of very talented engineers and a real maestro. Congratulations to the team that pulled this off,” he said.

Read more: Robot tax could ease drawbacks of automation

Bravo, Maestro Yumi

According to ABB, Yumi achieved a very high level of fluidity of gesture, with an incredible softness of touch and expressive nuancing.

There is a limit to what Yumi can do, however. After all, it can only replicate a conductor’s movement and not respond to the music or tell musicians in the orchestra to change tone.

While it is unlikely that robots will take over from humans in the symphonic arts, the aim of the exercise was to help develop industrial robots that are easier to use and perform better with less human intervention. ABB supplies industrial robots to customers worldwide, including automotive company BMW and electronics contract manufacturer Foxconn.