Hong Kong Story | Human Machine Interactive Concert: Singing Different "Oriental Pearls"

Xinhua News Agency, Hong Kong, July 17th: Human -machine interaction concert: singing different "Oriental Pearls"

Xinhua News Agency reporter Lu Min

"Xiaohe bent to the south, flow to Xiangjiang to take a look ..."

At the sound of the singing, the colorful 3D audio and painted on the big screen "dance" with the melody, the crystal beads are constantly fingering out of the city, seaport, ship sails, moonlight and lighting. Traveling, huge waves rolled out of the rainbow ...

A unique concert was held on the 14th in the Concert Hall of the Hong Kong Hall of Hong Kong. The commander of the Hong Kong Baptist University Symphony Orchestra Pan Minglun. Under the live accompaniment of the live band, more than 300 virtual human vocal chorus "Pearl of the Orient" is equipped with 3D audio and paintings created by artificial intelligence as songs. It is magnificent and magnificent.

Guo Yi, an artificial intelligence expert and vice president of the Baptist University, accepted an exclusive interview with Xinhua News Agency. Xinhua News Agency reporter Wu Xiaochu Photo

"This should be the world's first similar human -machine cooperation performance." Guo Yi Ke, an artificial intelligence expert and vice president of the Baptist University, said, "People and machines jointly create and complete an art work, we have done it."

This is the first transcript of the "Hong Kong Human -Appearance Symbby Art Creation Platform Technology Construction" project of the Baptist University. This project obtained more than 52 million Hong Kong dollars from the Hong Kong Special Administrative Region Government last year to conduct a five -year art and technology research.

Establish an artificial intelligence "choir"

This year marks the 25th anniversary of the return of the motherland in Hong Kong. The project team first thought of the song "The Pearl of the East". "The use of human -machine interaction can be re -created to send our blessings." Guo Yi Ke, who was in charge of the project.

The first challenge was an artificial intelligence "choir" composed of 320 virtual singers.

By collecting the recording songs of professional singers, they extracted and decomposed the main characteristics of vocal music, developed the generating model of vocal music, and successfully created a four -sound chorus. At the same time The virtual singer, "forming" a group.

The virtual singers are not good at singing "singing", they will sing wrong, sing treble, or even break the tone. "The sound is the frequency. It is not too difficult to teach them to learn to sing, but to continuously improve the frequency by adjusting the frequency." Guo Yi Ke said.

During the rehearsal, Pan Minglun, the music director of the Hong Kong Baptist University Symphony Orchestra, was tied to many sensors, and he communicated with the technical team in the audience from time to time. Xinhua News Agency reporter Wu Xiaochu Photo

The second challenge is the human -computer interaction during the performance. On the stage, the command and orchestra is a real person. The virtual singers must learn to "listen" instructions to seamlessly connect with the live performance of the symphony orchestra, which is not easy.

The reporter saw at the rehearsal scene that Pan Minglun took off his suit. His waist, shoulders, arms, and wrists were tied with a strap with sensors. Essence

The role of the sensor is to "simultaneously interpret" the body movement of the command to the artificial intelligence chorus system. "The difficulty lies in this. The gesture of the conductor is not fixed. It needs a vague logic for the system to learn. Starting is chaotic, the more you practice now." Guo Yi Ke said.

After half a year of scientific research and hammer, the immersed artificial intelligence choir successfully completed the cooperation with the real symphony orchestra.

"Oriental Pearl" in the eyes of artificial intelligence

The biggest breakthrough comes from "creation" from artificial intelligence. In conjunction with the artistic conception of the song, the 3D audio and paintings of artificial intelligence independently described the "Oriental Pearl" in the eyes with the imagination and expressive power of the sky.

In this 3D audio -visual with a length of about 4 minutes, nearly 3,000 images are generated by artificial intelligence, and the logic between the pictures is smooth, and it matches the lyrics and melody of "Oriental Pearl". "According to the continuous changes in the content of the lyrics, various scenes are dynamically generated. The lens designed a virtual flight perspective. The speed and height change with the melody, and the volume is high. Say.

The Symphony Orchestra of the Hong Kong Baptist University played in the rehearsal. Xinhua News Agency reporter Wu Xiaochu Photo

At the beginning, the team just allowed artificial intelligence to generate a painting according to the lyrics and simply connect them. With the deepening of exploration, they began to try to "create" three -dimensional videos of artificial intelligence. "This requires a systematic understanding of the lyrics, and the logic must be coherent and automatically generated. At this time, our innovation begins." Guo Yi Ke said.

To do this, we must first let the machine learn to understand the text connotation of the lyrics. The "church" machine understands some corresponding relationships, such as Xiangjiang represents Hong Kong, the image of "pearl" must become the main line of the sound and painting. Based on this, let the machine imagine independently and create by themselves. After the machine completes the creation, it requires people to feedback without beauty and meaning, and help it correction.

"This is the creation of human -machine interaction." Guo Yi Ke said, "We have developed an artificial intelligence -driven media artist model. The" Oriental Pearl "sound and painting is an important step for our goal to the super artificial intelligence artist."

Let technology empower art

In the past artificial intelligence art creation, a traditional idea was machine imitation. Guo Yi can disagree with this. He believes that in the field of art creation, artificial intelligence has its own unique creative ability. With the advancement of human -computer interaction technology, its creation can be different from people, can be understood and appreciated, and may continue to improve with the needs of people's aesthetics. "Scientific breakthroughs often come from concepts first. I think that we Chinese have this ability and do not turn around in the discourse system of others. Our innovation is that the concept has changed, so we have found a new breakthrough point." Guo Yi can say.

Guo Yi Ke (second from right), an artificial intelligence expert and vice president of the Baptist University, was at the rehearsal site. Xinhua News Agency reporter Wu Xiaochu Photo

The first edition of the "Oriental Pearl", the entire image is projected on a bead, and then the beads are opened layer by layer. When "flowing to Xiangjiang to see", it suddenly turned into a bauhinia. "We are all stunned. The concept of" Xiangjiang "will actually be displayed in the form of a bauhinia. It is very wonderful. This is the creativity of the machine." He said.

If you believe that the machine can create, what will the future art look like? This issue excited Guo Yi Ke and his team. Guo Yico believes that technology empowering art, human -machine reached more interaction and exchanges, bringing new possibilities to the development of art.

In the next step, human -machine interaction must explore deeper levels, such as connection consciousness and emotions. "We can encode beauty, let the machine understand beauty, and then make beauty become part of its creation. This is an important direction for our research." Guo Yi Ke said.

Guo Yi Ke hopes that in the future, he can create an opera in cooperation with human -machines to compile, guide, act, and sing. "It is impossible to explore endlessly," he said.

Video reporter: Lin Ning, Liang Jiajun

- END -