Electronic music has a performance problem, and this artist is trying to solve it

During a rehearsal at SXSW last month, Chagall van den Berg encountered an unusual problem: his digital knee escaped the wrong way. "My friend laughed and said," Wow, this is a problem that no SXSW artist has ever had, "van den Berg said. The edge.

van den Berg is a musician who wears motion-tracking gloves and an integral combination covered with sensors that, during this SXSW performance, not only control the projection of a digital avatar that appears behind it, but also almost all instruments. and effect in music and his voice. As she moves on stage, her avatar, floating in space, moves in a synchronized manner. When she extends her arms over her head, audio noises slow down and stutter. Every movement of the hand and body has a cause and an effect, creating a dreamscape impregnated with pop and hypnotic to watch.

Carrying these sensors, van den Berg can import chords and melodies with a flicked hand gesture or distort the video of itself in a whimsical way by raising an arm. Because every movement can create audio or visual changes, its performance is very physical, but elegant and deliberate. "I can do all the moves and look like an air traffic controller," says van den Berg. "It would work, but it's not very performative. All the songs I play have movements that are functional and meaningful. "

van den Berg uses motion tracking gloves in 2017. The performance starts at 3:50.

Born in Amsterdam, the Netherlands, van den Berg was obsessed with both music and computers, but he had always seen them as separate things. She played instruments in bands, was a singer / songwriter and worked traditionally with other producers to create rhythms for her songs. Then in 2011, she entered for the first time in a "real" studio. Watching the producers wander through his songs for remixes, he clicked "I can do it". She taught herself to produce music and soon after, in 2012, she released her first EP. "I immediately noticed how much freedom and independence I had," says van den Berg. "I no longer needed other people to produce my songs. My musical expression became so much more direct because I could just make the sounds I had in my head instead of explaining them to another human being. "

van den Berg had solved a problem for herself, but releasing music created another: how would she interpret the songs she had written? The nature of most electronic tracks meant that she had two options: getting behind a table with a bunch of gear, knobs and faders, or playing an accompaniment track and singing over it. Neither was acceptable to her. "I had that choice," says van den Berg. "Either it was going to be real and live, but boring to watch and away from the audience, or I was playing a recording and could dance on stage. Dancing and being one with the audience was much more appealing, but the musician in me did not really like the idea of ​​singing on a track. So I had a dilemma. "

This dilemma faced by van den Berg is a problem faced by many DIY and electronics artists – how to incorporate movement and expressiveness when playing essentially standing in front of a desk, using an interface that the audience will probably never see? And then make it interesting? Groups like The Glitch Mob use electronic drum kits and Microsoft Surfaces hacked to the public, while several startups, such as Enhancia and Genki Instruments, rely on rings of MIDI controllers. Van den Berg's initial solution was Mi.Mu, a pair of movement tracing gloves created by the musician Imogen Heap. Each glove has nine fully customizable sensors and triggers. Almost all movements can be assigned to any musical setting. You can, for example, lower your arm to add reverb or pinch in the air to add chorus to your voice. Van den Berg was so inspired by the Mi.Mu gloves that she wrote "Sappho Song" the day she received them.

C & # 39; was in 2014 and she had launched a quest for van den Berg to seamlessly bridge the gap between the movements of her body on stage and the control of audio and visual effects. While the gloves allowed van den Berg to control the audio and video with her hands, she wanted to do more: how could she use it? the whole body instead of just his hands? The iteration she is working on now combines gloves with a combination of motion capture, the kind of thing typically used to record the movement of people in video games or movies. For the moment, she has programmed a song to use all the technologies she wears, but the goal is to expand her program to a full one-hour show. It is not an easy task. Along the way, she had to learn C ++, find a company that would lend her sensors (she does not have the system, estimating the total cost at around $ 12,000) and constantly experimenting with new combinations of existing technologies. on all platforms to connect everything. together.

Van den Berg's configuration looks elegant and minimal on stage at SXSW, but a complex network of hardware and software is needed to make it work. She wears Mi.Mu gloves on her hands, then a custom body with 15 Xsens 3D motion tracking sensors covering her arms, shoulders, head, pelvis, legs and feet. On the side are three computers – a Mac laptop running three applications to control playback and audio effects, a Mac mini using the openFrameworks code creation tool to handle all the visuals controlled by the Mi.Mu gloves and a Windows laptop. that runs the combination's sensors and connects them in real time to the Unity game engine using the VR VRee wireless platform. There is also additional equipment, such as microphones and an audio interface. To make sure there is no problem, the show runs on its own Wi-Fi network, which forces sites to disable Wi-Fi from their home or move it to a different frequency to avoid interference. That's a lot. But the result – an accomplishment of five years of work connecting body, voice, music and video in one reactive thing – is incredibly exciting.

The whole process must be much simpler if other electronics musicians adopt it, says van den Berg. "[My setup] That's not how we are going to change the world of electronic music performance, because no one will spend all that money and will not worry about IP addresses and networking during the soundcheck, "explains -t it. "Who wants to tour with three expensive computers and put everything in place during a 20-minute change at a festival? It's crazy. "

Image: Jan Mulders

van den Berg is not the only electronic artist to think about the link between his physique and technology. The musician Laura Escudé also uses motion-based controllers and reactive visuals, but she is one of the most ambitious. And although each hack is selfish in the sense that it brings her closer to the show she has in mind, van den Berg hopes that her experiment will inspire others to think about how their performances can be more immersive, and why they are not. it can make it easier for artists to do the same thing.

"My dream is that I'm not the only one in the world doing this kind of thing," says van den Berg. "This performance of electronic music is becoming more human and more expressive." This means being able to handle all the technical headaches, so that others do not have to.

"I'm a kind of guinea pig," she says with a laugh. "But that suits me."

van den Berg performs at SXSW. Video credit: New Dutch Wave.