In late May, a small crowd at St. Dunstan’s church in East London’s Stepney district gathered for two hours of traditional Irish music. But this event was different; the tunes it featured were composed, in part, by an artificial intelligence (AI) algorithm, dubbed folk-rnn, a stark reminder of how cutting-edge AI is gradually permeating every aspect of human life and culture—even creativity.
Developed by researchers at Kingston University and Queen Mary University of London, folk-rnn is one of numerous projects exploring the intersection of artificial intelligence and creative arts. Folk-rnn’s performance was met with a mixture of fascination, awe, and consternation at seeing soulless machines conquering something widely considered to be the exclusive domain of human intelligence. But these expeditions are discovering new ways that man and machine can cooperate.
How Does AI Create Art?
Like many other AI products, folk-rnn uses machine learning algorithms, a subset of artificial intelligence. Instead of relying on predefined rules, machine learning ingests large data sets and creates mathematical representations of the patterns and correlations it finds, which it then uses to accomplish tasks.
Folk-rnn was trained with a crowd-sourced repertoire of 23,000 Irish music transcripts before starting to crank out its own tunes. Since its inception in 2015, folk-rnn has undergone three iterations and has produced more than 100,000 songs, many of which have been compiled in an 14-volume online compendium.
Flow Machines, a five-year project funded by the European Research Council and coordinated by Sony’s Computer Science Labs, also applied AI algorithms to music. Its most notable—and bizarre—achievement is “Daddy’s Car,” a song generated by an algorithm that was trained with lead sheets from 40 of The Beatles’ hit songs.
Welcome Mistakes
Algorithms can mimic the style and feel of a musical genre, but they often make basic mistakes a human composer would not. In fact, most of the pieces played at folk-rnn’s debut were tweaked by human musicians.
“Art is not a well-defined problem, because you never know exactly what you want,” says Francois Pachet, who served as the lead researcher at Flow Machines and is now director of Spotify’s Creator Technology Research Lab. But, he adds cheerfully, “it’s good actually that art is not well defined. Otherwise, it would not be art.”
The generated lead sheet for “Daddy’s Car” was also edited by a human musician, and some tracks were added by hand. “There was pretty much a lot of AI in there, but not everything,” Pachet says, “including voice lyrics and structure, and of course the whole mix and production.”
“The real benefit is coming up with sequences that aren’t expected, and that lead to musically interesting ideas,” says Bob Sturm, a lecturer in digital media at Queen Mary, University of London who worked on folk-rnn. “We want the system to create mistakes, but the right kind of mistakes.”
No Right Answers
“The thing that makes art wonderful for humanity is that there is no right answer—it’s entirely subjective,” says Drew Silverstein, CEO and co-founder of Amper Music, an AI startup based in New York. “You and I might listen to the exact same piece of music, and you might like it, and I might hate it, and neither of us is right or wrong. It’s just different.
“The challenge in the modern world is to build an AI that is capable of reflecting that subjectivity,” he adds. “Interestingly, sometimes, neural networks and purely data-driven approaches are not the right answer.”
Oded Ben-Tal, senior lecturer in music technology at Kingston University and a researcher for folk-rnn, points out another challenge AI faces in respect to creating music: Data does not represent everything.
“In some ways, you can say music is information. We listen to a lot of music, and as a composer, I get inspired by what I hear to make new music,” Ben-Tal says. “But the translation into data is a big stumbling block and a big problem in that analogy. Because no data actually captures all the music.”
