How Recent Innovations in AI-Assisted Motion-Capture Technology Might Impact Dance Artists

By now, it’s a familiar behind-the-scenes moment: Artists wearing special suits covered in sensors perform in front of a green screen, while cameras collect data from their body’s motions. Later, that data is transformed into an otherworldly screen animation—from fighting aliens in Avatar to a dancing antihero in Venom: The Last Dance.

Motion capture has been a staple of film, TV, and gaming for multiple decades. And dancers, as expert movers, are often involved in the recording process. But, recently, artificial intelligence has led to advances in performance-capture and animation technology, making it simpler to record, analyze, and digitally re-create a person’s movements. (Those clunky sensors might soon be a thing of the past.) While these changes have the potential to streamline the motion-capture process, they also introduce new questions about the future of artistic production—and dance artists’ role in it.

Moving Forward, With Guardrails

Some new AI motion-capture systems, including Move AI and QuickMagic, are “markerless.” That means that they can recognize key body points—for example, a limb or joint—in a regular 2-D video and use them to generate a 3-D rendering of the performer’s body, without the need for the physical “markers” or suits typically used in motion capture. With the assistance of machine learning, they can then create an animated rendering of someone’s movement.

In the film or video game industries, markerless motion capture can have upsides for the performer. Not only does it require far less set-up time, but it also allows complicated movements and stunts to be captured more true to form, instead of being impacted by the restrictions of a suit. And unlike AI programs like ChatGPT, Dall-E, or Midjourney, most of these motion-capture systems do not create based on text prompts; for now, at least, you can’t ask them to make a video of a dancer jumping. The aim, says Jack Broome, the head of customer success at Move AI, is not to generate movement but, rather, to make animating movement quicker, cheaper, and more accessible. “We’re not looking to replace creators or take anything away from that industry work,” Broome says.

Still, it’s not hard to see why these developments have made performers—especially movement-based artists—worried about their jobs. As AI technology has progressed, SAG-AFTRA has sought to address growing concerns regarding AI use in interactive media projects, like video games, with a new contract outlining AI security, safety provisions, and a wage increase for voice actors and movement artists. Earlier this year, a new contract was signed, focusing on consent, transparency, and compensation, and requiring proper communication and permissions regarding the use of one’s likeness. Since AI can be used to combine performances from multiple movement artists, splicing together the best bits into one motion-capture sequence, the new contract also includes rules about crediting.

“We did not come from an approach of prohibiting the use of this technology,” says Sarah Elmaleh, the interactive media agreement negotiating committee chair for SAG-AFTRA. “Looking back through history, attempting to try and stop a high-speed train when it comes to tech is maybe not the way to go.” And, she notes, some of the union’s members do participate regularly in projects that involve AI motion capture.

In Academia and Beyond

Outside of the entertainment industry, dancers have been using AI tech in combination with motion capture to explore new artistic and research opportunities. Ben Baker, a professor at Colby College, recently had a machine-learning model analyze a data set of dance videos collected by Google researchers, helping to identify the subtle but distinct attributes of 10 different hip-hop dance genres, including krump, breaking, and popping. The model was then able to identify the style of a new piece of movement with 76 percent accuracy, compared to the 38 percent accuracy rate of human subjects. Baker believes his research has potential applications for dance notation, animation, archiving, and training.

“Being able to have someone dance in front of a couple cameras and say, ‘These are some of the ways that their movement changed, these are some of the differences in the positions they get versus the positions you’re looking at now’ ” has great value, he says. “You can even turn that into visual feedback to have a computation model of what an individual’s body might move like if it were a little bit more trained in this genre.” Now, Baker is developing his own dance data set at Colby, using markerless motion capture to record dancers performing various genres. That data will be amalgamated with other available data to train a larger and more capable 3-D model.

Then there are artists like Rashaad Newsome, whose approach to AI motion capture is more targeted. He trained his specialized machine-learning humanoid, called Being (The Digital Griot), on Black feminist theory and abolitionist texts—and also used performance capture to record and incorporate the movements of vogue dancers. His Afro-futurist cyborg­ then became the centerpiece of his 2022 work Assembly, and the 2025 documentary of the same name.

But while machine learning informs Being’s language, the movement itself is animated by Newsome. “I don’t think an AI can create the characters and worlds that I have spent decades creating,” he says. “It would take so long to train it to do it that I just did it myself.”

That’s telling commentary on the limitations of existing AI-assisted motion capture. As Elmaleh says, fully AI-derived representations of humans can feel like a shortcut, eliminating the individual expression of an artist.

“The fact that real human bodies, real human flesh and bone and tissue, are executing these extraordinary flips and falls safely, with characterization on top, is actually really mind-blowing,” Elmaleh says. “The lengths that we go to, the hard work that is required to overcome our conventional limitations to be something greater, is affecting and meaningful.”