This Taiwanese performance-art piece creates real-time 3D visual sculptures from a dancer's movements

Stereoscopic projections, 3D models, DNA strands, twisted bodies and puppetmaster hands help Hsin-Chien Huang explore the past

Blending complex imagery with dance, 3D printing and Xsens motion capture, The Inheritance has been earning rave reviews for a story about the spiritual pains that remain in families after a state of national unrest.  

Created by Storynest director Hsin-Chien Huang, The Inheritance wrestles with a time when Taiwan was under martial law (1949 to 1987). Closed borders, and the inability to communicate with people outside of Taiwan left many stranded abroad. Aid attempts discovered by the Investigation Bureau resulted in harassment at the home or on the street, leading to a state of constant fear in the populace.

Today Huang is still intrigued by state-sanctioned history and everything it conceals. Through The Inheritance, he grapples with his family’s struggle, using dance and various forms of new media to turn memories into a form of visual sculpture.  

Set in front of a giant screen, the dancers navigate nine chapters of stereographic projections.

The projections are governed by the dancers and the real-time motion capture data they are transmitting.

“We wanted to give the audience a real sense of three-dimensionality,” said Huang. “This way the dancers’ performance seems to extend to the tip of the audience’s nose.”

To achieve this, the team employs the Xsens MVN inertial motion capture suit during the event. With the ability to work outside of a studio, and a 240 FPS capture rate, the team is able to register even the most intricate of details from its dancers.

In The Inheritance, the computer-controlled virtual characters have been mapped to dancer movements, enabling an interactive experience that builds on each character’s unique AI.

Additional premade animation and physics simulations are also loaded into the system, giving Huang the opportunity to use predefined motions when it suits the story.

“The future can only be changed if we invent fresh and sincere ways to understand our past,” said Huang. “I want to find new ways to document my private memories.”

Advertisement. Article continues below

“Exploration is the only way for me to understand my personal history,” said Huang. “I want to use my hands to experience that journey, and by using the technologies like motion capture and 3D printing, I can transform the trails of my hands into tangible sculptures.”

To manipulate the show in real-time, the performance data is streamed to the crew’s rendering and interactive software.

There, the mocap data can be duplicated, mirrored, randomised, delayed, or accelerated in the moment, enabling collaboration between Huang and the dancers, even when communication is impossible. 

Xsens MVN is a full-body, 3D inertial motion tracking system that the company claims wil delivers fast, production-ready 3D data to professional animators. 

Using this, Huang can record and edit the data and then attach it to whatever character he needs to.

“It allows me to replay a dancer’s mocap data again and again, changing what I need to within my interactive program,” said Huang. "It’s an excellent way to collaborate with dancers when I’m not in their presence."

Advertisement. Article continues below

Huang also sees a chance to make something physical out of the projections - he uses a 3D printer to recreate many of the visuals the audience sees in the show. What are interpretations of memories for him become active pieces that people can hold and engage with in a new way. 

“With accurate mocap data, human hands become a 3D scanner,” explained Huang. “Our walking paths record the contours of the terrain we have travelled. When I covert these paths into 3D models and print them out with a 3D printer, these once ambiguous memories become visible. They can be shared and re-touched.”

 The Inheritance is still in development.

The project was sponsored by the Crossover Fund between Digital and Visual Art from National Taiwan Fine Art Museum, and produced by Ms. You-Chiou Chen of White Egret Foundation. The Choreography Adviser was Le Gend Lin of Legend Lin Dance Theatre, and its first performance was presented at the Chiang Kai-shek Auditorium of the old Taiwan Air Force headquarters in Taipei.

The Storynest is a team led by Huang Hsin-Chien. The team creates stories, digital prints, website art, digital music and interactive installations.

Applying digital technology, Storynest combines photography, narratives, music and images into its projects and represents them with unprecedented styles.