I was inspired by The Weeknd's "Over Now" music video and its psychedelic vibe, and I attempted to capture a similar feeling with this project. I was going for a surrealist aesthetic that still had an underlying story but felt dreamlike and disorienting. I chose "Strawberry Fields Forever" in collaboration with my actress, as we wanted a psychedelic vibe, and that is the entire theme and feeling of this song.
The concept centered on creating maximum creative freedom during editing. I directed my actress to dance around, run into a fake tree, punch the air, and more. I wanted as many varied actions as possible so I could have a wide palette of movements to work with in post-production. The downloaded avatar was a regular person character, which made the most sense for the story (watch it to understand more)!
We recorded all our footage in one four-hour motion capture session at the NYU Tandon Integrated Design and Media Ballrooms. There were no real challenges, as we had great training from our professor Matthew Rader, but it was time-consuming and required precise effort. Using OptiTrack cameras, we recorded into Motive software, capturing a wide range of movements, including dancing, running, punching, and various expressive actions. The goal was to capture as much varied movement as possible to maximize creative freedom during editing.
After recording, we took the data from Motive and imported it into Unreal Engine after creating RigidBody setups. While there was some trouble at this step, I got it to work. The technical pipeline from mocap to Unreal Engine required careful attention to detail, but once the data was properly imported, it opened up endless creative possibilities.
I built the psychedelic environment as well as the house at the beginning of the video. This was a lot of work in both Blender and Unreal Engine 5. I created this brutalist structure for the house, but then for the surreal room, it was a lot of trying to create this liminal, weird, surreal space. It needed to feel both familiar and alien, grounding the motion capture performance in a space that enhanced the song's dreamlike quality.
In Unreal Engine 5, I created the environment, lighting, visual effects, and camera angles. The editing process was quite time-consuming, and Unreal got quite difficult to work with during editing. While I didn't think creating the video would take that much time, it took hours upon hours upon hours of work to edit and render the video, but it came together in the end.
At the end of the project, I was actually somewhat unhappy with it. I felt as though it could have been far better, but as I have looked back upon it, it isn't that bad given it was my first time doing motion capture in a longer setting. It has its faults, but I learned a lot, and I learned a brand new incredible skill that was incredibly fun to play with. The technical challenges, including importing mocap data, creating environments in Blender and Unreal, and managing the rendering pipeline, all taught me patience and problem-solving in a complex 3D workflow. Overall, while it's not my finest work, it's important and interesting work for sure! This project represents my first deep dive into motion capture technology and real-time 3D animation, opening up entirely new creative possibilities for future projects.