Exploration

A man awakes in a bare room. While exploring it, the man realizes that the room reacts to him and that he can transform it. But after a while the room develops a life of its own and resists the man’s actions. Exploration combines a site-specific mixed reality projection with interactive elements. By integrating the spatial environment and the audience into the story, actor, audience, and environment become equal protagonists.

In this research project the CTech student Marcel Bückner examines the interactive staging of films outside the cinema. It was developed in interdisciplinary cooperation with students of Creative Technologies, Screenwriting/Dramaturgy, Acting, Set Design, Animation and Cinematography.

The language and aesthetics of modern film derive from the spatial setup of a movie theatre in which a static audience consumes a story passively. Challenging this caging has a long tradition and became an artistic movement in the 70s with the idea of Expanded Cinema. Artists experimented with film in different environments and forms. At the same time the audience’s involvement was questioned and interactive storytelling investigated, for example with the Kinoautomat.

With the possibilities of modern technology the exploration of environments and interaction for film is more interesting than ever. With algorithmic control, powerful hardware and photorealistic computer-generated content film can merge with, and expand any environment. Site-specific content generation picks up on classic Trompe-l’œil elements in painting and architecture, creating optical illusions of space. Similar to del Caso’s famous painting Escaping Criticism, which escapes the borders of a closed canvas, film can escape the black box of a movie theatre. Similarly, sensors enable a variety of audience participations, creating novel relationships between the audience, the environment and the story.

The interactive mixed reality project Exploration investigates these new possibilities, combining interactive film with the site-specific expansion of the film's environment through Trompe-l’œil effects. The projection creates the illusion of a three-dimensional space on the two-dimensional surface and presents a framework for interactive story elements. The audience’s gestures, movements and sounds control loop-based sequences of the film. For example, in the beginning of the film, a man sleeps leaning with his back to a wall. The audience can wake the man by poking him with their own shadow, which falls from a back projection onto the same surface as the projection of the film. Only when the man is successfully wakened, a linear film sequence follows.

The story is carried out by a single actor who interacts with dance-like movements with the room he is in, integrating the room as protagonist into the story. The story reflects on the phases of a creative process. At first the man analyses and playfully explores the room, leading then to a stressful struggle, with the man being almost swallowed by the room’s closing walls. Only through the encouragement and help of the audience he reaches at the very end a beautiful union with the room and is able to exit it.

All in all, Exploration investigates new paradigms of storytelling and film by combining a site-specific mixed reality projection with interactive elements. Exploration integrates the spatial environment and the audience into the story, making actor, audience and environment equally to protagonists.

For the festival's party Marcel and his company Xenorama also presented the mixed-reality video projections Metamutations.

Hardware: Projector Barco F90-4K13, Kinect 2

Software: vvvv, 3dsMax, Nuke

#mixed_reality #projection_mapping #interaction #film #storytelling

Time and Place: Sehsüchte Festival 2018, 25.-28.04.2018

Advisor: Prof. Dr. Lena Gieseke

[Projects]  [Creative Technologies]