In this experimental enactment I created a minimalist stage like set consisting of a chair and a table on which rests a book.
The video below illustrates some of the issues and problems associated with navigating the set and possible interactions between the projected character and the virtual objects.
Problems and issues:
1. Projected body mask and perspective
As the performer moves away from the kinect, the virtual character shrinks in size such that the projected body mask no longer matches the performer. Additional scripting to control the size of the avatar or altering the code in the camera script might compensate for these problems, though there may be issues associated with the differences between movements and perceived perspectives in the real and virtual spaces.
2. Colocation and feedback
The lack of three dimensional feedback in the video glasses results in the performer being unable to determine where the virtual character is in relationship to the virtual objects and thereby unable to successfully engage with the virtual objects in the scene.
3. Real/virtual interactions
There are issues associated with interactions between the virtual character and the virtual objects. In this demonstration objects can pass through each other. In the Unity games engine it is possible to add physical characteristics so that objects can push against each other, but how might this work? Can the table be pushed or should the character be stopped from moving? What are the appropriate physical dynamics between objects and characters? Should there be additional feedback, perhaps in the form of audio to represent tactile feedback when a character comes into contact with an object?
How might the book be picked up or dropped? Could the book be handed to another virtual character?
Rather than trying to create a realistic world where objects and characters behave and interact ‘normally’ might it be more appropriate and perhaps easier to go around the problems highlighted above and create surreal scenarios that do not mimic reality?