Tag Archives: intermediality

13. PopUpPlay

On Thursday 26th February 2015 I attended the launch of Pop Up Play at De Montford University, a free “Open Source” mixed reality toolkit for schools.

The experience of PopUpPlay was described as a hybrid mix of theatre, film, game and playground.

It was extremely refreshing and inspiring to witness the presentation of the project and experience a live hands-on demonstration of the toolkit.

The presentation included case studies with videos showing how children used the system and feedback from teachers and workshop leaders on its power and effectiveness.

Feedback from the trials indicated how  easily and rapidly children took to the technology, mastering the controls and creating content for the system.

What was especially interesting in the light of iMorphia was the open framework and inherent intermedial capabilities presented by the system. A simple interface  enabled the control of background images, webcam image input and kinect 3D body sensing, as well as control of DMX lights and the inclusion of audio and special effects .

The system also supported a wireless iPad tablet presenting a simplified and robust control interface designed for children, rather than the more feature rich computer interface. The touchable interface also enabled modification of images through familiar touch screen gestures such as pinch, expand rotate and slide.

Research Aims (from the PopUpPlay website)

“The overarching aims of this research project were to understand how Arts and cultural organisations can access digital technology for creative play and learning, and how we can enable children and young people to access meaningful digital realm engagement.

In response to this our specific objectives were to create a mixed reality play system and support package that could:

  • Immerse participants in projected images and worlds
  • Enable children to invest in the imaginary dimensions and possibilities of digital play
  • provide a creative learning framework, tools, guides and manuals and an online community
  • Offer open source software, easy to use for artists, learning officers, teachers, librarians, children and young people”

Two interesting observations drawn by the research team from the case studies were the role playing of the participants and the design of a set of ideation cards to help stimulate creative play.

Participants tended to adopt the roles of Technologist, Director, Player, Constructor and Observer. Though they might also swap or take on multiple roles throughout the experience.

DSC_7774

The ideation cards supplied suggestions for activities or actions based on four categories

Change, Connect, Create and Challenge.

Change – change a parameter in the system.

Connect – carry out an action that makes connections in the scene.

Create – create something to be used in the scene.

Challenge – a new task to be carried out.

DSC_7773

An interesting observation was that scenes generally did not last more than 3 minutes before the children became bored and something was required to change the scene in some way, hence the use of the ideation cards.

The use of ideation cards as a means of shaping or catalysing performative practise echoes one of the problems Jo Scott mentioned when a system is too open, that there would be nowhere to go and some shaping or steering mechanism was required.

A number of audience members commented on the lack of narrative structure, though the team felt that children were quite happy to make it up as they went along and the system embodied a new ontology, an iterative process moving between moment to moment which represented a new practise within creative play.

Through the Looking Glass

One of the weaknesses of the system I felt was the television screen aspect where participants watched the mixed reality on a screen in front of themselves, as if looking upon a digital mirror, which tended to cause a breakdown of the immersive effect when participants looked at each other. I felt there were problems with this approach and one of the interesting aspects of iMorphia was the removal of the watched screen, instead one watched oneself from the perspective of the audience. It would be interesting to combine Pop Up Play with the third person viewing technique utilised in iMorphia.

The lack of support for improvisation within iMorphia could be potentially addressed by the Pop Up Play interface. Though the system enables individual elements to be loaded at any time it does not currently support a structure that would enable scenes or narrative structures to be created or recalled, nor transitions between scenes to be created in the form of a trajectory. Though advertised as OpenSource, the actual system is implemented in MaxMSP which would require a license to be able to modify or add to the software.

Though very inspiring, I was viewing the system from the perspective of questioning how it might be used in live performance. Apart from the need for a hyper structure to enable the recall of scenes another problematic aspect was the need for the subject to be brightly illuminated by a very bright white LED lamp. This is a problem I also encountered when testing out face tracking, it would only work when the face was sufficiently illuminated. The Kinect webcam requires sufficient illumination to be able to “see”, unlike its inbuilt infra-red 3D tracking capability. This need for lighting then clashes with the projectors requirement of a near dark environment. Perhaps infra-red illumination or a “nightvision” low lux webcam might solve this problem.

12. Live Performance

Inspired by the intermedial performance work of Jo Scott, I am beginning to formulate an outline for a series of experimental live performance experiments as a means of testing the hypothesis as to whether it is possible to evoke the uncanny through intermedial performance. Intermedial being used here to highlight the mutually dependent relationship between the performer and the media being used.

Jo Scott improvises with her technology as a means of being present and evoking liveness, giving her the ability to move her performance in any direction at any time, responding to feedback from her system, the generated media and the audience. In comparison the iMorphia system as it currently stands does not support this type of live improvisation, characters are selected via a computer interface to the unity engine and once chosen are fixed.

How might a system be created that supports the type of  live improvisation offered by Jo’ s system?  How might different aspects of the uncanny be evoked and changed rapidly  and easily? What form might the performance take? What does the performance space look like? What is the content and what types of technology might be used to deliver a live interactive uncanny performance?

How does the performance work of Jo Scott compare to other intermedial perfomances – such as the work of Rose English, Forced Entertainment and Forkbeard Fantasy? Are there other examples that might be used to compare and contrast?

I am beginning to imagine a palette of possibilities,  a space where objects, screens and devices can be moved around and changed. An intimate space with one participant, myself as performer/medium and the intermedial technology of interactive multimedia devices, props, screens and projectors – a playful and experimental space where work might be continually created, developed and trialled over a period of a number of weeks.

The envisaged performance will require the development of iMorphia to extend body mapping and interaction in order to address some of the  areas of future research mapped out following the workshops – such as face mapping, live body swapping and a mutual interactive relationship between performer, participant and the technology:3way

Face projection, interactive objects and the heightened inter-relationship between performer and virtual projections are seen as key areas where the uncanny might be evoked.

There will need to be  a balance between content creation and technical developments in order that the research can be contained and released.

 

Face tracking/mapping

Live interactive face mapping is a relatively new phenomena and is incredibly impressive, with suggestions of the uncanny, as the Omote project demonstrates (video August 2014):

Omote used bespoke software written by the artist Nobumich Asai and is highly computer intensive (two systems are used in parallel) and involves complex and labour intensive procedures of special make up and reflective dots for the effect to work correctly.

Producing such an effect may not be possible due to the technical and time limitations of the research,  however there are off-the-shelf alternatives that achieve a less robust and accurate face mapping effect including a face tracking plugin for Unity and a number of webcam based software applications such as the faceshift software.

The ability to change the face is also being added to mobile devices and tools for face to face communication such as skype as the recently launched (December 2014) software Looksery demonstrates:

Alternatively, rather than attempting to create an accurate face tracking system, choreographed actors and crafted content can produce a similar effect:

 

 

 

 

10. Fascinate, Evaluate and Praxis

My last post, 9. Improvisation Workshop – Two Performers, was over two months ago in early August, this post, at the end of October is a brief status update.

At the end of August I attended the Fascinate conference in Falmouth where I presented a paper and ran a small workshop on iMorphia. The production of the paper served as a process of reflection and re-evaluation. The possible directions for the development of iMorphia seemed to be continually expanding and it was becoming clear I needed to try and draw the threads together and focus. I was also becoming highly aware of the increasing tension between art making and PhD thesis production. As an artist  I wanted to use iMorphia to make a piece of work, a production a live performance, something that would be surreal and uncanny. My supervisors called for focus and rigour and the production of a paper that might be presented at HCI conferences and also be of interest to the performing arts.

Early on in my proposal I had expressed the desire to use practise as a main component and had found resonance with the Manifesto for Performative Research (Haseman 2006).

How might Art as Mode of Inquiry, a method I had used in my previous research in Computer Related Design at the RCA  (1995-2001) be used as a basis for PhD Research?

Serendipitously I discovered Practise As Research in the Arts (Robin Nelson 2013) and a Kindle version enabled me to make copious notes. This was the method I wished to employ. It is highly recommended reading for any would-be Practise as Researchers, especially those, like me, based in the more traditional positivist, science and empirical branches of academia. After reading this I felt enlightened and equipped with methods and evidence to create a new direction of melding practise with theory – praxis.

The next task was to  find an appropriate research field where my work might be located, a territory that might offer similar examples of practise and associated theories that I might use to synthesise my praxis.

I began reading the thesis’s of other practitioner researchers, references taking me into Critical Theory, the fuzzy words and worlds of Continental Philosophy (Derrida et al), Object Orientated Ontology and Speculative Realism, Phenomenology and  Machinic Philosophies.

One area  rich area with exemplars of practise performance and technology is that of Intermediality; though also rather sprawling and contested, I found resonance with other practitioners and inspiring examples of practise. One example in particular being the work of Jo Scott, someone I had referred to very earlier on in my early PhD proposal.

Jo Scott is an artist and research student investigating “New Forms of Liveness in Live Intermedial Performance”

“This practice-as-research project addresses liveness in performance and investigates its construction and manifestation, specifically within an intermedial context. Taking as its starting point definitions of liveness posited by Peggy Phelan, Philip Auslander and Erika Fischer-Lichte, the focus of the research is to interrogate such definitions through practice.” (Jo Scott 2013)

In her paper, “Dispersed and Dislocated: The construction of liveness in live intermedial performance”, Scott (2012) discusses how her practice based research informs theories on liveness and intermediality, arguing that two essential elements to creating a sense of liveness is the real-time nature of the technology enhanced performance unfolding in space and time and the unpredictability of its direction at any moment in time.”

In her writing Jo made reference to “Intermediality in Theatre and Performance” (Chapple, Freda and Kattenbelt, Chiel 2006)  though unable to find a copy of the book, an updated online version Mapping Intermediality in Performance was available and here I found a resource full of writings and examples that produced further resonance with my own rather foggy directions of where I wanted to head in my research in ‘multi-modal performance and technology’.

Rather than the traditional approach of producing a written thesis, PaR advocates a PhD submission which includes practise (live performance), documentary evidence of practise (DVD) and “complementary writing”. In addition notes, copies of journals and a website such as this are all recognised as evidence of research.

Rigour not only extends to knowledge of the field (writing, practise) but also evidence of criticality and self awareness of ones own approach to the research, especially making the assumed or hidden approaches of the artist (tacit knowledge) visible and explicit.

I am currently working on my ‘complementary writing’ with references and acknowledgements to practises and theory. How might iMorphia be informed and refined through the discourse of intermediality? Are there  other discourses that I might use? What form might the new practise take? What theories might be imbricated to produce praxis?

Where next? What next?  How?
Theory and Practise = Praxis.