Improvements – Mirroring Piece

An improvement I received when testing my piece was to mirror the video capture. It came to my awareness that at the moment it captures the video the wrong way round, so when users put up their right hand, on the screen their hand goes up on the left. This makes the interaction quite confusing which could deter people from interacting with my piece. I have found this improvement quite difficult to do and have had quite a bit of difficulty with the programming. Through reading online forums, I managed to gain some help from others who had similar problems. Here are the steps I went through and the various problems I encountered:

Screen Shot 2015-01-13 at 12.58.02To start with I managed to get my piece to mirror by adding this code:
Screen Shot 2015-01-12 at 12.23.50

This got the video capture to mirror but meant the brightness thresholding wouldn’t work correctly and so the video capture was in colour. This meant that obstructing the falling objects didn’t work as well as it was harder to detect the threshold on colours. Also, I felt that the black and white capture looked better as the coloured shapes stand out more.

As I was having difficulty, I decided to start my piece from scratch as originally I had used the brightness thresholding example. I felt that starting from scratch would help me to solve the problems easier as I would be writing out all the code step by step, in a way that I understood. I managed to resolve the problem by adding in videoMirror and a for loop to go through each of the video pixels. Then, instead of using the normal formula of x+y*width to access the array of pixels, I set it equal to [(width-(x+1))+y*width].

Screen Shot 2015-01-13 at 10.28.16

This flipped the video capture and the brightness thresholding still worked correctly. However, this caused the an error with the falling shapes. Now, when you put up your right hand to obstruct the shapes, it obstructs them up the left hand side. This meant I had to flip the shapes too, which I managed to do by changing the x co-ordinate of the shapes to (width-1-x).

Screen Shot 2015-01-13 at 13.04.08

Carrying out user testing with my family and friends was very useful and now my piece is mirrored correctly. This shows that I have followed an iterative design process as I created my piece, got feedback and then made improvements. I have learnt a lot more by carrying out this improvement, especially in relation to the advantages and disadvantages of hacking/programming. I feel that now I have programmed from scratch, my knowledge and understanding has advanced and I’m more confident with resolving any further problems I may encounter.  Next, I will go on to explore some other suggested improvements, such as adding more of each shape and altering speeds. Then, I will go on to test my piece in the studio room, capturing full body movement using a white screen.

Processing Forum, 2014. Mirroring a Webcam in a Sketch. Processing [online]. Available from: [Accessed12 Jan].

Processing Forum, 2014. Flip the Video Capture on the X-axis.Processing [online]. Available from: [Accessed11 Jan].

Processing Workshop – Video

In todays workshop, we learnt how to set up basic video capturing. This is useful to me as I need to understand this for my piece. The video capture setup is quite basic but you can add additional code to set it so that if the camera doesn’t work, an image will load instead, which could be useful for my piece. In the workshop, I set up the video capture and set it so if its bright, the sketch goes white and if it is dim the sketch goes black. Here is the code:

Screen Shot 2014-12-11 at 14.36.24

Overall, learning the basics of video capturing in this workshop was useful to me and from this, I will go on to advance my knowledge to learn more about motion and brightness thresholding, of which I will be using for my interactive piece.

Idea For Interactive Piece

The brief we have been set is to…

‘Create a piece of interactive information design for a shared public space, which is intended to elucidate/explain some an idea or concept you perceive as key to our 21st century media experience.’

The interactive element of my piece will be related to motion, I will video capture the public space and detect the brightness threshold of motion. The piece will feature objects that gravitate towards the bottom of the screen; when motion is detected, the objects are then obstructed from falling. The video capture will be in contrasting black and white, with brightly coloured objects, and the people will be captured as silhouette like figures rather than in full detail. The interactive element of my idea gives the users a sense of agency, which hopefully will help to keep them engaged. A piece that relates to my idea is Text Rain, produced by Camille Utterback and Romy Achituv in 1999:

It features coloured falling letters which respond to users motions and can be caught, lifted, and then let fall again.

‘If a participant accumulates enough letters along their outstretched arms, or along the silhouette of any dark object, they can sometimes catch an entire word, or even a phrase. The falling letters are not random, but form lines of a poem about bodies and language’ (Utterback 2014).

I like how the text produces a poem and think that this piece is very effective. For my piece, I could experiment with using different objects, for example I could possibly use shapes, texts or graphics. Relating it to the public space, I feel the idea would be suitable as the piece can be experienced singularly or by multiple people.

The concept that I could apply my piece to is surrealism. Surrealism is a

‘movement, which began in the 1920s, of writers and artists who experimented with ways of unleashing the subconscious imagination’ (Tate, 2014).

Duval (2014) describes surrealism as a form of expression that surpasses realism, hence in this case the surreal can be seen as being better than ‘real’. Surrealists were inspired by Freud, who

‘identified a deep layer of the human mind where memories and our most basic instincts are stored. He called this the unconscious, since most of the time we are not aware of it. The aim of surrealism was to reveal the unconscious and reconcile it with rational life.’ (Tate, 2014).

Thus, surrealism is related to the blurring the distinction between reality(conscious mind) and dreams(unconscious mind). This could link to Baudrillards’ idea of hyperreality, of which I am studying in my Consumer Culture theory unit. He defines hyperreality as the blurring distinction between reality and simulations of reality, of which in the case of surrealism, dreams and imagination could be classed as simulations of reality. Baudrillard (1970, p.12) suggests that

‘Instead of reality, people are treated to simulations involving the constant recombination of various signs, of elements of the code.’

This highlights the blurring boundaries between reality and simulation of reality, of which can strongly be applied to contemporary postmodern culture. It can relate to my idea as users may think it is reality as they are present on the screen, but in fact it is a re-presentation of reality, with the juxtapositioning of random falling objects. This also relates to Saussures’ earlier ideas regarding semiotics: the signifier (object/words/pieces) and signified (concept/meaning), and as Barthes (1964 p.42) suggest:Saussure-Signified-signifier

‘the signified is not ‘a thing’ but a mental representation of the ‘thing’’

My idea could also relate to the concept of audiences, which I studied a lot in my first year theory modules, relating to the changes from passivity to activity, audiences to users and consumption to production (see previous post – Active Audiences).

Next, I will go on to explore some surreal art work and photography that I find of interest, and will relate it to my idea, as well as further exploring my chosen concept of surrealism. I will also go on to develop my processing skills, by watching tutorials and independently experimenting with trial and error, as well as attending the workshops.


Barthes, R, 1964. Elements Of Semiology. New York: Hill and Wang.

Baudrillard, J, 1970. The Consumer Society: Myths & Structures [online]. London: Sage Publications.

Duval, A, 2014. Surrealism: What Is Surrealism? [online]. Available from: [Accessed 27 November 2014].

Tate, 2014. Surrealism [online]. Available from: [Accessed 27 November 2014].

Utterback, C, 2014. Text Rain [online]. Available from: [Accessed 27 November 2014].

Processing – Experimenting with Video

I have been looking at some processing video examples that I find interesting in order to help me develop ideas for my interactive piece. I have also been watching some of Daniel Shiffmans’ processing tutorials in order to improve my understanding of past processing skills that I have learnt as well as develop new skills. I have learnt more about the Capture class, and the steps required to setup video capturing. I am going to analyse some video capturing examples, and explore how I could make it interactive within public spaces.

  • Video Pixelation – Likewise to images, you can also get the pixels of a video capture. Similarly, you have to use for loops and the pixel array formula (x+(y*width)) but in this case you load the video capture rather than an image. An idea relating to video pixelation could be that if people are moving at a certain speed, they will become pixelated, whereas if they are stood still in the space they become in focus.
  • Motion Detection – This is something I would like to incorporate into my interactive piece, but I need to be aware that at busy periods, there will be lots of movement within the public space. An idea related to motion is that when someone walks into the pubic space a new class/object will be created, which then follows the fastest movement within the public space (motion tracking).
  • Colour Tracking – Instead of tracking motion within the space, I could set it so the camera tracks a certain colour. For example, when someone walks past wearing red, an object could follow them on the public screen. However, this would become complex if there was lots of people wearing the same colours.

From looking at these examples, it has helped me to improve my processing knowledge about video capturing and motion detection/tracking. From this, I would like to incorporate motion in order to make my piece interactive, but still need to continue to fully develop an idea. To do this, I am next going to look at some existing interactive pieces that I find interesting.

Processing (2014). Processing [online] Available from:[Accessed 11 November 2014].