Post 28: Other Projected Sets

  I have always loved old barns and decaying architecture, peeling paint and graffiti, and have tons of pictures. 
 So I am also proposing this "Wooden Red Barn" set, using the same basic plywood panels:

  The view looking in with the doors open would be:  

   The reverse view looking out from the inside would be:

  I also came up with a Graffiti Projected Set with metal doors:

   The view looking in with the doors open looking into a big warehouse could be: 

  The reverse view looking out from the inside could be:

Post 27: A set made of Functional doors and windows?

  I have been struggling with ideas for a flexible set that could be used by Mary and Randy as a green screen in a first stage, but also used for a Projection/Dance piece at Light Dreams, and later adapted to other stages, swapping and adding panels. 
  Both Mary and I love doors. She has used them on stage before, as well as frames. I have photographed a lot of them. Open/close, Hide/see through, Behind/in front, Going in/Going out, Leaving/Coming back, a lot of visuals and feelings hinge on doors(pun intended).
   Staying away from hinged screens, what if for a start we used 3 free standing 4ft x 8ft x 3/4" birch plywood panels lined up about a foot apart:

 Two could be cut out for double doors, and one for window shutters, and all the doors and shutters hinged in place using double screen hinges so they can open both in and out.

   The panels will need supports that don't get in the way of the dancers. I propose bolting them with strong angle brackets to some 1/8" x 3" x 36" steel flat bars.
   There could be also two narrow hinged panels to fill the gap between the three main panels. It would look something like this:

   The doors,shutters and narrow "gap panels" could be opened or closed in various ways, and hide or show through to the back wall.
   We could project any kind of image mapped to the set. For example, I made a "Marrakesh Set" mapping these three panels:

 If we close the gaps between the panels and open doors, it could look like this, showing through the doorways into the bright blue courtyard projected on the back wall:
  

 Somebody could crack the shutters open and look out. We could have extra panels with doors and window and mix them in different configurations.
  An idea I like a lot is to turn the virtual set inside out, and project the inside of the courtyard on the front panels and the street outside on the back wall, suddenly reversing the stage 180 degrees while the actors change side :

 The point of view reversal could take place as the "actors" are in the doorways, and would require a 180 rotation in place for them during a very short black out. Walk in becomes walk out.
  Actually, this set feels as much like Mexico or Guatemala than Marrakesh. No wonder actually, the Arabs occupied Spain for centuries after all. How about a Frida Kahlo storyline then? Now that's rich subject, and would make for very colorful sets:

"interesting" costumes:

 and accessories...

   Right down my alley…

Post 26: What Next?

   Well, now that Mike has figured out the basics, we need to refine it, and I would like to actually participate. He used a Game Building software called Unity, that ia actually cross platform. The full Pro version costs $1500, but there is a free limited version which I downloaded and installed on my Mac. Mike sent me his project folder, and I could open it, but in the end couldn't get the Kinect to connect. He offered to loan me a windows machine, but I would rather run Windows on my Mac with Parallel Desktop. Since all I have is the old Windows XP, I need to upgrade to Windows 7, that Mike recommended over Windows 8.
   Not so simple though, as Microsoft has discontinued Windows 7, and non OEM copies are rare and expensive, over $160 on eBay. I suppose I could download the much cheaper OEM version, and buy it again if I change machine… 
   I may also have to upgrade to Parallel 9.

Post 25: Hope for the Kinect at Last!

   BIG NEWS: Corey loaned Mike a Kinect for the week end, and he got it to work and do pretty much what I was asking for in my emails to them last week:

Hi Guys,

   I am playing with the idea of a set that could be projected on at a roughly 30 degrees angle WITH A SINGLE PROJECTOR. I made a cardboard model to test it:

    And it works. My 5K could cover a 20 ft wide stage in the dark, floor and walls.

    It can be mapped with an image like this, which requires precise projector positioning and accurate alignment:

     Or I can use MadMapper with an image like this, and map each panel separately, which makes it a lot easier:

     

 Either way, this is the way it ends up looking on the model:

   As you can see, the image projected on the "dancers" is mostly of the floor(which could be interesting as long as they stay at the front of the stage).

   Now, WOULD IT NOT BE NICE if I could use the Infra Red Kinect camera to map BOTH the shape and movement of the dancers, and replace it image in image, on the fly, with either a solid color or an entirely different image. That would be like having several precisely fitted infinitely adjustable projector/spotlights following them around. I expect some kind of delay due to processing, but the movements could be kept slow.

   I can dream it up, but I honestly don't know how to do it (yet), and won't have time to learn the Processing language and the Kinect Hacks in the book I bought until after Light Dreams.

   So, do you see any way you could figure it out?

   Keeping you on your toes… Let me know what you think.

   Thanks.

JJ

  This is a neat idea.  I think it can be done without a lot of work, but there will definitely be spillover from the dancers to the ground/walls (both from latency and registration errors).

   I'll run a couple of tests to see what the issues look like; maybe we can come up with a way that they either aren't very apparent, or otherwise work into the overall projection.

Corey Shum

Technical Director, Enabling Technology Laboratory
School of Engineering, Dept of Mechanical Engineering
University of Alabama at Birmingham — Knowledge that will change your world

Corey,

  I am more concerned about the latency than the registration errors, but that in itself could create interesting effects, with the Dancer sometimes getting ahead of her projected image. 

  Because of the way the IR sensor in the Kinect works (projected dot pattern reflected towards the IR camera), the limited density of dots, and the fact that surfaces at almost 90 degrees (like the edges of a body) don't reflect the dots, there is some raggedness around the edges, but that can be minimized with a flat distant backdrop, and could be smoothed and feathered to some extent.

  They are also going to come out with a new much improved higher definition Kinect with skeletal tracking, that is also supposed to act like a green screen and key out the background:

       http://www.ign.com/articles/2013/11/07/how-next-gen-kinect-is-more-accurate-than-ever

  That would be fabulous, and I hope it comes out soon, and can be hacked for the Mac…

  Thanks for your help. Keep me posted.

JJ

PS: When we get this one figured out, I would like to find a way to harness the amazing point cloud rotating 3D imaging capability of the Kinect:

      The best software I have found for the Mac to visualize the images is cocoaKinect.:

                      http://fernlightning.com/doku.php?id=randd:kinect

       Synapse does a pretty good job at recognizing the skeleton:

                               http://synapsekinect.tumblr.com/post/6610177302/synapse

JJ,

   Corey was kind enough to loan me a Kinect over the weekend.  I played around with it and was able to create a proof of concept application.  I like your idea.  It lets you do a green screen effect without the green screen.  I wrote some code that scans through the depth data from the device.  For each pixel that belongs to the player, I replaced with the color information from the RGB camera on the device.  All other pixels were replaced with a background image. I also went ahead and added the particle effect we were talking about last time.  If you look at the screen shot, you can see that the particles are being emitted from my hand:

    The latency wasn't nearly as big of an issue as I thought it would be.  But, I wasn't moving that fast.  A dancer may see a little more.  The only major issue I ran into happened when the Kinect lost me and had to reacquire.  Sometimes when it reacquires a body, it will stop sending depth and color information from the device.  The tracking still worked though.  So, in the application, you could still see the particles but the image of me disappeared.

    BTW, we can add any 3D objects to the scene.  not just particles.  Here is a scene with me holding the ghost and PacMan models from my game in each hand.
   Thanks,

 Mike

Good Morning my Friends,

  Whaooooo!… Look at that! Super cool! That is some great news to wake up to and start a new week.

  Mike, you are a Godsend. You are the quiet man that delivers… And quickly!

  The images look great by the way, and I think you really need to do something with the Kinect either for PacMan, or another interactive game.

  My big problem is that I use a Mac, so can it work for me. I actually did manage to move an object one time with Synapse and Quartz composer, but now keep getting an error message.

  In any case, great job Mike, as always.

  Thanks a bunch to both of you.

JJ

Post 24: What Music?

   The Great thing about cooperating with Mary on this project is that we bring two totally different points of view to the game, and yet communicate and understand each other very well. I am the "Half Nerd Artist" who tends to think of visuals, images and technical possibilities afforded by the new hardware and software available today to play with. Mary is the reminder, as Myrna was in the workshop, not to lose track of "content"(I hate that word, it makes Art sound like it belongs in a box, and my greatest desire is to get out of the box!). She thinks in terms of music, rhythm, meaning, style, choreography, feelings, expression, movement.
   Which brings me to our current discussion about "music". I love "Good Music" in all genres. I was brought up on "serious" classical music mostly because my parents had gotten me in a "record of the month club" that didn't offer any rock and roll, and because my high school Music Teacher was in love with the great Russian composers. This is definitely a bit of an acquired taste, but it stuck for many many years, even though I also loved the Beatles, Ray Charles, John Lee Hooker, Cat Stevens, Leonard Cohen, as well as Georges Brassens, Jean Ferrat, and Jacques Brel. 
  There was a Music from the Andes period in the late 60's culminating in a trip to Peru and Bolivia in 72.
  After moving to the US to be with Rachel, I got into Renaissance Music in the 70's, and baroque music in the 80's. The high point was definitely hearing Mozart's Requiem in Prague in the very church where it premiered.
  In the 90's, I started to concentrate on Vivaldi's Religious and Operatic Music, almost to the point of obsession. I researched and tracked down every recording by RV numbers. I would play it loudly non stop all day when I was in the studio painting, from the Stabat Mater to Orlando Furioso. Don't mention the Four Seasons please...
   Things changed around 1999, which is when I started spending most of my time either inside on the computer, or outside building our house, which took several years. I found that either activity required a kind of concentration that tuned out music, so there was actually no point playing any. At the same time, I was restless and searching for a new way to express myself as an Artist, and the Internet offered a novel marvelous new way of doing research. No more poring through shelves of books at the library, everything became readily available from my desk with just a few clicks. It was a game changer. I was already a "hunter", I became a maniac, one site leading to another, click after click, procuring rapidly masses of information, knowledge and images… eBay changed the face of collecting, and the house filed up with wondrous old stuff from all over the world.
  My hunting included music, and I discovered all kinds of new music, mostly singers: Rokia Traoré, Souad Massi, Missia, Elena Frolova, Agnès Jaoui, Daby Balde, Etta Scollo, Yasmin Levi, Cesaria Evora, Chavela Vargas, Sopor Aeternus, Devendra Benhart, Katie Melua, Paolo Conte, Daan. I suppose the common thread is "singing from the guts".
  I became particularly impressed by the oratorio like work of Belgian composer Nicholas Lens: Flamma-Flamma, Terra-Terra, Amor Eaternus. Unfortunately, his work is becoming more minimalistic and challenging over the years, and I still consider his early Flamma-Flamma his best. 
  My work in Animation, Video and Architectural Mapping required sound tracks, and I started collecting thousands of sounds to go with the thousands of images I already had. I used Nicholas Lens work many times as a base. But what works for a sound track is often not the music you would listen casually. It has to have power, expression, rhythmic variety, originality, and the beat drives the movement of the images on the screen. A sound track doesn't have to be what we would call music, it is more like a mix of layered interesting sounds: music, sounds, noise, speech. 
   Which brings me finally to our ongoing conversation with Mary about what kind of "music/ sound" we want to work with. We saw the Koresh Dance Company the other day, and the strongest piece had no music per se, just spoken words and sounds. The track for Bridgman-Packer's last piece "Voyeur" is a very complex forever changing layering of music and sounds. I liked both.
   We definitely don't want "pretty music", nor traditional "dance music"(if I hear another Bolero, I will puke! I swear), nor boring repetitive "minimalistic music". I would like to have beautiful harmonies, but also some dissonances, a strong changing beat, some jarring unexpected sounds(to wake up the audience…), natural sounds, words. I want unexpected "things" to happen. I want the listener to be surprised, a little confused, a little shocked, never bored.