This is the Adaptive Use Musical Instrument Webapp

AUMI Together

AUMI Together is using HTM5 and Javascript audio and video processing to recreate the same experience that the iOS and Desktop versions provide.
It's simpler, but also more adaptable, because of the platform it runs on.
This version is using the same color tracker, roughly, as iOS. To simplify it, instead of finding the largest contiguous area of the selected colors, it just chooses all of them and hopes you are using a single, very color-contrasty object to track.

How to do it:

• Choose a video source from the drop down. The browser will ask for permission to use the video cameras the first time you do this per session. Assent to its use.
• To choose colors to track, click and drag over the target area on the video screen. While gathering colors, "Gathering colors" will appear on screen in the status area. When you are done, a code will appear in the status area describing the colorset.
• Choose an instrument - by deffault, it' the toy piano. When the cursor passes into a yellow box, it plays a note!
• The switches on top help to visualize what's going on.
  • Mirror: Flips the video image. On by default.
  • Rotate: rotates the video image, to correctly orient the camera. External cameras may be sideways or upside down!
  • Letterbox or Fill: either fills the image in the canvas, clipping it, or letterboxes/curtains it.
  • Size: you can choose a smaller size video screen to make it more efficient.
  • Zoom: you can Zoom in to make it more efficient and to concentrate on part of the image.
  • Show Color As Contrast: This helps visualize the current tracked color set.
  • Show RGB4 Colors: This helps visualize truncated color space I use for tracking colors.

It seems to be OK in Safari and Chrome. It doesn't seem to work on my iPhone, but on the iPad (gen 5), it's good - you need to double tap on the screen to set the colors though.