Monthly Archives: August 2015

Bright Lights

Hi All,

This was by far one of the most frustrating things that I have ever done.  I was a situation of knowing what the computer had to do in English, but completely over complicating it and forgetting simple stuff when coding. (Ex. “Serial.begin(9600)”) Dumb. Not having power plugged in.  Dumb.  In the end, with a serious amount of help from Scott, it worked.

There are a whole bunch of next steps.  I think the visual feedback is really important as a guide. Like the glove I made, the LED tells you which scale you a playing.  Creating a trellis like board that would support the 4 pronged through hole NeoPixels seems like a lot of work and skills that I don’t have, but Mike’s arcade buttons inspired me.  An arcade button MIDI controller with customizable LED colors.  Get a microSD card reader and a speaker in there. Awesome. Get some bluetooth or other wireless protocol and have two of them talking to each other. Awesome.  So many ideas.

Anyway, here is the video:

Here’s the Arduino Code:

Here’s the processing code:

Thank you all for such an awesome class. I had a bitchin’ time.





Thanks for everything guys

Mood Display Board

I’m very inspired by the website called “We feel fine.”


It is a website that collects people’s posts about feelings from the internet. I want to do a physical mood display. Different color pastels represent different moods. User can pick up the color that stands for current feeling, and use it as a brush to write or draw anything on the screen. After that, users can also take a photo with what they write or draw and save it for keeping or sharing later:)


The complete code:


int potVal;

void setup() {
// put your setup code here, to run once:

void loop() {
// put your main code here, to run repeatedly:
potVal = analogRead(A0);



// Smoother color tracking
// based on Dan Shiffman & Dan O’Sullivan’s examples


// Variable for capture device
Capture video;

PGraphics drawing; /// a contect to hold the “drawing”
PImage vidMirror;

import processing.serial.*;
Serial mySerial;
float diam;

// A variables for the color we are searching for.
color trackColor;
float thresh = 20;
int avgX, avgY; //this is what we are trying to find
int prevAvgX, prevAvgY;

void setup() {
size(1280, 720);
video = new Capture(this, width, height);
// Start off tracking for red
trackColor = color(255, 0, 0);

// instance of the drawing
drawing = createGraphics(width, height);

vidMirror = new PImage(video.width, video.height);

// Print out the list and look for port your Microcontroller is on
// Finding the Arduino is not easy because they have weird names like “/dev/tty.usbmodem1421″ or COM1
String portName = Serial.list()[2];
mySerial = new Serial(this, portName, 9600);


void draw() {
// Capture and display the video
if (video.available()) {;
// image(video, 0, 0);

for(int x = 0; x < video.width; x++){
for(int y = 0; y < video.height; y++){
vidMirror.pixels[x+y*video.width] = video.pixels[(video.width-(x+1))+y*video.width];


//we are going to find the average location of changed pixels so
//we will need the sum of all the x find, the sum of all the y
// find and the total finds
int totalFoundPixels= 0;
int sumX = 0;
int sumY = 0;

// Begin loop to walk through every pixel
for (int x = 0; x < vidMirror.width; x ++ ) {
for (int y = 0; y < vidMirror.height; y ++ ) {
int loc = x + y*vidMirror.width;
// What is current color
color currentColor = vidMirror.pixels[loc];
float r1 = red(currentColor);
float g1 = green(currentColor);
float b1 = blue(currentColor);

// colors we are looking for
float r2 = red(trackColor);
float g2 = green(trackColor);
float b2 = blue(trackColor);

// Using euclidean distance to compare colors
float difference = dist(r1, g1, b1, r2, g2, b2);
// We are using the dist( ) function to compare the current
// color with the color we are tracking.

// If current color is more similar to tracked color than
// closest color, save current location and current difference
if (difference < thresh) {
sumX = sumX + x;
sumY= sumY + y;

// average the locations of all the pixels that were close enough.
if (totalFoundPixels > 0) {
avgX = sumX/totalFoundPixels;
avgY = sumY/totalFoundPixels;
// Draw a circle at the tracked pixel
stroke(0, 127);
ellipse(avgX-10, (avgY-10), 20, 20);

drawing.line(prevAvgX, prevAvgY, avgX, avgY);
blend(drawing, 0, 0, width, height, 0, 0, width, height, ADD);

prevAvgX = avgX;
prevAvgY = avgY;
void serialEvent(Serial _port){
if (mySerial == null) return; //this is a hack to cover a bug where the port does not get set up in time.
//this says if the port is not set up yet, bail (for now.)
String input = mySerial.readStringUntil(‘\n’);
if (input != null) { //if a ‘\n’ character has in fact now arrived
input = input.trim(); //Take off the ‘\n’ character, get rid off the white space
diam = float(input); //Turn it into number

void mousePressed() {
// Save color where the mouse is clicked in trackColor variable
int loc = mouseX + mouseY*vidMirror.width;
trackColor = vidMirror.pixels[loc];

void keyPressed() {
//for adjusting things on the fly
if (key == ‘q’){

if (key == ‘[‘) {
println(“Threshold ” + thresh);
} else if (key == ‘]’) {
println(“Threshold ” + thresh);


Hi again,

It was wonderful to meet and learn from all of you,  and also thank you Scott for being a fantastic professor. I hope we all stay in touch, or that in the future I see all of your work advertised on billboards all over the city (ideally, both scenarios). Best of luck to you all in the coming semester and forward.

Here is the link to my final:

Thanks guys!

not my final – but a missing post

Hi all,

Here is assignment of combining an existing processing sketch with arduino controls. I believe I showed it in class (maybe over the break?) but now it can also live permanently on the site.


Final Project – Big Brother Time!

Had a great time in the class. It was great meeting each of you and getting the change to learn something new!

My project uses a FSR to manage the color of a box on a grid that correspond to the day and hour of the week. There is a check periodically through each hour, and when there is pressure when checked, it moves the color of the box from white to black gradually. Scott got me going on the hour and day portion which was really helpful.



Processing code:

import processing.serial.*; // import the serial library
import java.util.Date; // import the java date library

Serial mySerial; // an instance of the serial class

int thisBox = 255; // the variable that describes the box color

int prevHour; // a variable to hold the previous hour
// use this to check for a change in the hour
// and reset the color variable

int xPos =0; // a variable for the xPosition of a box (if you want to go that route)
int yPos =0; // a variable for the yPosition of a box (if you want to go that route)

void setup() {
size(700, 600); // a larger window so that it’s more legible
println(Serial.list()); // list our serial ports

String portName = Serial.list()[1]; // get the port name
// open the serial port at 9600 baud
mySerial = new Serial(this, portName, 9600);
// when we receive anewline char, trigger the serialEvent fxn

// white background

// draw a grid in black
for (int i = 0; i<700; i=i+100) {
for (int j = 0; j<1200; j=j+25) { // CD- back to 800
rect(i, j, 100, 25);

void draw() {
// variables to hold the hour and day of the week
int hour = hour(); /// 24 hour clock 9 10 11 12 13 14 15 16 17 18 etc….
int day=new Date().getDay(); //0 = Sunday, 1 = Monday, 2 = Tuesday, 3 = Wednesday, 4 = Thursday, 5 = Friday, 6 = Saturda

// if the hour has changed
if (hour !=prevHour) {
thisBox=255; // make the new fill color white
prevHour=hour; // set the previous hour to this hour

fill(thisBox); // fill color for the box you’re currently writing to

// you can set this up to change a variable for the box x,y position
// so that if day == 1, xPos = 0. if day==2, xPos=100 (etc).
// and for the hours if hour ==9, yPos=0. if hour=10, yPos =100 (etc.)

if (hour == 1) { // if 9am CD-back to 9
yPos =0;
} else if (hour==2) { // if 10 am
} else if (hour==3) { // if 10 am
} else if (hour==4) { // if 10 am
} else if (hour==5) { // if 10 am
} else if (hour==6) { // if 10 am
} else if (hour==7) { // if 10 am
} else if (hour==8) { // if 10 am
} else if (hour==9) { // if 10 am
} else if (hour==10) { // if 10 am
} else if (hour==11) { // if 10 am
} else if (hour==12) { // if 10 am
} else if (hour==13) { // if 10 am
} else if (hour==14) { // if 10 am
} else if (hour==15) { // if 10 am
} else if (hour==16) { // if 10 am
} else if (hour==17) { // if 10 am
} else if (hour==18) { // if 10 am
} else if (hour==19) { // if 10 am
} else if (hour==20) { // if 10 am
} else if (hour==21) { // if 10 am
} else if (hour==22) { // if 10 am
} else if (hour==23) { // if 10 am
} else if (hour == 0) { // if 9am CD-back to 9
yPos =575;

if (day == 1) { // if Monday
xPos =0;
} else if (day ==2) { // if Tuesday
xPos =100;
} else if (day ==3) { // if Tuesday
xPos =200;
} else if (day ==4) { // if Tuesday
xPos =300;
} else if (day ==5) { // if Tuesday
xPos =400;
} else if (day ==6) { // if Tuesday
xPos =500;
} else if (day ==0) { // if Tuesday
xPos =600;
} /// continue for the other days of the week thru Fri

// draw the box
rect(xPos, yPos, 100, 25);
// serialEvent only fires when a message is received from the Arduino
// in this case it’s only once a minute (because that’s how often the
// Arduino fires
void serialEvent(Serial port) {
if (port == null) return; // make sure there’s information

String inString = port.readStringUntil(‘\n’); // read until the newline character

if (inString != null) { // as long as there’s an actual word in there
inString = trim(inString); // trim off any whitespace:
String check = “working”;
if (inString.equals(check)) { // if the arduino sent working”
thisBox = thisBox-25; // make the color for the box a little darker
// 4*60=240, which would be pretty close to black if someone is in the chair the full hour
} else {
// nothing to do if someone’s not working


Arduino code

int fsrVal; // variable to hold the senspr value

void setup() {
// put your setup code here, to run once:
Serial.begin(9600); // start serial communication

void loop() {
// put your main code here, to run repeatedly:
fsrVal = analogRead(A0); // read the sensor, save in variable

if (fsrVal < 999) { // if the sensor value is less than 999
Serial.println(“working”); // send a “working” message
} else if (fsrVal >= 1000) { // if it’s greater or equal to 1000
Serial.println(“notWorking”); // send the “not working” message
delay(600); // wait one minute before the next read


JS Final – Sliding Images Music of Rachels and Images of Egon Schiele

Here is Version 3 of my Slide Show algorithm!


This Program works simply by using one of the two slide potentiometers to control the picture on screen as well as the music in the background. The volume level of the sample just called as well as the volume level of the background music can be controlled.


It’s the prototype for an Art Installation that requires user participation to “mix” the music with these selectable fragments/


Problems: Overall it was not too challenging getting everything working, the only hurdle encountered was truly getting a grasp for how the serial commands work. The handshake between the serial port and processing can be tricky which is why a number of lines of code are needed to make sure the data streams are lined up properly. I was accidently streaming data in before the system was ready to accept any data, that lead to a number of issues that were cleanly resolved with a if statement that checked if the data stream was truly ready to be read in.


In regards to the program I will hit some of the highlight functions. I built a custom class duoPA that was able to load in the music files as well as the pictures and pair them together. So whenever my sample selector would select a point it would be referencing two files. Also have a pixel scrambler running in the background as well as a function that randomly selects sections of a loaded in picture and throw it into the main screen. Everything is controlled by millis() and modulo division looking for zero for timing.


It was a ton of fun getting this all working and I can’t wait to ship out v4!





Prismatic Box

When I started this project, I wanted to make a controller that could manipulate an image in Processing.  The design of this controller would be similar to this one from HitBox, but with slightly different button positions:


I bought 12 buttons from Video Games New York and took a sturdy cardboard box from my apartment.  Here’s a stock image of the Light Blue buttons:

From there, I brought my materials to the ITP 4th Floor to get wires, solder, and various tools.  I measured the size of the buttons to determine the size of the holes I would need to cut into the box.  Once the measurements were complete, I slid the buttons in.

Box with Buttons

Next came the wiring.  I’ve actually never used solder before this project, but I learned very quickly with a little help.  I started by soldering the wires from the 4th floor to the buttons on one side and the Arduino cables on the other.  I was only able to wire half of the buttons before Tuesday’s class since I also wanted to work on the Processing program.

Wires for Six

The initial thought was for Processing was to display what button is being pressed.  However I thought this would be boring, so I tried to make something more exciting.  Since this controller design is intended for video games, I decided to make a circle that would move around based on button presses.  However, it became difficult for me to implement animations in a way that I wanted.

As exciting as this would have been, I came to the realization that the important part of my project was the hardware.  Still, I didn’t want to just show button presses in Processing.

First DemoWhile showing my progress in class, Drew suggested that I should have the colors of the buttons on my controller blend in Processing.  I really liked this idea because it was simpler than the animated circle, but more exciting than the basic buttons.  I decided that blending colors would be the basis of my new program.

The next day, I got back to wiring.  I had to secure some of the original solder joints with heat shrink wraps.  After that I soldered the rest of the wires to the other six buttons.  Then figured out how properly attach my buttons to the breadboard in circuit.  I also decided that didn’t need the white buttons anymore because they were having trouble staying in the breadboard.  On top of that, I didn’t want to use that color in my new Processing program.

Wires in Breadboard

Once everything was wired, I programmed the Arduino to communicate with my computer.  Here’s the code for that:

Now the only thing left was the program in Processing.  I decided to alter the “ArrayBallsAddSubtract” code from class to allow for Arduino input.  I also changed numerous parameters including as the colors, sizes, and opacity of the balls.  You’ll find my code and a video demo below:



I learned so much over the course of this project between hardware design, soldering, and programming.  But if there’s one thing I learned this week that trumps everything else, it was to know my limits.

I’m the type of person who has grand elaborate ideas.  These ideas are great in theory, but practically I don’t always have the skills, knowledge, or time to complete them.  Eventually, I get so fixated on these ideas that I become frustrated when I can’t finish them.  But during this project I was able to change course when I noticed things weren’t working, specifically with Processing.

Despite being disappointed in this change originally, I realized that I had already created something I was really proud of.  I’ve wanted to make a controller like this for over a year, and the fact that I was able to do so is amazing to me.  On top of that, I was still able to program software to demonstrate this controller.  Sure, it wasn’t the software I originally had in mind, but it’s something I was within my abilities.

At the end of the day (or week as it may be), I’m ecstatic that I was able to develop something I’ve thought about for a long time.  But more importantly, I’m happy that I could surpass the such a big creative obstacle: myself.

“I Miss You” Exorcism Teddy Bear

For our final project we initially wanted to make a phone case that would alert a third party if the user was in danger and unable to text.  However, we realized that this information wouldn’t be particularly useful unless the recipient was given a way to act on the information; we thought that having the button press on the phone case send the GPS coordinates of its location would be a good solution, but we didn’t feel that that was a reasonable goal given our experience.

Our next idea was to pitch the same basic messaging system a a means of communication for parents and children who are too young to use cell phones but whose parents would want to hear from them throughout the day.  After many iterations of this design, we settled on a teddy bear that would, when squeezed or hugged, send a message to the parent’s computer saying “I miss you” along with an audio file to get the parent’s attention if they are away from their computer.  When the parent clicks the window, an LED lights up in the bear to show the child that their parent has received their message.  Somewhere between inadvertently settling on red and black as a color scheme for the bear and having a little too much fun mixing the audio file, however, we seemed to have taken a turn for the satanic but decided to run with it in the name of memorability.

The benefits of teamwork: saving time and combining efforts.  Anna is responsible for the teddy-bear design, embedding the Arduino, figuring out the circuit schematic, working on the basics of the coding, and building a prototype; Sam then built a second prototype using a smaller breadboard, added to, refined, and debugged the code, and added the audio file.  Both of us spent a considerable amount of time working together to ensure that the Arduino was embedded properly and the teddy was working consistently.



Draw a line based on a color in a video