Scratch

The Kinect has an infrared transmitter and camera - that is what is used to "see" you.

The regular camera is just used in the Xbox to take pictures or video conference. I don't think it contributes to the "sensing" of where you are. I don't think we have access to that yet in scratch.

Others have described in more detail than I have tried to follow what the Kinect output is. Try a quick Google search.

The net result at the Scratch level, is that by using Stephen Howell's program, you get variables in Scratch that give you the x and y coordinates of each body part/joint.

Then you can do things like use the x position of the right hand to pick a keyboard note to play; as you move your hand, it's like you're playing a piano (at the hand level, not the finger level).

I did a quick slide-show where if you crossed your hands (rhand x < lhand y) it switched the background.

I don't know if there are any libraries yet that sense higher level actions (jumped, threw ball, turned around, kicked with the left foot). In theory, those could all be built in scratch itself, and broadcast when they detect a certain type of motion.

I also haven't seen a z position. Stephen, is that available? That could be useful for detecting walking towards or away from the camera. Otherwise, maybe you could use the distance from the head to the hips as a proxy for how close/far the subject is.

-Rich

markyparky56 wrote:

SHowell wrote:

ihaveamac wrote:

I assume the Kinect has a usb plug?

Standalone Kinects (bought without an XBox) have, there is an addon cable available for the bundled ones.

What if you had two, reasonably good quality, web-cams? could you emulate it that way? If I think about it, it would mean basicly making your own kinect, what kind of output do you get for the Kinect? do you get something that needs to be de-coded or just points of the body? (and I geuss a whole lot more)

(For those residents of the UK, the Kinect is £127!)
http://www.amazon.co.uk/Kinect-Sensor-Project-Natal-Adventures/dp/B0036DDW2G/ref=sr_1_1?ie=UTF8&qid=1294765435&sr=8-1

No, there are IR transmitters and very complex algorithms at work, easier to buy a Kinect (if you can afford one) I'm afraid.

OK, new version of the Kinect/Scratch program for everyone to try.

This has a simple GUI which allows you to launch the Kinect and Connect to Scratch by clicking buttons. It gives some feedback in a log window. Currently does not tell you if you are posing correctly, working on that.

New features include: Mirror mode (x-axis reflected), Multi-user begun (flaky, works for 2 players mostly, needs more testing and my 2 year old refused to stand in the calibration pose long enough to ensure it worked consistently), and z-axis (for richardatscratch). Z axis is a value that decreases (negatively) the farther you get from the Kinect sensor.

Screenshot: http://27.media.tumblr.com/tumblr_levyu1ROgU1qg7yt4o1_500.png

New version available on my blog at http://stephen-howell.tumblr.com

Hope you all enjoy it (at least, those of you with Kinects!)

Stephen

SHowell wrote:

Z axis is a value that decreases (negatively) the farther you get from the Kinect sensor.

As in there is a set 0, or 0 is where you start off at?

SHowell wrote:

markyparky56 wrote:

SHowell wrote:

Z axis is a value that decreases (negatively) the farther you get from the Kinect sensor.

As in there is a set 0, or 0 is where you start off at?

0 is the closest it can recognise, about 1 meter from the sensor.

Ok. And the furthest?

Last edited by markyparky56 (2011-01-12 12:41:13)

SHowell wrote:

Well, I got to about -200 before I hit the wall.

I geuss its not possible to find the maximum then.

SHowell wrote:

Well, it is if you have a big room, the sensor will eventually be unable to see you.
I'll test it later, currently have a bigger issue, multi-user is not working properly!

Whats wrong? Is it getting them mixed up?

In terms of the z-axis, I got to -256 before it lost me (and I hadn't hit the wall yet). When I walk right up to the Kinect, I get 56 (so the range is -256 to 56, give or take). It lost me when I was that close.

For me x ranged from 400 to -400
y ranged from -250 to 250

Maybe I could have gotten a little more, but the kinect was having trouble tracking me reliably.

If you have OpenNI loaded, you can use:

OpenNI\Samples\Bin\Release\NiUserTracker.exe

to see what Kinect "sees". It seems that the easiest way to get "calibrated and tracked" is to start close to the Kinect and walk away slowly, and then stand in the calibration position. It usually takes only a few seconds for me.

I don't think we have motor control yet. I usually prop the Kinect up on a small pad or pencil to it is angle correctly. It's a bit difficult to angle it for both me and my 2 year old daughter. One could hook it up to the xbox, adjust he tilt there and then put it back on the PC.

-Rich

-Rich

New version guys and girls, try this: http://stephen-howell.tumblr.com/post/2734703739/new-version-of-openni2scratch

There are some brief instructions there as well for setting up some scripts in Scratch to use this.

Hi, I am really interested in this project, but I only have a Mac! Can anyone post a tut on how to get this up and running on Mac?

johnnydean1 wrote:

SHowell wrote:

richardatscratch wrote:

The regular camera is just used in the Xbox to take pictures or video conference. I don't think it contributes to the "sensing" of where you are. I don't think we have access to that yet in scratch.

Well, we have access to the pixel data (what the camera sees) but to send it to Kinect would involve sending each pixel individually, Scratch has no network support yet for blobs of data (that I can see).
Stephen

You could always put the data into a list and send it that way.

The remote sensor network communications document I found only mentions sending broadcast and sensor-updates for named global variables, I see no way to send a list. Can you link to some further documentation?

SHowell wrote:

terminator355 wrote:

Hi, I am really interested in this project, but I only have a Mac! Can anyone post a tut on how to get this up and running on Mac?

Sadly, I don't have a Mac, and the code uses .net so it isn't simple to port. I suggest it's about a days work for a good mac C developer to port if there were OpenNI drivers available. Any old pcs that you can use?

Actually, I've got an old PC, and could run windows 2000 on my Mac through boot camp, but that's really old, and I'm not sure it would work that well. Or would it work okay?

johnnydean1 wrote:

SHowell wrote:

johnnydean1 wrote:

You could always put the data into a list and send it that way.

The remote sensor network communications document I found only mentions sending broadcast and sensor-updates for named global variables, I see no way to send a list. Can you link to some further documentation?

There is no documentation I am afraid - this forum is probably the expert. What I mean is if you have a list
#( 'R' 'R' 'G' 'B' )
and you turn it into a single string.
"R,R,G,B"
you could send that by the sensor-update to set a (|list| sensor value) block.
You could then use Scratch by saying:
replace item 1 of SCRATCHList with (letter 1 of (|list| sensor value) )
replace item 2 of SCRATCHList with (letter 3 of (|list| sensor value) )
replace item 3 of SCRATCHList with (letter 5 of (|list| sensor value) )
replace item 4 of SCRATCHList with (letter 7 of (|list| sensor value) )

so your ScratchList would have all the list transfered.
I hope this helps.


NOTE:
To send lists to Scratch/Incorperate them into strings, would require modifying Scratch's Source Code Found Here

Two things.

1. R[RGB? Why is there two Rs?

2. RGB as far as I know can be upto 3 digits long, e.g. 255-255-255 is pure white and 0-0-0 is pure black.

Last edited by markyparky56 (2011-01-15 15:53:55)