Face detect and UDP

Hi,
You probably know Johnny Lee, a researcher at Carnegie Mellon who has fun with the wiimote. See here. http://www.youtube.com/watch?v=Jd3-eiid-Uw

I make a small webcam/application that after detection of a face moves the position of it via UDP.
Here http://www.zinfo972.net/avatar/FACE_UDP.RAR
Need help for the Blitz test ;-)

JP
PS: it's not a..."trojaner" topic. ;-)

;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Graphics3D 800,600,32,2
AppTitle "Face UDP Detect 3D "
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
; Face UDP (c) zjp+++++++++++++++++++++++++++++++++++++++++
Global recu$
Global UDP_Serveur
Global VX=160
Global VY=120
Global VR=50
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
; UDP read face detect
Function recu_UDP()
	;recu$=""
	IP_Client = RecvUDPMsg(UDP_Serveur)
	If IP_Client Then 
		recu$ = ReadLine(UDP_Serveur)
		; parce
		; found X
		i=Instr(recu$,"X(",1)+2
		C$=Mid$(recu$,i,4)
		VX=C$
		; found Y
		i=Instr(recu$,"Y(",1)+2
		C$=Mid$(recu$,i,4)
		VY=C$
		; found R
		i=Instr(recu$,"R(",1)+2
		C$=Mid$(recu$,i,4)
		VR=C$
	End If 
End Function
UDP_Serveur = CreateUDPStream(26000) ; <<<<<<< Quake UDP PORT ;-)
If UDP_Serveur <>0 Then 
	Print "Server Start."
Else
	Print "Can't start Server"
	End
End If
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
light=CreateLight()
cam=CreateCamera()

Type box
	Field xpos#,ypos#,zpos#
	Field spinspeed#
	Field entity
	Field alpha#,r,g,b
End Type

CubeType=CreateCube()
For x= 1 To 6 
	For y = 1 To 6
		For z = 1 To 6
			b.box=New box 
			b\entity = CopyEntity(CubeType)
			b\xpos#=x*4
			b\ypos#=y*4
			b\zpos#=z*4
			rc=Rnd(10,200)
			gc=Rnd(10,205)
			bc=Rnd(10,205)
			EntityColor b\entity,rc,gc,bc
			EntityAlpha b\entity,.8
			PositionEntity b\entity,b\xpos#,b\ypos#,b\zpos#
		Next
	Next
Next
;
MoveEntity cam,12,10,0
HideEntity CubeType 
;
Repeat
	; move the cam. Fx to build here
	RotateEntity cam,vy/6,-vx/6,0 ; dummy test
	;++++++++++++++++++++++++++++++++++++++++++
	UpdateWorld
	RenderWorld
	recu_udp ; get Face position
	Text 10,10,"Face x "+VX+", Face Y "+VY+", Face Radius "+VR
	Flip
Until KeyHit(1)=1
End

Hi

Are you doing head-tracking based on image recognition with a web cam?

I supose you are evaluating differences with the actual frame with tespect of the previous.

Why don't use the wiimote?

I'm not at home now, let me download, do a test with my webcam and will see if I can help in some way

edit
hey, cool!
works fine, let me test different conditions like light, contrast, distance.
Are you testing some feature in particular?


best regards


Juan

Hi,
"...Are you doing head-tracking based on image recognition with a web cam?.."
> Yes

Need some help to build a "fantastic" demo with Blitz3D. I'm not a ... Blitz3D guru ;-)

JP

It doesn't work on my webcam :(
It's all black in the webcam window

it works here! very cool... maybe you could make it so that it can do that perspective thing that guy on youtube did with the wii remote

edit oops didnt see that :p good work

perhaps this should go in the codearchives?

is easiest to do with the wiimote, it sends 100 packets of coord's info each second (position of the led's inside the sensor bar). The code johnny gives is so simple to translate to blitz, the math part very short the rest is easily done with the 3d blitz engine.

I'm geting poor frames per second but i'm testing your face detect in my laptop (an ancient one), i'll try nex week in a newer (obiously not mine!)

best regards

Juan

"...It doesn't work on my webcam :(..."
This program use an UDP/port : 26000. Check your firewall

"..I'm geting poor frames per second.."
Updated. Speed of detection has improved.

The aim of the program is not to replace the wiimote. Wiimote does not detect faces.You can use it to detect your "sleep" ;-))

JP

I try to simulate a face with a ball, and other mimics and your soft didn't fail!
If one put a hand or other thigs in the face, the soft stop detecting the face, that sort of things was the first I try to do, but with the intention of test seriously your work. I supose you are watching some particullar aspects of the face... (2 eyes, 1 nose, .. and the like)

about speed, i supose that was not the focus of your post.

and of course, face detection nothing has to do with the wiimote, great job.

I'll continue to see, what are the important aspects in your face recogn.

Juan

i made a pong game you control with your face :)

here is the code.. very basic and messy tho :( I am trying to make a callibrator for each kind of camera. you may have to tweak this code to make it work for your camera..

;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Graphics 640,480,0,1
AppTitle "Face UDP Detect 3D "
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
; Face UDP (c) zjp+++++++++++++++++++++++++++++++++++++++++
Global recu$
Global UDP_Serveur
Global VX=160
Global VY=120
Global VR=50
Global TX#
Global TY#
Global RX#,RY#,RR#,OX#,OY#
Global chng
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
; UDP read face detect
Function recu_UDP()
	;recu$=""
	IP_Client = RecvUDPMsg(UDP_Serveur)
	If IP_Client Then 
		chng = 1
		recu$ = ReadLine(UDP_Serveur)
		; parce
		; found X
		i=Instr(recu$,"X(",1)+2
		C$=Mid$(recu$,i,4)
		VX=C$
		; found Y
		i=Instr(recu$,"Y(",1)+2
		C$=Mid$(recu$,i,4)
		VY=C$
		; found R
		i=Instr(recu$,"R(",1)+2
		C$=Mid$(recu$,i,4)
		VR=C$
	End If 
	If chng = 1 Then
    ox# = rx#
    oy# = ry#
    EndIf
	rx# = -vx + 353
	ry# = -vy + 223
	rr# = vr
	If chng = 1 Then
		tx# = rx#
		ty# = ry#
	Else
;		tx# = tx# + (rx#-ox#)/80
	;	ty# = ty# + (ry#-oy#)/80
	EndIf
	chng = 0
End Function
UDP_Serveur = CreateUDPStream(26000) ; <<<<<<< Quake UDP PORT ;-)
If UDP_Serveur <>0 Then 
	Print "Server Start."
Else
	Print "Can't start Server"
	End
End If
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
SetBuffer BackBuffer()
Global px = 720
Global bx,by,bdx,bdy
bx = 320
by = 240
bdx = Rnd(-2,2)
bdy = Rnd(-3,-1)
;
;
Repeat
  Cls
	If tx > 50 Then
	px = px + (tx-50)/50+2
	EndIf
	If tx < -50 Then
	px = px + (tx+50)/50-2
	EndIf
	If px < 0 Then px = 0
	If px > 560 Then px = 560
	
	bx = bx + bdx
	by = by + bdy
	
	If by < 0 Then bdy = -bdy
	If bx < 0 Then bdx = -bdx
	If by > 480 Then bdy = -bdy
	If bx > 640 Then bdx = -bdx
	If by > 449 And by < 453 And bx > px And bx < px+80 Then bdy = -bdy
	
	Rect px,450,80,10
	Oval bx-5,by-5,10,10,1
	

	Text 10,10,"Face x "+Int(tX)+", Face Y "+Int(tY)+", Face Radius "+VR
	Flip
	recu_UDP
Until KeyHit(1)=1

CloseUDPStream(UDP_Serveur)
End

can someone test this to see if it works the same on every computer. Just move your face left and right and it moves the pong paddle :)



@ZJP - i will help if you want me to... wow i have made at least 3 of my previous games head controlled :) this is fun

can u guys add the camera part to this so it works for my webcam via udp?

what do you mean by adding a camera part?

so that it actually turns on my webcam, and reads my face, then moves the platform w/ my face movement, as u said above?

oh you must download and run the camera program in the first post in this thread. When you exit the window, it makes itself a process and keeps your camera on as well as doing facerecognition constantly

Here it is
http://www.zinfo972.net/avatar/FACE_UDP.RAR

can u release the source so i can try to mod it if needed? if u need my email, ill be glad to give it to u :)

oo im with darkshadowwing on this one. any chance you will release the source?

It doesn't work with my Webcam :(
Some random dots in random colors appear in the screen.
I don't know what kind of webcam it is, because it's built in to my laptop screen.

u need to download the file in the 1st post on this topic, THEN compile the source. i really wish he will release this source. it would be a great contribution to the community. besides, we might be able to help make it better! :)

"...u need to download the file in the 1st post on this topic, THEN compile the source. i really wish he will release this source. it would be a great contribution to the community. besides, we might be able to help make it better! :) ..."

This is not a BlitzBasic source. Build in C++
I'm working on extra options. Webcam number ,ip number (local or not), udp port. It can detect a face via Wan or Lan

JP

could u turn it to a dll THEN we could create the userlib for it, and create it? :)

AHAHA
and the first person you test on is francky vincent
ahahaha :))

;-)