Joystick Causing Lag

Im getting lagg upon first starting my game. I've traced it down to the player control loop.

If KeyDown(KEY_UP) or JoyY(0) = -1 Then
	_up = True
Else
	If KeyDown(KEY_DOWN) or JoyY(0) = 1 Then 
		_Down = True 
	EndIf
EndIf

Im checking for movement on the joystick along with keys. All is fine when there is a joystick attached. Unplug the joystick and the first 5-10 seconds of loops causes a large lag.

I presume I must split the detection of keys and joystick and even have to get the player to select joystick as the means of inout on the main menu ? - Never had todo this with any other language tho'

Making an input manager is a good way to go, so you can just check inputMgr.MovingRight() and such. That way you don't have to worry about input devices, keyconfigurations and such.