get_token word parse

Hi Guys.

Ive just added to the code archives this simple word token finder, i wonder if its possable to speed it up?

http://www.blitzbasic.com/codearcs/codearcs.php?code=2011

kev

I think so:

showTokens("this is a line of text, using a simple lexer to display each word.")
Function showTokens(SLine$,identifier$=" ")
	Local m$, currentWord$
	For loop = 1 To Len(SLine)
		m = Mid(SLine, loop, 1)
		If m = " " Then
			displayToken(currentWord)
			currentWord = ""
		Else
			currentWord = currentWord + m
		End If
	Next
	If currentWord <> "" Then displayToken(currentWord)
End Function

Function displayToken(tokenName$)
	Print tokenName
End Function