String splitting to Arrays

BlitzMax Forums/BlitzMax Programming/String splitting to Arrays

In my module I have got 2 methods for splitting Strings and send them to an Array. I have made 2 functions out of them to share them with you.

The first one called Return_strip devides the string as often it finds the delivered ASCII code in the String.

The second one called Return_strip2 devides the string as long as the delivered _strip:string is. i.e: the stripstring is
" :" the main string will be divided two times.
("Load Map:Test.dat" will result in 1. "load" and 2. "Map:Test.dat" with the first function but with the second one you will get : 1."Load" 2."Map" 3."Test.Dat")


Strict

Local I:Int



Print "Return_strip2 : ('Load Map:Test.dat')"

Local Strip:String[]
Strip = Return_Strip2("Load Map:Test.dat"," :")
For I = 0 To Strip.length-1
Print I + " : " + Strip[I]
Next

Print "Return_strip  : ('Load Map:Test.dat')"

Local Strip2:String[]
Strip2 = Return_Strip("Load Map:Test.dat",32)
For I = 0 To Strip2.length-1
Print I + " : " + Strip2[I]
Next




Function Return_Strip2:String[](_String:String,_strip:String)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Mid(_strip,I+1,1))
			If sp_p = - 1 Or I > _strip.length -1 Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function



I added something similar to the code archives except it returns a list rather than an array.

Well, I haven't looked in the Codearchives for a long time.
And in an other topic someone searched for this.

Yours is nice, I wished I had seen it before.

As long as yours works it doesn't really matter.

Hmm... it might be interesting to speed test them to find out which is faster.

Would be nice but it seems both needs 0 millisecs.

Tested with this code:


Local I:Int
Global time:Int
Global time2:Int
Global time3:Int


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
Time2 = MilliSecs()
time3 = time2 - time

Print "needed time: " + time3

Rem
For I = 0 To Strip2.length-1
Print I + " : " + Strip2[I]
Next
End Rem

Print "Pertubatios :"

time = MilliSecs()
Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
Time2 = MilliSecs() 
time3 = time2 - time

Print "needed time: " + time3 
Rem
If myList Then
	For a$ = EachIn myList
		Print a$
	Next
EndIf
End Rem

Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()
time3 = time2 - time
Print "needed time: " + time3




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function



Results :

1: 0
2: 0
3: 14 (last one was a test if my test is correct)

well, a better test would be to do it a thousand times:
Local I:Int
Global time:Int
Global time2:Int
Global time3:Int


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
For Local a:Int = 1 To 1000
	Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2



Print "Pertubatios :"

time = MilliSecs()
'Local myList:TList
For Local b:Int = 1 To 1000
	Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2


Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function


This shows that mine is slower (which is what I suspected).
It is sometimes however handy to have the list functionality.

*EDIT*
Mine can be sped up slightly by not creating the list each loop (i.e. create it outside the loop and the assign to it inside).

Actually, this is weird, mine performs better if you run it first:


Local I:Int
Global time:Int
Global time2:Int
Global time3:Int


Print "Pertubatios :"

time = MilliSecs()
'Local myList:TList
For Local b:Int = 1 To 1000
	Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
For Local a:Int = 1 To 1000
	Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2






Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function




*EDIT*
solved it:


Local I:Int
Local time:Int
Local time2:Int


Print "Pertubatios :"

time = MilliSecs()
'Local myList:TList
For Local b:Int = 1 To 1000
	Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
	myList= Null
	FlushMem
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
For Local a:Int = 1 To 1000
	Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
	strip2 = Null
	FlushMem
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2






Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function


Flushmem is your friend.

I get:
Pertubatios :
needed time: 25
Return_strip :
needed time: 15
Testloop :
needed time: 43
in Debug Mode

and:
Pertubatios :
needed time: 18
Return_strip :
needed time: 14
Testloop :
needed time: 11
in Release.

Yours is still faster, but it makes me feel better that mine isn't quite as horrendously slow as I thought.

mind if i ask why you start your vars with underscore, is this a basic convention?

@Hakea :If you mean '_' this with underscore then I actually use it because I have seen somewhere else and it looks much better for me and it helps me not to loose the overview in greater projects.

@Pertubatio : Yeah that flushmem() works wonders ;)