Managing External Objects

Ok, this is a tricky one, and not even easy to explain or find a suitable title for ;)

I have an object, which is managed externally. I want to create my own BlitzMax objects to wrap it. The external objects are given integer handles, so all I need is an Int field and off I go.

Now the external objects are "managed". What I mean by this is that very often you'll create a new one but an existing one will already fit the bill so it will return the int handle to this one instead of creating a new one. So if I set my BMax object's destructor to destroy it ( It's an integer handle remember, so it's not reference-counted, it has to be explicitly destroyed ) then the other BMax objects I have referencing the same external object will now point to an object which does not exist. When new external objects are created, they may even end up referencing completely different objects.

And that is no good. That'll crash heaps if you're referencing objects which don't exist.

So the sensible way to handle this is to reissue my BMax objects as well. So if the handle I get back when creating a new external object is one that's already been issued and assigned to a previous object, I don't create a new one at all, I resupply the handle to the last one. And that's great because when the BMax object is destroyed, I note that the corresponding external object is now dead, kill it, and if the handle comes through again, I'll know that it's a new external object which is just reusing the same handle.

Problem is.. this means I can't have MY BMax objects managed automatically. Because in order to give out the handles, I need to have copies of them don't I? Probably in an array, since that allows me to retrieve them quickly by looking up the external object's handle as the array index. And if I have a copy of every BMax object then simply letting the handles go out of scope or nulling them won't work, will it? Because there's always that one leftover handle I keep internally preventing the reference count hitting zero and the GC taking it all away to Junksville.

Soooo.. assuming that my description has made any sense at all ( and if it hasn't please tell me how I can clarify ) is there any possible way to set this up so that I can still have my BMax objects cleaned up automatically without explicitly destroying them, but at the same time, reissue existing BMax objects so that the external objects are not destroyed before all my BMax objects have finished with them?

Never mind, that idea didn't pan out. This is trickier than I thought.

I don't suppose there's a way to store handles to BlitzMax objects in such a way that the Garbage Collector cannot see them, is there? IE: A way I could keep a list of objects but the garbage collector will delete them as soon as all *other* references are broken? That would solve it, but I don't guess that would be a very sensible language feature.

I am trying to get handle on what you mean but not sure I get it, what you are saying. Do you want your Blitzmax object to be destroyed when the external object is destroyed but the keep the blitzmax object alive if the external object is alive?

Doug Stastny

MemAlloc()?

I'm not sure I know what you're talking about either but isn't handlefromobject ignored by GC?

Yes it is ignored. But the object would still be freed if there are no references within BM anymore.

But you can clone the object data to an unmanaged version.

somemem = MemAlloc(SizeOf(theObject))
MemCopy (varptr theObject),somemem, sizeof(theObject)

(written out of head, can't test atm)

Nope, almost the opposite. I'm in control of when the external objects are destroyed. I want to destroy each automatically when the last BMax object which references it is destroyed, but not before.

Unfortunately, the nature of these objects is that you may create a new one only to get back the handle to an existing one instead, because the external "libary" determines that a new one is not needed. There is no way to know this is going to happen. So I can either have multiple BMax objects pointing to the same external object or manage my BMax objects the same way, reissuing the same BMax object I issued the first time this external object was requested.

But both ways seem to make it impossible to automatically destroy. If I maintain a list of all the BMax objects I've issued, those objects will never go out of scope, and so will never be destroyed.

If I create a new BMax object every time, there's no way to know whether there are any more BMax objects exist which reference the same external object. If I destroy the external object in the destructor, the external object will be destroyed when there are BMax objects out there still referencing it. If I don't put it in the destructor, it doesn't get automatically destroyed.


Is it? Then that *should* be just the ticket. Cast the objects to ints and store those. Then I can reissue the same object by casting the integer back to an object handle, but they won't stop the objects being automatically destroyed.

I'll have to code it up to be sure, but that sounds good.


That's good, I want that. I want to be able to maintain a list of them that doesn't stop them being freed, so that's ideal.

the handle returned by handlefromobject is managed by the GC. internally it calls the BBRETAIN() function which increments the reference counter.

Ah ok, that won't work then. Thanks for saving me the testing time on that one.

i wouldnt recommend this... but this is interesting. make sure debug is on:

Framework BRL.Blitz

Extern
	Function bbObjectRelease(pBBObject:Object)
EndExtern

Type test
	Field x:Int=5
	Method Delete()
		DebugLog("deleting")
	EndMethod
EndType

Local mytest:test=New test
bbObjectRelease(mytest)
GCCollect()
DebugLog(mytest.x)

Epp GMAN, thats a disaster waiting to happen as the object memory was moved to the freelist so although you still are pointing to it major problem.

check out this minor change of variable from local to global. The compiler deals with them differently. Local variables are not refernce counted. They are release by the going out of scope on the Program Stack.

Framework BRL.Blitz

Extern
	Function bbObjectRelease(pBBObject:Object)
EndExtern

Type test
	Field x:Int=5
	Method Delete()
		DebugLog("deleting")
	EndMethod
EndType

Global  mytest:test=New test
bbObjectRelease(mytest)
GCCollect()
DebugLog(mytest.x)
mytest = Null
GCCollect()

As for Gabriels clarification, I think I understand what he wants to do, I just need to noodle on implmentation.

Basically need to create a reference counted intermidary list to manage the Handles. When the handle count is reduced to zero destroy the object in the library.

I'll noodle on it and see if I can produce some code to demonstrate concept. Its hard since I dont have the library in question. What lib is this Gabriel?

Doug Stastny

yup. i figured it would be :) i was surprised it worked really.

Gman did you see my other post? Wanted to know if you tested that yet with the VTable?

Doug Stastny

i have not gotten deep into it yet. i did run it quick on the machine that had the issue with the last and it worked fine. i will post more detail in the proper post.

Gabriel,

I slept on you question and this is what I came up with. Bascially it creates intermidiary layer that does the reference counting. Kinda fun little coding over morning coffee. I tried to comment it enough to make sense.

SuperStrict

Rem 
  TExternal is only to simulate an externally managed object that can
  be returned multiple times per function call.  We are just going to store 
  objects in simple stringlist,  nothing fancy or fast

  Our handle is nothing more than memory address of the first element of data in the 
  object.  it will be consistent for all instances pointing to this object.  


End Rem

Type TExternal
   	Field _Text:String    
   	Method Handle:Int()
		Return Int(Byte Ptr(Self))
   	End Method
End Type

Global ExternalObjects : TList=New TList


Rem
  A function that gets external handle for string in string list
  function will return handle to same item if already exists
  
End Rem

Function GetExternal:Int(Text:String)
  For Local e:TExternal=EachIn ExternalObjects
	If e._Text=Text
  	    Print "Reusing "+Text
		Return e.Handle()
	End If
  Next  
  Local newitem:TExternal= New TExternal
  newitem._Text=Text
  ExternalObjects.AddFirst(newitem)	
  Print "Adding "+Text
  Return newitem.Handle()
End Function

Rem 
  A Function to Delete the object when we are done with the handles and truely want
  to delete the object  
End Rem

Function DeleteExternal(Handle:Int)
  For Local e:TExternal=EachIn ExternalObjects
	If e.Handle()=Handle
	   ExternalObjects.Remove(e)
	   Print "Deleting "+e._Text
	   Return
	End If
  Next  	
  Throw "Unable to find item to delete"
End Function


Rem
  Test to see if our external string to handle works  this just test the idea
  of external object manager than returns handles more than  once if item 
  already exists in collection or adds it if it doesnt
End Rem

'Local a:Int=GetExternal("APPLE")
'Local b:Int=GetExternal("CHERRY")
'Local c:Int=GetExternal("APPLE")

'Print "a="+a
'Print "b="+b
'Print "c="+c

'DeleteExternal(a)
'DeleteExternal(b)
'DeleteExternal(c) ' This will throw since item does not exists


Rem 
	Here is the implementation to manage this.  It uses a TMap with a special Type to manage
	reference counting of the instance of the Handle given out.
	
	When that count reachs zero it will remove object from the map 
	and delete the external object


End Rem



Rem
	This object is used to manage the handles we have already gotten
	with a TMAP.  Since TMAP requires object we can make it use 
	the key and behavior by overriding the Compare Method
End Rem


Type THandle
	Field _handle:Int

	Method Compare:Int(o:Object)
		Return _handle-THandle(o)._handle
	End Method
		
	Function Create:THandle(Handle:Int)
		Local	t:THandle
		t=New THandle
		t._handle=Handle
		Return t
	End Function
End Type

Rem
	This is the Type we use to reference count the external items	
End Rem

Type TExternalMapItem
	Global _Map:TMap= New TMap
	Field _Handle:Int
	Field _RefCount:Int
		
	
	Method AddRef()
		_RefCount:+1
	End Method
	
	Method ReleaseRef()
		_RefCount:-1
		If _RefCount= 0
			DeleteExternal(_Handle)	
			_Map.Remove(THandle.Create(_Handle))
			_Handle=0
		End If
	End Method
	
	
	Function GetExternalMapItem:TExternalMapItem(Text:String)
	    ' create a Thandle 
		Local key:THandle= THandle.Create(GetExternal(Text))
		' if the Thandle is in the map we just add a reference
		' otherwise we insert it in map and add reference
		Local mapitem:TExternalMapItem=  TExternalMapItem(_Map.ValueForKey(key))
		If (mapitem=Null)
			mapitem=New TExternalMapItem
			mapitem._handle=key._handle
			_Map.Insert(key,mapitem)
		End If
		mapitem.AddRef()
		Return mapitem
	End Function				
	
End Type


Rem
	This is the wrapped item.  we implement a Dispose method if we want to
	ensure deletion before Delete is called since we cant control when the GC
	will clean up this takes care safely wacking it, under our control or GC control.
End Rem


Type TExternalWrapper
	Field _ExternalMapItem:TExternalMapItem
	Method Handle:Int()
		Return _ExternalMapItem._Handle
	End Method
	Method Dispose()
		If _ExternalMapItem _ExternalMapItem.ReleaseRef()
		_ExternalMapItem=Null
	End Method
	Method Delete()
		Dispose()		
	End Method	
	
	' our Create function for the wrapper
	Function Create:TExternalWrapper(Text:String)
		Local wrapper:TExternalWrapper= New TExternalWrapper
		wrapper._ExternalMapItem=TExternalMapItem.GetExternalMapItem(Text)
		Return wrapper
	End Function    
End Type



Global a:TExternalWrapper=TExternalWrapper.Create("APPLE")
Global b:TExternalWrapper=TExternalWrapper.Create("CHERRY")
Global c:TExternalWrapper=TExternalWrapper.Create("APPLE")

Print "a's Handle ="+a.Handle()
Print "b's Handle ="+b.Handle()
Print "c's Handle ="+c.Handle()
b=Null
a=Null
Print "Before GCCollect Only CHERRY should be released"
GCCollect()
c=Null
Print "Before GCCollect Only APPLE should be released"
GCCollect()
Global d:TExternalWrapper=TExternalWrapper.Create("PEAR")
Print "d's Handle ="+d.Handle()
Print "Force Deletion"
d.Dispose()
Print "Gone!"
d=Null
Print "Before GCCollect nothing will happen"
GCCollect()
Print "See nothing happend since we already disposed"
Print "Done"

Maybe it will help, good luck

Doug Stastny