BliztGet issue when download image

zoom*

Hi everybory! i use an improvement of the BlitzGet function to download image (i don't remember who did it). And i have an issue: the images are partially downloaded. A part of them is not downloaded (10% or less approx.). Why? Someone has already got this problem before? Thanks.

jfk EO-11110

Please post your sourcecode.

zoom*

Sure, sorry, but i think the issue doesn't come from the code.

Function wec_BlitzGet (webFile$, saveDir$, saveFile$,fileKind$)

    ; -------------------------------------------------------------------------
    ; Strip "http://" if provided
    ; -------------------------------------------------------------------------
    If Left (webFile$, 7) = "http://" Then webFile$ = Right (webFile$, Len (webFile$) - 7)

    ; -------------------------------------------------------------------------
    ; Split into hostname and path/filename to download
    ; -------------------------------------------------------------------------
    slash = Instr (webFile$, "/")
    If slash
        webHost$ = Left (webFile$, slash - 1)
        webFile$ = Right (webFile$, Len (webFile$) - slash + 1)
    Else
        webHost$ = webFile$
        webFile$ = "/"
    EndIf
        
    ; -------------------------------------------------------------------------
    ; Add trailing slash to download dir if not given
    ; -------------------------------------------------------------------------
    If Right (saveDir$, 1) <> "" Then saveDir$ = saveDir$ + ""

    ; -------------------------------------------------------------------------
    ; Save filename -- get from webFile$ if not provided
    ; -------------------------------------------------------------------------
    If saveFile$ = ""
        If webFile = "/"
            saveFile$ = "Unknown file.txt"
        Else
            For findSlash = Len (webFile$) To 1 Step - 1
                testForSlash$ = Mid (webFile$, findSlash, 1)
                If testForSlash$ = "/"
                    saveFile$ = Right (webFile$, Len (webFile$) - findSlash)
                    Exit
                EndIf
            Next
            If saveFile$ = "" Then saveFile$ = "Unknown file.txt"
        EndIf
    EndIf

    ; DEBUG
    ; RuntimeError "Web host: " + webHost$ + Chr (10) + "Web file: " + webFile$ + Chr (10) + "Save dir: " + saveDir$ + Chr (10) + "Save file: " + saveFile$

    www = OpenTCPStream (webHost$, 80)

    If www
    
        WriteLine www, "GET " + webFile$ + " HTTP/1.1" ; GET / gets default page...
        WriteLine www, "Host: " + webHost$
        WriteLine www, "User-Agent: wecreader"
        WriteLine www, "Accept: */*"
        WriteLine www, ""
        
        ; ---------------------------------------------------------------------
        ; Find blank line after header data, where the action begins...
        ; ---------------------------------------------------------------------
                
        Repeat
            header$ = ReadLine (www)
            If Left (header$, 16) = "Content-Length: "    ; Number of bytes to read
                bytesToRead = Right (header$, Len (header$) - 16)
            EndIf
        Until header$ = "" Or (Eof (www))
        
        If bytesToRead = 0 Then Goto skipDownLoad
        
        ; ---------------------------------------------------------------------
        ; Create new file to write downloaded bytes into
        ; ---------------------------------------------------------------------
        save = WriteFile (saveDir$ + saveFile$)
        If Not save Then Goto skipDownload

        ; ---------------------------------------------------------------------
        ; Incredibly complex download-to-file routine...
        ; ---------------------------------------------------------------------

        For readWebFile = 1 To bytesToRead
        
            If Not Eof (www) Then WriteByte save, ReadByte (www)
            
            ; Call BytesReceived with position and size every 100 bytes (slows down a LOT with smaller updates)
            
            tReadWebFile = readWebFile

			;tCurrent = MilliSecs()
			;If (tCurrent-tStart>=tDelta) Then
	
				;receivedBytes = (tReadWebFile-posByteBefore)/1000
				;posByteBefore = tReadWebFile
				;tStart = tCurrent
		
			;EndIf

            If tReadWebFile Mod 5000 = 0 Then wec_BytesReceived (readWebFile, bytesToRead,fileKind$)

        Next

        CloseFile save
        
        ; Fully downloaded?
        If (readWebFile - 1) = bytesToRead
            success = 1
        EndIf
        
        ; Final update (so it's not rounded to nearest 100 bytes!)
        wec_BytesReceived (bytesToRead, bytesToRead,fileKind$)
        
        .skipDownload
        CloseTCPStream www
        
    Else
    
        RuntimeError "Failed to connect"
        
    EndIf
    
    Return success
    
End Function

; -----------------------------------------------------------------------------
; User-defined update function, called every 100 bytes of download -- alter to suit!
; -----------------------------------------------------------------------------
; TIP: Pass a user-defined type instead, with all data (this stuff plus URL, local filename, etc)
; -----------------------------------------------------------------------------
Function wec_BytesReceived (posByte, totalBytes, fileKind$)
    ; Example update code...

    Cls
	;Color 40,40,40
	;Rect 10,10,GraphicsWidth()-20,GraphicsHeight()-40,1
	;Color 55,55,55
	;Rect (GraphicsWidth()-204)/2,(GraphicsHeight()-6)/2,204,6,0
	;Rect 10,10,GraphicsWidth()-20,GraphicsHeight()-40,0
	Color 175,0,0;255,128,0
	;Rect (GraphicsWidth()-200)/2,(GraphicsHeight()-2)/2,wec_Percent (posByte, totalBytes)*2,2,1
	Local alphaend# = wec_Percent (posByte, totalBytes)*360/100
	Local radius = 35
	For alpha# = 0 To alphaend# Step .1
	
		Line GraphicsWidth()/2,GraphicsHeight()/2,GraphicsWidth()/2+radius*Cos(alpha#),GraphicsHeight()/2+radius*Sin(alpha#)
		
	Next
	
	DrawImage imgLoad,(GraphicsWidth()-ImageWidth(imgLoad))/2,(GraphicsHeight()-ImageHeight(imgLoad))/2
	
	Color 100,100,100
	If fileKind$="image" Then
	 
		;Text (GraphicsWidth()-204)/2,(GraphicsHeight()-6)/2-2-12,"Buffering image " + (imageIndex+1) + " on " + webimageListSize + "..."; [" + receivedBytes + "kb/s]";remaing time: ~" + Abs(((totalBytes-posByte) * (tCurrent-tStart)/posByte)/1000) + " s)" ;" + posByte/(tCurrent-tStart) + " kb/s)" ;
    
	ElseIf fileKind$="list" Then 
	
		;Text (GraphicsWidth()-204)/2,(GraphicsHeight()-6)/2-2-12,"Loading list..."; (remaing time: ~" + Abs(((totalBytes-posByte) * (tCurrent-tStart)/posByte)/1000) + " s)" ;" + posByte/(tCurrent-tStart) + " kb/s)" ;
	
	EndIf
	;Text 20, 20, "Downloading file -- please wait..."
    ;Text 20, 40, "Received: " + posByte + "/" + totalBytes + " bytes (" + Percent (posByte, totalBytes) + "%)"
    wec_DisplayWECWareLogo()
	Flip


End Function

; -----------------------------------------------------------------------------
; Handy percentage function
; -----------------------------------------------------------------------------
Function wec_Percent (part#, total#)
    Return Int (100 * (part / total))
End Function

jfk EO-11110

What exactly do you mean, "partially downloaded" ? Do you say some are downloaded and some are not? Or did you mean every image is downloaded 90% but then aborts downloading?

If it's the first thing, then: Did you test this with multiple servers? Are some of them run by other people?

Note: some webserver do not allow hotlinking of images. They usually check the browser referrer and if it looks as if you're not watching the corresponding webpage then they won't deliver the image you're trying to access. Maybe that's the problem?

zoom*

i mean every picture are downloaded and displayed, but all the content of picture isn't downloaded. You can see a screen here:

Sometimes pictures are wholly downloaded and sometimes not.

jfk EO-11110

Strange. Probably some kind of timeout. I wouldn't use this per-byte reading code:

	For readWebFile = 1 To bytesToRead
		
			If Not Eof (www) Then WriteByte save, ReadByte (www)
			
			; Call BytesReceived with position and size every 100 bytes (slows down a LOT with smaller updates)
			
			tReadWebFile = readWebFile
			If tReadWebFile Mod 100 = 0 Then BytesReceived (readWebFile, bytesToRead)

		Next

but maybe try to read the entire lot of bytesToRead to a bank, eg:

bank=createbank(bytesToRead)
readbytes(bank,0,bytesToRead)
wr=writefile("mypic.bmp")
writebytes wr,bank,0,bytesToRead
closefile wr
freebank bank

ehrm, it's pseudo code, make sure the parameters are correct before you run it.

jfk EO-11110

PS I see your idea is to provide a progress bar for the download. Do you really need it? well you still can use ReadBytes, reading blocks of 100 bytes instead of single bytes.

zoom*

Thanks i'll test you code. It's very stange 'cause before this code worked well.