[dancer-users] Starman large file upload

[Warren Young]

On Apr 18, 2017, at 9:01 AM, Zahir Lalani <ZahirLalani at oliver.agency> wrote:
> 
> We are also looking into chunked uploads

For anything taking more than a couple of seconds or 2 GB (whichever comes first) you should indeed be using HTML5’s new File API to send chunks of the file individually rather than try to send it all at once.

> we should be looking at the content range header or similar to prevent this

No, you should be sending query parameters that identify which chunk number this is and how many more chunks there will be, so that the Dancer route handler you’re sending this to knows when it has received the last chunk.

Dancer pseudocode:

    post ‘/upload’ => sub {
        my $chunkNumber = params ‘chunkNumber’;
        return { error => ‘no chunk number’ }
            unless isNaturalNumber($chunkNumber);
        my $chunkCount = params ‘chunkCount’;
        my $chunk = request->upload(‘chunk’);

        # Do something intelligent with $chunk*
    }

HTML pseudocode:

    <input type=“file” onchange=“sendFile(this.files[0]”>

jQuery pseudocode:

    function sendFile(file) {
        var chunkSize = X;   // a base-2 “round” number suitable to your app
        var fileBytes = file.size;
        var numChunks = Math.floor(fileBytes / chunkSize + 1);
       
        function sendChunk() {
            var start, end;  // calculated using available values
            var chunk = file.slice(start, end);

            var fd = new FormData();
            fd.append(‘chunk', chunk);
            fd.append(‘chunkNumber', chunksSent);
            fd.append(‘chunkCount', chunksTotal);
            
            $.ajax(‘/upload’, {
                data: fd,
                success: function() {
                    if (++chunksSent < numChunks) sendChunk();
                },
            });
        }        

       	sendChunk();
    }

That will certainly not run as-is, but if it should be sufficient to get you to working code.