Download & Upload using Greasemonkey & PHP

Question

I need to transfer a file (.zip) from a clients third-party-website to mine.

GM-Snippet (commented out other trys)

 GM_xmlhttpRequest({
    method: "GET",
    url: dlurl,
    timeout:0,
    onload: function(resp) {
    var conti=resp.responseText;

alert("goup");
GM_xmlhttpRequest({
    method: "POST",
    url: "http://absolut-fair.com/wbb_back/api.php?id=3",
    overrideMimeType: 'text/plain; charset=x-user-defined',
    data:"chk=1&data="+base64_encode(conti),
    headers: {
            "Content-Type": "application/x-www-form-urlencoded"
    },
    onload: function(resp) {
        var conti=resp.responseText;

        if(conti=="1") alert("upload erfolgreich");
        else alert(conti);
//....

PHP-Snippet

$name="test.txt";
uploadfile(base64_decode($_POST["data"]),detect_mime($name),$name);

Problem

All of the decoded files are corrupted when I try to open them. The problem is sending binary data. Sending raw text worked fine!

//Update

I just downloaded the healthy and the invalid zip files and it appears that the invalid one handles a lot of space filling useless code segments. So i got the feeling that base64 is not able to fully encode/decode binary data. Anyone has any knowledge?

I attached a screenshots of the source to make it clear!

The valid file:

the valid file

The invalid one:

the invalid one

There is a size limit for _POST data. What you should do is upload the files like a regular HTTP upload, and access the file in php using _FILES — Michael, Dec 06 '12 at 21:08
Of course. Please see the question and the supported code before asking. //edit: I cant automaticly download the file over JS/GM, otherwise i'd already done it this way. — James Cameron, Dec 06 '12 at 21:11
What I'm saying is the POST data is kept in memory on the webserver until the whole request is received, while a file upload (multipart/form-data) is usually cached to disk. Your current code will work for smaller files, but for bigger files it's either going to run out of memory or exceed the request limit. — Michael, Dec 06 '12 at 21:16
I'm not too familiar with the client js you're using for the http post request. But the first step is to compare the base64 encoded data on the clientside with the data the script receives in the _POST variable. — Michael, Dec 06 '12 at 21:18
I know, but the files are under 1MB, so neither the cache of the client, nor the memory of the webserver will be exceeded. — James Cameron, Dec 06 '12 at 21:18
You need to stop posting this quickly ;) In the base64 encodement it appears to have a big loss of data. — James Cameron, Dec 06 '12 at 21:19
Base64 usually appends == signs at the end of data. It might mess up the http request variables. Try to URL-encode it client-side before you append it to the request. — Michael, Dec 06 '12 at 21:21
Tried `data:"chk=1&data="+encodeURIComponent(base64_encode(conti)),`, doesnt work either. A test showed that the data sent equals the data received (before decoding base64 again). — James Cameron, Dec 06 '12 at 21:37

score 1 · Accepted Answer · edited May 23 '17 at 12:08

As i wasn't able to get help here i asked an experienced user named "Kugelfisch23" at the german community "gulli" for help and sent him this topic.

Now that we solved the problems i want to provide additional informations for other users trying to send binary files over Greasemonkey and receive it with PHP.

If you work at some kind of this too, i recommend sending first a small exemplary file like this, as it contains all binary characters. Therefore you'll be able to check where the problem occurs and if probably, what is most likely, the encoding/decoding went wrong.

1st Problem: Receiving binary data

As you can see in my example in the beginning, i overwrote the mime-type when SENDING binary data, but not when receiving. Therefore, the object responseText only contains UTF-8 characters and corrupts the data. => One needs to overwrite the mime-type when receiving too.

GM_xmlhttpRequest({
    method: "GET",
    url: dlurl,
    timeout:0,
    overrideMimeType: 'text/plain; charset=x-user-defined',
    //...

2nd Problem: Encoding binary data to base64

As its not possible to simply send binary data as a POST-parameter, one needs to encode the data to make it sendeable. In the first trys i used this base64_encode - Function to encode my data. As i later found out, does this function only provides insufficient conversion to send binary data and should be mostly used to encode text. This function (refering to the function base64Encode(str), don't confuse it with base64ArrayBuffer) instead is able to encode binary data.

3rd Problem: Sending base64

Sending the base64-encoded result over post isn't enough, as it contains a lot of characters that the PHP-Endpoint can't handle or converts to invalid characters. Use encodeURIComponent on the base64-result before sending it.

4th Problem: Receiving and decoding

Even though it's rather simple decoding the incoming data, i'll provide you my snippet.

if(empty($_POST["data"])) die("no data sent");  
$name="test.zip";
$binary=base64_decode($_POST["data"]);

Result

Greasemonkey/JS

        var dlurl = "http://website.zip";
        GM_xmlhttpRequest({
            method: "GET",
            url: dlurl,
            overrideMimeType: 'text/plain; charset=x-user-defined',
            timeout:0,
            onload: function(resp) {
                var conti=resp.responseText;
                var encoded = encodeURIComponent(base64Encode(conti));
                GM_xmlhttpRequest({
                    method: "POST",
                    url: "http://absolut-fair.com/wbb_back/api.php?id=3",
                    overrideMimeType: 'text/plain; charset=x-user-defined',
                    data:"data="+encoded,
                    headers: {
                            "Content-Type": "application/x-www-form-urlencoded"
                        },
                    onload: function(resp) {
                     //...

PHP

See code for the 4th problem

Download & Upload using Greasemonkey & PHP

1 Answers1