1

I tried to search for solution (for 2 days now) bot nothing helps me.

Because of many reasons I was forced to write own library in JavaScript to build a multipart/dorm-data body.

I checked how other libraries (for node) works and I read the spec. Then I wrote the code (https://github.com/jarrodek/ChromeRestClient/blob/files/app/elements/payload-editor/multipart-form-data.js) which is quite simple because the spec is.

However, my test server do not recognise parameters in the request. Any parameters (text or files).

My library generates generating the following HTTP message:

([NL] represents new line character which is \r\n)

POST /post HTTP/1.1[NL]
HOST: localhost:8081[NL]
content-type: multipart/form-data; boundary=-------------805520824430161118507807[NL]
content-length: 17789[NL]

-------------805520824430161118507807[NL]
Content-Disposition: form-data; name="image"; filename="arc-icon.png"[NL]
Content-Type: image/png[NL]
[NL]
�PNG[NL]
[binary data for about ~17700 bytes)[NL]
[NL]
-------------805520824430161118507807--[NL]

This is snapshot what is actually passed to a socket. The library builds the HTTP message and converts it into the ArrayBuffer which is an argument in socket send function.

Now, I know there's some issue with my code, I just can't find it. I tried to add new line after message part (as in the code above) or not (after image data I always adding new line). Maybe someone can see an issue with the message here and point it out because I have no more ideas how to fix it :)

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Pawel Uchida-Psztyc
  • 3,735
  • 2
  • 20
  • 41
  • _"The library builds the HTTP message and converts it into the `ArrayBuffer` which is an argument in socket send function"_ Where is `ArrayBuffer` at `javascript` at Question? _"However, my test server do not recognise parameters in the request"_ Can you include full `javascript` at Question and reproduce issue at jsfiddle http://jsfiddle.net or plnkr http://plnkr.co? – guest271314 Feb 03 '17 at 21:30
  • It actually doesn't matter. The HTTP message in the question is converted to string message sent to server. I need to find an issue with the HTTP message to correct error in code. Analysing the code would much more difficult than the message (in this case) – Pawel Uchida-Psztyc Feb 03 '17 at 21:33
  • Why can you not use `FormData`? – guest271314 Feb 03 '17 at 21:34
  • I can't use any of this services because it's a chrome app that is using sockets API which isn't available in regular JS API. Like I said, I need to find an issue with the message, not the code. However, full code is linked in the question - @guest271314 – Pawel Uchida-Psztyc Feb 03 '17 at 21:35
  • Because it can't be converted to multipart message in a low level so I'd be able to pass it to socket. Again, the code doesn't matter here. I am asking to help me find an issue with the HTTP message. - @guest271314 – Pawel Uchida-Psztyc Feb 03 '17 at 21:36
  • _"Because it can't be converted to multipart message in a low level"_ Not sure what you mean by "in a low level"? `FormData` object can be constructed using `fetch()`. Is `boundary` portion correct? See [How to upload files in Web Workers when FormData is not defined](http://stackoverflow.com/questions/13870853/how-to-upload-files-in-web-workers-when-formdata-is-not-defined/13970107#13970107), [Javascript Web Workers File Upload](http://stackoverflow.com/questions/12615622/javascript-web-workers-file-upload) – guest271314 Feb 03 '17 at 21:40
  • 1
    Not sure how I can get HTTP body message from FormData :) I can only do it using Fetch API and the request object. But I can't until https://bugs.chromium.org/p/chromium/issues/detail?id=688100 is fixed - @guest271314 – Pawel Uchida-Psztyc Feb 03 '17 at 21:46
  • _"Not sure how I can get HTTP body message from FormData :)"_ See [Get HTTP Body of Form in JavaScript](http://stackoverflow.com/questions/40111982/get-http-body-of-form-in-javascript) – guest271314 Feb 03 '17 at 21:51
  • You can use `.text()` to get `FormData` – guest271314 Feb 03 '17 at 23:43
  • If you are only passing `File` instance, why is creating `"multipart/form-data"` necessary? You can use `FileReader` to create an `ArrayBuffer` from `File` object. Or parse `.text()` at resulting `Promise` value string. – guest271314 Feb 04 '17 at 02:29
  • Sorry for late reply. I agree with other ways to do it. But they will not work for this particular use case. I need to produce valid body for multipart request without using Request object because it's buggy right now (see referenced issue report). I can't use Fetch/XHR in my case because I'm working directly on socket. What I'm asking to help me find an issue with the message in question :) – Pawel Uchida-Psztyc Feb 05 '17 at 16:08
  • Why can you not use `Request`, `Response` where the report uses `Request`? Not sure that the result is actually a bug. Would suggest filing an issue at `fetch` specification at github if you believe that there is a bug with the `fetch` implementation. – guest271314 Feb 05 '17 at 16:10
  • Again, there's a bug in the Body (which Request extends) and it returns empty ArrayBuffer when using FormData as an argument for body. See referenced bug. – Pawel Uchida-Psztyc Feb 05 '17 at 16:12
  • _"Body (which Request extends) and it returns empty ArrayBuffer"_. The resulting `ArrayBuffer` at linked bug report is not empty. The Answer converts properties of `FormData` to `ArrayBuffer`s. – guest271314 Feb 05 '17 at 16:19
  • What is input? If expected result is `ArrayBuffer` what is purpose of first creating `FormData` object? Why do you not use `FileReader` to convert `File` object to `ArrayBuffer`? – guest271314 Feb 05 '17 at 16:33
  • Not clear from Question if expected result is `multipart/form-data` or `ArrayBuffer`? Filed issue here https://github.com/whatwg/fetch/issues/480, then closed. If you believe there is a bug with `Request` or `fetch` implementation, consider describing issue there. – guest271314 Feb 05 '17 at 16:39
  • The expected result is `ArrayBuffer` containing multipart message body. But the question is to find an issue with the message in the question and not about how the message was created. I just can't see why this message is not going through. – Pawel Uchida-Psztyc Feb 05 '17 at 17:29
  • Once `FormData` is created you can use `TextEncoder` to encode the message body http://stackoverflow.com/questions/40111982/get-http-body-of-form-in-javascript text as `TypedArray`. Then use `TextDecoder` to decode the `TypedArray`. Though with for example `.png` files it may be necessary to first parse the raw binary data within message body. The simplest approach would be to create `ArrayBuffer`s from fields of `form`. Not sure why `FormData` is necessary within process? – guest271314 Feb 05 '17 at 17:31
  • The simplest approach would be to pass `data URI` of `File` object to `FormData`, to avoid having to parse binary representation of file within message body. That is, use `FileReader` to pass `data URI` to `FormData`, get `multipart/form-data` from reading `Response` or `Request`, then create `TypedArray` from the message. When reading `TypedArray` as text, parse the `data URI` from the original message body. Or, not use `FormData` at all. Use `FileReader` to convert form fields to `ArrayBuffer`s, negating the need to parse, re-parse `FormData` – guest271314 Feb 05 '17 at 17:41

2 Answers2

1

Finally found it. The issue was with the boundary. According to spec body parts are separated from each other with two dash (-) signs and the boundary string defined in content type. This two dashes missing in my message.

So the correct version of the message body is:

POST /post HTTP/1.1[NL]
HOST: localhost:8081[NL]
content-type: multipart/form-data; boundary=-------------805520824430161118507807[NL]
content-length: 17789[NL]

---------------805520824430161118507807[NL]
Content-Disposition: form-data; name="image"; filename="arc-icon.png"[NL]
Content-Type: image/png[NL]
[NL]
�PNG[NL]
[binary data for about ~17700 bytes)[NL]
[NL]
---------------805520824430161118507807--[NL]
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Pawel Uchida-Psztyc
  • 3,735
  • 2
  • 20
  • 41
0

Not sure how I can get HTTP body message from FormData :) I can only do it using Fetch API and the request object.

You can iterate FormData object, call .blob(), first, then pass Blob to new Response() with .arrayBuffer() chained.

<input name="files" type="file" accepts="image/*" multiple="multiple">
<script>
  document.querySelector("input[type=file]")
    .onchange = (e) => {
      const [files, fd, arr] = [e.target.files, new FormData(), Array()];

      for (let file of files) {
        console.log(file);
        fd.append(`file-${[...fd.keys()].length}`, file, file.name);
      }

      for (let [prop] of fd) {
        let request = new Request("/", {
          method: "POST",
          body: fd.get(prop)
        });
        arr.push(
          request.blob()
          .then(blob => new Response(blob).arrayBuffer())
        )
      }

      Promise.all(arr)
        .then(buffers => {
          for (let ab of buffers) {
            console.log(ab);
            let img = new Image;
            img.onload = () => {
              document.body.appendChild(img)
            }
            let url = URL.createObjectURL(new Blob([ab]));
            img.src = url;
          }
        })
    }
</script>

To read "multipart/form-data" use .text() chained to Request() then parse returned data using String or RegExp methods.

guest271314
  • 1
  • 15
  • 104
  • 177
  • You are right and this is what I did before. But in Chrome 56 there is a bug repoted by me earlier this week ( bugs.chromium.org/p/chromium/issues/detail?id=688100) that makes this impossible. And you don't need to iterate over FormData because the Request object can accept FormData and the `arrayBuffer` function produces with full multipart message body. But because of this bug I can't use it. So I'm looking for different way. – Pawel Uchida-Psztyc Feb 05 '17 at 16:04
  • @PawełPsztyć _"But in Chrome 56 there is a bug repoted by me earlier this week ( bugs.chromium.org/p/chromium/issues/detail?id=688100) that makes this impossible."_ Not sure what you mean by "impossible"? Does `javascript` at Answer not return expected result? Not sure that the result is a bug. – guest271314 Feb 05 '17 at 16:08
  • Your answer do not answers my question. The problem is not with creating ArrayBuffer from file. The question is about producing valid multipart message from form data. – Pawel Uchida-Psztyc Feb 05 '17 at 16:11
  • You can use `fetch`, `Request` or `Response` to create a valid `multipart/form-data` message, as demonstrated at Answer and linked Questions. – guest271314 Feb 05 '17 at 16:12
  • Why can you not use `fetch` where bug report uses `Request`? – guest271314 Feb 05 '17 at 16:14
  • Because the app is working directly on socket and it require ArrayBuffer (not FormData or any other). The app is an API testing tool and it uses low level network access, so until bug in the Request object is not fixed I can't use any of methods in your answers. – Pawel Uchida-Psztyc Feb 05 '17 at 16:17
  • Are you sure that there is a bug? And not that the result is expected? Have you asked the authors of `fetch` what the expected result is? – guest271314 Feb 05 '17 at 16:18
  • It's not expected result, spec says something else, the bug is confirmed in crbug. – Pawel Uchida-Psztyc Feb 05 '17 at 17:30
  • Can you share link to the specification section which you are referencing? – guest271314 Feb 05 '17 at 17:48
  • For creating an array buffer from Request object: https://fetch.spec.whatwg.org/#dom-body-arraybuffer. For handling FormData in the Request class: https://fetch.spec.whatwg.org/#bodyinit But still I don't understand why you need this? The question is not about this but the multipart spec and is generated by me message is OK with multipart spec. – Pawel Uchida-Psztyc Feb 05 '17 at 18:17
  • @PawełPsztyć Perhaps not gathering requirement correctly, here. – guest271314 Feb 05 '17 at 20:52