0

I need to interact with a remote HTTP server at the lowest possible level (i.e.: at socket level) because my target is a very small embedded system with no support for higher level libraries (it's a bare-metal uController wit no O.S. at all and talking to a GSM modem via serial line; modem has some support for sockets, but nothing above that).

Basic need is to upload a "file" using POST.

I have all needed Header/Body in place and it "usually works".

Problem is I randomly get a "HTTP/1.1 502 Bad gateway error" response and this is more likely to happen as the size of "file" increases.

I understand this means there's some problem between the reverse proxy frontend (nginx, apparently) and the backends, but I have absolutely no control on those (actually I dont't really know how the atual setup besides what can be gleamed from (light) probing).

My current strategy is to open a plain socket and send the folowing sequence (dots represent binary data):

POST /path/to/websend.php HTTP/1.0
Host: host.domain.tld
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:33.0) Gecko/20100101 Firefox/33.0
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Type: multipart/form-data; boundary=AaB03x
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Accept: */*
Content-Length: <full_length>

--AaB03x
Content-Disposition: form-data; name="IV"
Content-Type: application/data
Content-Transfer-Encoding: binary

000102030405060708090A0B0C0D0E0F
--AaB03x
Content-Disposition: form-data; name="S_TXT_FILE"; filename="FILENAME_s.txt"
Content-Type: application/data
Content-Transfer-Encoding: binary

..............................................................
..............................................................
...... several 512byte blocks ................................
..............................................................
..............................................................
--AaB03x--

Is there something I could do to enhance reliability? I already do multiple retries and this actually works, but sometimes I need to retry six or more times to have a positive answer (200 OK).

Note I send exactly the same sequence on rety and it succeeds... eventually.

I need to send two parts because content is encrypted and first part is the neded "Initialization Vector".

Clifford
  • 88,407
  • 13
  • 85
  • 165
ZioByte
  • 2,690
  • 1
  • 32
  • 68
  • If you don't already have ruled it out, I would suggest to verify that the server is not causing the problem first, for example by using multiple cURL commands such as: `curl -X POST --data-binary @path/to/my-file.txt http://example.com/`.[A Simple Explanation of a 502 Bad Gateway Error & How to Resolve It](https://blog.hubspot.com/marketing/502-bad-gateway) is a good read. Capturing a network trace with Wireshark or tcpdump in order to verify that your request is well formed may be useful as well. – Frant May 11 '21 at 13:32
  • See [this post](https://stackoverflow.com/questions/19116016/what-is-the-right-way-to-post-multipart-form-data-using-curl) for how to post using multipart using cURL: `curl -X PUT URL --header 'Content-Type: multipart/form-data; boundary=---------BOUNDARY' --data-binary @file` - this is a `PUT` example. – Frant May 11 '21 at 13:43
  • Thanks @Frant I already tried sending using cURL. I sill get problems sometimes, but they are *way* less frequent. My guess is there's a huge difference in speed between DSL-connected PC and a GSM-connected embedded uController. I cannot capture anything because there's no "network" in the normal meaning; I just have a "smart GSM Modem" ( "SIM900_ATC", if it matters; I use AT+CIPSTART/AT+CIPSEND) – ZioByte May 11 '21 at 14:03
  • Anyway you can duplicate the remote nginx server (a minima) on a PC with Internet access you would run your own server on, so that you may see what is going on on the network by capturing traces on the PC, modify the server log level, and consult the logs on the server side at the time the problem occurs ? – Frant May 11 '21 at 14:10
  • I can think about that, but I don't really know how the real remote is actually configured. I don't think I can probe it enough to prepare a significant duplicate, can I? – ZioByte May 11 '21 at 14:41
  • If your issue is caused by a huge difference in speed between DSL-connected PC and a GSM-connected embedded uController, the precise configuration of the remote nginx server may not matter that much. A minimalist nginx+php setup with a the minimal amount of code required for handling a multipart POST would suffice to collect more information I guess. This could be a minimal spring-boot REST app as well, it probably does not matter. The right choice should probably be guided by the WEB technology you are the most familiar with. – Frant May 11 '21 at 14:46
  • When you say "usually works", what is the successes/failures ratio you are observing ? – Frant May 11 '21 at 15:14
  • @Frant: short messages (^200 bytes) go through 4 out of 5 without retries. Long messages (>10k bytes) sometimes need 6/7 retries to go through. I also see a circadian cycle most likely due to network status. – ZioByte May 11 '21 at 15:28
  • Would a scenario where you would send your messages to a relay you would 100% control be possible ? – Frant May 11 '21 at 15:39
  • @Frant: not sure. I will inquire tomorrow. That *might* be a nice idea... if at all feasible. – ZioByte May 11 '21 at 21:29
  • It is definitively feasible, please let me now in the case you would need more information. – Frant May 12 '21 at 17:15

0 Answers0