What's the effective way for compress URL of 1200-1500 characters?

Question

I have many problematics in this question. I have to compress URI which doesn't have a static architecture then it's hard to do a things like that :

repos=aaa,bbb,ccc&
labels=ddd,eee,fff&
milestones=ggg,hhh,iii&
username=kkk&
show_open=0&
show_closed=1&
show_commented=1&
show_uncommented=0

extract:

aaa,bbb,ccc|ddd,eee,fff|ggg,hhh,iii|kkk|0110

The request are made a little bit in a GraphQL way so we send structure and have answers for this structure. For the moment the compression we use is juste changing some wording which are in all requests (function: is f:, ..). We want to do a generic compression, not something we have to maintain if the architecture change(in the following example we don't want to change quizz.campaign to #12 per example). We do this kind of request one to two times per connection so what is the most expensive between download a library at the connection which will compress my request from 1500 to maybe 300 characters OR do some generic compression like change the [ to ( which will compress from 1500 to 900.

The generic compression could look like something like that:

str.replace(/\*/g, "%#111")
        .replace(/\!/g, "%#222")
        .replace(/\(/g, "%#333")
        .replace(/\)/g, "%#444")
        .replace(/\~/g, "%#555")
        .replace(/\_/g, "%#666")
        .replace(/\'/g, "%#777")
        .replace(/\,/g, "*")
        .replace(/\:/g, "!")
        .replace(/\[/g, "(")
        .replace(/\]/g, ")")
        .replace(/\{/g, "~")
        .replace(/\}/g, "_")
        .replace(/\"/g, "'")
        .replace(/\%#111/g, ",")
        .replace(/\%#222/g, ":")
        .replace(/\%#333/g, "[")
        .replace(/\%#444/g, "]")
        .replace(/\%#555/g, "{")
        .replace(/\%#666/g, "}")
        .replace(/\%#777/g, '"');

The json of a request looks like that:

    [{
    "f": "find-qzc-by-id",
    "k": "qzcpn",
    "a": {
        "quizz.campaign/id": "quizz-kdfjdslk"
    },
    "n": [{
        "f": "quizz.campaign/activeQuizzQuestions",
        "k": "quizz.campaign/activeQuizzQuestions",
        "n": ["fanQuestion/id", "fanQuestion/questionText", "fanQuestion/questionType", {
            "f": "fanQuestion/choices",
            "k": "fanQuestion/choices",
            "n": ["quizz.question.choice/bgImg", "fanQuestion.choice/id", "fanQuestion.choice/adminLabel", "fanQuestion.choice/text"]
        }, "bs.model/scid", "bs.model/uiConfig"]
    }, "quizz.campaign/id", "quizz.campaign/gameLogic", "quizz.campaign/quizzConfig", "quizz.campaign/bgImg", "quizz.campaign/previewImage", "quizz.campaign/title", "quizz.campaign/description", "campaign/publicOrganizerName", {
        "f": "campaign/organization",
        "k": "campaign/organization",
        "n": ["org/id"]
    }, "campaign/bsCpnType", "bs.model/scid", "bs.model/uiConfig", "bs.model/bsCustomContent", "bs.time/created", "bs.model/nl"]
}]

Then the encoded request looks like that :

?q=%5B%7B%22f%22%3A%22find-qzc-by-id%22%2C%22k%22%3A%22qzcpn%22%2C%22a%22%3A%7B%22quizz.campaign%2Fid%22%3A%22quizz-kdfjdslk%22%7D%2C%22n%22%3A%5B%7B%22f%22%3A%22quizz.campaign%2FactiveQuizzQuestions%22%2C%22k%22%3A%22quizz.campaign%2FactiveQuizzQuestions%22%2C%22n%22%3A%5B%22fanQuestion%2Fid%22%2C%22fanQuestion%2FquestionText%22%2C%22fanQuestion%2FquestionType%22%2C%7B%22f%22%3A%22fanQuestion%2Fchoices%22%2C%22k%22%3A%22fanQuestion%2Fchoices%22%2C%22n%22%3A%5B%22quizz.question.choice%2FbgImg%22%2C%22fanQuestion.choice%2Fid%22%2C%22fanQuestion.choice%2FadminLabel%22%2C%22fanQuestion.choice%2Ftext%22%5D%7D%2C%22bs.model%2Fscid%22%2C%22bs.model%2FuiConfig%22%5D%7D%2C%22quizz.campaign%2Fid%22%2C%22quizz.campaign%2FgameLogic%22%2C%22quizz.campaign%2FquizzConfig%22%2C%22quizz.campaign%2FbgImg%22%2C%22quizz.campaign%2FpreviewImage%22%2C%22quizz.campaign%2Ftitle%22%2C%22quizz.campaign%2Fdescription%22%2C%22campaign%2FpublicOrganizerName%22%2C%7B%22f%22%3A%22campaign%2Forganization%22%2C%22k%22%3A%22campaign%2Forganization%22%2C%22n%22%3A%5B%22org%2Fid%22%5D%7D%2C%22campaign%2FbsCpnType%22%2C%22bs.model%2Fscid%22%2C%22bs.model%2FuiConfig%22%2C%22bs.model%2FbsCustomContent%22%2C%22bs.time%2Fcreated%22%2C%22bs.model%2Fnl%22%5D%7D%5D

If you think that a library would be the better way for the cost question, do you have some recommandations for compress in js and decompress on the jvm ? Thanks.

we can't really answer without knowing what library would be used (and its size). In your scenario, does the library get loaded only once or for each request? If the first, using a library seems right. Also, what is the reason for this URL compression and why not pass data with POST? — Kaddath, Jan 30 '18 at 09:01
Because using GET is more optimized in html and when you do any request except GET you have 2 calls, the first is a request of type OPTIONS then it do the POST request. We really want to optimize all the things that can be optimized. And for the moment i don't have any library in mind, i'm not closed but it could be a thing like lz-string i think. — user3659739, Jan 30 '18 at 09:16
Seems to be more about esoteric optimizations, at least, we can safely say that this isn’t even remotely connected to the [jvm] tag. — Holger, Jan 30 '18 at 09:23
Ok i see, actually for OPTIONS in POST, this happens only if you set a custom header or if content-type is not among some certain values, [see here](https://stackoverflow.com/a/13030629/7393478) — Kaddath, Jan 30 '18 at 09:25
That was only examples but for my problem i really have to use the GET method, that's a constrains. — user3659739, Jan 30 '18 at 09:33
@Kaddath after some tests i can assume that the library i'll use will weight approximatively 5ko and the request will weight 750o. What's the best between dowload 5ko and send a 750o request OR download 0ko and send a 925o request ? Thanks — user3659739, Jan 30 '18 at 15:16
It still depends if the library is loaded only once, for multiple requests, or for all requests. It also depends on what you want to be the focus: time or data weight? If the implementation is not too long, i would suggest doing a little benchmark to compare the solutions, by measuring what you want (time/weight) directly in your environment (you can also compare with gzip as proposed in an answer) — Kaddath, Jan 30 '18 at 16:05

score 0 · Answer 1 · answered Jan 30 '18 at 09:24

Simply using POST doesn't require OPTION request, check https://stackoverflow.com/a/27924344/158037

If you really want to use compression you don't have to do it in JS. For direction "server -> browser" enabling deflate compression ("Accept-Encoding: deflate", gzip or br) should be enough and won't require any changes in code, maybe just enabling it in server configuration (apache, tomcat, whatever You use).

For "browser->server" and actually using the query string using HTTP 2.0 would be simplest solution as it compressed all headers, including "Path:".

This is an interesting point but for some constraints we must use the GET method, especially for the http cache per example. — user3659739, Jan 30 '18 at 09:32

What's the effective way for compress URL of 1200-1500 characters?

1 Answers1