How to format url with multiple question marks and ensure that the last query string is used

Question

In my node.js App, I am consuming an API that accepts a call as follows

http://myapi.com/action?url= https://www.test.com/play?action=jump

Then based on the parameter passed into the URL parameter(example – ‘action=jump’) the API returns a specific response.

The problem I am experiencing is, in some cases, the URL sent to the API may already have a question mark with a query string(?song=portrait)

http://myapi.com/action?url= https://www.test.com/play?song=portrait?action=jump

In this case, the API is sending back the incorrect response because it’s using the very first question mark parameter from the query string. Example ?song=portrait

Without modifying the API how can I ensure that the API reads the parameter at the end - example – action=jump

Is there a way to escape the first question mark yet keeping a valid URL or how can I use url.format() or url.parse() to solve this problem

I forgot to add that I am already encoding the URL that is passed into the URL parameter.

How does the API read parameters? It automatically takes second parameter and considers it 'jump', or it actually checks the names of the parameters passed? — David H., Jul 11 '16 at 15:57
You really need to use parse/encode, otherwise there are security risks where your site could be exploited to perform queries it wasn't designed for. — mksteve, Jul 18 '16 at 07:22
Out of curiosity, why do you need the whole url as one parameter? would it be easier to create requests like "http://myapi.com/action?url=www.test.com&params=song:portrait&params=action:jump" — Jared Beach, Jul 18 '16 at 13:01
When you say you are *consuming* an API, on first read I thought you were sending requests to that API. If so, you are going to have to understand the oddities of how that API parses, and build/rebuild the request line to match. When you say "in some cases" the request line is not in a form the API parses (as wanted), it sounds like you are neither the producer or the consumer. If you are neither, there is nothing you can do. Please clarify, are you are making requests of this API, or did you implement this API? — Preston L. Bannister, Jul 18 '16 at 15:45

score 4 · Answer 1 · edited May 23 '17 at 11:51

Consider use the url.parse()/url.format() to handle correctly urls in node.js.

You can try with:

var url = require('url');
  , apiurl = 'http://myapi.com/action';
  , urlObject = url.parse(apiurl);

// Add query string parameters
urlObject.query = {
  action: 'jump', 
  url: 'https://www.test.com/play?song=portrait'
};

// Generate valid url
var urlString = url.format(urlObject);
console.log(urlString);

And it will build a correct url: http://myapi.com/action?action=jump&url=https%3A%2F%2Fwww.test.com%2Fplay%3Fsong%3Dportrait

Also, remember that any other question marks after the first, are treated as literal.

score 3 · Answer 2 · answered Jul 06 '16 at 21:30

3

You should use encodeURIComponent (works in the browser and Node).

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent

answered Jul 06 '16 at 21:30

lucasjackson

1,515
1
11
21

1

If you are already encoding the url param, the action param should just be appended using an ampersand. ex: `'http://myapi.com/action?url=' + encodeURIComponent('https://www.test.com/play?song=portrait') + '&action=jump'` – lucasjackson Jul 06 '16 at 21:37
The api doesn't read the query string if I do &action=jump... only when I use the question mark – user1526912 Jul 06 '16 at 21:40

score 1 · Answer 3 · edited May 23 '17 at 12:22

Instead of adding ?action=jump You'd want to add &action=jump to this url: https://www.test.com/play%3F.

Notice the %3F-part, it is a encoded ?.

This does not work since it would just add that part as another parameter to the http://myapi.com/action-url.

And I know that you're already encoding your url, but it does not encode the & so we will have to do that manually.

You can solve this by changing the & to %26 when you add your parameters.

Example:

http://myapi.com/action?url=https://www.test.com/play%3Fsong=portrait%26action=jump

And on the server-side you'd want to use decodeURIComponent(req.query.url) to parse the %26 back to &.

This way, you'll have a complete correct url like this:

https://www.test.com/play?song=portrait&action=jump

With this string, you could use the function from this thread: How can I get query string values in JavaScript?

function getParameterByName(name, url) {
    name = name.replace(/[\[\]]/g, "\\$&");
    var regex = new RegExp("[?&]" + name + "(=([^&#]*)|&|#|$)"),
        results = regex.exec(url);
    if (!results) return null;
    if (!results[2]) return '';
    return decodeURIComponent(results[2].replace(/\+/g, " "));
}

And call it with:

getParameterByName('action', 'https://www.test.com/play?song=portrait&action=jump')

This will return jump.

Now you can parse your url and extract the parameter "action" to always ensure you get the correct parameter, disregarding the position.

score 0 · Answer 4 · answered Jul 16 '16 at 20:33

0

Maybe I'm oversimplifying but couldn't you use lastIndexOf():

var url = 'http://myapi.com/action?url= https://www.test.com/play?song=portrait?action=jump';
var params = url.slice(url.lastIndexOf('?')+1);
console.log(params) // action=jump

answered Jul 16 '16 at 20:33

Donnie D'Amato

3,832
1
15
40

score 0 · Answer 5 · answered Jul 18 '16 at 12:15

When u constructing the url with query string values then first call encodeURIComponent method by passing the querystring values then frame the URL.

Ex:
var encodedURLComp="http://myapi.com/action?url="+window.encodeURIComponent("https://www.test.com/play?song=portrait?action=jump"); alert(encodedURLComp);

It Encodes the following literals @#$&=:/,;?+'

How to format url with multiple question marks and ensure that the last query string is used

5 Answers5