4

I am using a combined batch and java script I found to retrieve the html from a web site using a batch file and one we address is not returning the desired output as it appears when I use the url in firefox.

The script I am using to pull the html is:

@if (@This==@IsBatch) @then
@echo off
rem **** batch zone     *********************************************************

setlocal enableextensions disabledelayedexpansion

rem Batch file will delegate all the work to the script engine 
if not "%~1"=="" (
    cscript //E:JScript "%~dpnx0" %1
)

rem End of batch area. Ensure batch ends execution before reaching
rem javascript zone
exit /b

@end
// **** Javascript zone     *****************************************************

// Instantiate the needed component to make url queries
var http = WScript.CreateObject('MSXML2.ServerXMLHTTP.6.0');

// Retrieve the url parameter
var url = WScript.Arguments.Item(0)

// Make the request

http.open("GET", url, false);
http.send();

// If we get a OK from server (status 200), echo data to console

if (http.status === 200) WScript.StdOut.Write(http.responseText);

// All done. Exit
WScript.Quit(0);

The url I am trying to feed the script is http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=["Arabian+Nights"]

or alternativly http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=["Arabian Nights"]

the problem seems to be the space/+ as none of the other urls I feed it are using a space or +

The way I am calling the script to pull the html is:

call callurl.cmd "http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=["Arabian+Nights"]"

edit: found original thread the script is from Open a URL without using a browser from a batch file

only change i made was Msxml2.XMLHTTP.6.0 was changed to MSXML2.ServerXMLHTTP.6.0 because original script couldn't load sites due to security from what I found.

Community
  • 1
  • 1

3 Answers3

5

In this case the problem is that the windows scripting host consumes the double quotes included in the arguments.

npocmaka has shown one of the solutions: encode the quotes in the url. From my point of view it is the correct one (double quote is an unsafe character and should be encoded).

Another solution is to not to pass the URL as an argument to the script, but to store it in a environment variable and then in the javascript part retrieve the value from the variable

@if (@This==@IsBatch) @then
@echo off
rem **** batch zone *********************************************************

    setlocal enableextensions disabledelayedexpansion

    rem Ensure we get a correct reference to current batch file
    call :getFullBatchReference _f0

    rem Batch file will delegate all the work to the script engine 
    if not "%~1"=="" (
        set "URL=%~1"
        cscript //nologo //E:JScript "%_f0%"
    )

    rem Ensure batch ends execution before reaching javascript zone
    exit /b %errorlevel%

:getFullBatchReference returnVar
    set "%~1=%~f0"
    goto :eof

@end
// **** Javascript zone *****************************************************
// Instantiate the needed component to make url queries
var http = WScript.CreateObject('MSXML2.ServerXMLHTTP.6.0');

// Retrieve the url parameter from environment variable
var url = WScript.CreateObject('WScript.Shell')
            .Environment('Process')
            .Item('URL');

var exitCode = 0;

    try {
        // Make the request
        http.open("GET", url, false);
        http.send();

        // If we get a OK from server (status 200), echo data to console
        if (http.status === 200) {
            WScript.StdOut.Write(http.responseText);
        } else {
            exitCode = http.status;
        };

    } catch (e) {
        // Something failed
        WScript.StdOut.Write('ERROR: ' + e.description );
        exitCode = 1;
    };

    // All done. Exit
    WScript.Quit( exitCode );

Now, it can be called as

geturl.cmd "http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=["Arabian+Nights"]"
Community
  • 1
  • 1
MC ND
  • 69,615
  • 8
  • 84
  • 126
  • Thank you for the help but I still can't get the output i want the html should list a bunch of lines with text such as – red death68 Sep 20 '16 at 17:55
  • @reddeath68, tested as `geturl.cmd "http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=["Arabian+Nights"]"` and I get the indicated output. What do you get? – MC ND Sep 20 '16 at 18:00
  • http://pastebin.com/YRdJrsnS is the output i get and its worth noting when added to cmd line it added the url as "http://gatherer.wizards.com/Pages/Search/Default.aspx?outpu??t=spoiler&method=vis??ual&action=advanced&??set=["Arabian+Nights??"]" note the double question marks – red death68 Sep 20 '16 at 18:07
  • @reddeath68, for some reason (I don't know why) there are non visible characters when the command line is copied from the previous comment. I've included it in the answer. Can you test it? – MC ND Sep 20 '16 at 18:17
  • that did it so it was passing hidden characters to the script in the first place? – red death68 Sep 20 '16 at 18:21
  • @reddeath68, no, I copied the URL from your question and it worked well. But the text in the comment (copied and pasted from my console) included some hidden characters. If you copied the pasted command, your url was invalid and that led to the wrong page output. – MC ND Sep 20 '16 at 18:30
  • good to know thanks for the help this was driving me nuts for a few days. – red death68 Sep 20 '16 at 20:06
3

call the cscript like that:

cscript //E:JScript "%~dpnx0" "%~1"

I dont think the spaces needs to be encoded but rather the double quotes (with %22) though this could require to parse the whole command line (%*) you can try something like

setlocal enableDelayedExpansion
set "link=%*"
set "link=!link:"=%%22!"
....
 cscript //E:JScript "%~dpnx0" "%link%"

You can also try with named arguments and pass the whole command line to the script.

npocmaka
  • 55,367
  • 18
  • 148
  • 187
  • The script is a hybrid between javascript and batch as the first post shows and it is saved as callurl.cmd as the post said to name it – red death68 Sep 20 '16 at 08:09
  • I guess he means within the batch-file where you move to the JS-part. – geisterfurz007 Sep 20 '16 at 08:12
  • @geisterfurz007 - Yes.I know this technique.Rather the problem is in the double quotes and script thinks there are two arguments. – npocmaka Sep 20 '16 at 08:16
  • the changes lead to the following output C:\Users\reddeath68\Desktop\test>test.bat 'method' is not recognized as an internal or external command, operable program or batch file. 'action' is not recognized as an internal or external command, operable program or batch file. The syntax of the command is incorrect. C:\Users\reddeath68\Desktop\test\callurl.cmd(39, 1) Microsoft JScript compilatio n error: Syntax error – red death68 Sep 20 '16 at 08:22
  • @reddeath68 - try to remove `echo %link%` line. – npocmaka Sep 20 '16 at 08:25
  • @reddeath68 now the `&` is taken as control symbol and leaved unquoted will harm the script. – npocmaka Sep 20 '16 at 08:26
  • 1
    Your approach is correct (change the quotes in the URL to `%22`), but using named arguments will not solve anything. The reason for the problem is that the windows script host remove any double quote in the arguments, so the inner javascript code will not include them in the URL making the request fail – MC ND Sep 20 '16 at 08:40
  • and how do we solve this then? I have limited batch knowledge and next to no javascript knowledge – red death68 Sep 20 '16 at 08:42
0

simply replace the space or plus-sign + with a URL encoded space %20.

e.g. http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=["Arabian%20Nights"]

Fraser
  • 15,275
  • 8
  • 53
  • 104
  • the script does not seem to interpret the %20 properly maybe because its being fed from a batch file? – red death68 Sep 20 '16 at 07:45
  • 2
    You need to escape the `%` with another `%`, so use `%%20` instead. – SomethingDark Sep 20 '16 at 07:48
  • its returning the site like this http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=[%22Arabian%%20Nights%22] instead of like this http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=visual&action=advanced&set=[%22Arabian%20Nights%22] ill edit first post to show how I am calling the script from the batch file maybe im calling it wrong – red death68 Sep 20 '16 at 08:05
  • @SomethingDark - there are also double quotes in the link. I don't think the space is the problem – npocmaka Sep 20 '16 at 08:14