Batch script get html site and parse content (without wget, curl or other external app)

Question

I need to work with windows cmd functionality only. I need two vars/strings from a website to use in the batchscript for validate actions with it. To not make it too simple this website needs authentification in addition.

I found this somewhere:

@set @x=0 /*
:: ChkHTTP.cmd
@echo off
setlocal
set "URL=http://www.google.com"
cscript /nologo /e:jscript "%~f0" %URL% | find "200" > nul
if %ErrorLevel% EQU 0 (
echo Web server ok % Put your code here %
) else (
echo Web server error reported
)
goto :EOF

JScript */
var x=new ActiveXObject("Microsoft.XMLHTTP");
x.open("GET",WSH.Arguments(0));x.send();
while (x.ReadyState!=4) {WSH.Sleep(50)};
WSH.Echo(x.status)

But I'm not sure if it's possible to get the site content this way too instead of status answer and the more I don't know how to implement website authentification to this.

The above code does not work correctly as it will always produce error because of the pipe, but this seemed nearer to my needs of parsing the content I hoped.

Do note that the script you listed uses JScript. Are you able to use any other scripting languages besides just Batch (Powershell, JScript, VBScript, Etc...)? Also +1 to Prof Pickle — David Ruhmann, Mar 13 '13 at 21:24
powershell is not installed at the moment and would be a security risk i guess but the above code is running on the machine well, so jscript is possible and vbscript will work too through wsh i guess. the original script is pure batch and i need two strings from the website to be red in and used as vars to validate against some file deletion action. — peet, Mar 14 '13 at 06:42
I keep coming back to this one... I want to provide an answer, but I just... can't seem to figure out why are you doing this, lol. — rud3y, Mar 14 '13 at 15:21
as i told above the script is running over a share to clean up left over directories which are empty, to not disturb the process which is using and leaving those directories i have to first check which array it uses actually to not touch that area when deleting the unneeded ones. — peet, Mar 14 '13 at 16:10
Well, I'm glad you got an answer! Very interesting and specific piece of code :P — rud3y, Mar 15 '13 at 12:23

rojo · Accepted Answer · 2013-03-14T20:33:05.290

7

I've only ever used wget to fetch web content from a Windows batch script. Using an XHR via JScript was a fantastic idea!

But the script you're trying to plunder appears to be intended for checking whether a web server is responding, not for fetching content.

With some modifications, you can use it to fetch a web page and do whatever processing you need.

@if (@a==@b) @end /*

:: fetch.bat <url>
:: fetch a web page

@echo off
setlocal
if "%~1"=="" goto usage
echo "%~1" | findstr /i "https*://" >NUL || goto usage

set "URL=%~1"
for /f "delims=" %%I in ('cscript /nologo /e:jscript "%~f0" "%URL%"') do (
    rem process the HTML line-by-line
    echo(%%I
)
goto :EOF

:usage
echo Usage: %~nx0 URL
echo     for example: %~nx0 http://www.google.com/
echo;
echo The URL must be fully qualified, including the http:// or https://
goto :EOF

JScript */
var x=new ActiveXObject("Microsoft.XMLHTTP");
x.open("GET",WSH.Arguments(0),true);
x.setRequestHeader('User-Agent','XMLHTTP/1.0');
x.send('');
while (x.readyState!=4) {WSH.Sleep(50)};
WSH.Echo(x.responseText);

edited Mar 14 '13 at 20:33

answered Mar 14 '13 at 20:15

rojo

24,000
5
55
101

very cool rojo, now i could find the string, strip and use as var, but i need to authenticate to fetch the page. the site is protected with basic htaccess. is it possible to implement this in the jscript part anyway please? – peet Mar 14 '13 at 20:55
1

Easiest way would be to pass the auth info via the URL, like `http://username:password@www.site.com/etc`. I'm not positive that that will work. If it doesn't, then I can [add some code](http://zanstra.home.xs4all.nl/inTec/ServerXMLHTTP.htm) for `x.setRequestHeader('Authorization',etc)`. Or apparently the `x.open` method also supports [two optional arguments](http://www.4guysfromrolla.com/webtech/110100-1.2.shtml) to supply auth info. Let me know if you need me to pursue this. – rojo Mar 15 '13 at 01:10
you're great, i almost forgot about this easy way of authentification as ie and i guess most other browsers don't support this way anymore, but with the script it did work well for me. so the only intersting option would be if with other ways it would be possible to use a hash instead of clear password. anyway great work, thanks a lot man. – peet Mar 15 '13 at 04:59
hello rojo, i tried to parse but don't see the forest because of so much tree's around, here i asked for help [link]http://stackoverflow.com/questions/15493297/parse-batch-line-by-line maybe you could show me how to parse the returned lines right please? – peet Mar 19 '13 at 21:57
could you take a look please – peet Mar 19 '13 at 22:57
@peet - Jeez, settle down. :) OK, [as you wish](http://stackoverflow.com/a/15512544/1683264). Incidentally, if you want to put a link in a comment, do `[text to display](url of link)` with no space between, where the text is enclosed in brackets and the URL in parentheses. To link as you wished in your previous comment, you would type `[here I asked for help](http://stackoverflow.com/questions/15493297/)`. – rojo Mar 19 '13 at 23:43
you are totally right, i am fighting with syntax here sometimes, will change my use. i still did not get a markup working inside of code blocks like the `markup` syntax help sites always do and like possible in comments by backtic. could you explain to me what the first line @a==@b does? in the original code this was x, while x is used in the Jscript i can imagine some sence but your code does not use a or b? – peet Mar 22 '13 at 06:33
3

@peet - The `if (@a==@b) @end` line is a valid `if` statement in both Windows batch language and JScript. It's valid, but it's intentionally false. The interesting bit of that line is the `/*` at the end. That begins a multiline comment in JScript, so JScript ignores everything after `/*` until it encounters a `*/`. And since, in fact, `@a` does **not** equal `@b`, the Windows cmd interpreter does not bother trying to execute `@end /*`, but happily continues to process the next lines which JScript ignores as comments. – rojo Mar 22 '13 at 10:53
now i got it, cool, thank you very much for explaining. i got it working now except i don't manage to get the results passed back to the calling batch file. i tried different ways (except errorlevel retun) but am missing something, mybe you could gimme a light [here](http://stackoverflow.com/questions/15560198/returning-var-back-to-calling-batch-file)? – peet Mar 22 '13 at 17:39
1

This is awesome. So much better than wget. – JaseC Mar 21 '14 at 12:56

Batch script get html site and parse content (without wget, curl or other external app)

1 Answers1

Linked