How to powershell curl emailrep.io

Question

When I try to use Powershell to perform a curl command on emailrep.io I do not get the proper output.

I have tried the following three commands, however they do not return the expected output, namely a proper formatted JSON:

curl emailrep.io/bsheffield432@gmail.com
Invoke-RestMethod emailrep.io/bsheffield432@gmail.com
Invoke-WebRequest emailrep.io/bsheffield432@gmail.com

The output of the last command looks as follows

StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html lang="en">
                      <head>
                        <meta charset="utf-8">
                        <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
                        <meta name="description" content="">...
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Accept-Ranges: bytes
                    Content-Length: 16661
                    Content-Type: text/html
                    Date: Wed, 07 Aug 2019 13:44:10 GMT
                    ETag: "5d3f381c-4115"
                    Last-Modified: Mon, 29 Jul 20...
Forms             : {search-form}
Headers           : {[Connection, keep-alive], [Accept-Ranges, bytes], [Content-Length, 16661], [Content-Type,
                    text/html]...}
Images            : {@{innerHTML=; innerText=; outerHTML=<IMG style="WIDTH: 35px" src="./assets/img/logo-light.png">;
                    outerText=; tagName=IMG; style=WIDTH: 35px; src=./assets/img/logo-light.png}, @{innerHTML=;
                    innerText=; outerHTML=<IMG src="./assets/img/social_twitter.png" width=19>; outerText=;
                    tagName=IMG; src=./assets/img/social_twitter.png; width=19}, @{innerHTML=; innerText=;
                    outerHTML=<IMG src="./assets/img/github_logo.png" width=20>; outerText=; tagName=IMG;
                    src=./assets/img/github_logo.png; width=20}, @{innerHTML=; innerText=; outerHTML=<IMG
                    class=company-logo-img src="">; outerText=; tagName=IMG; class=company-logo-img; src=}...}
InputFields       : {@{innerHTML=; innerText=; outerHTML=<INPUT id=search-input class="search-input form-control
                    form-control-lg" name=search-input placeholder="Enter an email">; outerText=; tagName=INPUT;
                    id=search-input; class=search-input form-control form-control-lg; name=search-input;
                    placeholder=Enter an email}}
Links             : {@{innerHTML=<IMG style="WIDTH: 35px" src="./assets/img/logo-light.png"> ; innerText= ;
                    outerHTML=<A id=home-link href="https://emailrep.io"><IMG style="WIDTH: 35px"
                    src="./assets/img/logo-light.png"> </A>; outerText= ; tagName=A; id=home-link;
                    href=https://emailrep.io}, @{innerHTML=API; innerText=API; outerHTML=<A class=nav-link
                    href="#api">API</A>; outerText=API; tagName=A; class=nav-link; href=#api}, @{innerHTML=Contact;
                    innerText=Contact; outerHTML=<A class=nav-link href="/contact">Contact</A>; outerText=Contact;
                    tagName=A; class=nav-link; href=/contact}, @{innerHTML=Docs; innerText=Docs; outerHTML=<A
                    class=nav-link href="/docs">Docs</A>; outerText=Docs; tagName=A; class=nav-link; href=/docs}...}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 16661

I installed curl for windows and I can get it to work with the following command. However there should be a powershell alternative to perform this.

curl.exe emailrep.io/bsheffield432@gmail.com
{
  "email": "bsheffield432@gmail.com",
  "reputation": "none",
  "suspicious": true,
  "references": 0,
  "details": {
    "blacklisted": false,
    "malicious_activity": false,
    "malicious_activity_recent": false,
    "credentials_leaked": false,
    "credentials_leaked_recent": false,
    "data_breach": false,
    "last_seen": "never",
    "domain_exists": true,
    "domain_reputation": "n/a",
    "new_domain": false,
    "days_since_domain_creation": 8760,
    "suspicious_tld": false,
    "spam": false,
    "free_provider": true,
    "disposable": false,
    "deliverable": true,
    "accept_all": false,
    "valid_mx": true,
    "spoofable": true,
    "spf_strict": true,
    "dmarc_enforced": false,
    "profiles": []
  }
}

This is HTML and not JSON.

Welcome to SO! Which kind of request (GET, POST) do you want to send? Could you also please clarify what is command and what is error message? — B--rian, Aug 07 '19 at 14:13
I am using a GET request. The expected output would be in JSON format however when I try using one of the power shell native commands such as Invoke-WebRequest I get the html from the website. — httpSteve, Aug 07 '19 at 14:28
It may be easier if I add the DOC API: https://emailrep.io/docs/?shell#query-an-email — httpSteve, Aug 07 '19 at 14:31

B--rian · Answer 1 · 2019-08-07T14:56:07.910

I am not sure why it is necessary to use curl if you have Powershell-native commands.

Assume you want to perform the following GET-request

curl -X GET "http://me:mypassword@myserver.com:1234/"

The same thing in PowerShell is slightly longer:

$user = 'me'
$pass = 'mypass'

$pair = "$($user):$($pass)"

$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))

$basicAuthValue = "Basic $encodedCreds"

$Headers = @{
    Authorization = $basicAuthValue
}

Invoke-WebRequest -Uri 'http://myserver.com:1234' -Headers $Headers

If you do not need authentication, it would be

curl -X GET "http://myserver.com:1234/"

which is then also easy in PowerShell:

Invoke-WebRequest -Uri 'http://myserver.com:1234'

Further reading on parsing JSON with PowerShell:

How to parse JSON from the Invoke-WebRequest in PowerShell?

The solution offered there to receive nice JSON back is

$response = Invoke-WebRequest -Uri "https://myserver.com:1234/"
$jsonObj = ConvertFrom-Json $([String]::new($response.Content))

There is no authentication required. So I would assume the following would work `Invoke-WebRequest emailrep.io/bsheffield432@gmail.com` — httpSteve, Aug 07 '19 at 14:31

Mathias R. Jessen · Accepted Answer · 2019-08-10T21:11:38.443

emailrep.io seems to be doing a naive check for curl in the user-agent string. Override the user-agent string with Invoke-WebRequest and you should get the expected response:

$rep = Invoke-WebRequest https://emailrep.io/bill@microsoft.com -UserAgent not_actually_curl |ConvertFrom-Json

# Since we piped it to `ConvertFrom-Json`, $rep is now a structured object:
if($rep.reputation -ne 'high' -or $rep.suspicious){
    Write-Warning "<$($rep.email)> is not to be trusted!"
}

I reached out to the maintainer of emailrep.io and they actually updated the API to always return json for non-browsers (including Windows PowerShell web cmdlets), so the above should no longer be necessary

How to powershell curl emailrep.io

2 Answers2