Why do Azure decode Requested URL before sending request to app

Question

I requested this url:

https://dataapi.kristiansand.onegde.no/api/case/2020000138/2020000138-25/202318/S%C3%B8rveivtestFil%2Fmed%2FSash%20og%20space.PDF

and this is the log stream from the receiving server

<html xmlns="http://www.w3.org/1999/xhtml">

<head>
    <title>IIS Detailed Error - 404.0 - Not Found</title>
    <style type="text/css">
        <!--body{margin:0;font-size:.7em;font-family:Verdana,Arial,Helvetica,sans-serif;}code{margin:0;color:#006600;font-size:1.1em;font-weight:bold;}.config_source code{font-size:.8em;color:#000000;}pre{margin:0;font-size:1.4em;word-wrap:break-word;}ul,ol{margin:10px 0 10px 5px;}ul.first,ol.first{margin-top:5px;}fieldset{padding:0 15px 10px 15px;word-break:break-all;}.summary-container fieldset{padding-bottom:5px;margin-top:4px;}legend.no-expand-all{padding:2px 15px 4px 10px;margin:0 0 0 -12px;}legend{color:#333333;;margin:4px 0 8px -12px;_margin-top:0px;font-weight:bold;font-size:1em;}a:link,a:visited{color:#007EFF;font-weight:bold;}a:hover{text-decoration:none;}h1{font-size:2.4em;margin:0;color:#FFF;}h2{font-size:1.7em;margin:0;color:#CC0000;}h3{font-size:1.4em;margin:10px 0 0 0;color:#CC0000;}h4{font-size:1.2em;margin:10px 0 5px 0;}#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS",Verdana,sans-serif;color:#FFF;background-color:#5C87B2;}#content{margin:0 0 0 2%;position:relative;}.summary-container,.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}.content-container p{margin:0 0 10px 0;}#details-left{width:35%;float:left;margin-right:2%;}#details-right{width:63%;float:left;overflow:hidden;}#server_version{width:96%;_height:1px;min-height:1px;margin:0 0 5px 0;padding:11px 2% 8px 2%;color:#FFFFFF;background-color:#5A7FA5;border-bottom:1px solid #C1CFDD;border-top:1px solid #4A6C8E;font-weight:normal;font-size:1em;color:#FFF;text-align:right;}#server_version p{margin:5px 0;}table{margin:4px 0 4px 0;width:100%;border:none;}td,th{vertical-align:top;padding:3px 0;text-align:left;font-weight:normal;border:none;}th{width:30%;text-align:right;padding-right:2%;font-weight:bold;}thead th{background-color:#ebebeb;width:25%;}#details-right th{width:20%;}table tr.alt td,table tr.alt th{}.highlight-code{color:#CC0000;font-weight:bold;font-style:italic;}.clear{clear:both;}.preferred{padding:0 5px 2px 5px;font-weight:normal;background:#006633;color:#FFF;font-size:.8em;}-->
    </style>
</head>

<body>
    <div id="content">
        <div class="content-container">
            <h3>HTTP Error 404.0 - Not Found</h3>
            <h4>The resource you are looking for has been removed, had its name changed, or is temporarily unavailable.
            </h4>
        </div>
        <div class="content-container">
            <fieldset>
                <h4>Most likely causes:</h4>
                <ul>
                    <li>The directory or file specified does not exist on the Web server.</li>
                    <li>The URL contains a typographical error.</li>
                    <li>A custom filter or module, such as URLScan, restricts access to the file.</li>
                </ul>
            </fieldset>
        </div>
        <div class="content-container">
            <fieldset>
                <h4>Things you can try:</h4>
                <ul>
                    <li>Create the content on the Web server.</li>
                    <li>Review the browser URL.</li>
                    <li>Create a tracing rule to track failed requests for this HTTP status code and see which module is
                        calling SetStatus. For more information about creating a tracing rule for failed requests, click
                        <a href="http://go.microsoft.com/fwlink/?LinkID=66439">here</a>. </li>
                </ul>
            </fieldset>
        </div>
        <div class="content-container">
            <fieldset>
                <h4>Detailed Error Information:</h4>
                <div id="details-left">
                    <table border="0" cellpadding="0" cellspacing="0">
                        <tr class="alt">
                            <th>Module</th>
                            <td>AspNetCoreModuleV2</td>
                        </tr>
                        <tr>
                            <th>Notification</th>
                            <td>ExecuteRequestHandler</td>
                        </tr>
                        <tr class="alt">
                            <th>Handler</th>
                            <td>aspNetCore</td>
                        </tr>
                        <tr>
                            <th>Error Code</th>
                            <td>0x00000000</td>
                        </tr>
                    </table>
                </div>
                <div id="details-right">
                    <table border="0" cellpadding="0" cellspacing="0">
                        <tr class="alt">
                            <th>Requested URL</th>
                            <td>https://testdataapiwin:80/api/case/2020000138/2020000138-25/202318/SørveivtestFil/med/Sash og space.PDF</td>
                        </tr>
                        <tr>
                            <th>Physical Path</th>
                            <td>D:\home\site\wwwroot\api\case\2020000138\2020000138-25\202318\SørveivtestFil\med\Sash og space.PDF</td>
                        </tr>
                        <tr class="alt">
                            <th>Logon Method</th>
                            <td>Anonymous</td>
                        </tr>
                        <tr>
                            <th>Logon User</th>
                            <td>Anonymous</td>
                        </tr>
                    </table>
                    <div class="clear">
                    </div>
                </div>
            </fieldset>
        </div>
        <div class="content-container">
            <fieldset>
                <h4>More Information:</h4>
                This error means that the file or directory does not exist on the server. Create the file or directory and try the request again.
                <p>
                    <a href="http://go.microsoft.com/fwlink/?LinkID=62293&amp;IIS70Error=404,0,0x00000000,14393">View more information &raquo;</a>
                </p>
                <p>Microsoft Knowledge Base Articles:</p>
            </fieldset>
        </div>
    </div>
</body>

</html>
2020-01-22 16:51:18.785 +00:00 [Information] Microsoft.AspNetCore.Hosting.Internal.WebHost: Request starting HTTP/1.1 GET https://dataapi.kristiansand.onegde.no/api/case/2020000138/2020000138-25/202318/SørveivtestFil/med/Sash og space.PDF
2020-01-22 16:51:18.785 +00:00 [Trace] Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware: All hosts are allowed.
2020-01-22 16:51:18.785 +00:00 [Trace] Microsoft.AspNetCore.HttpsPolicy.HstsMiddleware: Adding HSTS header to response.
2020-01-22 16:51:18.786 +00:00 [Debug] Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware: The request path /api/case/2020000138/2020000138-25/202318/SørveivtestFil/med/Sash og space.PDF does not match an existing file
2020-01-22 16:51:18.786 +00:00 [Debug] Microsoft.AspNetCore.Routing.Matching.DfaMatcher: No candidates found for the request path '/api/case/2020000138/2020000138-25/202318/SørveivtestFil/med/Sash og space.PDF'
2020-01-22 16:51:18.786 +00:00 [Debug] Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware: Request did not match any endpoints
2020-01-22 16:51:18.786 +00:00 [Information] Microsoft.AspNetCore.Hosting.Internal.WebHost: Request finished in 0.9629ms 404
2020-01-22T16:51:18  PID[17896] Information Sending response: 200.0 OK

Azure stats:

App Service
Stack: .NET Core
HTTP version: 1.1
OS: Windows
Region: North Europe

As you can see, the Requested URL has been decoded before arriving the application.

Is this something I can disable in Azure?

A URL and headers are never encrypted. Error 404 is based on the URL and header and not on the content. — jdweng, Jan 22 '20 at 17:58
@jdweng This is not about encryption or decryption. It's about encoding and decoding. The requested URI is encoded using encodeURIComponent(filename) to encode the slashed as part of the filename — orsolli, Jan 22 '20 at 18:10
The HTML: is bytes. The bytes get displayed using the viewer like notpade. Characters 0x00 to 0x7F always get displayed the same. Encoding characters 0x80 to 0xFF get mapped to a unicode character depending the encoding used. HTML doesn't use encoding instead the mapping is done use the language specified in the header. — jdweng, Jan 22 '20 at 18:15
as the friends above had mentioned, the problem here is the special chars from Western European in your URL — Thiago Custodio, Jan 22 '20 at 20:26
The file D:\home\site\wwwroot\api\case\2020000138\2020000138- 25\202318\SørveivtestFil\med\Sash og space.PDF does not exist, that's why I get 404, but the route https://testdataapiwin:80/api/case/{caseNumber}/{document}/{fileId}/{filename} do exist and does not return 404 whatever filename you provide — orsolli, Jan 23 '20 at 08:27

itminus · Answer 1 · 2020-01-23T02:27:02.873

It is not about Azure but because the slash (/) that represents a directory in the URL has been also encoded to %2F by your javascript encodeURIComponent(filename) function.

In your case, / is not part of the filename but a separator for sub path. Note Linux/Windows OS will forbid us to use / as part of filename. In other words, the slash character (/) should NOT be encoded.

If you do want to allow your filename to represent a path to some file like /202318/SørveivtestFil/med/Sash og space.PDF, use encodeURI(filename) instead of encodeURLComponent(filename):

> encodeURI(`/202318/SørveivtestFil/med/Sash og space.PDF`)   
"/202318/S%C3%B8rveivtestFil/med/Sash%20og%20space.PDF"

> // note the `/` is encoded
> encodeURIComponent(`/202318/SørveivtestFil/med/Sash og space.PDF`)  
"%2F202318%2FS%C3%B8rveivtestFil%2Fmed%2FSash%20og%20space.PDF"

Note encodeURI(filename) won't escape A-Z a-z 0-9 ; , / ? : @ & = + $ - _ . ! ~ * ' ( ) # (See this thread). If you want to encode everything except the slash (/), do it as below:
```
> filename.split('/').map(e => encodeURIComponent(e)).join('/')
"/202318/S%C3%B8rveivtestFil/med/Sash%20og%20space.PDF"
```

As I chose to use encodeURIComponent, it solved the problem locally because I don't have a web server that encodes the requiest URI. Although slash(/) was replaced by underscore(_) somewhere along the way, but that is just fine because as you say, slash is not a valid filename. In this case, the filename is decided to be the title of a case, so I have the option to modify it. I think I'll just replace the slash before using it as a filename — orsolli, Jan 23 '20 at 07:34

Why do Azure decode Requested URL before sending request to app

1 Answers1