Is it possible to map all static files or url routes of a website?

Question

I want to make a few static files available in my server, but I don't want people to access it unless they know the exact URL.

This is meant to work like a game, solve a puzzle and move to the next phase by finding out the URL. Surely there are several similar games in the web.

What I'm worried is if someone could just avoid the puzzle by "mapping" all the static files in a server. Is that possible? [Assuming the puzzle solution would lead to a static file served]
I would like to extend the same question for URL routes, is it possible to discover all the url routes of a website? For example I have mysite.com and mysite.com/hidden/solution.html if there are no links to the second URL, can someone get this information? [Except bruteforcing]

I'm hosting this page using AWS lightsail (django and apache). Is there something I can/need to do in the django/apache side to prevent this?

xyres · Accepted Answer · 2021-05-02T05:45:53.213

is it possible to discover all the url routes of a website? [Except bruteforcing]

It's not possible without brute-forcing (or URL fuzzing) but your users can share the links, so...

I see two ways to do this:

1. If users are authenticated

If users are authenticated, then the solution is straight-forward. Save the solved puzzle ids in the database. If a user wants to view a solution, check the database to see if the user has solved the puzzle and serve the response.

2. If users are not authenticated

You can use some sort of cryptographically signed token. When a user solves the puzzle, you'll send them a signed token which would mean that "This user has solved this puzzle". Later, when the user wants to view the solution, they'll need to send the token to view it.

See Django's Cryptographic signing docs.

import time
from django.core.signing import Signer

signer = Signer()

token = signer.sign_object({
    'solution_id': <solution-id>, # could be the name of the html file
    'expire_at': time.time() + 900 # token expiry time (15 minutes from now) 
})

print(token) # -> 'eyJtZXNzYWdlIjoiSGVsbG8hIn0:Xdc-mOFDjs22KsQAqfVfi8PQSPdo3ckWJxPWwQOFhR4'

# Send the token to the client

Later if a user wants to view /solution.html, they'll need to send the token in the url.

/solution.html?token=<token value>

In your view, you can check the token like this:


token = request.GET.get('token')

if not token:
    # return HTTP 403 Permission Denied

try:
    data = signer.unsign(token)
except signing.BadSignature:
    # invalid signature
    # return HTTP 403 Permission Denied

# check if token expired
if data['expire_at'] > time.time():
    # return HTTP 403

# check if token's solution_id matches the requested solution
# ...
# return the response

The drawback of this approach is that even if a user solves a puzzle, they can only view the solution until the token expires.

What about the static files, are they safe from "mapping" as well? I know some older pages you usually can even access the static directory as a folder. Not sure if I need to config apache to prevent that or if it's default. Thanks @xyres — Justcurious, May 02 '21 at 13:40
@Justcurious Maybe `X-Sendfile` can be useful for restricting staticfiles access. See this: https://stackoverflow.com/questions/24631585/xsendfile-with-apache-and-django — xyres, May 02 '21 at 14:00
I don't really want to restrict it, they can be public available. I don't mind if people share the url, I just don't want someone to scan/map the static files and just find it there. — Justcurious, May 02 '21 at 14:02

Is it possible to map all static files or url routes of a website?

1 Answers1