this is actually a pretty reasonable idea IF you use a large, and randomly generated url. there are many systems that actually work like this already.
for example, in google docs, you can create a link that anyone with that link can edit the document. It's long enough that you could never feasibly guess that link. Also, password reset links are basically this, except they are (hopefully) only usable once. (see below)
You'll need to ensure that the secret is not leaked. That means using https, not logging accesses, or returning the secret in other api calls.
That said, as many above commenters mention, a URL is stored all sorts of insecure places on your computer, but if an adversary has access to your computer you are already screwed. It's pretty typical to assume that your end user device is secure.
Also, any secret is only secret inversely proportional to how many people know it. It may be tempting to share a url with other people who require access. A much better system might be to make each URL work once, but add a cookie to the user's browser, which is the actual token.
Basically, just like a password reset flow/email confirm flow, except without passwords.