The best answer that I found was to the question linked to above as 'duplicate'. However since the best answer (in my opinion) is not the accepted answer; here is a link to the answer I found most helpful (https://stackoverflow.com/a/5760645/3114742)
When you ask why a standard is the way it is, there are two possible
interpretations of your question: (a) what are the perceived benefits
of the design choice that was made, and (b) what was the historical
sequence of events that led to this design being adopted over other
designs.
Clearly, using HTTP-like URIs gives uniqueness. But that could also
have been achieved in other ways, for example by inventing a new URI
scheme. I think that if you trace the arguments that were made at the
time, you will find that many of the people advocating the use of
HTTP-style URIs were also advocating that namespace URIs should be
dereferencable, perhaps to a schema or to some other kind of document.
Indeed, it's W3C policy for its own namespaces that there is always a
document that can be retrieved when you type the namespace URI into a
browser, and that it should tell you something useful about the
namespace.
So I suspect the fact that we use URIs that look deferenceable but
don't define any semantics for what happens when they are dereferenced
is in fact the result a committee compromise between two or more
opposing camps. But you'd have to do some proper historical research
to confirm that.