Sure. Just exit
after your 401 header. No need for the header("Location...")
at all.
if(badCrawler()){
header("HTTP/1.1 401 Unauthorized");
exit;
}
Side note: 401 typically is used in conjunction with an authentication request.
From the specs:
The response MUST include a WWW-Authenticate header field (section 14.47) containing a challenge applicable to the requested resource.
It might be better to use 403 Forbidden
to deny access, or even 404 Not Found
if you want the bad crawler to think the page doesn't exist any longer:
header("HTTP/1.0 404 Not Found");
exit;
Sending content
Note that your 404 response might result in a blank page in some browsers, see the top answer in this thread for a full explanation of why that is happening. Basically the header is working but it's just up to you to display any HTML content).
The solutions is simple, echo your content (or include a separate file) right before the exit
statement.