Because, from what I understand of your question the mod_rewrite will always goto a page that does exist (category.php), you have to set the 404 error generation within the category.php page,
Don't bother setting the 404 error header separately, simply
if (no match with categories){
//header("Location:404.php",TRUE, 404);
header("Location:404.php");
exit;
}
From what I know of .htaccess it can't read if the data that it is processing (the category URL) is valid or not, as in, it can't tell you if the category is genuine or false because that is something that can only be defined by the PHP script.
sorry I missed out the Location:
part of the header
update
I think the loop that you reference, David, is that while the PHP file redirects to the 404.php page, the .htaccess is trying to match this to your rewrite rules, so you need to add checks to only rewrite URLs that are not referencing files or folders that already exist so, in your .htaccess :
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^/\.]+)/?$ category.php?s=$1 [L,QSA]
This will only rewrite URLs if they do not point to a file or folder that exists before the rewrite.
Update edit 2
The issue develops that now the OP wants the errors of the 404 pages to be noticed as 404 errors by the SEO aspects of the website, this is not done by header("Location:")
because by its nature header location is a redirect.
So:
Reading up on here: Why won't my PHP app send a 404 error? I think the issue is that because PHP sets the status code to 404, file not found, that then it becomes that PHP pages responsibility to output a 404 file. So rather than setting a header Location redirect, the header should be set as a 404 error header and then after the header the 404 error page can be output on the category.php page as an include in the category page,
So, solution, when the category.php page has a invalid category call, it needs to output a header with a 404 in it:
header('HTTP/1.0 404 Not Found');
immediately followed by the category page outputting the message to the browser, which will be including the 404.php
such as :
<?php
if (!category match){
header('HTTP/1.0 404 Not Found');
include "404.php";
}
I do not know how your 404.php is set up but you may need to adjust the code to output the include file with a return value in the 404.php
,
This should then cause the incorrect category codes to be displayed with 404 error headers for SEO purposes.