I would be using word boundaries for your sample input and expected result. No lookarounds and no capture groups necessary.
Code: (Demo)
$str = 'this i"s a "test" word i"s"s';
echo preg_replace('~\b"\b~', '', $str);
Output:
this is a "test" word iss
Sometimes users on Stackoverflow ask for one specific thing, but are actually open to different interpretations -- this is the reason for me stretching the interpretation of the question requirements. It may help the OP and/or it may help future readers.
Or if we are being super literal, then this pattern is best for matching double quotes that are neither preceded or followed by spaces: ~(?<! )"(?! )~