Maybe.
Allowing cross domain XHR opens up a few attacks. E.g. see https://stackoverflow.com/a/7615287/841830. See also Is CORS a secure way to do cross-domain AJAX requests?
But this tends not to come up with the normal use cases for Phantom: whether you are testing your own web site, or screen-scraping, you tend to go to pre-decided URLs and links, and are not sending secret information, and are not going to be tricked by a new and suspicious link. You are unlikely to be logged into your bank, or Facebook, while testing your site or scraping google search results. (BUT, if you are scraping google pages that force you to sign into Google first, be a bit more careful - perhaps set up a dedicated gmail account just for your scraping.)
So, in summary, the attacks are a bit more obscure and unlikely, compared to a normal desktop browsing session, but they are still there, so only use --web-security=no
when your script otherwise will not work.