Both: as for any image, it depends if this is an image that conveys information or a decorative image.
Image that conveys information
It MUST be an HTML image with a meaningful text alternative (ex: a link made of only this image. A link must never be empty so the image can't be a CSS one otherwise the link would be empty.
Decorative image
A background image or part of a sprite or a custom font character or an HTML image with empty alternative text (the latter if it's an image an author will only use once as decoration in content and that you won't add to the stylesheet)
Appropriate Use of Alternative Text (edit: by WebAIM)
edit: one case where the user won't see any image is if he uses High Contrast Mode. Then if he can't see any link in a menu made of large "icons" only with no text, that's bad. If he can't see bullets in front of link items but text of these links a re still very clear, then OK.
There are many other reasons why images wouldn't appear on a page whether it is flaky Wi-Fi, slow connection, cost on mobile, buggy proxy or parental controls, user decision, etc