It should not be necessary for a user who requires enhanced accessibility features to install plugins or alter settings for them to use your website/service. By meeting the WCAG guidelines on contrast (and other things) you can improve the experience for those users in a vanilla browser and operating system.
There are many contrast checkers online which help with this, I use this one. This SO question explains what the different ratings mean very well.
I like to think that when it comes to accessibility it is worth investing time in doing it properly so that more people can use your site or service. Not making your site/service accessible when it is relatively easy to do so is locking these users out. They will shop elsewhere rather than battle with browser/OS settings in order to get your website usable for them.
If it requires user customization of your site just to make it usable, that is a fail on your part.
A side benefit is that for people with nominal eye site, your site will be easier to read and look at for long periods of time if foreground items contrast well with the background. Which is something you probably want, right?