Changing current culture (How to change CurrentCulture at runtime?) may impact exception messages but it will also change normal code behavior. Consider doing so only for debugging.
Most of the code does not bother with localizing exceptions and simply have hardcoded strings as error messages. Smaller number of libraries will use localizable resource strings for all or just potentially user-facing exceptions but unlikely to localize in many languages. Even smaller number will have all exceptions localized for all languages (I believe Microsoft have all .Net Framework libraries localized for all languages).
So unless if library you are concerned about is from that smaller set with localized exceptions there is nothing you can do to change exception text. You can usually quickly experiment with it by writing code that is setting current culture (CurrentCulture and CurrentUICulture) to language you are interested in and triggering particular exception. Or you can decompile code and see if exception messages are hardcoded.
Instead of changing culture I would catch more specific exceptions and write code to handle them especially if library produces information shown to user or handles any serialized data.
Warning: changing current culture will modify behavior of all string parsing and formatting functions. So if code you call tries to read or write any documents/settings or produces any UI strings I'd be very concerned with changing current culture.
