text.replacingOccurrences
can be used with regular expressions so using the group ['`´] could work here (I am not aware of any meta character for this). As @Rob mentioned in the comments it might be worth expanding the pattern further like [‘’'`´] or [‘’‛'′´`❛❜] or use "\p{Quotation_Mark}"
let trim = text.replacingOccurrences(of: "['`´]", with: "", options: .regularExpression)
It doesn't replace è or é which is good I suppose
let text = "name's are Josè and André, strings are `abc` and ´def´"
let trim = text.replacingOccurrences(of: "['`´]", with: "", options: .regularExpression)
print(trim)
yields
names are Josè and André, strings are abc and def