Add \\ to escape non-UTF 8 characters within a string using regex

Question

I am working with a dataset that has free text containing special characters. I need to clean the text before use strsplit for a subsequent function, but would prefer to add escapes (\\) before the special characters rather than delete them altogether.

For example, the string that looks like this:

Do you love great hair? Wind it up! Your curls are your gift- set them free and help preserve your natural curl with bounce and definition. Cleanse hair without weighing it down while reducing frizz. Infused with pineapple, argan oil and quinoa. Let your natural beauty shine bright!

Should look like this:

Do you love great hair\\? Wind it up\\! Your curls are your gift\\- set them free and help preserve your natural curl with bounce and definition. Cleanse hair without weighing it down while reducing frizz. Infused with pineapple, argan oil and quinoa. Let your natural beauty shine bright\\!

I've figured out how to remove a list of several special characters (~!@#$%^&*(){}|<>/), but can't find the right tutorial for adding \\ before them.

Note: I am not looking to remove ALL punctuation because some characters are used for subsequent delimiting logic. Instead, I want to address a specific subset of special characters.

Sample data:

>dput(tar$clean.text[1:10])
list(c("Dove go fresh Cucumber and Green Tea Beauty Bar combines the refreshing scent of cucumber and green tea with Dove's gentle cleansers and _ moisturizing cream. Dove Beauty Bar is proven to be more gentle and mild on skin than ordinary soap. It can be used on your hands and as a mild facial cleanser, so if you're also after a fresh face and refreshed hands throughout the day, why not try adding Dove Beauty Bar go fresh Cucumber and Green Tea to your skin care routine? Light, hydrating feel and refreshing formula that effectively nourishes skin. A refreshing shower can be just what you need to start the day off right. Dove's go fresh range blends nourishing ingredients and light, fresh scents in a formula that's gentle on your skin. Dove go fresh beauty bars give you a feeling of hydrating freshness that leaves you and your skin feeling blissfully revived. For best results: Your hands are one of the driest parts of your body so give them a boost and lather your Dove beauty bar between wet hands. Once you've covered your body with the rich lather, making sure to avoid contact with your eyes, rinse away thoroughly. At Dove, our vision is of a world where beauty is a source of confidence, and not anxiety. So, we are on a mission to help the next generation of women develop a positive relationship with the way they look - helping them raise their self-esteem and realize their full potential.", 
    "Scent: Cucumber", "Health Facts: Sulfate-free", "Suggested Age: 5 Years and Up", 
    "Wellness Standard: Aluminum-free, paraben-free", "Recommended Skin Type: Normal", 
    "Beauty Purpose: Moisturizing, basic cleansing", "Package Quantity: 1", 
    "TCIN: 10819409", "UPC: 011111611023", "Item Number (DPCI): 049-00-0604"
    ), c("Me! Bath Bath Bomb Papaya Nectar 6 ct is a great idea to add to a spa gift basket. These bath bombs are like scoops for your bath to make mini bath ice cream that gives you super soft skin.", 
    "Scent: Fruit", "Health Facts: Vegan, paraben-free, aluminum-free", 
    "Product Form: Bath bomb", "Suggested Age: Adult Use Only", "Wellness Standard: Aluminum-free, cruelty-free, paraben-free, vegan", 
    "Recommended Skin Type: Normal", "Sustainability Claims: Cruelty-free", 
    "TCIN: 18828570", "UPC: 858858000358", "Item Number (DPCI): 037-08-1164"
    ), NA_character_, NA_character_, c("Aura Cacia pure essential oils in 4 fl oz Body Oil has a lavender and cocoa butter scent. This natural skin care oil shows skin tone improvement that you can feel.", 
    "Scent: Lavender, Cocoa Butter", "Health Facts: Contains lavender, butylparaben-free, phthalate-free, formaldehyde donor-free, formaldehyde-free, nonylphenol ethoxylate free, propylparaben-free, Sulfate-free, paraben-free, dye-free, aluminum-free", 
    "Product Form: Lotion", "Suggested Age: All Ages", "Recommended Skin Type: Normal", 
    "Beauty Purpose: Skin tone improvement", "Sustainability Claims: Not tested on animals, cruelty-free", 
    "TCIN: 50030689", "UPC: 051381911720", "Item Number (DPCI): 037-05-1378"
    ), c("Deep clean pores with the Facial Cleansing Brush from Eco", 
    "Tools. This compact brush features soft bristles for moderate exfoliation, leaving you with soft, supple skin. Your serums and moisturizers can more effectively penetrate your skin once all the dead skin cells are out of the way. The compact size is ideal for packing in your weekend tote or suitcase for cleansing on the go.", 
    "Material: Nylon", "Suggested Age: All Ages", "Beauty Purpose: Basic cleansing, exfoliating", 
    "TCIN: 52537254", "UPC: 079625074864", "Item Number (DPCI): 037-08-2254"
    ), c("Deep Steep Rosemary Mint Sugar Scrub gently exfoliates dead skin cells while moisturizing, leaving smooth, radiant, polished skin. This formula is made up of a smooth blend of shea butter, cocoa butter and carefully sourced sugar to give you light, blissful fragrance with just the right amount of exfoliation and no harsh scratching. Apply desired amount of Deep Steep Rosemary Mint Sugar Scrub to wet skin from shoulders to ankles. Massage in a circular motion. Rinse.", 
    "Scent: Rosemary", "Health Facts: Contains argan oil, contains coconut oil, contains shea butter, formaldehyde donor-free, gluten-free, dye-free, ethyl alcohol-free, paraben-free, phthalate-free, vegan", 
    "Product Form: Scrub", "Suggested Age: All Ages", "Recommended Skin Type: Dry, normal", 
    "Beauty Purpose: Exfoliating", "TCIN: 53242409", "UPC: 674749101153", 
    "Item Number (DPCI): 037-08-2123"), NA_character_, c("Want to feel gorgeously soft skin every day? Transform your daily shower into an irresistible treat with the exquisitely fragranced Caress Evenly Gorgeous body wash. Indulge your skin with a rich exfoliating lather delicately scented with burnt brown sugar and karite butter that makes this body wash smell good enough to eat. Subtle notes of soft crisp apple and berry open up to a bold floral heart, while rich scents of warm tonka bean, vanilla and balsam together round out the lush lather to leave you with perfectly buffed and glowing skin. Caress Evenly Gorgeous is a revitalizing body wash that blends rich, luxurious lather with expertly crafted fine fragrance It is a body wash that gently cleanses your skin to leave it delicately fragrant, beautifully soft.", 
    "Lather up and indulge in a deeply cleansing and reviving shower experience. With fine floral fragrance and gentle exfoliates, Caress Evenly Gorgeous will leave you feeling delicately perfumed and silky-smooth, making this the perfect body wash for every day? and every night. Caress body wash and beauty bar fragrances are crafted by the world's best perfumers to transform your daily shower into an indulging experience that will make you feel special every day?Scent: Fresh", 
    "Health Facts: Aluminum-free, paraben-free, fluoride-free", "Product Form: Liquid", 
    "Suggested Age: 5 Years and Up", "Wellness Standard: Aluminum-free, paraben-free", 
    "Recommended Skin Type: Normal", "Beauty Purpose: Basic cleansing", 
    "Package Quantity: 1", "TCIN: 13446229", "UPC: 011111014909", 
    "Item Number (DPCI): 049-00-0806"), c("Maintain a sanitary and healthy atmosphere with the MEDLINE n/a READYBATH, PREMIUM,FRAG FREE, 8/PK - 24pks. These sterile swab sticks are pre-treated with povidone-iodine for preparing skin for incision and other medical issues. Comes in disposable packages of 3.", 
    "Scent: Unscented", "Health Facts: No fragrance added", "Suggested Age: Adult Use Only", 
    "Recommended Skin Type: Normal", "Beauty Purpose: Basic cleansing", 
    "Package Quantity: 1", "TCIN: 14339945", "UPC: 080196731445", 
    "Item Number (DPCI): 037-13-0198"))`

Code that removes a list of symbols: tar$clean.text<-str_replace_all(tar$clean.text, "~|!|@|#|$|%|^|&|\\*|\$|\$|\\{|\\}|_|\\\\|<|>|\\?|\\[|\\]|-", "") # Removes a ton of non-UTF characters

I'm sure there is a simple modification to my regexp, but can't seem to figure it out. All previous answers I've found are more specific to fixing a specific text pattern, rather than generally replacing across a lot of different variations.

Maybe a dupe of [Is there an R function to escape a string for regex characters](https://stackoverflow.com/questions/14836754/is-there-an-r-function-to-escape-a-string-for-regex-characters). The thing is that you may replace the whole pattern with `"\\\\\\0"` - `cat(str_replace_all(string, "[~!@#$%^&*(){}_\\\\<>?\\[\\]|-]", "\\\\\\0"))` — Wiktor Stribiżew, Jul 18 '18 at 18:01
I checked out the function, but it seems that using `"(\\W)"` replaces all spaces as well, which I don't want to do. For the solution above, are you saying that the command is: `string2<-"\\\\\\0" - cat(str_replace_all(tar$clean.text, "[~!@#$%^&*(){}_\\\\<>?\\[\\]|-]", "\\\\\\0"))` That results in the error `non-numeric argument to binary operator`, I'm sure I'm just applying the wrong way and being obtuse. — roody, Jul 18 '18 at 18:19
`cat(str_replace_all(string, "[~!@#$%^&*(){}_\\\\<>?\\[\\]|-]", "\\\\\\0"))` works on my end. What is `tar$clean.text`? — Wiktor Stribiżew, Jul 18 '18 at 18:22
So sorry. That's the real code I'm working on (that doesn't match the example). I'll add proper data above. — roody, Jul 18 '18 at 18:25
It worked! Thank you. I will improve the question to make it clear that I am looking for a subset of characters, not all punctuation. — roody, Jul 18 '18 at 18:41
If you add double escapes to some characters, how do you add double escapes to an escape ? If you add double escapes to a character already escaped, how do you know it's already escaped ? — , Jul 18 '18 at 20:29

Wiktor Stribiżew · Accepted Answer · 2018-07-18T19:06:01.537

You may use

str_replace_all(x, "[~!@#$%^&*(){}_\\\\<>?\\[\\]|-]", "\\\\\\0")

A base R approach:

gsub("([]\\~!@#$%^&*(){}_<>?[|-])", "\\\\\\1", "~!@#$%^&*(){}_\\<>?[]|-")

See the regex demo.

Details

[ - start of a character class matching any of the following chars:
- ~ - ~
- ! - !
- @ - @
- # - #
- $ - $
- % - %
- ^ - ^ (if you put it at the start, escape with \\)
- & - &
- * - * (no need to escape inside a character class)
- ( - (
- ) - )
- { - {
- } - }
- _ - _ (note it is a word char, and \W would not match it)
- \\\\ - a \ char (a literal \ escaped with another literal \)
- < - a <
- > - >
- ? - ?
- \\[ - a [ char (in ICU regex, must be escaped inside a character class
- \\] - a ] char (ibid.)
- | - a | char (it is not an OR operator inside a character class)
- - - a - char
] - end of the character class.

The "\\\\\\0" string replacement pattern is parsed as two literal backslashes that defines a singular literal backslash and a \0 literal string that is a backreference to the whole match in the ICU regex in R.

Note that gsub TRE regex is a bit trickier: ] must be the first char in the character class, [ should not be escaped, literal \ should only be single (no regex escape sequences are supported inside TRE patterns), and - must be at the end. Also, there is no support for the whole match backreference, hence, you need to wrap the whole pattern with a capturing group and replace with \1 backreference.

score -1 · Answer 2 · answered Jul 18 '18 at 19:14

-1

let dat = tar$clean.text[1:10] then you can do:

 Map(gsub,"([[:punct:]])","\\\\\\1",dat)

answered Jul 18 '18 at 19:14

Onyambu

67,392
3
24
53

Add \\ to escape non-UTF 8 characters within a string using regex

2 Answers2