Trouble with gsub and regex in R

Question

I am using gsub in R to add text into the middle of a string. It works perfectly but for some reason, when the location gets too long it throws an error. The code is below:

gsub(paste0('^(.{', as.integer(loc[1])-1, '})(.+)$'), new_cols, sql)

Error in gsub(paste0("^(.{273})(.+)$"), new_cols, sql) :  invalid
  regular expression '^(.{273})(.+)$', reason 'Invalid contents of {}'

This code works fine when the number in the brackets(273 in this case) is less but not when it is this large.

This produces the error:

sql <- "The cat with the bat went to town. He ate the fat mat and wouldn't stop til the sun came up. He was a fat cat that lived with a rat who owned many hats.The cat with the bat went to town. He ate the fat mat and wouldn't stop til the sun came up. He was a fat cat that lived with a rat who owned many hats."  
new_cols <- "happy" 
gsub('^(.{125})(.+)$', new_cols, sql)  #**Works
gsub('^(.{273})(.+)$', new_cols, sql)

Error in gsub("^(.{273})(.+)$", new_cols, sql) :    invalid regular
  expression '^(.{273})(.+)$', reason 'Invalid contents of {}'

What is the content of `loc`, `new_cols` and `sql`can you make a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example/5965451#5965451) please? — Paul Rougieux, May 19 '16 at 13:04
I test it and it works until 255 and doesn't work for values above 255. Maybe `gsub` only accepts `{n}` values only up to a size of one byte in regexs?! — FlorianSchunke, May 19 '16 at 13:19
Sorry, it was just an artifact from my actual code, I use it to build the regex. Same error with or without the paste. — Soxman, May 19 '16 at 13:20

score 15 · Accepted Answer · edited Jun 20 '20 at 09:12

15

Background

R gsub uses TRE regex library by default. The boundaries in the limiting quantifier are valid from 0 till RE_DUP_MAX that is defined in the TRE code. See this TRE reference:

A bound is one of the following, where n and m are unsigned decimal integers between 0 and RE_DUP_MAX

It seems that the RE_DUP_MAX is set to 255 (see this TRE source file showing #define RE_DUP_MAX 255), and thus, you cannot use more in {n,m} limiting quantifier.

Solution

Use PCRE regex flavor, add perl = TRUE and it will work.

R demo:

> sql <- "The cat with the bat went to town. He ate the fat mat and wouldn't stop til the sun came up. He was a fat cat that lived with a rat who owned many hats.The cat with the bat went to town. He ate the fat mat and wouldn't stop til the sun came up. He was a fat cat that lived with a rat who owned many hats."
> new_cols <- "happy"
> gsub('^(.{273})(.+)$', new_cols, sql, perl=TRUE)
[1] "happy"

edited Jun 20 '20 at 09:12

Community

1
1

answered May 19 '16 at 13:25

Wiktor Stribiżew

607,720
39
448
563

2

Thanks! Works great! – Soxman May 19 '16 at 13:30
1

@David, `perl=T` = `perl=TRUE`. – Wiktor Stribiżew May 19 '16 at 13:33
3

[You don't say...](http://www.r-bloggers.com/r-tip-avoid-using-t-and-f-as-synonyms-for-true-and-false/) – David Arenburg May 19 '16 at 13:34
1

I might be wrong, but it always worked. Looks like the issue similar to using `$&` in Perl. – Wiktor Stribiżew May 19 '16 at 13:35
@WiktorStribiżew try: `T<-FALSE; T && TRUE` and understand why it's dangerous to use abreviation without warning – Tensibai May 19 '16 at 13:36
@Tensibai: :) I have never done that, that is why it always worked. – Wiktor Stribiżew May 19 '16 at 13:37
@WiktorStribiżew and that's why it's advised to not use it in SO answers (or to drop a word about it) :) (out of this I upvoted, nice digging on the root cause) – Tensibai May 19 '16 at 13:38

Trouble with gsub and regex in R

1 Answers1

Background

Solution

Linked