7

When loading a .csv with sqldf, everything goes fine until I load data.table. For example:

library(sqldf)
write.table(trees, file="trees.csv", row.names=FALSE, col.names=FALSE, sep=",")
my.df <- read.csv.sql("trees.csv", "select * from file", 
        header = FALSE, row.names = FALSE)

works, while

library(data.table)
my.df <- read.csv.sql("trees.csv", "select * from file", 
        header = FALSE, row.names = FALSE)
# Error in list(...)[[1]] : subscript out of bounds

Doesn't. When loaded, data.table informs you that

The following object(s) are masked from 'package:base':

   cbind, rbind

So, I tried this

rbind <- base::rbind  # `unmask` rbind from base::
library(data.table)
my.df <- read.csv.sql("trees.csv", "select * from file", 
        header = FALSE, row.names = FALSE)
rbind <- data.table::rbind # `mask` rbind with data.table::rbind

which works. Before I litter all my code with this trick:

What is the best practise solution to deal with masking conflicts in R?

EDIT: There is a closely related thread here, but no general solution is suggested.

Community
  • 1
  • 1
Ryogi
  • 5,497
  • 5
  • 26
  • 46
  • 8
    filing a bug report to the maintainer of `data.table` is what I'd suggest. – baptiste Oct 15 '11 at 04:15
  • 2
    I agree with @baptiste **data.table** shouldn't clobber a basic piece of R like `rbind`. It is fine to replace `rbind` with something else as long as it works like the original. However, I would confirm the the error is coming from a call to `rbind()` - just to be sure - before filing a bug report with the maintainers of data.table. – Gavin Simpson Oct 15 '11 at 08:33
  • 1
    The `data.table` project has an r-forge mailing list where this would be an appropriate post. Their "help-help" webpage says they do not have moderators who can review posting from nonsubscribers: http://lists.r-forge.r-project.org/mailman/listinfo/datatable-help , so you would need to first subscribe. (However, I do not think Matthew Dowle would mind a bug report that wasn't. He's very evenhanded in responding to user input.) – IRTFM Oct 15 '11 at 13:34
  • Yeah, it's a bit funny that there isn't simply a `cbind.sqldf`-ish method for whatever class `my.df` is, but maybe there's a reason the author didn't want to create a new class. – Carl Witthoft Oct 15 '11 at 16:27
  • The consensus seems to be bug-report for `data.table`. From @CarlWitthoft: are you instead suggesting that could be an shortcoming of `sqldf`? – Ryogi Oct 15 '11 at 18:12
  • @RYogi: no, sorry -- I didn't know which package `data.table` belonged to. – Carl Witthoft Oct 15 '11 at 19:15
  • Starting in a fresh R session try reading the file using `library(sqldf); my.df <- read.csv.sql(...whatever...)` and only issue the `library(data.table)` command *after* the file has been read in. Then `data.table` can't interfere since its not loaded at the time that the file is read. – G. Grothendieck Oct 15 '11 at 21:08
  • @G.Grothendieck that works perfectly well when only one file is loaded at the beginning of the script, but not when dealing with several large files (so that pre-loading all of them is not an option). – Ryogi Oct 15 '11 at 21:13
  • 2
    @RYogi, (1) Each time you want to use `sqldf` issue `detach()` assuming that `data.table` was the last package you attached. If that does not work try unloading it as well, i.e. `detach(unload = TRUE)`. Once you have read in your data issue `library(data.table)` again. (2) You also might try your original code in R 2.14 and R 2.15 since there have been some changes in how R handles such conflicts and it might be that this is automatically handled by R in the way you want in those newer versions. – G. Grothendieck Oct 15 '11 at 21:19
  • 1
    @RYogi, (3) I noticed that your original code does work with the R-Forge version of data.table. – G. Grothendieck Oct 17 '11 at 16:41

1 Answers1

2

As per the comments, yes, please file a bug report :

bug.report(package="data.table")

That way it won't be forgotten, you'll get an automatic email each time the status changes and you can reopen it if the fix proves to be insufficient.

EDIT:

Now in v1.6.7 on R-Forge :

  • Compatibility with package sqldf (which can call do.call("rbind",...) on an empty ...) is fixed and test added. data.table was switching on list(...)[[1]] rather than ..1. Thanks to RYogi for reporting #1623.
Matt Dowle
  • 58,872
  • 22
  • 166
  • 224