1

I have a column of parcel identifier numbers which data.table::fread reads in as integer64. It was throwing errors some months ago so I was converting to character, but after it stopped throwing errors I saw no downside to leaving it as it was. But after some processing with package sp, I was trying to revert from SpatialPointsDataFrame back to data.table with x <- as.data.table(x) and got:

Error in as.data.frame.default(x[[i]], optional = TRUE) : 
  cannot coerce class ""integer64"" to a data.frame

Reading in as character is the consensus recommendation here. But I am trying to understand what's going on. They could have been numbers not IDs. Seems like sp::coordinates is able to take in integer64 numbers from a data.table into its own type of data frame but from there on it's a one-way street?

EDIT: I was going to do traceback() etc. suggested by @MichaelChirico in the comments, realized R was hanging after the error message, decided to try on a small sample. To my surprise it worked with no errors/warnings. So far, the largest single table to which I added spatial data and converted back had circa 2.25 million rows. I kept upping the sample size to see where it would break. The end result was (to me) twilight zone - I was able to convert the whole SPDF if I randomized the rows, not if not! https://imgur.com/aghsdDA This is not a "solution" so I'm going to leave this up here for someone who knows what's going on under the hood.

EDIT_2: Replaced inline illustrative image in previous edit with a link. Link to 12MB SPDF file is here. Variable of interest is integer64 'Prop'- see above; trying to revert from SPDF back to data.table. Reproducible on my Mac 10.10.5, running Rstudio v1.1.453 on R v3.3.3, packages data.table v1.10.4-3 and sp v1.3-1. Substitute PATH_TO_FILE with your download location below:

x <- readRDS("PATH_TO_FILE/x_SPDF.rds")

z <- x[sample(nrow(x), 2535323),] # all rows but randomized
z <- as.data.table(z) # works
x1 <- x[1:2535322, ] # un-randomized subset w/ nrow() minus 1
x1 <- as.data.table(x1) # works
x <- as.data.table(x) # stops with error noted in initial question

Being on an old clunker of a Mac may have something to do with it, but I figure if it works on subsets and randomized subsets of any size, it should work generally.

armipunk
  • 458
  • 2
  • 13
  • 3
    Easiest may be to install & load the `bit64` package, I guess – MichaelChirico Apr 07 '19 at 16:50
  • for why you're experiencing this particular error, maybe try examining the `traceback()` and/or `debugonce(as.data.frame.default)` or `debugonce(as.data.table)` – MichaelChirico Apr 07 '19 at 16:52
  • Please do not post an image of code/data/errors: it cannot be copied or searched (SEO), it breaks screen-readers, and it may not fit well on some mobile devices. Ref: https://meta.stackoverflow.com/a/285557/3358272 (and https://xkcd.com/2116/). Further, please make this question *reproducible*. This includes sample code (including listing non-base R packages), sample data (e.g., `dput(head(x))`), and expected output. Refs: https://stackoverflow.com/questions/5963269, https://stackoverflow.com/help/mcve, and https://stackoverflow.com/tags/r/info. – r2evans Apr 08 '19 at 15:48

0 Answers0