R crashes after big for loop, apply or foreach

Question

I wrote a script which works very correctly on small sample datasets, but when I try the same thing on much larger - and real - datas, R Studio Session crashes with a fatal error, as does R session when I run the script without using RStudio. Here is my sessionInfo() :

R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=French_France.1252  LC_CTYPE=French_France.1252    LC_MONETARY=French_France.1252
[4] LC_NUMERIC=C                   LC_TIME=French_France.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] rsconnect_0.4.3 tools_3.3.1

I tried building the same script using a for loop, foreach package, apply functions, plyr's one... Nothing worked. I am sorry not being able to give a reproducible example, but the scripts and data are very large, and when they are just an extract, it works...

Do anyone know what is the problem ? I precise that I don't get any error message...

EDIT

Here's my code, if it help anyone finding solution :

     # pairing des sections
  print("Pairing...")
  pair <- pairing(revs)

  # établissement de la struture en "text" et "diff"
  revs <- structure(revs)

  # initialisation de la boucle
  if(init) revs <- init(revs)
  else revs[[1]] <- start

  depart_id <- max(revs[[1]][length(revs[[1]]) - 1][[1]]$diff$id)

  print("Versionning... ")
  for(i in 2:length(revs)) {

    print(i)

    old <- revs[[i-1]]
    new <- revs[[i]]

    pairSmall <- filter(pair, pairing == i)

    revs[[i]] <- one.status(new, old, pairSmall)
    revs[[i]] <- two.status(revs[[i]], old, pairSmall, depart_id)
    revs[[i]] <- three.status(revs[[i]],old, pairSmall, calcul.id(revs[[i]], old))
    revs[[i]] <- four.status(revs[[i]], old, pairSmall, calcul.id(revs[[i]], old))
    revs[[i]] <- five.status(revs[[i]], old, pairSmall)
    revs[[i]] <- six.status(revs[[i]], old, pairSmall, calcul.id(revs[[i]], old))
    revs[[i]] <- seven.status(revs[[i]], old, pairSmall, calcul.id(revs[[i]], old))
    revs[[i]] <- eight.status(revs[[i]], old, pairSmall, calcul.id(revs[[i]], old))

    }

more than likely it might be happening because your data is too big to fit inside RAM, which results in crash. — abhiieor, Aug 14 '16 at 14:16
My data is about 50 mb whereas i have 8gb ram... Multiples assignation of revs((i)) can cause a problem like this ? — Léo Joubert, Aug 14 '16 at 14:19
If I understand correctly, you do not initialize `revs`. You should preset it with the correct size. Have a look [here](http://stackoverflow.com/questions/2908822/speed-up-the-loop-operation-in-r)... And reproducible code would also help. — Christoph, Aug 14 '16 at 14:24
Revs store my datas, here is only the code of the crashing function, which take revs as an argument. Here i modified the revs value with an i index — Léo Joubert, Aug 14 '16 at 14:27

score 0 · Accepted Answer · answered Aug 15 '16 at 11:09

0

In fact it was a problem with the eight.status function, so please don't take this question into account it was not well formulated.

answered Aug 15 '16 at 11:09

Léo Joubert

522
4
17

R crashes after big for loop, apply or foreach

1 Answers1