0

I am attempting to use gganimate with parallel processing, but when using anim_save, I get a "Provided file does not exist" error." It also says "Called from: png_dim(frames1)". The error seems to be coming from the aaa.R file from the package, and only happens in some cases. I don't know if it is relevant, but I am using RStudio and a Mac.

I am using a modified version of gganimate from a Github pull request as described in the answer to this question: How to use multiple cores to make gganimate faster. I call this new package gganimatep. The way I did this, based on the answer, was to download the zipped file from the link in the answer, change the file name in DESCRIPTION (to gganimatep), delete the vignettes folder, and then do RStudio / Build / Install and restart. (I am not very familiar with Github, but I noticed that the gganimate parallel page (https://github.com/HenrikBengtsson/gganimate/tree/feature/parallel-animate) says This branch is 2 commits ahead, 42 commits behind thomasp85:master. I don't know if this means that when I install it, I am basically installing an old version of gganimate.)

First example (with base code taken from https://ggplot2.tidyverse.org/reference/ggsf.html):

library(tidyverse)
library(sf)
library(gganimate)
library(future)

nc <- sf::st_read(system.file("shape/nc.shp", package = "sf"), quiet = TRUE)

set.seed(1)
nc1 <- nc %>% mutate(value = AREA, year = 1)
nc2 <- nc %>% mutate(value = AREA + runif(1, -.2, .2), year = 2)
nc3 <- nc2 %>% mutate(value = value + runif(1, -.2, .2), year = 3)

nc_new <- bind_rows(nc1, nc2, nc3)

b <- ggplot(nc_new) +
    geom_sf(aes(fill = value)) +
    labs(title = 'Year: {frame_time}') +
    transition_time(year) 

If we use the base gganimate (without parallel processing), it works:

b

or

gganimate::anim_save("test.gif", b, end_pause=8, width = 600, height = 400, duration=5, nframes=50)

But if we use gganimatep with parallel processing, it doesn't work:

library(gganimatep)
plan(multisession, workers = 7)
b

or

plan(multisession, workers = 7)
gganimatep::anim_save("test.gif", b, end_pause=8, width = 600, height = 400, duration=5, nframes=50)

Screenshot:

enter image description here

Because both b and anim_save give the same or very similar errors, I conclude that it is not a problem with anim_save, but more likely something having to do with gganimatep.

Another example. Using example code from the gganimate website (https://gganimate.com/), this does work:

library(gganimatep)
library(gganimate)
library(ggplot2)
library(gapminder)
library(future)
library(countrycode)
#devtools::install_github("rensa/ggflags")
library(ggflags)

g <- ggplot(gapminder, aes(gdpPercap, lifeExp, size = pop, colour = country)) +
    geom_point(alpha = 0.7, show.legend = FALSE) +
    scale_colour_manual(values = country_colors) +
    scale_size(range = c(2, 12)) +
    scale_x_log10() +
    facet_wrap(~continent) +
    labs(title = 'Year: {frame_time}', x = 'GDP per capita', y = 'life expectancy') +
    transition_time(year) +
    ease_aes('linear')

plan(multisession, workers = 7)
gganimatep::anim_save("test.gif", g, end_pause=8, width = 600, height = 400, duration=5, nframes=50)

When I do a slightly different thing (changing dots to flags), the base gganimate (not parallel processing) works:

gapminder2 <- gapminder
gapminder2$country2 <- countrycode(as.character(gapminder2$country),"country.name","iso2c")
gapminder2$country2 <- tolower(gapminder2$country2)
gapminder2$country2 <- as.character(gapminder2$country2)

g2 <- ggplot(gapminder2, aes(gdpPercap, lifeExp, size = pop, country=country2)) +
    geom_flag(show.legend=FALSE) +
    scale_colour_manual(values = country_colors) +
    scale_size(range = c(2, 12)) +
    scale_x_log10() +
    facet_wrap(~continent) +
    labs(title = 'Year: {frame_time}', x = 'GDP per capita', y = 'life expectancy') +
    transition_time(year) +
    ease_aes('linear')

gganimate::anim_save("test.gif", g2, end_pause=8, width = 600, height = 400, duration=5, nframes=50)

BUT, when I switch to gganimatep, it does not work:

plan(multisession, workers = 7)
gganimatep::anim_save("test.gif", g2, end_pause=8, width = 600, height = 400, duration=5, nframes=50)

I get the following error:

enter image description here

As mentioned above, it is the "Provided file does not exist" error." And "Called from: png_dim(frames1)". It appears to be from the aaa.R file.

In another instance, it said:

Inserting image 300 at 29.90s (100%)...
Encoding to gif... done!
Warning message:
In file.create(to[okay]) :
  cannot create file 'tmean_2020.gif', reason 'Read-only file system'

In another instance, it said:

enter image description here

Note that another thing (not a MWE) that also doesn't work with gganimatep is:

a <- ggplot(weather2) + 
    geom_sf(aes(fill = temp, geometry=geometry, group=NAME)) + 
    scale_fill_viridis_c(name = NULL) + 
    theme_map() +
    transition_time(as.integer(date)) +
    labs(subtitle = paste('Date: {frame_time}')) 

setwd("/")
plan(multisession, workers = 7)
gganimatep::anim_save("tmean_2020.gif", a, end_pause=8, width = 600, height = 400, duration=30, nframes=300)

What can I do to fix this? Thanks!

bill999
  • 2,147
  • 8
  • 51
  • 103
  • I haven't used `plan()` to do parallel processing, but when using `doParallel` I specify the packages each worker needs to complete the code. Are the workers loading the `ggflags` package? Maybe load `furrr` like here: https://byuistats.github.io/M335/parallel_furrr.html – Skaqqs Sep 20 '21 at 12:48
  • If I understand the process of how you created the `gganimatep` package on your machine correctly, I think, as you mentioned, it is a older version of `gganimate` with some custom modifications. Judging from the code examples you provide, you seem to create the ggplot object with functions from `gganimate` and then try to render it with `gganimatep`. `gganimatep` should contain all functions you need to create the animation, try not to load `gganimate` and hope that there weren't any significant API changes. – PhJ Sep 21 '21 at 13:34
  • @Skaqqs, I am not sure. I haven't used `doParallel` before. – bill999 Sep 21 '21 at 14:45
  • @PhJ, to try to fix it, I just downloaded the newest version of `gganimate` from github and manually changed the code that was changed for the parallel version (as in https://github.com/thomasp85/gganimate/pull/403/files). I then loaded this package (basically, the newest version of `gganimate` with the code updated to handle parallel processing). I also started R from scratch and only loaded the the new version and not the base `gganimate`. I get the same errors as before, unfortunately. Note that when I load the new, parallel version, it says things like: – bill999 Sep 21 '21 at 14:49
  • ... `Loading required package: ggplot2 Registered S3 methods overwritten by 'gganimatepar': method from c.TransmuteFactory gganimate`. And also note that when I try to run the code to render the plot using the parallel version, it actually does do parallel computing, but it does seem like `gganimate` is overwriting `gganimatepar` methods: `egistered S3 methods overwritten by 'gganimate': method from c.TransmuteFactory gganimatepar ggplot_add.EaseAes gganimatepar`. This is despite `gganimate` not being loaded. – bill999 Sep 21 '21 at 14:49
  • I just uninstalled `gganimate`, but I still get the `Provided file does not exist` error. (It also may not be doing the parallel processing anymore.) I'm at a loss as to what to do. – bill999 Sep 21 '21 at 14:56
  • Must you use `gganimate()` or are you open to other packages/methods? – Skaqqs Sep 21 '21 at 14:57
  • I didn't think there was a good substitute to `gganimate`, but if there is, I would be interested to know about it. (But I am not interested in manually generating individual frames and then stitching them together.) – bill999 Sep 21 '21 at 14:58
  • Ok, my suggestion was going to be to run `gganimate()` in parallel, with each stream animating a subset of your data, then stitching the result of each stream together. It sounds like that's not what you're after? – Skaqqs Sep 21 '21 at 15:52
  • 1
    That is an interesting idea. Thank you. I'd probably prefer to avoid having to do this if possible – bill999 Sep 21 '21 at 15:58
  • @bill999 what I think might happen is that some change to the `gganimate` code between when the pull request for `gganimatep` was created an now is breaking the extra code from `gganimatep`. – PhJ Sep 21 '21 at 16:40
  • @PhJ, that is a good point. Based on your earlier comment, I downloaded the latest version of the `gganimate` code from Github and then manually changed this new `gganimate` code based on the changes from the `gganimate parallel` pull request (https://github.com/thomasp85/gganimate/pull/403/files). Call this version `gganimatepar`. This has the same problems as before. – bill999 Sep 21 '21 at 16:51
  • 2
    @bill999 and just to be 100% sure that you did not try that: You build the package `gganimatep` as described before (https://stackoverflow.com/a/67350496/7912251), and then try your examples **without** ever loading in `gganimate`, but **only** loading `gganimatep`. `gganimatep` should then provide you with the functions from `gganimate` such as `transition_time`, etc. These functions should be from when the pull request was made, and therefore hopefully be compatible with the parallel rendering. – PhJ Sep 21 '21 at 18:07
  • @PhJ, yes, I did that, and it doesn't work. To be sure, I just uninstalled (or confirmed that all versions of `gganimate` were uninstalled (including the original version). I then built the package `gganimatep` (or similarly named) as in (https://stackoverflow.com/a/67350496/7912251). I then loaded this package (and not `gganimate`, which isn't even installed on the computer) and still get the same error as before. ... – bill999 Sep 21 '21 at 18:31
  • @PhJ, but in doing this exercise again, I find interesting (and hopefully useful) things: When I **do** first run `plan(multisession...` and then run either `b` or `gganimatep::anim_save...`, then I get the same `Provided file...` error as described above. When I **don't** first run `plan(multisession, workers = 7)`, it runs, but doesn't actually render anything: a) no graph is produced in RStudio or b) the .gif file is 814 bytes and blank. I also get dozens of warnings (not errors): `there is no package called ‘gganimate’`. So sthg is going on with `gganimatep` trying to use `gganimate`. – bill999 Sep 21 '21 at 18:37
  • 2
    @PhJ, to rule out that the issue is what the package is named, I uninstalled all versions of `gganimate` and followed the steps to install the branched parallel version as described in the other post. BUT this time I did not rename it, so it retains its name of `gganimate`. If I **do not** first run `plan(multisession, workers = 7)`, then it actually works now, and uses parallel processing. Yay! Note that if I do first run `plan...`, then I get the same `Provided file...` error as above and this warnings: `no applicable method for 'st_normalize' applied to an object of class "list"`. – bill999 Sep 21 '21 at 19:01
  • 1
    @PhJ, I don't completely understand everything, but I am probably have figured out enough to do what I need. Thanks for the help. Because you led me to the right direction, if you'd like to write an answer with what ended up working, I'll give you the bounty. – bill999 Sep 21 '21 at 19:15
  • 1
    @bill999 I don't completely understand it either, but I am glad it worked for you! – PhJ Sep 21 '21 at 20:32
  • 1
    Glad you figured it out. I'm interested in a benchmark comparison between the parallel and nonparallel version, if anyone is up for it. – Skaqqs Sep 21 '21 at 22:50

1 Answers1

2

From trial and error, the following seems to be the problem: the package needs to be build from the GitHub pull request (a) (as described here) and needs to be named gganimate to satisfy internal dependencies.

After setting up the environment correctly, parallel processing seems to work without using the plan(multisession, workers=7) command, but using it appears to throw an error.

(see the discussion under the question for more details)

PhJ
  • 721
  • 7
  • 13