`materialize': Could not find

Question

I'm seeing the following error it only is appearing in cron jobs using the whenever gem. The application is working correctly otherwise. The scheduled job doesn't run. But I can run it manually and it does work.

/usr/local/bundle/gems/bundler-2.2.27/lib/bundler/definition.rb:496:in `materialize': Could not find rails-5.2.6, rake-12.3.3, puma-4.3.8, mysql2-0.5.3, sass-rails-5.1.0, uglifier-4.2.0, coffee-rails-4.2.2, jquery-rails-4.4.0, jbuilder-2.11.2, sdoc-0.4.2, bootsnap-1.8.1, blacklight_oai_provider-6.1.1, whenever-1.0.0, active-fedora-13.2.4, active-triples-1.1.1, blacklight-6.24.0, hydra-head-12.0.1, ldp-1.0.3, nokogiri-1.12.4-x86_64-linux, solrizer-4.1.0, rsolr-2.3.0, devise-4.8.0, devise-guests-0.7.0, rdf-3.1.13, byebug-11.1.3, rspec-rails-5.0.2, jettywrapper-2.0.5, listen-3.7.0, fcrepo_wrapper-0.9.0, solr_wrapper-2.2.0, factory_bot_rails-6.2.0, web-console-2.3.0, spring-3.0.0, actioncable-5.2.6, actionmailer-5.2.6, actionpack-5.2.6, actionview-5.2.6, activejob-5.2.6, activemodel-5.2.6, activerecord-5.2.6, activestorage-5.2.6, activesupport-5.2.6, railties-5.2.6, sprockets-rails-3.2.2, nio4r-2.5.8, sass-3.7.4, sprockets-3.7.2, tilt-2.0.10, execjs-2.8.1, coffee-script-2.4.1, rails-dom-testing-2.0.3, thor-1.1.0, json-1.8.6, rdoc-4.3.0, msgpack-1.4.2, oai-1.1.0, chronic-0.10.2, deprecation-1.1.0, faraday-0.17.4, faraday-encoding-0.0.5, ruby-progressbar-1.11.0, rdf-vocab-3.1.14, bootstrap-sass-3.4.1, globalid-0.5.2, kaminari-1.2.1, twitter-typeahead-rails-0.11.1.pre.corejavascript, hydra-access-controls-12.0.1, hydra-core-12.0.1, http_logger-0.6.0, json-ld-3.1.9, rdf-isomorphic-3.1.1, rdf-turtle-3.1.3, slop-4.9.1, racc-1.5.2, xml-simple-1.1.9, builder-3.2.4, bcrypt-3.1.16, orm_adapter-0.5.0, responders-3.0.1, warden-1.2.9, hamster-3.0.0, link_header-0.0.8, rspec-core-3.10.1, rspec-expectations-3.10.1, rspec-mocks-3.10.2, rspec-support-3.10.2, childprocess-4.1.0, i18n-1.8.10, rubyzip-1.3.0, rb-fsevent-0.11.0, rb-inotify-0.10.1, retriable-3.1.2, factory_bot-6.2.0, binding_of_caller-1.0.0, websocket-driver-0.7.5, mail-2.7.1, rack-2.2.3, rack-test-1.1.0, rails-html-sanitizer-1.4.2, erubi-1.10.0, arel-9.0.0, marcel-1.0.1, concurrent-ruby-1.1.9, minitest-5.14.4, tzinfo-1.2.9, method_source-1.0.0, sass-listen-4.0.0, coffee-script-source-1.12.2, faraday_middleware-0.14.0, multipart-post-2.1.1, autoprefixer-rails-10.3.3.0, sassc-2.4.0, kaminari-actionview-1.2.1, kaminari-activerecord-1.2.1, kaminari-core-1.2.1, blacklight-access_controls-6.0.0, cancancan-1.17.0, htmlentities-4.3.4, json-canonicalization-0.2.1, multi_json-1.15.0, ebnf-2.2.1, rexml-3.2.5, diff-lcs-1.4.4, ffi-1.15.4, debug_inspector-1.1.0, websocket-extensions-0.1.5, mini_mime-1.1.1, loofah-2.12.0, thread_safe-0.3.6, amazing_print-1.3.0, sxp-1.1.0, unicode-types-1.7.0, crass-1.0.6 in any of the sources (Bundler::GemNotFound)
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler/definition.rb:234:in `specs_for'
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler/runtime.rb:18:in `setup'
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler.rb:149:in `setup'
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler/setup.rb:20:in `block in <top (required)>'
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler/ui/shell.rb:136:in `with_level'
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler/ui/shell.rb:88:in `silence'
from /usr/local/bundle/gems/bundler-2.2.27/lib/bundler/setup.rb:20:in `<top (required)>'
from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /usr/local/bundle/gems/spring-3.0.0/lib/spring/commands.rb:33:in `<module:Spring>'
from /usr/local/bundle/gems/spring-3.0.0/lib/spring/commands.rb:4:in `<top (required)>'
from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /usr/local/bundle/gems/spring-3.0.0/lib/spring/server.rb:9:in `<top (required)>'
from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /usr/local/bundle/gems/spring-3.0.0/lib/spring/client/server.rb:9:in `call'
from /usr/local/bundle/gems/spring-3.0.0/lib/spring/client/command.rb:7:in `call'
from /usr/local/bundle/gems/spring-3.0.0/lib/spring/client.rb:30:in `run'
from /usr/local/bundle/gems/spring-3.0.0/bin/spring:49:in `<main>'

Update: Whenever sets up the crontab using the file /config/schedule.rb relevant section

every 1.hour do command 'cd /home/hydra && bin/rails r import/cron_import.rb folklife' end

We are running this inside docker the dockerfile

FROM ruby:2.6.8

# Install our dependencies and rails
RUN \
    gem install bundler \
    && gem install rails \
    && mkdir -p /home/hydra

# update and install dependencies
RUN apt-get update && apt-get -y install cron

# Node.js
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash - \
    && apt-get install -y nodejs

WORKDIR /home/hydra
ADD ./hydra /home/hydra
RUN bundle install

ADD ./startup.sh /usr/bin/
RUN chmod -v +x /usr/bin/startup.sh
ENTRYPOINT ["/usr/bin/startup.sh"]

startup.sh

#!/bin/bash

# set terminal 
export TERM=vt100

# start cron and update whenever 
service cron start
bundle exec whenever
whenever --update-crontab

# remove PID and start the server
file="/home/hydra/tmp/pids/server.pid"
if [ -f $file ] ; then
    rm $file
fi

bin/rails s -p 3000 -b '0.0.0.0'

output from crontab -l (relevant sections)

# Begin Whenever generated tasks for: 
/home/hydra/config/schedule.rb at: 2021-09-21 18:38:41 +0000
PATH=\
GEM_HOME=/usr/local/bundle
BUNDLE_APP_CONFIG=/usr/local/bundle
RUBY_MAJOR=2.6
RUBYGEMS_VERSION=""
BUNDLE_BIN=""
BUNDLE_PATH=""
RUBY_VERSION=2.6.8
BUNDLER_VERSION=2.2.27

{section omitted}

* * * * * /bin/bash -l -c 'cd /home/hydra && bin/rails r import/cron_import.rb folklife >> /home/hydra/log/cron.log 2>> /home/hydra/log/cron_error.log'

# End Whenever generated tasks for: /home/hydra/config/schedule.rb at: 2021-08-20 20:22:02 +0000

Gemfile

source 'https://rubygems.org'

# Rails Dependencies
# =====================================================
gem 'rails', '~> 5.2.6'
gem 'bundler'
gem 'rake', '~> 12.3'
gem 'puma', '~> 4.3'
gem 'mysql2', '>= 0.4.4', '< 0.6.0'
gem 'sass-rails', '~> 5.0'
gem 'uglifier', '>= 1.3.0'
gem 'coffee-rails', '~> 4.2'
gem 'jquery-rails'
gem 'jbuilder', '~> 2.5'
gem 'sdoc', '~> 0.4.0', group: :doc

gem 'bootsnap', '>= 1.1.0', require: false

# OAI-PMH 
# =====================================================
gem 'blacklight_oai_provider'

# Automatic Import
# =====================================================
gem 'whenever', require: false

# Hydra Dependencies
# =====================================================
gem 'active-fedora'
gem 'active-triples'
gem 'blacklight'

gem 'hydra-head'
gem 'ldp'
gem 'nokogiri'
gem 'solrizer'

gem 'rsolr'
gem 'devise'
gem 'devise-guests', '~> 0.5'

gem 'rdf', '<= 3.1.13'

# Capistrano
# =====================================================
# gem 'capistrano-rails'

# DEVELOPMENT AND TESTS ONLY
# =====================================================
group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a   debugger console
  gem 'byebug'
  gem 'rspec-rails'
  gem 'jettywrapper'
  gem 'listen'
  gem 'fcrepo_wrapper', '~> 0.4'
  gem 'solr_wrapper', '~> 2.0'
end

group :test do
  gem 'factory_bot_rails'
end

group :development do
  gem 'web-console', '~> 2.0'
  gem 'spring'
end

Please share: 1) what command do you use to install the crontab? e.g. `whenever --update-crontab` 2) which user runs that command, and 3) for that user, what is the relevant output from `crontab -l`? — Jared Beck, Aug 20 '21 at 21:11

score 1 · Accepted Answer · answered Sep 23 '21 at 15:08

1

The issue was related to environment variables and not being able to find the correct path for the gems. I found a solution and updated the schedule.rb file.

ENV.each { |k, v| env(k, v) }

# set logs and environment
set :output, {:standard => "#{path}/log/cron.log", :error => "#{path}/log/cron_error.log"}
set :environment, ENV['RAILS_ENV']

every 1.minutes do
  command "cd #{path} && bin/rails r import/cron_import.rb folklife"
end

answered Sep 23 '21 at 15:08

Tracy McCormick

401
1
7
18

Info for this solution came from https://stackoverflow.com/questions/5835675/setting-path-for-whenever-in-cron-so-it-can-find-ruby – Tracy McCormick Sep 23 '21 at 15:14

`materialize': Could not find

1 Answers1