Cloudify 3.2.1 install workflow fail with error "install-nodejs.sh: not found" on aws ec2

Question

I'm a newbie for using cloudify. I have managed to bootstrap a Cloudify Manager on aws ec2. However, I have a problem when executing install workflow of the nodecellar.

My environments are: cloudify-windows-cli_3.2.1, cloudify-manager-blueprints 3.2, cloudify-nodecellar-example 3.2

input.yaml

image: 'ami-5189a661'
size: 't2.medium'
agent_user: 'ubuntu'

When executing install workflow, instances for nodecellar and mongodb have been successfully created, however there were some errors:

Executing workflow 'install' on deployment 'nodecellar' at management server 52.
19.100.72 [timeout=900 seconds]
Deployment environment creation is in progress!
Waiting for create_deployment_environment workflow execution to finish...
2015-08-25T11:31:27 CFY <nodecellar> Starting 'create_deployment_environment' wo
rkflow execution
2015-08-25T11:31:27 CFY <nodecellar> Installing deployment operations worker
2015-08-25T11:31:28 CFY <nodecellar> Sending task 'worker_installer.tasks.instal
l'
2015-08-25T11:31:28 CFY <nodecellar> Task started 'worker_installer.tasks.instal
l'
2015-08-25T11:31:28 CFY <nodecellar> Task succeeded 'worker_installer.tasks.inst
all'
2015-08-25T11:31:28 CFY <nodecellar> Starting deployment operations worker
2015-08-25T11:31:29 CFY <nodecellar> Sending task 'worker_installer.tasks.start'

2015-08-25T11:31:29 CFY <nodecellar> Task started 'worker_installer.tasks.start'

2015-08-25T11:31:34 CFY <nodecellar> Task succeeded 'worker_installer.tasks.star
t'
2015-08-25T11:31:34 CFY <nodecellar> Installing deployment operations plugins
2015-08-25T11:31:34 CFY <nodecellar> Sending task 'plugin_installer.tasks.instal
l'
2015-08-25T11:31:35 CFY <nodecellar> Task started 'plugin_installer.tasks.instal
l'
2015-08-25T11:31:43 CFY <nodecellar> Task succeeded 'plugin_installer.tasks.inst
all'
2015-08-25T11:31:50 CFY <nodecellar> Installing deployment workflows worker
2015-08-25T11:31:50 CFY <nodecellar> Sending task 'worker_installer.tasks.instal
l'
2015-08-25T11:31:50 CFY <nodecellar> Task started 'worker_installer.tasks.instal
l'
2015-08-25T11:31:52 CFY <nodecellar> Task succeeded 'worker_installer.tasks.inst
all'
2015-08-25T11:31:52 CFY <nodecellar> Starting deployment workflows worker
2015-08-25T11:31:52 CFY <nodecellar> Sending task 'worker_installer.tasks.start'

2015-08-25T11:31:52 CFY <nodecellar> Task started 'worker_installer.tasks.start'

2015-08-25T11:31:58 CFY <nodecellar> Task succeeded 'worker_installer.tasks.star
t'
2015-08-25T11:31:58 CFY <nodecellar> Starting deployment policy engine core
2015-08-25T11:31:58 CFY <nodecellar> Sending task 'riemann_controller.tasks.crea
te'
2015-08-25T11:31:58 CFY <nodecellar> Task started 'riemann_controller.tasks.crea
te'
2015-08-25T11:31:59 CFY <nodecellar> Task succeeded 'riemann_controller.tasks.cr
eate'
2015-08-25T11:31:59 CFY <nodecellar> 'create_deployment_environment' workflow ex
ecution succeeded
2015-08-25T11:32:00 CFY <nodecellar> Starting 'install' workflow execution
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_security_group_c7651] Creating
node
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_elasticip_3a4c9] Creating node
2015-08-25T11:32:01 CFY <nodecellar> [mongod_security_group_60680] Creating node

2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_elasticip_3a4c9.create] Sending
 task 'ec2.elasticip.allocate'
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_security_group_c7651.create] Se
nding task 'ec2.securitygroup.create'
2015-08-25T11:32:01 CFY <nodecellar> [mongod_security_group_60680.create] Sendin
g task 'ec2.securitygroup.create'
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_elasticip_3a4c9.create] Task st
arted 'ec2.elasticip.allocate'
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_security_group_c7651.create] Ta
sk started 'ec2.securitygroup.create'
2015-08-25T11:32:01 CFY <nodecellar> [mongod_security_group_60680.create] Task s
tarted 'ec2.securitygroup.create'
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_elasticip_3a4c9.create] Task su
cceeded 'ec2.elasticip.allocate'
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_security_group_c7651.create] Ta
sk succeeded 'ec2.securitygroup.create'
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_elasticip_3a4c9] Configuring no
de
2015-08-25T11:32:01 CFY <nodecellar> [nodecellar_security_group_c7651] Configuri
ng node
2015-08-25T11:32:02 CFY <nodecellar> [nodecellar_elasticip_3a4c9] Starting node
2015-08-25T11:32:02 CFY <nodecellar> [mongod_security_group_60680.create] Task s
ucceeded 'ec2.securitygroup.create'
2015-08-25T11:32:02 CFY <nodecellar> [nodecellar_security_group_c7651] Starting
node
2015-08-25T11:32:02 CFY <nodecellar> [mongod_security_group_60680] Configuring n
ode
2015-08-25T11:32:02 CFY <nodecellar> [nodejs_host_c340b] Creating node
2015-08-25T11:32:02 CFY <nodecellar> [nodejs_host_c340b.create] Sending task 'ec
2.instance.run_instances'
2015-08-25T11:32:02 CFY <nodecellar> [nodejs_host_c340b.create] Task started 'ec
2.instance.run_instances'
2015-08-25T11:32:02 CFY <nodecellar> [mongod_security_group_60680] Starting node

2015-08-25T11:32:03 CFY <nodecellar> [mongod_host_58700] Creating node
2015-08-25T11:32:03 CFY <nodecellar> [mongod_host_58700.create] Sending task 'ec
2.instance.run_instances'
2015-08-25T11:32:03 CFY <nodecellar> [mongod_host_58700.create] Task started 'ec
2.instance.run_instances'
2015-08-25T11:32:03 CFY <nodecellar> [nodejs_host_c340b.create] Task succeeded '
ec2.instance.run_instances'
2015-08-25T11:32:03 CFY <nodecellar> [nodejs_host_c340b] Configuring node
2015-08-25T11:32:04 CFY <nodecellar> [mongod_host_58700.create] Task succeeded '
ec2.instance.run_instances'
2015-08-25T11:32:04 CFY <nodecellar> [nodejs_host_c340b] Starting node
2015-08-25T11:32:04 CFY <nodecellar> [nodejs_host_c340b.start] Sending task 'ec2
.instance.start'
2015-08-25T11:32:04 CFY <nodecellar> [nodejs_host_c340b.start] Task started 'ec2
.instance.start'
2015-08-25T11:32:04 CFY <nodecellar> [mongod_host_58700] Configuring node
2015-08-25T11:32:04 CFY <nodecellar> [mongod_host_58700] Starting node
2015-08-25T11:32:04 CFY <nodecellar> [mongod_host_58700.start] Sending task 'ec2
.instance.start'
2015-08-25T11:32:04 CFY <nodecellar> [mongod_host_58700.start] Task started 'ec2
.instance.start'
2015-08-25T11:32:05 CFY <nodecellar> [nodejs_host_c340b.start] Task rescheduled
'ec2.instance.start' -> OperationRetry('Waiting server to be running. Retrying..
.',) [attempt 1]
2015-08-25T11:32:05 CFY <nodecellar> [mongod_host_58700.start] Task rescheduled
'ec2.instance.start' -> OperationRetry('Waiting server to be running. Retrying..
.',) [attempt 1]
2015-08-25T11:32:35 CFY <nodecellar> [nodejs_host_c340b.start] Sending task 'ec2
.instance.start' [attempt 2]
2015-08-25T11:32:35 CFY <nodecellar> [nodejs_host_c340b.start] Task started 'ec2
.instance.start' [attempt 2]
2015-08-25T11:32:35 CFY <nodecellar> [mongod_host_58700.start] Sending task 'ec2
.instance.start' [attempt 2]
2015-08-25T11:32:35 CFY <nodecellar> [mongod_host_58700.start] Task started 'ec2
.instance.start' [attempt 2]
2015-08-25T11:32:35 CFY <nodecellar> [nodejs_host_c340b.start] Task succeeded 'e
c2.instance.start' [attempt 2]
2015-08-25T11:32:35 CFY <nodecellar> [nodejs_host_c340b] Installing worker
2015-08-25T11:32:35 CFY <nodecellar> [mongod_host_58700.start] Task succeeded 'e
c2.instance.start' [attempt 2]
2015-08-25T11:32:36 CFY <nodecellar> [nodejs_host_c340b.install] Sending task 'w
orker_installer.tasks.install'
2015-08-25T11:32:36 CFY <nodecellar> [nodejs_host_c340b.install] Task started 'w
orker_installer.tasks.install'
2015-08-25T11:32:36 CFY <nodecellar> [mongod_host_58700] Installing worker
2015-08-25T11:32:36 CFY <nodecellar> [mongod_host_58700.install] Sending task 'w
orker_installer.tasks.install'
2015-08-25T11:32:36 CFY <nodecellar> [mongod_host_58700.install] Task started 'w
orker_installer.tasks.install'
2015-08-25T11:32:46 CFY <nodecellar> [nodejs_host_c340b.install] Task failed 'wo
rker_installer.tasks.install' -> RecoverableError('NetworkError: Timed out tryin
g to connect to 172.31.20.80 (tried 1 time)',) [attempt 1]
2015-08-25T11:32:46 CFY <nodecellar> [mongod_host_58700.install] Task failed 'wo
rker_installer.tasks.install' -> RecoverableError('NetworkError: Timed out tryin
g to connect to 172.31.24.55 (tried 1 time)',) [attempt 1]
2015-08-25T11:33:16 CFY <nodecellar> [nodejs_host_c340b.install] Sending task 'w
orker_installer.tasks.install' [attempt 2]
2015-08-25T11:33:16 CFY <nodecellar> [nodejs_host_c340b.install] Task started 'w
orker_installer.tasks.install' [attempt 2]
2015-08-25T11:33:16 CFY <nodecellar> [mongod_host_58700.install] Sending task 'w
orker_installer.tasks.install' [attempt 2]
2015-08-25T11:33:16 CFY <nodecellar> [mongod_host_58700.install] Task started 'w
orker_installer.tasks.install' [attempt 2]
2015-08-25T11:33:24 CFY <nodecellar> [nodejs_host_c340b.install] Task succeeded
'worker_installer.tasks.install' [attempt 2]
2015-08-25T11:33:24 CFY <nodecellar> [mongod_host_58700.start] Sending task 'wor
ker_installer.tasks.start'
2015-08-25T11:33:24 CFY <nodecellar> [mongod_host_58700.start] Task started 'wor
ker_installer.tasks.start'
2015-08-25T11:33:24 CFY <nodecellar> [mongod_host_58700.install] Task succeeded
'worker_installer.tasks.install' [attempt 2]
2015-08-25T11:33:24 CFY <nodecellar> [mongod_host_58700.start] Sending task 'wor
ker_installer.tasks.start'
2015-08-25T11:33:24 CFY <nodecellar> [mongod_host_58700.start] Task started 'wor
ker_installer.tasks.start'
2015-08-25T11:33:30 CFY <nodecellar> [nodejs_host_c340b.start] Task succeeded 'w
orker_installer.tasks.start'
2015-08-25T11:33:30 CFY <nodecellar> [mongod_host_58700.start] Task succeeded 'w
orker_installer.tasks.start'
2015-08-25T11:33:30 CFY <nodecellar> [nodejs_host_c340b.install] Sending task 'p
lugin_installer.tasks.install'
2015-08-25T11:33:30 CFY <nodecellar> [nodejs_host_c340b.install] Task started 'p
lugin_installer.tasks.install'
2015-08-25T11:33:30 CFY <nodecellar> [mongod_host_58700.start] Task succeeded 'w
orker_installer.tasks.start'
2015-08-25T11:33:31 CFY <nodecellar> [mongod_host_58700] Installing host plugins

2015-08-25T11:33:31 CFY <nodecellar> [mongod_host_58700.install] Sending task 'p
lugin_installer.tasks.install'
2015-08-25T11:33:31 CFY <nodecellar> [mongod_host_58700.install] Task started 'p
lugin_installer.tasks.install'
2015-08-25T11:33:36 CFY <nodecellar> [nodejs_host_c340b.install] Task succeeded
'plugin_installer.tasks.install'
2015-08-25T11:33:36 CFY <nodecellar> [mongod_host_58700.install] Task succeeded
'plugin_installer.tasks.install'
2015-08-25T11:33:44 CFY <nodecellar> [nodejs_host_c340b.install] Sending task 'd
iamond_agent.tasks.install'
2015-08-25T11:33:44 CFY <nodecellar> [nodejs_host_c340b.install] Task started 'd
iamond_agent.tasks.install'
2015-08-25T11:33:44 CFY <nodecellar> [nodejs_host_c340b.install] Task succeeded
'diamond_agent.tasks.install'
2015-08-25T11:33:44 CFY <nodecellar> [nodejs_host_c340b.start] Sending task 'dia
mond_agent.tasks.start'
2015-08-25T11:33:44 CFY <nodecellar> [nodejs_host_c340b.start] Task started 'dia
mond_agent.tasks.start'
2015-08-25T11:33:45 CFY <nodecellar> [nodejs_host_c340b->nodecellar_elasticip_3a
4c9|establish] Sending task 'ec2.elasticip.associate'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.install] Task succeeded
'diamond_agent.tasks.install'
2015-08-25T11:33:45 CFY <nodecellar> [nodejs_host_c340b->nodecellar_elasticip_3a
4c9|establish] Task succeeded 'ec2.elasticip.associate'
2015-08-25T11:33:45 CFY <nodecellar> [nodejs_host_c340b.start] Task started 'dia
mond_agent.tasks.add_collectors'
2015-08-25T11:33:45 CFY <nodecellar> [nodejs_host_c340b->nodecellar_elasticip_3a
4c9|establish] Task started 'ec2.elasticip.associate'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.install] Sending task 'd
iamond_agent.tasks.install'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.install] Task started 'd
iamond_agent.tasks.install'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.install] Task succeeded
'diamond_agent.tasks.install'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.start] Sending task 'dia
mond_agent.tasks.start'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.start] Task started 'dia
mond_agent.tasks.start'
2015-08-25T11:33:45 CFY <nodecellar> [nodejs_host_c340b->nodecellar_elasticip_3a
4c9|establish] Task succeeded 'ec2.elasticip.associate'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.start] Task succeeded 'd
iamond_agent.tasks.start'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.start] Sending task 'dia
mond_agent.tasks.add_collectors'
2015-08-25T11:33:45 CFY <nodecellar> [mongod_host_58700.start] Task started 'dia
mond_agent.tasks.add_collectors'
2015-08-25T11:33:47 CFY <nodecellar> [nodejs_host_c340b.start] Task succeeded 'd
iamond_agent.tasks.add_collectors'
2015-08-25T11:33:47 CFY <nodecellar> [nodejs_179d0] Creating node
2015-08-25T11:33:47 CFY <nodecellar> [nodejs_179d0.create] Sending task 'script_
runner.tasks.run'
2015-08-25T11:33:47 CFY <nodecellar> [nodejs_179d0.create] Task started 'script_
runner.tasks.run'
2015-08-25T11:33:47 CFY <nodecellar> [nodejs_179d0.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/FX2P2/i
nstall-nodejs.sh: not found\n',) [attempt 1]
2015-08-25T11:33:48 CFY <nodecellar> [mongod_host_58700.start] Task succeeded 'd
iamond_agent.tasks.add_collectors'
2015-08-25T11:33:48 CFY <nodecellar> [mongod_2751b] Creating node
2015-08-25T11:33:48 CFY <nodecellar> [mongod_2751b.create] Sending task 'script_
runner.tasks.run'
2015-08-25T11:33:48 CFY <nodecellar> [mongod_2751b.create] Task started 'script_
runner.tasks.run'
2015-08-25T11:33:49 CFY <nodecellar> [mongod_2751b.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/6YZLJ/i
nstall-mongo.sh: not found\n',) [attempt 1]
2015-08-25T11:34:17 CFY <nodecellar> [nodejs_179d0.create] Sending task 'script_
runner.tasks.run' [attempt 2]
2015-08-25T11:34:17 CFY <nodecellar> [nodejs_179d0.create] Task started 'script_
runner.tasks.run' [attempt 2]
2015-08-25T11:34:17 CFY <nodecellar> [nodejs_179d0.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/IMY6N/i
nstall-nodejs.sh: not found\n',) [attempt 2]
2015-08-25T11:34:19 CFY <nodecellar> [mongod_2751b.create] Sending task 'script_
runner.tasks.run' [attempt 2]
2015-08-25T11:34:19 CFY <nodecellar> [mongod_2751b.create] Task started 'script_
runner.tasks.run' [attempt 2]
2015-08-25T11:34:19 CFY <nodecellar> [mongod_2751b.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/3H050/i
nstall-mongo.sh: not found\n',) [attempt 2]
2015-08-25T11:34:47 CFY <nodecellar> [nodejs_179d0.create] Sending task 'script_
runner.tasks.run' [attempt 3]
2015-08-25T11:34:47 CFY <nodecellar> [nodejs_179d0.create] Task started 'script_
runner.tasks.run' [attempt 3]
2015-08-25T11:34:48 CFY <nodecellar> [nodejs_179d0.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/33K21/i
nstall-nodejs.sh: not found\n',) [attempt 3]
2015-08-25T11:34:49 CFY <nodecellar> [mongod_2751b.create] Sending task 'script_
runner.tasks.run' [attempt 3]
2015-08-25T11:34:49 CFY <nodecellar> [mongod_2751b.create] Task started 'script_
runner.tasks.run' [attempt 3]
2015-08-25T11:34:49 CFY <nodecellar> [mongod_2751b.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/I8UV9/i
nstall-mongo.sh: not found\n',) [attempt 3]
2015-08-25T11:35:18 CFY <nodecellar> [nodejs_179d0.create] Sending task 'script_
runner.tasks.run' [attempt 4]
2015-08-25T11:35:18 CFY <nodecellar> [nodejs_179d0.create] Task started 'script_
runner.tasks.run' [attempt 4]
2015-08-25T11:35:18 CFY <nodecellar> [nodejs_179d0.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/UYB9N/i
nstall-nodejs.sh: not found\n',) [attempt 4]
2015-08-25T11:35:19 CFY <nodecellar> [mongod_2751b.create] Sending task 'script_
runner.tasks.run' [attempt 4]
2015-08-25T11:35:19 CFY <nodecellar> [mongod_2751b.create] Task started 'script_
runner.tasks.run' [attempt 4]
2015-08-25T11:35:19 CFY <nodecellar> [mongod_2751b.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/AU51M/i
nstall-mongo.sh: not found\n',) [attempt 4]
2015-08-25T11:35:48 CFY <nodecellar> [nodejs_179d0.create] Sending task 'script_
runner.tasks.run' [attempt 5]
2015-08-25T11:35:48 CFY <nodecellar> [nodejs_179d0.create] Task started 'script_
runner.tasks.run' [attempt 5]
2015-08-25T11:35:48 CFY <nodecellar> [nodejs_179d0.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/GYXQI/i
nstall-nodejs.sh: not found\n',) [attempt 5]
2015-08-25T11:35:49 CFY <nodecellar> [mongod_2751b.create] Sending task 'script_
runner.tasks.run' [attempt 5]
2015-08-25T11:35:49 CFY <nodecellar> [mongod_2751b.create] Task started 'script_
runner.tasks.run' [attempt 5]
2015-08-25T11:35:50 CFY <nodecellar> [mongod_2751b.create] Task failed 'script_r
unner.tasks.run' -> RecoverableError('ProcessException: /bin/sh: 1: /tmp/BTGYZ/i
nstall-mongo.sh: not found\n',) [attempt 5]

I am wondering is there anything I need to check regarding to these errors? Thanks.

Answer 1

It's really hard to say because you haven't included the complete log. However, what comes to mind is that this particular error could be caused because the agent wasn't able to write the file to the /tmp folder because of a permissions issue. I would check that you have configured the agent user correctly.

Answer 2

I have managed to solve the problem. It's about line encoding of script files in a blueprint.

Basically, in Unix systems, the end of a line is a line feed (LF), while in windows, it's a carriage return (CR) and a line feed (LF). Therefore, when I cloned a cloudify-nodecellar-example repository from GitHub to my local windows, the line encoding in the script files was automatically changed. Therefore, when I uploaded the blueprint and executed the workflow, there was an error that /bin/sh not found in the .sh file.

Basically, you can disable the line encoding correction by:

$ git config --global core.autocrlf true
# Configure Git on Windows to properly handle line endings

This solves the problem, thanks to this question for giving me some ideas.