ipython terminates with 'Syntax error: end of file unexpected' inside Kubeflow Pod, not happening in Docker container

Question

I'm trying to create a Kubeflow Pipeline component. My command works fine in a local Docker container, but within a Pod in a Kubeflow Pipeline it terminates with:

ipython /tmp/input-postgresql.ipynb: 1: ipython /tmp/input-postgresql.ipynb: Syntax error: end of file unexpected

To reproduce on Docker:

docker run -it continuumio/anaconda3:2020.07 /bin/sh
curl -o /tmp/input-postgresql.ipynb https://raw.githubusercontent.com/IBM/claimed/master/component-library/input/input-postgresql.ipynb && ipython /tmp/input-postgresql.ipynb

To reproduce on Kubeflow 1.6.0 please use the pipeline.yaml file

How did you create file `/tmp/input-postgresql.ipynb`? Did you use [CRLF line terminations or LF](https://askinglot.com/what-is-difference-between-lf-and-crlf#:~:text=The%20term%20CRLF%20refers%20to,used%20to%20terminate%20a%20line.)? Did you see [this topic](https://stackoverflow.com/questions/6366530/bash-syntax-error-unexpected-end-of-file)? — Mikołaj Głodziak, Aug 17 '21 at 12:30

score 1 · Accepted Answer · answered Aug 18 '21 at 21:55

Had to change the component.yaml now it works

Changed syntax for command definition and therefore also needed to find out how to pass parameters:

implementation:
  container:
    image: continuumio/anaconda3:2020.07
    command:
    - sh
    - -ec
    - |
      host=$0
      database=$1
      user=$2
      password=$3
      port=$4
      sql=$5
      data_dir=$6
      output_data_csv=$7
      mkdir -p $output_data_csv
      wget https://raw.githubusercontent.com/IBM/claimed/master/component-library/input/input-postgresql.ipynb
      ipython ./input-postgresql.ipynb host=$host database=$database user=$user password=$password port=$port sql=$sql data_dir=$data_dir output_data_csv=$output_data_csv
    - {inputValue: host}
    - {inputValue: database}
    - {inputValue: user}
    - {inputValue: password}
    - {inputValue: port}
    - {inputValue: sql}
    - {inputValue: data_dir}
    - {outputPath: output_data_csv}

ipython terminates with 'Syntax error: end of file unexpected' inside Kubeflow Pod, not happening in Docker container

1 Answers1