0

I am using Tinaroo (University of Queensland) super computer

When I call to run my code using

qsub 70my_01_140239.sh

I get this error

autosklearn.util.dependencies.IncorrectPackageVersionError: found 'dask' version 2021.11.2 but requires dask version >=2021.12

cleary i need to upgrade dask library

but I don't have access to upgrade it

Here is the actual code for qsub 70my_01_140239.sh

#!/bin/bash
#
#PBS -A qris-jcu
#
#PBS -l select=1:ncpus=24:mem=120GB
#PBS -l walltime=06:00:00
#PBS -N 70my_01_140239


shopt -s expand_aliases
source /etc/profile.d/modules.sh
cd ${PBS_O_WORKDIR}
module load python
module load anaconda
python 70myb.py 8000 3 "E&V" 2 1 15 10000 120 24

I tried adding conda install dask in 70my_01_140239.sh but it does not work as i have no permission to upgrade the library.

does anyone know How to upgrade a python library in a supercomputer

asmgx
  • 7,328
  • 15
  • 82
  • 143
  • 2
    I've never used a supercomputer, but FWIW, I assume it'd be better to ask the people who manage it, not us. Apart from that, what access do you have exactly? – wjandrea Jan 28 '23 at 00:21
  • @wjandrea support do not reply, and i think most supercomuter use the same OS and structure. looking for someone with knowledge – asmgx Jan 28 '23 at 00:24
  • 2
    Are you happy with the python version itself? Then just install your own python virtual environment and add the modules you want. – tdelaney Jan 28 '23 at 00:27
  • @tdelaney how to do that? how can install my own pythin virtual env? – asmgx Jan 28 '23 at 00:31
  • Well what access do you have exactly? For example, if you don't have internet access on it or can't sideload files or w/e, then you'd need to get creative. – wjandrea Jan 28 '23 at 00:38
  • @wjandrea I am a student, I have access to the internet, i tried ping google.com and I got a response – asmgx Jan 28 '23 at 00:43
  • There are a lot of ways to install a virtual environment. Look at https://docs.python.org/3/library/venv.html to start. Conda is another good option https://docs.conda.io/en/latest/. – tdelaney Jan 28 '23 at 00:51
  • Yeah, if you have full access with `conda`, you're golden. Check out [this answer](/a/59737857/4518341) (TLDR: create an environment first and activate it, then install the package). You'll also need to activate the environment in your script. – wjandrea Jan 28 '23 at 01:06

0 Answers0