0

I am trying to read a GRIB2 file using pygrib. I manage to open files and read data without problem, but I need to automate the process, and unfortunately each file contains multiple very similar fields:

1:Total Cloud Cover:% (avg):regular_ll:unknown:level 0 214:fcst time 0-1 hrs (avg):from 201705200000
2:Total Cloud Cover:% (avg):regular_ll:unknown:level 0 224:fcst time 0-1 hrs (avg):from 201705200000
3:Total Cloud Cover:% (avg):regular_ll:unknown:level 0 234:fcst time 0-1 hrs (avg):from 201705200000
4:Total Cloud Cover:% (avg):regular_ll:unknown:level 0 10:fcst time 0-1 hrs (avg):from 201705200000
5:Total Cloud Cover:% (instant):regular_ll:unknown:level 0 244:fcst time 1 hrs:from 201705200000
6:Total Cloud Cover:% (avg):regular_ll:unknown:level 0 211:fcst time 0-1 hrs (avg):from 201705200000

The only difference (as seen by pygrib) for these file is the field typeOfFirstFixedSurface, but I have no idea what this field (and cannot find the relevant information on the site where I got the grib2 files). I have looked at all the key/value parameters for every messages, and did not find any other useful information I could use to differentiate the fields....

However, when using Panoply, I see much more parameters, e.g.:

float Total_cloud_cover_convective_cloud(time=1, lat=721, lon=1440);
  :long_name = "Total cloud cover @ Convective cloud layer";
  :units = "%";
  :abbreviation = "TCDC";
  :missing_value = NaNf; // float
  :grid_mapping = "LatLon_Projection";
  :coordinates = "reftime time lat lon ";
  :Grib_Variable_Id = "VAR_0-6-1_L244";
  :Grib2_Parameter = 0, 6, 1; // int
  :Grib2_Parameter_Discipline = "Meteorological products";
  :Grib2_Parameter_Category = "Cloud";
  :Grib2_Parameter_Name = "Total cloud cover";
  :Grib2_Level_Type = "Convective cloud layer";
  :Grib2_Generating_Process_Type = "Forecast";

I could definitively use the long_name or Grib_Variable_Id fields to differentiate between messages, but I cannot access these "parameters" using pygrib.

Is there a way to access these parameters using pygrib?

Holt
  • 36,600
  • 7
  • 92
  • 139
  • Check out [my answer to my own question](https://gis.stackexchange.com/questions/173775/how-to-extract-specific-information-from-grib-files) regarding GRIB data, it might be helpful... I pulled almost all my hair out figuring out GRIB data because documentation is just simply terrible :/ – errata May 29 '17 at 14:08
  • @errata Thanks for the alternative, unfortunately I tried to install `iris_grib` and did not succeed because I do not manage to install the `gribapi` python module... It is amazing how much people put effort into creating new formats while having nobody able to use them because the proposed API are either a pure mess to use or to install... – Holt May 29 '17 at 15:57
  • If you want, I can share with you the steps I had to do to install `iris`... It was complete mess, just like everything related to GRIB was :/ – errata May 29 '17 at 16:15
  • @errata I did manage to install `iris`, the problem is that I don't get how I can install `gribapi` for python3 (to be able to use `iris_grib`) - Installation from sources is a mess, and the available package for debian requires python3.5 and some python modules with versions I do not have access to. I have found a way to deal with the issue I have for the files I want to use (I can link the values of `typeOfFirstFixedSurface` to corresponding `long_name`), but maybe it would be a good idea to add step-by-step to your answer on your question, and I could mark this one as a duplicate. – Holt May 29 '17 at 16:18
  • I'll write you an answer, hopefully might be helpful to you or anyone else troubled with this... – errata May 29 '17 at 16:22

2 Answers2

2

I had lots of problems with figuring out this particular format and tools which could help me with reading/parsing it... Eventually, I ended up with using IRIS which ended up in extremely messy setup. I did this a year and a half ago on Ubuntu machine with Python 2.7. I am pasting the steps I needed to do in order to install IRIS and all possible dependencies, hopefully this will be of some use to you although there are probably lots of outdated versions in the whole setup...

Good luck!

# http://scitools.org.uk/iris/docs/latest/installing.html

# necessary steps for a clean machine
#
# sudo apt-get install gcc python-dev build-essential python-setuptools libpq-dev git unzip cmake
# pip install virtualenv
# pip install virtualenvwrapper
# mkdir ~/.virtualenvs
# nano ~/.bashrc
# export WORKON_HOME=$HOME/.virtualenvs
# source /usr/local/bin/virtualenvwrapper.sh
# . ~/.bashrc
# mkvirtualenv iris

pip install numpy
pip install biggus

sudo apt-get install libblas-dev liblapack-dev libatlas-base-dev gfortran
# OR
# sudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose

pip install scipy

# cartopy dependecies
pip install cython

# https://github.com/OSGeo/proj.4
wget http://download.osgeo.org/proj/proj-4.9.1.tar.gz
tar -xzf proj-4.9.1.tar.gz
cd proj-4.9.1/
./configure
make
make install

# OR
# sudo apt-get install libproj-dev
# sudo apt-get install libgeos-dev

# http://sourceforge.net/projects/pyke/files/pyke/
wget http://sourceforge.net/projects/pyke/files/pyke/1.1.1/pyke-1.1.1.zip/download
unzip download
cd pyke-1.1.1/
python setup.py build
python setup.py install

pip install cartopy

# netcdf4 dependecies
# https://code.google.com/p/netcdf4-python/wiki/UbuntuInstall
# wget https://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.16.tar (??????????????)
sudo apt-get install libhdf5-dev

pip install h5py

sudo apt-get install python-netcdf libnetcdf-dev libnetcdf4

git clone https://github.com/Unidata/netcdf4-python.git
cd netcdf4-python
python setup.py build
python setup.py install


# other iris dependencies
pip install cf_units

sudo apt-get install libudunits2-dev

nano ~/.bashrc
# check where exactly xml file is
export UDUNITS2_XML_PATH=/usr/local/share/doc/udunits/udunits2.xml
. ~/.bashrc

pip install pillow

# gribapi
# https://software.ecmwf.int/wiki/display/GRIB/GRIB+API+CMake+installation
wget https://software.ecmwf.int/wiki/download/attachments/3473437/grib_api-1.14.4-Source.tar.gz?api=v2
tar -xzf grib_api-1.14.4-Source.tar.gz?api=v2
mkdir build; cd build
cmake ../grib_api-1.14.0-Source -DENABLE_PYTHON=ON
make -j4
ctest -j4
make install

# OR
# sudo apt-get install libgrib-api-dev
# sudo apt-get install openjpeg-tools

cp -R /usr/local/lib/python2.7/site-packages/grib_api ~/.virtualenvs/iristest/lib/python2.7/site-packages/

# aaaand, here we go, iris!
git clone https://github.com/SciTools/iris.git
cd iris
python setup.py build
python setup.py install
# rejoice!

# pip freeze output:
# Biggus==0.12.0
# Cartopy==0.13.0
# cf-units==1.0.0
# Cython==0.23.4
# h5py==2.5.0
# Iris==1.10.0.dev0
# netCDF4==1.2.2
# numpy==1.10.2
# Pillow==3.0.0
# pyke==1.1.1
# pyshp==1.2.3
# scipy==0.16.1
# Shapely==1.5.13
# six==1.10.0
errata
  • 5,695
  • 10
  • 54
  • 99
0

In order to install iris, using the Anaconda distribution, I did:

conda install -c conda-forge iris

and all required packages were installed at once without problems. I have Anaconda 3 version.

Bart
  • 9,825
  • 5
  • 47
  • 73
Pere Munar
  • 49
  • 7