1

Hey guys I was wondering if it is possible to plot NetCDF files using Gnuplot. I was trying to use PyFerret on the command line, but I was unable to figure it out. Using PyFerret I was able to determine that the NetCDF file has the following data:

yes? show data
     currently SET data sets:
    1> ./CrawfordPoint_surface.nc  (default)
 name     title                             I         J         K         L
 T_2M     Air temperature at 2 m           ...       ...       ...       1:105193
 T_2M_O   Origin of air temperature data   ...       ...       ...       1:105193
 RH_2M    Relative humidity at 2 m         ...       ...       ...       1:105193
 RH_2M_O  Origin of relative humidity dat  ...       ...       ...       1:105193
 WS_10M   Wind speed at 10 m               ...       ...       ...       1:105193
 WS_10M_O Origin of wind speed data        ...       ...       ...       1:105193
 P        Snowfall rate                    ...       ...       ...       1:105193
 P_O      Origin of snowfall rate          ...       ...       ...       1:105193
 SRIN     Downward shortwave radiation     ...       ...       ...       1:105193
 SRIN_O   Origin of downward shortwave ra  ...       ...       ...       1:105193
 SROUT    Upward shortwave radiation       ...       ...       ...       1:105193
 SROUT_O  Origin of the upward shortwave   ...       ...       ...       1:105193
 LRIN     Downward longwave radiation      ...       ...       ...       1:105193
 LROUT    Upward longwave radiation        ...       ...       ...       1:105193
 SHF      Sensible heat flux               ...       ...       ...       1:105193
 LHF      Latent heat flux                 ...       ...       ...       1:105193
 G        Subsurface heat flux             ...       ...       ...       1:105193
 TS       Surface temperature              ...       ...       ...       1:105193
 MELT     Melt rate                        ...       ...       ...       1:105193

Then I was trying to plot the data set T_2M to which I tried the following

yes? show grid T_2M
    GRID GFP1
 name       axis              # pts   start                end                 subset
 normal    X
 normal    Y
 normal    Z
 TIME      TIME            105193 i   01-JUN-1998 00:00    01-JUN-2010 00:00   full

Which made me wonder, where is the temperature data I am looking for, since it appears there is only time data on the file. However using an app called Panopoly I was able to visualize the data, so there is indeed the temp. data I look for somewhere in the T_2M file. So I tried potting it using Gnuplot but I was unable to do so. I am new to dealing with these type of files.

RSM
  • 227
  • 2
  • 11
  • 1
    I forgot all what I knew about `feret`. Do you know a way of outputing data like in a way that each line looks like `latitude, longitude, timestamp, value variable 1, value variable 2, ...`. The commas are not so important, all we need is a one-value-of-each-variable-per-geospatial-coordinate-format. If you have data this way, I could help you with the conversion for gnuplot (basically using sed, awk, bash, powershell, or something). I faintly remember that the famous [GNUPLOT not so FAQ](http://lowrank.net/gnuplot/index-e.html) helped me back in the days. – B--rian Jan 30 '20 at 13:46
  • Yeah ! My supervisor told me to use GNUPlot instead but I am so new to Ferret I have no idea how to extract data from the netCDF file onto a .txt file or .dat one – RSM Mar 14 '20 at 17:47

1 Answers1

0

One-dimensional netcdf data: Convert with ncks --json (part of nco) and jq to standard gnuplot format

This answer only addresses one-dimensional data which corresponds to the prototypical gnuplot data file as following:

❯ cat WorldEnergy.dat # from the demo folder of the gnuplot source
Latitude Longitude Continent                  Coal   Oil     Gas    Nuclear Hydroelectric Renewable
55       85        "Asia (excl. Middle East)" 947639 400017  272284 117365  44165         577942
17       -90       "C. America \& Caribbean"  4868   179261  40297  2607    4199          29201
25       38        "Middle East \& N. Africa" 14229  1240170 260613 0       5653          12124
50       -95       "North America"            600275 489867  585206 220291  54923         91945
-10      -55       "South America"            29025  345297  72900  2888    43298         62553
-5       25        "Sub-Saharan Africa"       129456 187118  9203   3345    4302          195472
50       25        "Europe"                   391517 648678  741413 303542  61526         67725
-24      133       "Oceania"                  155000 25500   27000  0       1400          5000

I leave out multi-data-set files separated by blank lines and gnuplot formats for splot (grid, sparse matrix, uniform matrix, nonuniform matrix).

I do a roundtrip with the help of Pandas, Xarray, Scipy:

#!/usr/bin/env python3
import pandas as pd
import xarray as xr
df = pd.read_table("WorldEnergy.dat", comment='#', delim_whitespace=True)
ds = xr.Dataset.from_dataframe(df)
ds.to_netcdf("WorldEnergy.nc")

Now, we have a netcdf file which has following human readable representation:

$ ncdump WorldEnergy.nc
netcdf WorldEnergy {
dimensions:
    index = 8 ;
    string25 = 25 ;
variables:
    char Continent(index, string25) ;
        Continent:_Encoding = "utf-8" ;
    int index(index) ;
    int Latitude(index) ;
    int Longitude(index) ;
    int Coal(index) ;
    int Oil(index) ;
    int Gas(index) ;
    int Nuclear(index) ;
    int Hydroelectric(index) ;
    int Renewable(index) ;
data:

 Continent =
  "Asia (excl. Middle East)",
  "C. America \\\\& Caribbean",
  "Middle East \\\\& N. Africa",
  "North America",
  "South America",
  "Sub-Saharan Africa",
  "Europe",
  "Oceania" ;

 index = 0, 1, 2, 3, 4, 5, 6, 7 ;

 Latitude = 55, 17, 25, 50, -10, -5, 50, -24 ;

 Longitude = 85, -90, 38, -95, -55, 25, 25, 133 ;

 Coal = 947639, 4868, 14229, 600275, 29025, 129456, 391517, 155000 ;

 Oil = 400017, 179261, 1240170, 489867, 345297, 187118, 648678, 25500 ;

 Gas = 272284, 40297, 260613, 585206, 72900, 9203, 741413, 27000 ;

 Nuclear = 117365, 2607, 0, 220291, 2888, 3345, 303542, 0 ;

 Hydroelectric = 44165, 4199, 5653, 54923, 43298, 4302, 61526, 1400 ;

 Renewable = 577942, 29201, 12124, 91945, 62553, 195472, 67725, 5000 ;
}

You can recreate the netcdf file WorldEnergy.nc from this representation using $ ncgen WorldEnergy.cdl where WorldEnergy.cdl is just a plain text file containing the above text.

We can generate a json representation with ncks --json from nco and use jq to generate tsv file for gnuplot

$ ncks --json WorldEnergy.nc | jq -r '.variables | del(.. | .type?,  .shape?, .dimensions?, .attributes?) | ((keys_unsorted | map(tostring)), ([.[].data] | transpose[])) | @tsv'
Coal   Continent                 Gas    Hydroelectric Latitude Longitude Nuclear Oil     Renewable index
947639 Asia (excl. Middle East)  272284 44165         55       85        117365  400017  577942    0
4868   C. America \\& Caribbean  40297  4199          17       -90       2607    179261  29201     1
14229  Middle East \\& N. Africa 260613 5653          25       38        0       1240170 12124     2
600275 North America             585206 54923         50       -95       220291  489867  91945     3
29025  South America             72900  43298         -10      -55       2888    345297  62553     4
129456 Sub-Saharan Africa        9203   4302          -5       25        3345    187118  195472    5
391517 Europe                    741413 61526         50       25        303542  648678  67725     6
155000 Oceania                   27000  1400          -24      133       0       25500   5000      7

Now you can plot this data as usual with gnuplot. (Note the order of the columns has changed which can be fixed either in gnuplot or jq; I would recommend using named columns in gnuplot)

Both tools ncks and jq are easily available under macOS with homebrew and you probably want to install netcdf (ncgen, ncdump) as well:

$ brew install netcdf nco jq
Hotschke
  • 9,402
  • 6
  • 46
  • 53