1

I'm trying to use Level2File to plot data from February of 2002 and I'm getting a number of inconsistent errors.

Sometimes I get errors on the call to Level2File depending on which .Z file I'm attempting to plot. Sometimes it's TypeError: unsupported operand type(s) for /: 'float' and 'NoneType'. Sometimes it's OSError: [Errno 22] Invalid argument.

Sometimes it starts by throwing warnings Unable to read volume header. Attempting to read messages.

followed by a ton of Unknown message: and a number, before ending with Remaining buffered message segments for message type(s): 15 (1) and then erroring on the call to Level2File

Occasionally the error is on the the line elv_angle = raddata.sweeps[sweep][0][0].el_angle instead, in which case it is an IndexError: list index out of range.

I just upgraded to metpy 1.1.0 and am running Python 3.9 on Win10.

Note: Right away I noticed the extension for the desired Level II files from Feb 2002 is .Z (whereas I think recent data has an extension of .gz or .ar2v?). Is it possible that these files are just too old to be processed by Level2File?

Below is a simplified version of the code I'm running (though it's basically just copied & pasted from the example code on Github).

Thanks!

import matplotlib as mpl
mpl.use('Agg')
from metpy.io import Level2File
from metpy.plots import colortables

from matplotlib.cm import get_cmap
import numpy as np
import os
import sys
import matplotlib.pylab as plt
import cartopy.crs as ccrs
import cartopy.feature as cfeature
from cartopy.io.shapereader import Reader
import datetime
import matplotlib.patheffects as PathEffects
from time import time
from datetime import datetime, timedelta

def get_lonlat(x, y, nex_lon, nex_lat):
    from pyproj import Proj

    p = Proj(proj='aeqd', ellps='sphere',
     lon_0=nex_lon,
     lat_0=nex_lat)

    return p(x,y,inverse=True)   

def main(argv):

  sweep = 0
  varname = 'REF'

  raddir = "C:\\Users\\rconn\\Documents\\WORK\\Cases\\20020203_Lake_Effect\\"
  filename = "KMQT20020203_200100.Z"
  raddata = Level2File(raddir+filename)
  elv_angle = raddata.sweeps[sweep][0][0].el_angle 

  # Pull data out of the file
  radtime = raddata.dt
  timestr = str(radtime)
  timeint = int(timestr[0:4]+timestr[5:7]+timestr[8:10]+timestr[11:13]+timestr[14:16])

  # First item in ray is header, which has azimuth angle
  az = np.array([ray[0].az_angle for ray in raddata.sweeps[sweep]])

  # 5th item is a dict mapping a var name (byte string) to a tuple
  # Plotting options that change based on desired variable
  if varname=='REF':
    var_hdr = raddata.sweeps[sweep][0][4][b'REF'][0]
    var_range = np.arange(var_hdr.num_gates) * var_hdr.gate_width + var_hdr.first_gate
    var = np.array([ray[4][b'REF'][1] for ray in raddata.sweeps[sweep]])

  # Get center lat, lon for plotting
  lat_0 = raddata.sweeps[0][0][1].lat
  lon_0 = raddata.sweeps[0][0][1].lon
  lat_c = lat_0 # For plot extent
  lon_c = lon_0
  deltalat = 1.8
  latsouth = lat_c - deltalat/2.
  latnorth = lat_c + deltalat/2.
  deltalon = ((5./4.)*deltalat)/np.cos(np.deg2rad(lat_0))
  lonwest = lon_c - deltalon/2.
  loneast = lon_c + deltalon/2.

  # Turn into an array, then mask
  data = np.ma.array(var)
  data[np.isnan(data)] = np.ma.masked

  # Convert az,range to x,y
  xlocs = var_range * np.sin(np.deg2rad(az[:, np.newaxis])) * 1000
  ylocs = var_range * np.cos(np.deg2rad(az[:, np.newaxis])) * 1000
  lon, lat = get_lonlat(xlocs, ylocs, lon_0, lat_0)

  # Plot the data
  fig = plt.figure(5, figsize=(10,8))
  ax = fig.add_subplot(111, projection=ccrs.PlateCarree())
  ax.outline_patch.set_linewidth(1.)
  ax.outline_patch.set_zorder(6)

  # Plot the data
  p = ax.pcolormesh(lon, lat, data, cmap=get_cmap('viridis'), vmin=-20, vmax=80, zorder=3)

  # Set custom extent for map plot
  ax.set_extent([lonwest,loneast,latsouth,latnorth],ccrs.PlateCarree())

if __name__ == '__main__':
    main(sys.argv[1:])

1 Answers1

0

The problem is that .Z files are compressed using the ancient UNIX compress program, and Python does not have any support in the standard library to decompress it. This is why MetPy doesn't natively support it.

Your best bet is to recompress these files using gzip (which the commandline utility can handle .Z):

for f in *.Z; do
    gunzip -c $f | gzip > "${f%.Z}.gz" 
done;
DopplerShift
  • 5,472
  • 1
  • 21
  • 20
  • Thanks for the quick response! When trying to recompress a file, I encounter the error/warning: `gzip: compressed data not read from a terminal. Use -f to force decompression.` It still makes a new .gz file, but then if I run Level2File on that new .gz file I get: File "C:\Users\rconn\AppData\Local\Programs\Python\Python39\lib\site-packages\metpy\io\_tools.py", line 97, in unpack_from return self._create(super().unpack_from(buff, offset)) struct.error: unpack_from requires a buffer of at least 24 bytes for unpacking 24 bytes at offset 0 (actual buffer size is 0) – Ryan Connelly Oct 14 '21 at 17:28
  • The error from MetPy indicates the file you’re trying to read is empty: “buffer size is 0”. Looks like you’re on Windows, so I’m really not sure what’s going on with `gunzip`/`gzip`. You might need to spend some time with `gunzip -h` or `gzip -h` and see if there’s some option needed to make that recompression work, or find some other way to easily redo the file. – DopplerShift Oct 14 '21 at 21:55