How can I find the diagonals of all matrices in a file using NumPy?

Question

I have a file which consists of a number of matrices separated by blank line. How can I find the all diagonals of all matrices using NumPy? The matrices are not of fixed size

1 2 3
1 1 1 
2 2 2

3 4 
3 2 
4 2 


3 2 3
4 2 2

Answer 1

Here is a recipe for iterating over a file in chunks delimited by a regex pattern. This is perhaps a bit more powerful than is needed here; the regex pattern is merely '\n{2,}', i.e. two-or-more end-of-line characters. On the other hand, once you have this tool, there is no need to reinvent the wheel.

Once you have a chunk -- a multi-line string representing an array -- you can parse it with np.loadtxt and find all diagonals using the diagonals function:

import io
import re
import numpy as np

# http://stackoverflow.com/q/17508697/190597
def open_chunk(readfunc, delimiter, chunksize=1024):
    """
    readfunc(chunksize) should return a string.
    """
    remainder = ''
    for chunk in iter(lambda: readfunc(chunksize), ''):
        pieces = re.split(delimiter, remainder + chunk)
        for piece in pieces[:-1]:
            yield piece
        remainder = pieces[-1]
    if remainder:
        yield remainder

# Based on http://stackoverflow.com/a/6313407/190597
def diagonals(L):
    h, w = len(L), len(L[0])
    return [[L[h - p + q - 1][q]
             for q in range(max(p-h+1,0), min(p+1, w))]
            for p in range(h + w - 1) ]

with open('data', 'r') as f:
    for chunk in open_chunk(f.read, r'\n{2,}'):
        arr = np.loadtxt(io.BytesIO(chunk))
        print([d for d in diagonals(arr) if len(d) != 1])

yields

[[1.0, 2.0], [1.0, 1.0, 2.0], [2.0, 1.0]]
[[3.0, 2.0], [3.0, 2.0]]
[[3.0, 2.0], [2.0, 2.0]]