Python and GnuCash: Extract data from GnuCash files

Question

I'm looking for information on how to read GnuCash files using python. I have read about this python-gnucash which provides Python bindings to the GnuCash library, but it takes a lot of work at the moment (e.g. dependencies, headers, etc.). The instructions are tailored for the Linux environment, and a rather old GnuCash version (2.0.x). I am running GnuCash 2.2.9. Though I can operate the Linux command line, I am running GnuCash on Windows XP.

My main objective is to read (no plans to write yet) my GnuCash files so that I can create my own visual dynamic reports using matplotlib and wxpython. I'm not yet in the mood to learn Scheme.

I hope someone can point me to a good start on this. As far as I know about GnuCash and Python, I think someone probably knows solutions of the following types:

1. More recently updated documentation aside from this one from the GnuCash wiki
2. Some workaround, like exporting to a certain file format for which there is a more mature Python library that can read it.

You guys might have better suggestions in addition to those mentioned.

Answer 1

I published piecash, a python interface to SQL saved GnuCash books that uses SQLAlchemy as basis (https://github.com/sdementen/piecash).

With it you can easily access all the information contained in a book.

For instance, to iterate over all accounts in the book:

from piecash import open_book

# open a book
with open_book("some_book.gnucash", open_if_lock=True) as mybook:
    # iterate over all accounts of the book
    for account in mybook.accounts:
        print(account)

or to iterate over all the splits in the "Asset" account:

# open the book
with open_book("some_book.gnucash", open_if_lock=True) as mybook:
    # retrieve the account by its fullname
    asset = mybook.accounts(fullname="Asset")
    # iterate over all its splits
    for split in asset.splits:
        print(split)

Recent versions also allows to extract the split information directly to pandas DataFrames for easy plotting/analysis with

from piecash import open_book

# open a book
with open_book("some_book.gnucash", open_if_lock=True) as mybook:
    # extract all split information to a pandas DataFrame
    df = mybook.splits_df()

    # print for account "Asset" some information on the splits
    print(df.loc[df["account.fullname"] == "Asset",
                 ["transaction.post_date", "value"]])

Answer 2

GNUCash 2.4 is out.

Can export to SQL so it's very much easier than parsing XML.

Sqlite, MySQL and PostgreSQL are supported (how cool is that!)

Answer 3

Are you talking about the data files? From there wiki, it looks like they are just compressed XML files. WIth Python, you can decompress them with the gzip module and then parse them with any of the available XML parsers.

ElementTree Example

>>> import xml.etree.cElementTree as ET
>>> xmlStr = '''<?xml version="1.0" encoding="UTF-8" ?>
<painting>
<img src="madonna.jpg" alt='Foligno Madonna, by Raphael'/>
<caption>This is Raphael's "Foligno" Madonna, painted in
     <date>1511</date>?<date>1512</date>.
</caption>
</painting>
'''
>>> tree = ET.fromstring(xmlStr)  #use parse or iterparse to read direct from file path
>>> tree.getchildren()
[<Element 'img' at 0x115efc0>, <Element 'caption' at 0x1173090>]
>>> tree.getchildren()[1].text
'This is Raphael\'s "Foligno" Madonna, painted in\n    '
>>> tree.getchildren()[0].get('src')
'madonna.jpg'

Answer 4

As Chop Suey said, GnuCash 2.4 has its own database format. If you still want to use the XML files, you can use the following script to convert from XML to a database, and then write your reports on that (e.g. gnucashconvert filename.gnucash sqlite3:////home/username/export.sqlite):

#!/usr/bin/env python

import os
import gnucash

def convert_gnucash(src_uri, target_uri):
    """Converts gnucash databases at the given uris from src to target"""
    session = gnucash.Session(src_uri)
    try:
        new_session = gnucash.Session(target_uri, is_new=True)
        try:
            new_session.swap_data(session)
            new_session.save()
        finally:
            new_session.end()
            new_session.destroy()
    finally:
        session.end()
        session.destroy()

if __name__ == "__main__":
    import sys
    if len(sys.argv) > 2:
        src_uri, target_uri = sys.argv[1], sys.argv[2]
        src_uri = ("xml://%s" % os.path.abspath(src_uri) if "://" not in src_uri else src_uri)
        target_uri = ("xml://%s" % os.path.abspath(target_uri) if "://" not in target_uri else target_uri)
        convert_gnucash(src_uri, target_uri)
    else:
        print >>sys.stderr, "Syntax %s src target" % (sys.argv[0])

Answer 5

I just published some python code that can read and interpret the sqlite3 file format used in gnucash 2.6 and higher:

https://github.com/MatzeB/pygnucash

Answer 6

I've taken the sqlite approach in a django app I've written to do a similar thing (although for budgeting). See https://github.com/evandavey/OpenBudget/blob/master/openbudgetapp/management/commands/gnucash-import.py for the code.

In terms of the data itself, I've used the pandas library to handle its time series nature.