Python String parse

Question

Im working on a data packet retrieval system which will take a packet, and process the various parts of the packet, based on a system of tags [similar to HTML tags]. [text based files only, no binary files].

Each part of the packet is contained between two identical tags, and here is a sample packet:

"<PACKET><HEAD><ID><ID><SEQ><SEQ><FILENAME><FILENAME><HEAD><DATA><DATA><PACKET>"

The entire packet is contained within the <PACKET><PACKET> tags. All meta-data is contained within the <HEAD><HEAD> tags and the filename from which the packet is part of is contained within the, you guessed it, the <FILENAME><FILENAME> tags.

Lets say, for example, a single packet is received and stored in a temporary string variable called sTemp. How do you efficiently retrieve, for example, only the contents of a single pair of tags, for example the contents of the <FILENAME><FILENAME> tags?

I was hoping for such functionality as saying getTagFILENAME( packetX ), which would return the textual string contents of the <FILENAME><FILENAME> tags of the packet. Is this possible using Python?

Any suggestions or comments appreciated.

score 1 · Answer 1 · answered Dec 08 '09 at 21:49

1

Something like this?

import re
def getPacketContent ( code, packetName ):
    match = re.search( '<' + packetName + '>(.*?)<' + packetName + '>', code )
    return match.group( 1 ) if match else ''

# usage
code = "<PACKET><HEAD><ID><ID><SEQ><SEQ><FILENAME><FILENAME><HEAD><DATA><DATA><PACKET>"
print( getPacketContent( code, 'HEAD' ) )
print( getPacketContent( code, 'SEQ' ) )

answered Dec 08 '09 at 21:49

poke

369,085
72
557
602

Also a nice solution. How do you guarantee that a search for FILENAME will only look within the confinds of the HEAD tag, just in case the filename were also to appear outside of the HEAD tag, like in the DATA tag for example? – Donal.Lynch.Msc Dec 08 '09 at 22:32
You can't. See http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags. – Robert Rossney Dec 09 '09 at 01:04
You could do something like `getPacketContent( getPacketContent( code, 'HEAD' ), 'FILENAME' )` though, to only get the FILENAME tag within the HEAD tag. – poke Dec 09 '09 at 19:38

mjv · Accepted Answer · 2009-12-09T00:11:21.330

If the packet format effectively uses XML-looking syntax (i.e., if the "closing tags" actually include a slash), the xml.etree.ElementTree could be used.
This libray is part of Python Standard Library, starting in Py2.5. I find it a very convenient one to deal with this kind of data. It provides many ways to read and to modify this kind of tree structure. Thanks to the generic nature of XML languages and to the XML awareness built-in the ElementTree library, the packet syntax could evolve easily for example to support repeating elements, element attributes.

Example:

>>> import xml.etree.ElementTree
>>> myPacket = '<PACKET><HEAD><ID>123</ID><SEQ>1</SEQ><FILENAME>Test99.txt</FILE
NAME></HEAD><DATA>spam and cheese</DATA></PACKET>'
>>> xt = xml.etree.ElementTree.fromstring(myPacket)
>>> wrk_ele = xt.find('HEAD/FILENAME')
>>> wrk_ele.text
'Test99.txt'
>>>

Nice work man - thats perfect! to get at any tag you just say: wrk_ele = xt.find('HEAD/---AnyTagName---') and then print wrk_ele.text. — Donal.Lynch.Msc, Dec 08 '09 at 22:26
Don't congratulate _me_, just the Python community at large (and its leaders in particular, they'll recognize themselves), who've made Pyton `batteries included` — mjv, Dec 08 '09 at 22:42

score 1 · Answer 3 · answered Dec 09 '09 at 01:14

As mjv points out, there's not the least sense in inventing an XML-like format if you can just use XML.

But: If you're going to use XML for your packet format, you need to really use XML for it. You should use an XML library to create your packets, not just to parse them. Otherwise you will come to grief the first time one of your field values contains an XML markup character.

You can, of course, write your own code to do the necessary escaping, filter out illegal characters, guarantee well-formedness, etc. For a format this simple, that may be all you need to do. But going down that path is a way to learn things about XML that you perhaps would rather not have to learn.

If using an XML library to create your packets is a problem, you're probably better off defining a custom format (and I'd define one that didn't look anything like XML, to keep people from getting ideas) and building a parser for it using pyparsing.

Python String parse

3 Answers3