tokenizing and parsing with python

Question

I don't have a code to show to you because I have no idea on how to start. The current target is to at least be able to create tokens from a file that contains some data eg:

file.txt

Name : Sid
data : Lazy Developer

%description 
This is a packaging file 

%install
 Enter the location to install the package.

and the python code should be able to create the tokens from this file and then when required print the data based on the input.

if getData() is the function then

getData('name') should output "Sid" GetData('description') should give the text below it.

I'm voting to close this question as off-topic because this isn't a tutorial service — jonrsharpe, Jul 27 '15 at 13:23
Please describe your question more specifically. Adding code examples, as well as input and expected output will help us help you. — Aaron, Jul 27 '15 at 13:23
tried adding code and file that i wanted to parse last time and no one helped either and this may not be a tutorial service but you surely can help someone by telling him where to start. — DevX136, Jul 27 '15 at 13:25
*"you surely can help someone by telling him where to start"* - fair enough, start by reading [ask]. — jonrsharpe, Jul 27 '15 at 13:26
[Here's](http://effbot.org/zone/simple-top-down-parsing.htm) a good resource for generally all things python — Aaron, Jul 27 '15 at 13:26
I tried that ... it's not detailed enough and this site isn't helping either — DevX136, Jul 27 '15 at 13:28
what are you trying to parse? code? data? You're not giving me much to go on... — Aaron, Jul 27 '15 at 13:30
It's data at the moment but i wanna parse everything so yeah. — DevX136, Jul 27 '15 at 13:52
take rpm specs as an example for the parsing. Where the split can be done using data.split(':') and the end delimiter can be used as '\n' — DevX136, Jul 27 '15 at 13:53
To start, you open a file, then you iterate through it line by line, each line you split as `line.split(':')` and populate a dictionary accordingly. At which step did you stuck? — bereal, Jul 27 '15 at 14:01
that's done with but how do i get the function to retrieve the data? Using tuples? and suppose the text file also has another special keyword like %description\n"Some text" then how do i separate them? My guess would be using "\n" but the if there is another such keyword on the next line after this bunch of text then that will be taken as a single token as well so... Another problem. — DevX136, Jul 27 '15 at 14:05

clemtoy · Accepted Answer · 2015-07-28T06:21:58.627

To retrieve the data from the file.txt:

data = {}
with open('file.txt', 'r') as f: # opens the file
    for line in f: # reads line by line
        key, value = line.split(' : ') # retrieves the key and the value
        data[key.lower()] = value.rstrip() # key to lower case and removes end-of-line '\n'

Then, data['name'] returns 'Sid'.

EDIT: As the question has been updated this is the new solution:

data = {}
with open('file.txt', 'r') as f:
    header, *descriptions = f.read().split('\n\n')
    for line in header.split('\n'):
        key, value = line.split(' : ')
        data[key.lower()] = value.rstrip()
    for description in descriptions:
        key, value = description.split('\n', 1)
        data[key[1:]] = value
print(data)

You might have to adapt this if there are some whitespaces between lines or at the end of the keys...

A shorter way to do this might be to use a regex and the method re.group().

This works but i updated the question would be helpful if you'd see that too. Thanks — DevX136, Jul 28 '15 at 03:50

score -1 · Answer 2 · edited May 23 '17 at 12:24

-1

As the commentators stated, your question doesn't really fit in with the site. However I'm going to try and point you in the right direction.

Your file.txt is effectively a yaml document. See this answer

import yaml
with open('file.txt', 'r') as f:
    doc = yaml.load(f)
print(doc["Name"])

I'd also strongly recommend reading this section of Dive Into Python (as well as reading the entire book). In the future try some amount of code and share that with your question.

edited May 23 '17 at 12:24

Community

1
1

answered Jul 27 '15 at 14:29

Kerry Hatcher

601
6
17

not really yaml its a sample of the spec files that rpm uses to create packages – DevX136 Jul 27 '15 at 15:40
Then please give some more information in your question. The example text matches yaml format per the spec. If there is other text in the file then we would need to know that to make an informed answer. – Kerry Hatcher Jul 27 '15 at 17:02

tokenizing and parsing with python

file.txt

2 Answers2