7

I want to parse

'This,is,an,example,text'

like in findTokens

'This,is,an,example,text' findTokens: $, 
an OrderedCollection('This' 'is' 'an' 'example' 'text')

but cannot figure out how to do it with PetitParser, delimitedBy: and separatedBy: didn't helped me I tried

( #any asParser delimitedBy: $, asParser ) plus flatten parse:  'This,is,an,example,text'

but obviously not worked

casperOne
  • 73,706
  • 19
  • 184
  • 253
user1000565
  • 927
  • 4
  • 12

4 Answers4

3

You can use delimitedBy: in combination with withoutSeparators:

|text parser|

text := 'This,is,an,example,text'.
parser := (#word asParser plus flatten delimitedBy: ($, asParser)) withoutSeparators.

parser parse: text

Seems to be a recent improvement to PetitParser.

Helene Bilbo
  • 1,142
  • 7
  • 20
2

a #delimitedBy: b expands to a , (b , a) star, so your parser as-is is saying "give me one character separated by commas".

It's not very readable, but this does what you want:

((($, asParser not , #any asParser) ==> [:nodes | nodes second])
  plus flatten delimitedBy: $, asParser

The first clause says "parse anything that isn't a comma". So given '12,24' you get #('12' $, '24').

Frank Shearar
  • 17,012
  • 8
  • 67
  • 94
1

Try

(#word asParser plus flatten separatedBy: $, asParser) 
     ==> [:nodes| nodes copyWithout: $, ]

I hope I understood what you wanted

Norbert Hartl
  • 10,481
  • 5
  • 36
  • 46
  • Yes, the only problem with that one is that if the word contains any non alphabetical character that would break the parsing no? – user1000565 Oct 18 '11 at 16:10
  • 1
    Yes, it does. I was just resembling your example. You need to specify what you really need to parse. If it is everything you might be better off with $, asParser negate that Sean suggested – Norbert Hartl Oct 18 '11 at 18:06
1

I use this pattern all the time with PetitParser when I want to exclude something. Just define either "what I'm looking for" or "what I want to exclude" (whichever's easier to describe) as a parser, and then negate it, and process as necessary. 

s := 'This,is,an,example,text'.
separator := $, asParser ==> [ :n | nil ].
token := separator negate plus flatten.
p := (token separatedBy: separator) ==> [ :nodes |
    nodes copyWithout: nil ].
p parse: s.
Sean DeNigris
  • 6,306
  • 1
  • 31
  • 37
  • 1
    This `copyWithout:` really feels ugly... I suspect there is a way to avoid it by defining a variant of `separatedBy:` that would not add the separators to the output in the first place… Another solution could be with `foldLeft:` but as I see it it requires your own class for the collection of tokens. – Damien Pollet Jan 31 '12 at 19:41