10

How, in Python, can I use shlex.split() or similar to split strings, preserving only double quotes? For example, if the input is "hello, world" is what 'i say' then the output would be ["hello, world", "is", "what", "'i", "say'"].

tekknolagi
  • 10,663
  • 24
  • 75
  • 119

2 Answers2

16
import shlex

def newSplit(value):
    lex = shlex.shlex(value)
    lex.quotes = '"'
    lex.whitespace_split = True
    lex.commenters = ''
    return list(lex)

print newSplit('''This string has "some double quotes" and 'some single quotes'.''')
Peter Lyons
  • 142,938
  • 30
  • 279
  • 274
  • 1
    Only negligibly so, and the `lex.commenters` bit is actually something that my answer doesn't do. +1 for a different way to git-r-dun. – Matt Ball Jul 29 '11 at 04:05
  • I started with the source code to the shlex.split function from the python source and just tweaked it with the list of quotes characters. – Peter Lyons Jul 29 '11 at 17:36
  • It took me a while, but this is slightly different than the normal shlex.split which passes posix ... [`shlex.shlex(value, posix=True)`](http://hg.python.org/cpython/file/2a38df26e009/Lib/shlex.py#l274) – Pykler Sep 04 '13 at 16:49
8

You can use shlex.quotes to control which characters will be considered string quotes. You'll need to modify shlex.wordchars as well, to keep the ' with the i and the say.

import shlex

input = '"hello, world" is what \'i say\''
lexer = shlex.shlex(input)
lexer.quotes = '"'
lexer.wordchars += '\''

output = list(lexer)
# ['"hello, world"', 'is', 'what', "'i", "say'"]
Matt Ball
  • 354,903
  • 100
  • 647
  • 710