39

So I used shlex.split() recently to split a command as argument to subprocess.Popen() function. I recalled that long back I also used re.split() function to split a string with a specific delimiter specified. Can someone point out what is the essential difference in between them? In which scenario is each function best suited?

Yves M.
  • 29,855
  • 23
  • 108
  • 144
swordfish
  • 393
  • 1
  • 3
  • 5
  • For splitting a string? Use neither, instead use the builtin `string.split('delimiter')` – Tim Jan 08 '16 at 14:42
  • @TimCastelijns It depends very much on what you want to split. `string.split` is the simplest but also least capable option. – tdelaney Jan 08 '16 at 14:44

1 Answers1

71

shlex.split() is designed to work like the shell's split mechanism.

This means doing things like respecting quotes, etc.

>>> shlex.split("this is 'my string' that --has=arguments -or=something")
['this', 'is', 'my string', 'that', '--has=arguments', '-or=something']

re.split() will just split on whatever pattern you define. 

>>> re.split('\s', "this is 'my string' that --has=arguments -or=something")
['this', 'is', "'my", "string'", 'that', '--has=arguments', '-or=something']

Trying to define your own regex to work like shlex.split is needlessly complicated, if it's even possible.

To really see the differences between the two, you can always Use the Source, Luke:

>>> re.__file__
'/usr/lib/python3.5/re.py'
>>> shlex.__file__
'/usr/lib/python3.5/shlex.py'

Open these files in your favorite editor and start poking around, you'll find that they operate quite differently.

Wayne Werner
  • 49,299
  • 29
  • 200
  • 290