Split a list on a key word which may appear multiple times

Question

I've read the examples which seem similar but I am not at that level to understand the answers. I want to take the list output and write each interface as a separate line (aka list I write to a csv). I need to split the initial return list on the keyword 'interface Vlan*'

I want to split returned list vlanlist on keyword interface vlan* into separate lists

from ciscoconfparse import CiscoConfParse
import os

for filename in os.listdir():
    if filename.endswith(".cfg"):
        p = CiscoConfParse(filename)
        vlanlist=(p.find_all_children('^interface Vlan'))
        vlanlist.insert(0,filename)

        print(vlanlist)

This is one line of output. I need to split the list on keyword "interface vlanxxx" into separate lines

[ 'interface Vlan1', ' no ip address', ' shutdown', 'interface Vlan2003', ' description XXXXXX', ' ip address 10.224.6.130 255.255.255.224', ' no ip redirects', ' no ip unreachables', ' no ip proxy-arp', ' load-interval 60', ' arp timeout 420']

Desired OUTPUT (this may have 2-20 diferent interfaces I want to split on depending on config file)

['interface Vlan1' ' no ip address', ' shutdown']
['interface Vlan2003', ' description XXXXXX', ' ip address 10.224.6.130 255.255.255.224', ' no ip redirects', ' no ip unreachables', ' no ip proxy-arp', ' load-interval 60', ' arp timeout 420']

Could you simplify your question by separating the problematic part from all of this file stuff? Is it really relevant? When I read the question title I thought: oh, that's easy! ... and then I got distracted. — timgeb, Oct 23 '18 at 16:04

Austin Salonen · Answer 1 · 2018-10-23T16:32:59.160

Here's a solution that's highly coupled to your single test case. You'll have to evolve it with more tests if the full dataset isn't representative of your single test case.

def extract(items):
  result, filename, idx = [], items[0], -1

  for x in items[1:]:
    if x.startswith('interface Vlan'):
      idx += 1
      result.append([filename])
    result[idx].append(x)

  return result

# given & expected are your example and output 
assert expected == extract(given)

EDIT:
... and you've already changed the inputs and outputs.

def extract(items):
  result, idx = [], -1

  for x in items:
    if x.startswith('interface Vlan'):
      idx += 1
      result.append([])

    if not result: continue  # assuming possible unwanted items before 'interface Vlan'
    result[idx].append(x)

  return result

assert expected == extract(given)

r.ook · Accepted Answer · 2018-10-23T17:10:39.950

You can further separate your returned vlanlist before you append the file names:

# First, find the index in the list where "interface Vlan" exists:
# Also, append None at the end to signify index for end of list
indices = [i for i, v in enumerate(l) if v.startswith('interface Vlan')] + [None]

# [0, 3, None]

# Then, create the list of lists based on the extracted indices and prepend with filename
newlist = [[filename] + vlanlist[indices[i]:indices[i+1]] for i in range(len(indices)-1)]

for l in newlist: print(l)

# ['test.cfg', 'interface Vlan1', ' no ip address', ' shutdown']
# ['test.cfg', 'interface Vlan2003', ' description XXXXXX', ' ip address 10.224.6.130 255.255.255.224', ' no ip redirects', ' no ip unreachables', ' no ip proxy-arp', ' load-interval 60', ' arp timeout 420']

Explanation for the second list comprehension:

newlist = [
    [filename] +                   # prepend single-item list of filename
    vlanlist[                      # slice vlanlist
        indices[i]:                # starting at the current index
        indices[i+1]               # up to the next index
    ] 
    for i in range(len(indices)-1) # iterate up to the second last index so i+1 doesn't become IndexError
]

If you don't like the index approach, you can try zip instead:

lists = [[filename] + vlanlist[start:end] for start, end in zip(indices[:-1], indices[1:])]

Your did it! And you showed me my path. I just didn't know what I didn't know. You threw me with the l for my vlanlist but I finally grasped the logic and it's so elegant. I'll try the zip as well just for skill building. Thank you. — Seth, Oct 23 '18 at 17:26
You're welcome, if you're still learning make sure you check out the other answers as well as each of them show you a different approach that might be helpful in other use cases. For `zip` it's basically creating `tuple`s of the `list`s provided, i.e. `zip(x, y)` will return a zip object of `(x[0], y[0]), (x[1], y[1])...`. But in this case it just serves to make the code a bit more readable IMO. — r.ook, Oct 23 '18 at 17:46
Yep! trying them all. I am a beginner. I've come pretty far but it's just not intuitive to me. I do Cisco mostly but just have no end of uses for Python to help me solve problems with large networks. — Seth, Oct 23 '18 at 18:21

score 1 · Answer 3 · answered Oct 23 '18 at 16:48

A quick and straightforward solution. Check the list for interface Vlan items, if it is, it creates a new list, else appends on the old list and some .strip() for good measure.

output = ['interface Vlan1', ' no ip address', ' shutdown', 'interface Vlan2003', ' description XXXXXX', ' ip address 10.224.6.130 255.255.255.224', ' no ip redirects', ' no ip unreachables', ' no ip proxy-arp', ' load-interval 60', ' arp timeout 420']

results = []

for i in output:
    if 'interface Vlan' in i:
        results.append([i.strip()])
    else:
        results[-1].append(i.strip())

>> results
 [['interface Vlan1', 'no ip address', 'shutdown'],
 ['interface Vlan2003',
  'description XXXXXX',
  'ip address 10.224.6.130 255.255.255.224',
  'no ip redirects',
  'no ip unreachables',
  'no ip proxy-arp',
  'load-interval 60',
  'arp timeout 420']]

score 0 · Answer 4 · answered Oct 23 '18 at 16:30

This identifies a single split point, and divides your list into two lists as specified. The split_pos list will find all split positions; you could iterate through this if there are multiple split points. The split condition looks for a string starting with the given text and at least three more characters, the "xxx" in your posting.

vlanlist = ['sw01.cfg', 'interface Vlan1', ' no ip address', ' shutdown', 'interface Vlan2003', ' description XXXXXX', ' ip address 10.224.6.130 255.255.255.224', ' no ip redirects', ' no ip unreachables', ' no ip proxy-arp', ' load-interval 60', ' arp timeout 420']
target = "interface Vlan"

split_pos = [idx for idx, str in enumerate(vlanlist) if str.startswith(target) and \
                                                        len(str) >= len(target)+3][0]

out1 = [vlanlist[0]] + vlanlist[1:split_pos]
out2 = [vlanlist[0]] + vlanlist[split_pos:]

print(out1)
print(out2)

Output:

['sw01.cfg', 'interface Vlan1', ' no ip address', ' shutdown']
['sw01.cfg', 'interface Vlan2003', ' description XXXXXX', 
 ' ip address 10.224.6.130 255.255.255.224', ' no ip redirects',
 ' no ip unreachables', ' no ip proxy-arp', ' load-interval 60', ' arp timeout 420']

Would it be possible to have more than 2 `interface Vlan` in the list though? — r.ook, Oct 23 '18 at 16:37
Yes; in that case, do *not* simply grab the first (only) element of `split_pos`. Iterate through the list of split positions, grabbing the slices you need. — Prune, Oct 23 '18 at 16:40
Right, hence my solution which is very similar to yours in nature, just that it can handle *n* of splitpos. I note that your starting point is *after* the filename has already been prepended. In the end I think we're both going about it pretty much the same :) — r.ook, Oct 23 '18 at 16:44

Split a list on a key word which may appear multiple times

4 Answers4