Splitting by consecutive delimiter in text file

Question

I have a text file:

John|Hopkins|||31
Sage|Jen|42

And I want to read it into python and split by ‘|’

So I want something like:

[['John', 'Hopkins', '31'], ['Sage', 'Jen', '42']]

file = open('mytxt.txt', 'r')
file_2 = file.readlines()

lst=[]
for line in file_2:
    line=line.strip('\n')
    line=line.split('|')
    lst.append(line)
print(lst)

I’m getting:

[['John', 'Hopkins', '', '', '31'], ['Sage', 'Gen', '42']]

As seen, there are '' present in the first list due to consecutive ||.

How do I modify the split statement to cater for single | and consecutive |||?

You could just filter out the empty elements if your list. See [Remove empty strings from a list of strings](https://stackoverflow.com/questions/3845423/remove-empty-strings-from-a-list-of-strings). — MikaelF, Mar 06 '20 at 05:20
lst.append([x.strip('|') for x in line.strip().split('|') if x]) — Mohanlal Prajapati, Mar 06 '20 at 05:31

OneCricketeer · Answer 1 · 2020-03-06T07:20:57.110

3

Use regex to capture one or more

import re

with open(‘mytxt.txt’) as f:
    for line in f:
        print(re.split(r'\|+', line.rstrip()))

edited Mar 06 '20 at 07:20

answered Mar 06 '20 at 05:17

OneCricketeer

179,855
19
132
245

2

I think you should escape `|` in regex pattern, since it is special character in regex syntax. – Boseong Choi Mar 06 '20 at 05:49

score 2 · Answer 2 · answered Mar 06 '20 at 05:19

2

Add this below your line=line.split(‘|’) line

line = [word for word in line if word!='']

answered Mar 06 '20 at 05:19

Himanshu

666
1
8
18

You can just do `if word` – OneCricketeer Mar 06 '20 at 07:22

score 1 · Answer 3 · answered Mar 06 '20 at 05:18

1

You may try filtering out empty values:

line = filter(None, s.split("|"))

answered Mar 06 '20 at 05:18

Selcuk

57,004
12
102
110

score 0 · Answer 4 · answered Mar 06 '20 at 05:51

Reading the file as a dataframe in and fetch the rows as a list would also do the work.

df = pandas.read_csv("mytxt.txt",sep = "|")
rowlist = []
for index,rows in df.iterrows():
    one_row = [rows[column] for column in df.columns]            #get the data as list
    one_row = list(filter(lambda x: str(x) != 'nan', one_row))   #remove null values
    rowlist.append(one_row)                                      #append to main list

Output = [['John', 'Hopkins', '31'], ['Sage', 'Jen', '42']]

score 0 · Answer 5 · answered Mar 06 '20 at 05:52

Here is full codes:

filtering with list comprehension

with open('mytext.txt') as f:
    lst = [
        [word for word in line.rstrip().split('|') if word]
        for line in f
    ]
print(lst)

split with regex

import re

with open('mytext.txt') as f:
    lst = [
        re.split(r'\|+', line.rstrip())
        for line in f
    ]
print(lst)

outputs are same:

[['John', 'Hopkins', '31'], ['Sage', 'Jen', '42']]

Splitting by consecutive delimiter in text file

5 Answers5

Here is full codes: