Conditional skip of split of new lines

Question

My data looks like

04/07/16, 12:51 AM - User1: Hi
04/07/16, 8:19 PM - User2: Here’s a link for you
https://www.abcd.com/folder/1SyuIUCa10tM37lT0F8Y3D
04/07/16, 8:29 PM - User2: Thanks

Using the below code, I am able to split each message into each new line

data = []
for line in open('/content/drive/My Drive/sample.txt'):
    items = line.rstrip('\r\n').split('\t')   # strip new-line characters and split on column delimiter
    items = [item.strip() for item in items]  # strip extra whitespace off data items
    data.append(items)

However, I do not want to split the line where a newline character is followed by a link. For example, Line 3 & 4 are one single message but they split up because of newline character.

.

Is there a way to avoid splitting when a newline character is followed by http?

score 0 · Answer 1 · answered May 21 '19 at 14:32

It can probably be optimised, but it works:

data = []                                                                
prev = ''                                                                

with open('C:/Users/kavanaghal/python/sample.txt', 'r', encoding='utf-8') as f:            
    prev = f.readline().strip()                                          

    while True:                                                          
        nxt = f.readline().strip()                                       

        if 'http' in nxt:                                                
            data.append(prev + ": " + nxt)                               
            prev = f.readline()                                          
            continue                                                     

        data.append(prev)                                                
        prev = nxt                                                       

        if not nxt:                                                      
            break                                                        

print(data)                                                              


>> ['04/07/16, 12:51 AM - User1: Hi', 
    '04/07/16, 8:19 PM - User2: Here's a link for you: https://www.abcd.com/folder/1SyuIUCa10tM37lT0F8Y3D', 
    '04/07/16, 8:29 PM - User2: Thanks']

score 0 · Answer 2 · answered May 21 '19 at 14:33

One way to do it is to just append it to the last entry in the list:

import re
data = []
with open('sample.txt', 'r') as f: # use open so the file closes automatically
    for line in f.readlines():
        if len(data) >= 1 and re.match(r'https?://(?:[-\w.]|(?:%[\da-fA-F]{2}))+', line):
            data[len(data) - 1] += f" {line.strip()}"
        else:
            data.append(line.strip())
for x in data:
    print(x)

Ouput:

04/07/16, 12:51 AM - User1: Hi
04/07/16, 8:19 PM - User2: Here’s a link for you https://www.abcd.com/folder/1SyuIUCa10tM37lT0F8Y3D
04/07/16, 8:29 PM - User2: Thanks

Credit to: Regex to extract URLs... for the regex

score 0 · Answer 3 · answered May 21 '19 at 14:39

You'd need to read the entire file at once:

all_lines = []
for index, line in enumerate(split):
    next_index = index + 1
    if next_index < len(split) and "https" in split[next_index]:
        line += split[next_index]
        del split[next_index]
    all_lines.append(line)

score 0 · Answer 4 · answered May 21 '19 at 14:54

Operate ex-post

data = []
for line in open('/content/drive/My Drive/sample.txt'):
    items = [item.strip() for item in line.rstrip('\r\n').split('\t')]
### now it is different from your code ###############################
    if items[0].startswith('http'):
        data[-1].append(items[0])
    else:
        data.append(items)

You possibly want a better control on what is matched, using a regular expression or what else in place of .startswith(), but this should get you started.

Conditional skip of split of new lines

4 Answers4