8

I have the following pandas data frame:

import pandas as pd
df = pd.DataFrame({ 'gene':["1 // foo // blabla",
                                   "2 // bar // lalala",
                                   "3 // qux // trilil",
                                   "4 // woz // hohoho"], 'cell1':[5,9,1,7], 'cell2':[12,90,13,87]})
df = source_df[["gene","cell1","cell2"]]

It looks like this:

                 gene  cell1  cell2
0  1 // foo // blabla      5     12
1  2 // bar // lalala      9     90
2  3 // qux // trilil      1     13
3  4 // woz // hohoho      7     87

What I want to get is this:

   gene    cell1  cell2
0   foo       5     12
1   bar       9     90
2   qux       1     13
3   woz       7     87

Namely select 2nd element of the splited string by // as delimiter.

The best I can do is this:

df["gene"] = df["gene"].str.split(" // ")
df
Out[17]:
               gene  cell1  cell2
0  [1, foo, blabla]      5     12
1  [2, bar, lalala]      9     90
2  [3, qux, trilil]      1     13
3  [4, woz, hohoho]      7     87

What's the right way to do it?

neversaint
  • 60,904
  • 137
  • 310
  • 477
  • See: [pandas: How do I split text in a column into multiple rows?](http://stackoverflow.com/questions/17116814/pandas-how-do-i-split-text-in-a-column-into-multiple-rows). – agold Nov 09 '15 at 07:46

3 Answers3

21

Use the vectorised str.split this will be much faster than using apply on a large dataset:

In [13]:
df['gene'] = df['gene'].str.split('//').str[1]
df

Out[13]:
   cell1  cell2   gene
0      5     12   foo 
1      9     90   bar 
2      1     13   qux 
3      7     87   woz
EdChum
  • 376,765
  • 198
  • 813
  • 562
3

You can use regex and strip first and last spaces by strip:

df["gene"] = df["gene"].str.extract(r"\/\/([a-z ]+)\/\/")
df["gene"] = df["gene"].str.strip()

print df
  gene  cell1  cell2
0  foo      5     12
1  bar      9     90
2  qux      1     13
3  woz      7     87

\/\/([a-z ]+)\/\/ means:

    \/ matches the character / literally
    \/ matches the character / literally
    1st Capturing group ([a-z ]+)
        [a-z ]+ match a single character present in the list below
            Quantifier: + Between one and unlimited times, as many times as possible,
            giving back as needed [greedy]
            a-z a single character in the range between a and z (case sensitive)
             the literal character  
    \/ matches the character / literally
    \/ matches the character / literally

Or regex without strip:

df["gene"] = df["gene"].str.extract(r"\/\/\s*([a-z ]+)\s\/\/")

/\/\/\s*([a-z ]+)\s\/\// means:

    \/ matches the character / literally
    \/ matches the character / literally
    \s* match any white space character [\r\n\t\f ]
        Quantifier: * Between zero and unlimited times, as many times as possible, 
        giving back as needed [greedy]
    1st Capturing group ([a-z ]+)
        [a-z ]+ match a single character present in the list below
            Quantifier: + Between one and unlimited times, as many times as possible, 
            giving back as needed [greedy]
            a-z a single character in the range between a and z (case sensitive)
            the literal character  
    \s match any white space character [\r\n\t\f ]
    \/ matches the character / literally
    \/ matches the character / literally
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
1

You were pretty close, however selecting the element from the resulting split is a bit more difficult doing it your way.

Here's a solution with apply

>>> df['gene'] = df['gene'].apply(lambda s: s.split('//')[1])
>>> df

    gene  cell1  cell2
0   foo       5     12
1   bar       9     90
2   qux       1     13
3   woz       7     87
sohail288
  • 433
  • 4
  • 8
  • when I compared this to the top solution, this one turned out to be faster, even though in theory vectorized operations should be faster than apply. Interesting. – Alena Volkova Mar 31 '23 at 13:47