0

I would like to perform text analysis like world cloud and ngram on one of the text columns. I have broken down the sentence into tokens and want to join back it to the original table. For example here are my two rows:

      Code              Text 
      ST-441          Purpose of your visit mentioned
      St-432           Describe how and where it happened

after doing text cleaning on the text column I applied the following function

After applying the split function the sentence has broken down into words, one row has become n rows and wants to add it back to the original table using a unique identifier column.

def cleans(data):
    tokens = list(map(lambda data: data.split(' '), text))

Now I got the list of tokens like 'purpose', 'your', 'visit', 'mentioned', 'described' ...

I am looking for the below output

 Code              Text 
 ST-441          Purpose
 ST-441          your
 ST-441          Visit
 ST-441          mentioned
 ST-432          Describe
 ST-432          how

Any help would be much appreciated.

Dr. Prof. Patrick
  • 1,280
  • 2
  • 15
  • 27
Dhanya_mj
  • 3
  • 4
  • You need to provide more info. How have you used the apply method? I do not see it used in your code. Having said that, you do not use the apply method to append rows to a dataframe. You should at least do a basic search on that first. A starting point is :https://stackoverflow.com/questions/49916371/how-to-append-new-row-to-dataframe-in-pandas – Galo do Leste Jan 23 '23 at 11:54
  • Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. – Community Jan 23 '23 at 19:46
  • @Galo do Leste I have edited the question. Hope it helps in understanding my issue better. – Dhanya_mj Jan 24 '23 at 13:50
  • Where did `St-432` go? And how/why is `of` exlcuded? – MatBailie Jan 24 '23 at 13:51
  • Does this answer your question? [Split sentence into words pandas and keep tags](https://stackoverflow.com/questions/63313590/split-sentence-into-words-pandas-and-keep-tags) – MatBailie Jan 24 '23 at 13:52
  • I have added ST_432, but because of the limited characters, I dint add it earlier. Also, I have removed stop words in the text cleaning process hence there is no 'of'. @MatBailie – Dhanya_mj Jan 24 '23 at 13:53

0 Answers0