1

I'm taking a look at some CBOW Python implementations. The owner of the code used a function called "line_processing" from text_process lib. When I tried to run, I got that error:

ImportError: cannot import name 'line_processing' from 'text_process'

So I took a look at the lib implementation. There is no function called "line_processing".

That guy used this function to read each line from a .txt file, and write them in a variable, creating a "big string":

    text = 'file.txt'
    print(text)
    text = ''
    count = 0
    for i in open(text_file, 'r', encoding='utf-8'):
        text+=line_processing(i)+'\n'
        count += 1
        if count % 10000 == 0: break

Is there anyone who knows something about "line_processing" function, or about a function/lib I can use instead?

Thank you!

Ps.:

$ python CBOW.py
Building prefix dict from the default dictionary ...
Dumping model to file cache C:\"path_to"\AppData\Local\Temp\jieba.cache
Loading model cost 0.723 seconds.
Prefix dict has been built successfully.
Traceback (most recent call last):
  File "CBOW.py", line 1, in <module>
    from text_process import line_processing
ImportError: cannot import name 'line_processing' from 'text_process' (C:\"path_to"\miniconda3\lib\site\...\text_process)
leob
  • 23
  • 4
  • Nothing you've yet shown tries to import `line_processing`, so your excerpt couldn't have generated the error message. If you add a full error message (w/ stack traceback showing files & line-numbers), it'd help identify the actual code triggering the error. And, while there is a `text_process` package shown at the PyPI package repository, it has suspiciously little info (like docs or a project source code link) & I've never heard of it. So it might not be the same `text_process` your code was using, or even safe to install. You'll need to look at other context to determine author intent. – gojomo Sep 08 '20 at 03:32
  • I decided to uninstall "text_process" lib. I added the full output. I don't know if it'll help, but you can take a look. – leob Sep 08 '20 at 14:22
  • The error now shows that line #1 of a file named `CBOW.py` attempts `from text_process import line_processing`. There's no chance this can succeed if there's no module named `text_process` available. Whether the module named `text_process` shown on PyPI is what the original author used or not is unclear, given the lack of info there about that package. But, given that the Google query [`text_process line_processing`] suggests that might have been a local/custom/obscure package anyway - so if it's not obviously documented/present locally, it's best to just forget about it. – gojomo Sep 08 '20 at 16:19
  • It's hard to suggest a replacement for that function without seeing the full context of what kind of input it was being used one, where its return values were eventually being used, and for what final aim. It looks vaguely like it was doing some line-by-line text-handling... but leaving the text as a string that's re-concatenated with up to 10000 other lines into a giant string. That's a pretty inefficient thing to do; if your real aim is using CBOW or learning CBOW, you'd be much better using some well-made off-the-shelf library or example code, not unclear-quality mystery-meat source code. – gojomo Sep 08 '20 at 16:23

0 Answers0