0
\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u200e' in position 11: character maps to <undefined>"

I tried using,

f.write("{},{}".format(usernames[i].text.strip().encode("UTF-8", errors="ignore").decode("UTF-8", errors="ignore"), name_margin[i].text.strip().encode("UTF-8", errors="ignore").decode("UTF-8", errors="ignore")) + "\n")

Help me with solution for this problem

I am trying to to save data I scraped from website in a abc.csv file and getting this error.

rishabh11336
  • 34
  • 4
  • 1
    **What problem are you trying to solve** by encoding to UTF-8 and then immediately decoding from UTF-8 again? – Karl Knechtel Feb 05 '23 at 23:56
  • I am trying to save characters of different languages but when I am using encode I am getting result as "b'username". I am trying to remove 'b' using decode – rishabh11336 Feb 06 '23 at 00:03
  • 2
    Randomly throwing encodes and decodes is not a good way to program. You need to ask yourself, "what do I HAVE", and "what do I NEED"? You're clearly on Windows, since it refers to CP1252. Are you writing to a file, or to a terminal? What's the encoding of the file? – Tim Roberts Feb 06 '23 at 00:09
  • "but when I am using encode I am getting result as "b'username"" Did you try **not** doing that, then? In your own words, what do you think `.encode` is for? What do you think `.decode` is for? What do you expect will be the **type** of `usernames[i].text.strip()`? What kind of result do you think you need, and why? – Karl Knechtel Feb 06 '23 at 00:12
  • I am writing to "abc.csv" file. This 1st I am trying to do this. encoding solves my problem but give me another problem as result was added with "b'username" 'b'. – rishabh11336 Feb 06 '23 at 00:13
  • 2
    The problem occurs because you did not specify an encoding **when opening the file**, so Windows uses its platform default encoding, which **cannot handle one of the characters** in your string. See the linked duplicate for details. None of the `.encode` or `.decode` calls here is relevant and they cannot help solve the problem; they only add more code that confuses you further. – Karl Knechtel Feb 06 '23 at 00:14
  • Also, if you want to create a CSV file then *please use the standard library `csv` module*, rather than trying to put all the commas etc. yourself. CSV is a more complicated format than you think. – Karl Knechtel Feb 06 '23 at 00:15
  • with open("file1.csv", "w", encoding="utf-8") as f: using encoding="utf-8" worked. Now I am able save characters of different language – rishabh11336 Feb 06 '23 at 00:18

0 Answers0