0

I am getting the below error while reading the csv file which have multiple columns and it might have some meta-characters or something. However, i have used encoding='utf-8' as per recommended python3.

Please let me know if someone can help or faced same issue before.

Error code

UnicodeEncodeError: 'ascii' codec can't encode characters in position 38730-38731: ordinal not in range(128)

My code:

df = pd.read_csv('OneView.csv', encoding='utf-8')
df =  df.to_html(escape=False)
df = df.replace('<th>','<th class = "th-sm" style="text-align: left">').replace('border="1"','id="example"').replace('class="dataframe"','class="display"')

jsbody='''
<script type="text/javascript" src="https://code.jquery.com/jquery-3.3.1.js"></script>
<script type="text/javascript" src="https://cdn.datatables.net/1.10.20/js/jquery.dataTables.min.js"></script>
        <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/1.10.20/css/jquery.dataTables.min.css"/>
<script>
$(document).ready(function() {
    $('#example').DataTable( {
        "order": [[ 1, "asc" ]]
    } );
} );</script>
'''
df = jsbody + df
df=df.replace('<th>','<th class = "th-sm"')
filename = 'example.html'
f= open(filename, 'w+')
f.write(df)
f.close()

Pandas Version

>>> import pandas as pd
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : 67a3d4241ab84419856b84fc3ebc9abcbe66c6b3
python           : 3.6.8.final.0
python-bits      : 64
OS               : Linux
OS-release       : 3.10.0-1127.18.2.el7.x86_64
Version          : #1 SMP Mon Jul 20 22:32:16 UTC 2020
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : C
LOCALE           : None.None

pandas           : 1.1.4
numpy            : 1.19.4
pytz             : 2020.4
dateutil         : 2.8.1
pip              : 20.3
setuptools       : 39.2.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fsspec           : None
fastparquet      : None
gcsfs            : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pyxlsb           : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
numba            : None

Python Version

Python 3.6.8 (default, Sep 26 2019, 11:57:09)

Sample OneView.csv

Status  Resource Name   Type    Occurred    Appliance   Model   State   Description Corrective Action
Critical    enc8039, bay 3  server-hardware 2020-11-30T05:36:18.730Z    PV8007  Synergy Composer    Locked  The Product ID and/or Serial Number of the server enc8039, bay 3 has changed, and no longer matches the server expected in the profile {'name':'pvc8495','uri':'/rest/server-profiles/f13bc2cc-52dc-46b7-aeff-582c81b56283'}. HPE OneView could not automatically correct the issue because the server is powered on and may be running a workload which would be interrupted. This most commonly happens when the Product ID or Serial number is manually changed using the RBSU (ROM-Based Setup Utility) on the server. These settings are not typically modified except when the system board on the server is replaced and needs to be set to have the same identifiers as the original board. Whenever these identifiers are changed, the server must first be allowed to complete a full power on self test, then be powered off, and then the enclosure must be refreshed. Not following these steps can cause this type of mismatch to occur.  If the server is running a workload, schedule maintenance and gracefully bring down the work load. Power down the server and then refresh the enclosure {'name':'enc8039','uri':'/rest/enclosures/797740SGH821VSJL'}.
Critical    pvc8461 server-profiles 2020-11-30T04:11:39.546Z    PV8001  Synergy Composer    Locked  An error has occurred on connection 1.  Interconnect {'name':'enc8002, interconnect 3','uri':'/rest/interconnects/ac77f2fa-d58a-48f8-b282-413c4f46de71'} port 11 subport a is unlinked. If the server {'name':'enc8002, bay 11','uri':'/rest/server-hardware/39313738-3034-4753-4838-313253313442'} is power cycling or powered off, connectivity alerts may occur as the network adapter is either disconnected or negotiates connectivity with the interconnect. These alerts can be ignored and should clear automatically. If this server is booted up and running an operating system, this alert indicates a loss of connectivity between the network adapter and the interconnect. Verify the configuration of the operating system, health and link status of the downlink ports on which the connection depends. If the problem persists, please contact your authorized support representative and prPVide them with a support dump.
Critical    enc8002, interconnect 3 interconnects   2020-11-30T04:11:39.180Z    PV8001  Synergy Composer    Active  Connection on downlink port 11, subport a has failed. The subport is unlinked.  If the server {'name':'enc8002, bay 11','uri':'/rest/server-hardware/39313738-3034-4753-4838-313253313442'} is power cycling or powered off, connectivity alerts may occur when the network adapter is either disconnected or negotiates connectivity with the interconnect. These alerts can either be ignored or manually cleared and will normally clear automatically. If the server is booted and running an operating system, the alerts indicate a loss of connectivity between the network adapter and the interconnect. Verify that the CNA supports the current interconnect downlink speed. Verify the configuration of the operating system, as well as the health and link status of the downlink ports on which the connection depends. If the problem persists, contact your authorized support representative and prPVide a support dump.
Critical    pvc8461 server-profiles 2020-11-30T04:11:28.481Z    PV8001  Synergy Composer    Locked  An error has occurred on connection 2.  Interconnect {'name':'enc8003, interconnect 6','uri':'/rest/interconnects/b6d8da91-8967-42c0-b4b8-9f0fcd5b6235'} port 23 subport a is unlinked. If the server {'name':'enc8002, bay 11','uri':'/rest/server-hardware/39313738-3034-4753-4838-313253313442'} is power cycling or powered off, connectivity alerts may occur as the network adapter is either disconnected or negotiates connectivity with the interconnect. These alerts can be ignored and should clear automatically. If this server is booted up and running an operating system, this alert indicates a loss of connectivity between the network adapter and the interconnect. Verify the configuration of the operating system, health and link status of the downlink ports on which the connection depends. If the problem persists, please contact your authorized support representative and prPVide them with a support dump.
Critical    enc8003, interconnect 6 interconnects   2020-11-30T04:11:28.147Z    PV8001  Synergy Composer    Active  Connection on downlink port 23, subport a has failed. The subport is unlinked.  If the server {'name':'enc8002, bay 11','uri':'/rest/server-hardware/39313738-3034-4753-4838-313253313442'} is power cycling or powered off, connectivity alerts may occur when the network adapter is either disconnected or negotiates connectivity with the interconnect. These alerts can either be ignored or manually cleared and will normally clear automatically. If the server is booted and running an operating system, the alerts indicate a loss of connectivity between the network adapter and the interconnect. Verify that the CNA supports the current interconnect downlink speed. Verify the configuration of the operating system, as well as the health and link status of the downlink ports on which the connection depends. If the problem persists, contact your authorized support representative and prPVide a support dump.
Critical    enc8002, bay 11 server-hardware 2020-11-30T03:59:31.130Z    PV8001  Synergy Composer    Locked  The iLO for the server hardware in bay 11 is reporting that the server hardware has a critical internal health status.  Check the iLO System Information - Health Summary and Integrated Management Log for more information. RemPVe the server hardware in bay 11 and re-insert it.  If the issue persists, replace the server hardware.
Critical    ENC2034, bay 5  server-hardware 2020-11-29T18:15:54.141Z    PV2004  Synergy Composer    Locked  Unable to establish trusted communication with the server.  Try refreshing the server hardware. If the issue persists, and if the iLO has a valid self-signed certificate, try adding the iLO certificate into OneView trust store and refresh the server hardware. Alternatively, if the iLO has a valid CA signed certificate, ensure that the root certificate and any appropriate intermediate certificates are present in OneView's trust store. If the issue still persists, generate a support dump and contact your authorized support representative. Use the link prPVided below to add certificate(s) to OneView trust store.
Critical    enc4058, bay 2  server-hardware 2020-11-28T23:23:36.144Z    PV4003  Synergy Composer    Active  The BIOS/Hardware status is critical.   Log into the iLO and view the Integrated Management Log (IML) for additional details.
Critical    enc4058, bay 2  server-hardware 2020-11-28T23:21:40.498Z    PV4003  Synergy Composer    Locked  The iLO for the server hardware in bay 2 is reporting that the server hardware has a critical internal health status.   Check the iLO System Information - Health Summary and Integrated Management Log for more information. RemPVe the server hardware in bay 2 and re-insert it.  If the issue persists, replace the server hardware.
Critical    enc3056, bay 1  server-hardware 2020-11-28T04:32:38.760Z    PV3003  Synergy Composer    Active  Remote Insight/ Integrated Lights-Out self test error 8192. To decode the error-code look into cpqSm2CntlrSelfTestErrors description. You can also check iLO logs & Diagnostic page to get more details on the error.
Critical    enc8040, bay 3  server-hardware 2020-11-27T20:00:51.833Z    PV8007  Synergy Composer    Active  Remote Insight/ Integrated Lights-Out self test error 8192. To decode the error-code look into cpqSm2CntlrSelfTestErrors description. You can also check iLO logs & Diagnostic page to get more details on the error.
Critical    ENC4069, bay 9  server-hardware 2020-11-27T19:00:20.648Z    PV4007  Synergy Composer    Active  Service Event: The state of the processor in slot 1 is 65.  A support ticket has been opened by the HPE Support Center.
Critical    ENC4069, bay 9  server-hardware 2020-11-27T18:55:13.543Z    PV4007  Synergy Composer    Active  There are failed entries in the iLO Integrated Management Log.  Log into the iLO and view the Integrated Management Log (IML) for additional details. If the underlying hardware fault has already been fixed, set the status of the corresponding IML log entry to repaired.
Critical    pvc6195 server-profiles 2020-11-27T13:28:28.898Z    PV6001  Synergy Composer    Locked  Server profile settings conflict with the server hardware configuration.    If the error is the result of performing a refresh of the server hardware, reapply the server profile. If the error is caused by inserting server hardware, change the profile server hardware type to match that of the actual server hardware. If this error is the result of performing a restore from backup, unassign then reassign all affected profiles. The process of unassigning all profiles is required to avoid duplicate MAC or WWN addresses in your environment.
Critical    ENC6018, bay 2  server-hardware 2020-11-27T13:28:25.601Z    PV6001  Synergy Composer    Locked  Unable to inventory the network adapters.   "Try the following, one at a time, until the problem is resolved:
• Check if there is any active alert related to iLO connectivity or trusted communication for this server and enclosure and resolve the issue as per the resolution steps prPVided in the alert.
• Refresh the server.
• Reset the server iLO, wait two minutes for the iLO reset to complete, and refresh the server.
• Verify that the server hardware is configured with supported network adapters.
• Verify that CPUs are installed in the sockets required to support the populated mezzanine slots as detailed in the server hardware QuickSpecs document.
• Re-seat the server network adapters and refresh the server."
Critical    ENC6018, bay 2  server-hardware 2020-11-27T13:27:38.537Z    PV6001  Synergy Composer    Locked  The iLO for the server hardware in bay 2 is reporting that the server hardware has a critical internal health status.   Check the iLO System Information - Health Summary and Integrated Management Log for more information. RemPVe the server hardware in bay 2 and re-insert it.  If the issue persists, replace the server hardware.

Note: There is no issue if used to red this file simply as follows:

Below works with no error:

df = pd.read_csv('OneView.csv')
print(df)
user2023
  • 452
  • 5
  • 22
  • you have problem in first line? how OneView.csv looks like? – sygneto Dec 02 '20 at 19:17
  • @sygneto, what problem you see , i just edited the post with some portion of data from the file. – user2023 Dec 02 '20 at 19:34
  • @sygneto, it doesn't work even if i remove it or keep it, it gives same error. – user2023 Dec 02 '20 at 19:41
  • possible duplicate :https://stackoverflow.com/questions/41228697/pandas-to-csv-ascii-cant-encode-character https://stackoverflow.com/questions/16923281/writing-a-pandas-dataframe-to-csv-file – sygneto Dec 02 '20 at 19:41
  • @sygneto, not really, i went through those and those are also not working in my case, and moreover they are also suggesting `encoding=utf-8` – user2023 Dec 02 '20 at 19:45
  • try with encoding='gb2312' and sep='\t', so : `df = pd.read_csv('OneView.csv', encoding='gb2312' ,sep='\t')` – sygneto Dec 02 '20 at 19:48
  • diffrent error now, `UnicodeDecodeError: 'gb2312' codec can't decode byte 0xe2 in position 26571: illegal multibyte sequence` – user2023 Dec 02 '20 at 19:56
  • try encoding='ISO-8859-1' then, and try to use https://chardet.readthedocs.io/en/latest/usage.html to define encoding – sygneto Dec 02 '20 at 20:04
  • This seems problem with html code. – user2023 Dec 02 '20 at 20:10
  • If its html, its good way to use pandas.read_html – sygneto Dec 02 '20 at 20:12

0 Answers0