How to properly handle character encoding when reading a file in Karate

Question

Character Ñ is being converted to � when reading a csv file.

* def data = read("data.csv")
* print data

expected:
VALUE1,VALUE2,DASMARIÑAS CITY

actual:
VALUE1,VALUE2,DASMARI�AS CITY

Does this answer your question? [What is character encoding and why should I bother with it](https://stackoverflow.com/questions/10611455/what-is-character-encoding-and-why-should-i-bother-with-it) — tripleee, Mar 18 '22 at 09:49
It isn't quite accurate to state that "Character Ñ is being _converted_ to �....". That � is the replacement character, and is used because the character to be displayed has an invalid value and could not be rendered. So when you see � it is because the data to be rendered is being replaced (or "substituted") with � rather than being _"converted"_. — skomisa, Mar 18 '22 at 18:37

score 1 · Answer 1 · answered Mar 18 '22 at 09:43

1

Resolved by just saving the csv file with type CSV UTF-8 (Comma delimited) (*csv)

Though previously working filter using karate.jsonPath is now not working

* def filteredData = karate.jsonPath(read("data.csv"),'$[?(@.TC_ID=="'+TC_ID+'")]')

answered Mar 18 '22 at 09:43

Don

hmm, I think the version of karate matters, there was an issue with the BOM which was fixed at some point. for the benefit of others who come across encoding issues - also see: https://stackoverflow.com/a/69116714/143475 – Peter Thomas Mar 18 '22 at 10:01
I'm using the latest version v1.1.0 of Karate. for now it was resolved by saving as json file instead of csv. – Don Mar 18 '22 at 10:21

1 Answers1