Encoding issue with file upload

Rails
1

Hello,

I’ve got some issue while trying to upload and read csv file on my
application. For some file, i get some odd character appearing when i
print the result, but not in my logs. Here joined a screen to explain
what i’m talking about.

I think it’s an encoding issue, so I tried to convert file content in
UTF-8 with Iconv, but i get some trouble to determine the exact original
encoding.

Any help would be appreciated.

Thanks

2

Céd B. wrote:

Hello,

I’ve got some issue while trying to upload and read csv file on my
application. For some file, i get some odd character appearing when i
print the result, but not in my logs. Here joined a screen to explain
what i’m talking about.

I think it’s an encoding issue, so I tried to convert file content in
UTF-8 with Iconv, but i get some trouble to determine the exact original
encoding.

Any help would be appreciated.

Thanks

that’s an encoding issue with   spaces.

3

that’s an encoding issue with   spaces.

As a follow-up, it could also be due to a BOM (byte order mark) issue.

4

Thank you for answers. Is there an easy way to clean up my file before
parsing it?

5

Ok, i found a fix for this, using Iconv lib, and it seems to work
correcty.

But i still get an issue with accented character. If i get a surname
with accented character, like “Cédric”, Iconv returns me “C’edric”. I’ve
got similar problems with every accented character.

I read somewhere that i had to set $KCODE = ‘utf8’, but it didn’t do the
trick. And i really need those accented character, so i can’t convert
them into their similar letters (ie.: [éèêë] => e)

Thanks per advance.