Strings contain the same characters but are still different

Question

I am trying to read a .txt file and use each sentence as a name for a team, and at the same time use that name to seek out another .txt file to get its content. All the .txt files are at the root of my assets folder. The first .txt file works fine, I use assetmanager.open and readLine() to obtain the string, but when using that string as a parameter to get the second .txt I get a java.io.FileNotFoundException. However, when calling that same .txt file with a hardcoded String, everything works fine. Upon further inspection I found out that the hardcoded string and the one used as a parameter return false after using the equals() function on it.

This is the method calling the first.txt

private void loadTeams() {
    try {
        BufferedReader r = new BufferedReader(new InputStreamReader(assetManager.open("matches.txt")));
        String name, bio, trainer;
        for(int i = 0; i < 4; i++){
            name = r.readLine();
            bio = r.readLine();
            trainer = r.readLine();
            System.out.println(name+", "+bio+", "+trainer);
            teams[i] = new Team(name, bio, i, loadPlayers(name), trainer);
        }
        r.close();
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

using "name" as a parameter for the following method:

private Player[] loadPlayers(String teamName){
    Player[] players = new Player[11];

    try {
        String path = "team_Netherlands.txt";     //works
        String path2 = "team_"+teamName+".txt";     //doesn't work?
        System.out.println("are "+path+" and " +path2 +" the same? "+path.equals(path2));

        BufferedReader r = new BufferedReader(new InputStreamReader(assetManager.open(path2)));

        //perform operations on the obtained info
        r.close();
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    return players;
}

The first sentence in the file is "Netherlands" (without the quotes) which I think should lead to team_Netherlands.txt for the path2 variable. using this will however crash the app. Using the path variable it works just fine. The println confirms that the strings are not equal. (See first sentence of the logcat)

logcat:

05-26 11:18:23.152 2960-2960/com.myname.testapp I/System.out: are team_Netherlands.txt and team_Netherlands.txt the same? false
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err: java.io.FileNotFoundException: team_Netherlands.txt
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.content.res.AssetManager.openAsset(Native Method)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.content.res.AssetManager.open(AssetManager.java:354)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.content.res.AssetManager.open(AssetManager.java:328)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at com.myname.testapp.Poule_Activity.load_Players(Poule_Activity.java:144)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at com.myname.testapp.Poule_Activity.load_Teams(Poule_Activity.java:94)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at com.myname.testapp.Poule_Activity.onCreate(Poule_Activity.java:53)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.app.Activity.performCreate(Activity.java:5990)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2332)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2442)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.app.ActivityThread.access$800(ActivityThread.java:156)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1351)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:102)
05-26 11:18:23.152 2960-2960/com.myname.testapp W/System.err:     at android.os.Looper.loop(Looper.java:211)
05-26 11:18:23.153 2960-2960/com.myname.testapp W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:5373)
05-26 11:18:23.153 2960-2960/com.myname.testapp W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
05-26 11:18:23.153 2960-2960/com.myname.testapp W/System.err:     at java.lang.reflect.Method.invoke(Method.java:372)
05-26 11:18:23.153 2960-2960/com.myname.testapp W/System.err:     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1020)
05-26 11:18:23.153 2960-2960/com.myname.testapp W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:815)

Why aren't those strings equal and how to make them equal? (i.e., make the non-hardcoded string equal to the hardcoded one)

thank you for your quick response :) I think you are on to something: path length: 20, path2 length: 16, do you know what could have caused this? — LGunit, May 26 '16 at 10:06
or `"team_"+team_Name.trim()+".txt"`, I guess this should have been my first comment :) — Amit.rk3, May 26 '16 at 10:12
that one also returns false, but path is the hardcoded one and that one has length 20, whilst path2 has only 16 — LGunit, May 26 '16 at 10:14
I tried its coming both strings are equal. Check the string you are passing whether any special character is added — Preethi Rao, May 26 '16 at 10:19
Can u please put the chunk of code where you are generating team_Name. — Preethi Rao, May 26 '16 at 10:22
@PreethiRao the team_name is a simple textfile that I have written in notepad++, the .txt files function as a script — LGunit, May 26 '16 at 10:43
After a lot of saving and resaving I managed to get rid of the BOM in the encoding. not sure if I should give credits to Jordi Castilla or laalto(can't tag), but thank you everyone for helping me! — LGunit, May 26 '16 at 10:53

Jordi Castilla · Answer 1 · 2016-05-26T10:59:16.143

If I execute this code:

public static void main(String[] args) {
    load_Players("Netherlands");
}

private static void load_Players(String team_Name) {
    String path = "team_Netherlands.txt"; // works
    String path2 = "team_" + team_Name + ".txt"; // doesn't work?
    System.out.println("are " + path + " and " + path2 + " the same? " + path.equals(path2));
}

All works as expected, so... what's wrong?

As you can see in your log, first Netherlands is not blue, what means parser has found something different.

Suspicious uh?

CHECKS:

When I paste this part of code into my eclipse:

team_Netherlands.txt and team_Netherlands.txt

I get this error when saving:

If I choose option select first character it selects:

System.out.println("team_Netherlands.txt".equals("team_Netherlands.txt"));
                                                       ↑ this one!!!

So you're passing wrong encoding as you can check with this snippet:

public static void main(String[] args) {
    String rightString = "_Netherlands.txt";
    String wrongString = "_Netherlands.txt";

    System.out.println("WRONG HASH");
    System.out.println(rightString.hashCode());
    System.out.println("\nRIGHT HASH");
    System.out.println(wrongString.hashCode());

    System.out.println("\nRIGHT");
    printChars(rightString);

    System.out.println("\n\nWRONG");
    printChars(wrongString);

}

private static void printChars(String s) {
    for (Character c : s.toCharArray()) {
        System.out.print((int) c + " ");
    }
}

OUTPUT:

WRONG HASH
1109617587

RIGHT HASH
-428164238

RIGHT
95 78 101 116 104 101 114 108 97 110 100 115 46 116 120 116 

WRONG
95 65279 78 101 116 104 101 114 108 97 110 100 115 46 116 120 116 
// ↑ here!!

SOLUTION's: (source)

Use BOMInputStream from Apache IO Commons

Handle manually (fast easy way):

private static String clean(String s) throws Exception {
    InputStream is = new ByteArrayInputStream(s.getBytes());
    try (BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
        // BOM marker will only appear on the very beginning
        br.mark(4);
        if ('\ufeff' != br.read())
            br.reset(); // not the BOM marker

        return br.readLine();
    }
}

TEST IT!:

public static void main(String[] args) throws Exception {
    String rightString = "Netherlands.txt";
    String wrongString = "Netherlands.txt";

    System.out.println("\nCOMPARE");
    System.out.println(rightString.equals(wrongString));
    System.out.println("\nCLEAN COMPARE");
    System.out.println(clean(rightString).equals(clean(wrongString)));

    System.out.println("\nRIGHT");
    printChars(clean(rightString));

    System.out.println("\n\nWRONG");
    printChars(clean(wrongString));
}

private static String clean(String s) throws Exception {
    InputStream is = new ByteArrayInputStream(s.getBytes());
    try (BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
        // BOM marker will only appear on the very beginning
        br.mark(4);
        if ('\ufeff' != br.read())
            br.reset(); // not the BOM marker

        return br.readLine();
    }
}

private static void printChars(String s) {
    for (Character c : s.toCharArray()) {
        System.out.print((int) c + " ");
    }
}

OUTPUT:

COMPARE
false

CLEAN COMPARE
true

RIGHT
78 101 116 104 101 114 108 97 110 100 115 46 116 120 116 

WRONG
78 101 116 104 101 114 108 97 110 100 115 46 116 120 116

ooh i see, I think that makes sense, I'll change the encoding and let you know what comes out of it :) — LGunit, May 26 '16 at 10:18
unfortunately changing everything to UTF-8 has not helped, what would be the correct encoding needed? — LGunit, May 26 '16 at 10:31
you're welcome. I also posted a last update with a working code and solution, you found an interesting bug and I want to learn more about it... but **I agree accepted answer is @laalto one, I was guided by him and links provided. :)** — Jordi Castilla, May 26 '16 at 11:01

score 5 · Accepted Answer · edited May 23 '17 at 11:44

5

The team_name you read from file contains UTF-8 byte order mark octets in front

ef bb bf

and they are not visible as such in log output.

Either save the files without BOM, or remove the BOM in your code.

edited May 23 '17 at 11:44

Community

1
1

answered May 26 '16 at 10:17

laalto

150,114
66
286
303

how you check it taking the string from here? I can see equals and hashCode does not match, but cannot find this hidden chars... – Jordi Castilla May 26 '16 at 10:27
1

@JordiCastilla copy-pasted to hexdump – laalto May 26 '16 at 10:29
1

... though now it seems the BOM was lost in @ErikKralj's edit. Anyway, it's there in the original version. – laalto May 26 '16 at 10:40

score 1 · Answer 3 · answered May 26 '16 at 10:41

1

it's must be have some Invisible characters.

could you show us path.getBytes() and path2.getBytes(),output array datas.

answered May 26 '16 at 10:41

star

321
2
9

turned out to be the encoding that caused the unequal strings, which would have resulted from that as well. Sorry for the late response, I was working through all the possible solution and had not seen your post yet. thank you! – LGunit May 26 '16 at 11:00

Android Nerd · Answer 4 · 2016-05-26T10:29:25.160

0

You can do something like:

    String s1="abc    def";
    String s2="abc def";
    s1=s1.trim().replaceAll("\\s+", " ");
    System.out.println(s1.equals(s2));

Apparently, the String from the text files seems good to go but you can still have options to do something like above to precise the equals!

edited May 26 '16 at 10:29

answered May 26 '16 at 10:18

Android Nerd

625
1
8
13

score 0 · Answer 5 · edited May 23 '17 at 12:01

0

You can find the answer to your problem here. The strings you are comparing are different objects that is why you get false. You are not actually comparing the value that those strings contain.

edited May 23 '17 at 12:01

Community

1
1

answered May 26 '16 at 10:20

Bouramas

1,077
1
17
36

score 0 · Answer 6 · answered May 26 '16 at 10:23

0

The problem is with the encoding. The names you are reading from the text file are UTF encoded.

What you can try and do is

String path3 = new String(path, "UTF-8");
String path4 = new String(path2, "UTF-8");

Do a string compare on path3 and path4 now and check.

answered May 26 '16 at 10:23

madhusudan rao Y

51
5

1

android studio says it cannot resolve the constructor with two strings, and removing the quotes at UTF-8 does not work either – LGunit May 26 '16 at 10:40

Strings contain the same characters but are still different

6 Answers6

Linked