java.lang.OutOfMemoryError on String.replace

Question

I have one text file which contains 234 lines of string1@string2@string3.

kya@きゃ@キャ
kyu@きゅ@キュ
kyo@きょ@キョ
sha@しゃ@シャ
shu@しゅ@シュ
...so 234 lines

I am writing converter which converts word with string2 or string3 to the word with string1.

InputStream is = context.getResources().openRawResource(R.raw.kanatoromaji);
InputStreamReader isr = new InputStreamReader(is);
BufferedReader reader = new BufferedReader(isr);
String line;
try {
    while ((line = reader.readLine()) != null) {
        String[] parts = line.split("@");
        String romaji = parts[0];
        String hiragana=parts[1];
        String katakana = parts[2];

        if (hiragana!=null&&word.contains(hiragana)) {
                word = word.replace(hiragana, romaji);//in this line getting outOfMemory error
        }
        if (word.contains(katakana)) {
            word = word.replace(katakana, romaji);
        }
    }

} catch (IOException e) {
    e.printStackTrace();
}

I am calling this method many times (200k-300k times in one run). So this causing this error: 07-24 00:52:32.859 10848-10848/net.joerichard.test E/AndroidRuntime﹕ FATAL

EXCEPTION: main
    java.lang.OutOfMemoryError
            at java.lang.AbstractStringBuilder.enlargeBuffer(AbstractStringBuilder.java:94)
            at java.lang.AbstractStringBuilder.append0(AbstractStringBuilder.java:132)
            at java.lang.StringBuilder.append(StringBuilder.java:124)
            at java.lang.String.replace(String.java:1367)
            at net.joerichard.test.Converter.kanaToRomaji(Converter.java:32)
            at net.joerichard.test.MainActivity.migrateKanaKanji(MainActivity.java:222)
            at net.joerichard.test.MainActivity.access$700(MainActivity.java:29)
            at net.joerichard.test.MainActivity$10.onClick(MainActivity.java:131)
            at android.view.View.performClick(View.java:4240)
            at android.view.View$PerformClick.run(View.java:17721)
            at android.os.Handler.handleCallback(Handler.java:730)
            at android.os.Handler.dispatchMessage(Handler.java:92)
            at android.os.Looper.loop(Looper.java:137)
            at android.app.ActivityThread.main(ActivityThread.java:5103)
            at java.lang.reflect.Method.invokeNative(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:525)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
            at dalvik.system.NativeStart.main(Native Method)

What is problem in my code? How to solve it?

Update:

According to JFPicard's answer, I converted my String word to StringBuilder sbWord. Then I tried to replace string2 and string3 using StringBuilder. I never replaced StringBuilder before. I search in Google and found this solution to replace my StringBuilder parts. Now my code looks like this:

public static String kanaToRomaji(Context context, String word) {

    String kana = word;
    StringBuilder sbWord = new StringBuilder(word);

    InputStream is = context.getResources().openRawResource(R.raw.kanatoromaji);
    InputStreamReader isr = new InputStreamReader(is);
    BufferedReader reader = new BufferedReader(isr);
    String line;
    try {
        while ((line = reader.readLine()) != null) {
            String[] parts = line.split("@");
            String romaji = parts[0];
            String hiragana=parts[1];
            String katakana = parts[2];

            if (hiragana!=null&&word.contains(hiragana)) {
                sbWord = replaceAll(sbWord, hiragana, romaji);//in this line getting outOfMemory error
            }
            if (word.contains(katakana)) {
                sbWord = replaceAll(sbWord, katakana, romaji);
            }
        }

    }
    catch (IOException e) {
        e.printStackTrace();
    }

    char[] chars = kana.toCharArray();
    for (char character: chars)
    {
        if(sbWord.toString().contains(String.valueOf(character)))
        {
            new MyLog(kana+":"+sbWord.toString());
            break;
        }
    }

    return word;
}

Method where I replace my StringBuilder parts:

public static StringBuilder replaceAll(StringBuilder builder, String from, String to)
{
    int index = builder.indexOf(from);
    while (index != -1)
    {
        builder.replace(index, index + from.length(), to);
        index += to.length(); // Move to the end of the replacement
        index = builder.indexOf(from, index);
    }
    return builder;
}

Now I am getting this error: 07-24 01:23:52.890 13512-13512/net.joerichard.test E/AndroidRuntime﹕ FATAL

EXCEPTION: main java.lang.OutOfMemoryError at java.lang.AbstractStringBuilder.move(AbstractStringBuilder.java:397) at java.lang.AbstractStringBuilder.insert0(AbstractStringBuilder.java:356) at java.lang.AbstractStringBuilder.replace0(AbstractStringBuilder.java:442) at java.lang.StringBuilder.replace(StringBuilder.java:637) at net.joerichard.test.Converter.replaceAll(Converter.java:65) at net.joerichard.test.Converter.kanaToRomaji(Converter.java:33) at net.joerichard.test.MainActivity.migrateKanaKanji(MainActivity.java:222) at net.joerichard.test.MainActivity.access$700(MainActivity.java:29) at net.joerichard.test.MainActivity$10.onClick(MainActivity.java:131) at android.view.View.performClick(View.java:4240) at android.view.View$PerformClick.run(View.java:17721) at android.os.Handler.handleCallback(Handler.java:730) at android.os.Handler.dispatchMessage(Handler.java:92) at android.os.Looper.loop(Looper.java:137) at android.app.ActivityThread.main(ActivityThread.java:5103) at java.lang.reflect.Method.invokeNative(Native Method) at java.lang.reflect.Method.invoke(Method.java:525) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553) at dalvik.system.NativeStart.main(Native Method)

What to do in this situation?

Where do you create the word variable? Also, have you tried to run the code on a JVM and outside of Android? — JP Moresmau, Jul 23 '15 at 20:14
@JPMoresmau, this is data migration app to prepare data for another app. I am running this on Android — Joe Rakhimov, Jul 23 '15 at 20:15
Yes, but maybe you can check the behavior on an Oracle JVM to be sure your code is in cause, and not the limitations or a bug of the android VM. — JP Moresmau, Jul 23 '15 at 20:24

JFPicard · Answer 1 · 2015-07-23T20:20:01.067

6

I think I have an idea.

This line word = word.replace(hiragana, romaji);

and this one word = word.replace(katakana, romaji);

create a new String each time. The String is immutable so it consume memory. A lot in your case.

Try to use a StringBuilder instead.

edited Jul 23 '15 at 20:20

answered Jul 23 '15 at 20:13

JFPicard

5,029
3
19
43

thank you for your answer. I tried what you suggested but even after I changed my String word to StringBuilder sbWord, unfortunately I am still getting outOfMemory exception. For more info, see my updated question – Joe Rakhimov Jul 23 '15 at 20:37
Try also to replace the split (String[] parts = line.split("@");) with a StringBuilder – JFPicard Jul 23 '15 at 21:49
Thank you, I found the solution with your help.- http://stackoverflow.com/a/31598833/2255924 – Joe Rakhimov Jul 23 '15 at 22:03

score 1 · Answer 2 · answered Jul 23 '15 at 20:05

1

You never release your resources when working with InputStream. You should close the stream when it's not needed anymore in a finally block.

Also, if you call this method 200k times, maybe you shouldn't parse the file all the time? As I understand the file is always the same, why don't you cache the results somewhere after first run?

answered Jul 23 '15 at 20:05

Gennadii Saprykin

4,505
8
31
41

i tried like that - i copied them to arraylist of objects before going to this method. no effect - same error. then i thought it is because of another thing – Joe Rakhimov Jul 23 '15 at 20:07
also tried to close buffered reader, input stream reader and input stream. unfortunately no effect – Joe Rakhimov Jul 23 '15 at 20:08

score 1 · Answer 3 · edited May 23 '17 at 12:02

I found this solution with the help of JFPicard, which I appreciate. My problem was in this line as I stated in question:

word = word.replace(hiragana, romaji);

My bad that I did not notice about empty string2 lines. I am really sorry, I did not know this may cause problem. I had 26 lines with strin2 empty.

wi@@ウィ

So error happened because it replaced 26 times because it is empty and could get through my if clause

        if (hiragana != null && word.contains(hiragana)) {
            word = word.replace(hiragana, romaji);
        }

Now I changed my code to this and it is working =)

        String[] parts = line.split("@");
        String romaji = parts[0];
        String hiragana = parts[1];
        if(hiragana.equals(""))
        {
            hiragana=null;
        }
        String katakana = parts[2];
        if(katakana.equals(""))
        {
            katakana=null;
        }

        if (hiragana != null && word.contains(hiragana)) {
            word = word.replace(hiragana, romaji);
        }
        if (katakana!=null && word.contains(katakana)) {
            word = word.replace(katakana, romaji);
        }

score 0 · Answer 4 · answered Nov 06 '21 at 09:10

Well the thread is kinda old but here is how I solved the problem:

The problem here is, that some applications may demand a high usage of the heap. If thats the case you should of course first check if you can lower the demand, but if its not possible you can activate the largeHeap option in your AndroidManifest.xml as described in the following article: https://pspdfkit.com/blog/2019/android-large-memory-requirements/#large-heap at least that worked for me. Here is, how the manifest could look like:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="<your-package-id>">

    <application
        android:allowBackup="true"
        android:fullBackupContent="true"
        android:icon="@mipmap/ic_ds99"
        android:label="@string/app_name"
        android:largeHeap="true"
        android:roundIcon="@mipmap/ic_ds99"
        android:supportsRtl="true"
        android:theme="@style/AppTheme"
        android:usesCleartextTraffic="true">
        <activity android:name=".UpdateActivity"></activity>

        <activity
            android:name=".MainActivity"
            android:configChanges="orientation|screenSize">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>

        <service
            android:name=".UpdateChecker"
            android:exported="false"></service>
    </application>
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />

    <uses-permission android:name="android.permission.INTERNET" />

</manifest>

The manifest file is usually found here: app/src/main/AndroidManifest.xml

java.lang.OutOfMemoryError on String.replace

4 Answers4