How to replace characters in a very large string using java causing large memory?

Question

I have a large String in which I need to replace a few string pathern.

I have written code as below and I am getting very large memory usage in deployed environment.

I have read online to see if there is any more optimal way this can be re-written but could not find conclusive answer.

Can anyone give any suggestions for code below.

    String response = "@very #large  & string % STUFF";
    System.out.println(response);
    String[] SPECIAL_CHARACTERS = {"&","%","#","STUFF"};
    for(int count = 0;count<SPECIAL_CHARACTERS.length;count++)
    {
        if(response.contains(SPECIAL_CHARACTERS[count]))
        {
            response = response.replace(SPECIAL_CHARACTERS[count],"");
        }
    }
    System.out.println(response);

Ahh, the bothersome immutability of Java/.NET world. https://stackoverflow.com/questions/279507/what-is-meant-by-immutable — PTwr, Jun 04 '20 at 11:58
What is the source of the string (for example Web Request), and what is the destination of the output(for example File / database)? — samabcde, Jun 04 '20 at 12:05
I am getting data from database having these special characters and then in java code replacing it and returning it in REST API response. I can't do replacement in DB sql due to performance consideration. — satish, Jun 04 '20 at 12:32

Stephen C · Answer 1 · 2020-06-04T13:22:49.327

2

I have a large String in which I need to replace a few characters.

I would avoid getting into that situation. Stream your data! (And how come you have a "very large" string stored in the database? That doesn't sound good.)

If you can't, this is the most memory efficient way to do what you are doing:

   int len = response.length();
   for (int i = 0; i < len; i++) {
       char ch = response.charAt(i);
       switch (ch) {
           case '&': case '%': case '#': case '@':
               break;
           default:
               System.out.print(ch);
       }
   }
   System.out.println();

In some circumstances, it may be better to use a StringBuilder so that you can do a single write operation.

   int len = response.length();
   StringBuilder temp = new StringBuilder(len);
   for (int i = 0; i < len; i++) {
       char ch = response.charAt(i);
       switch (ch) {
           case '&': case '%': case '#': case '@':
               break;
           default:
               temp.append(ch);
       }
   }
   System.out.println(temp.toString());

but that allocates more memory.

edited Jun 04 '20 at 13:22

answered Jun 04 '20 at 13:17

Stephen C

698,415
94
811
1,216

Thanks. Actually I missed mentioning that the text to be replaced can be a string (thus I created a String array). I am trying to think how would this code snippet would work in that case as I have to write a generic code to replace a specific text. I have updated my question to reflect same. In this case, I guess I would need to use String.replace() function which causes memory issue. Do you think same? – satish Jun 04 '20 at 16:01
You can easily modify the snippets to handle that. You might end up replacing the switch with a sequence of `if else` statements or a loop. But I have answered the question **that you asked**. Avoid using `String.replace()` because each time you call it you copy the entire string. – Stephen C Jun 05 '20 at 00:42
The key to modifying the snippets will be to understand how they work. Read the code. It is pretty obvious when you think about it. – Stephen C Jun 05 '20 at 00:51

score 1 · Answer 2 · answered Jun 04 '20 at 12:07

1

I would solve this kind of problem by reading packets from the input and append them to the output. So, assuming that the very large String is called input, I would always read easily manageable packets from input into temp, do the algorithm of replacements for temp and then append temp to output, which should be initialized before your cycles as empty String.

The problem you encounter is that you are always replacing your whole String. So, instead of replacing response each time, do all the replacements on smaller packets and add the result to a String variable.

answered Jun 04 '20 at 12:07

Lajos Arpad

64,414
37
100
175

I am not not sure I am following it. A code sample would be handy. – satish Jun 04 '20 at 12:34
@satish in order to do that I would need to test it before posting it here, but I'm lazy to do so. If you want, you can try implementing it and you may ask questions when you are stuck. – Lajos Arpad Jun 04 '20 at 17:28

How to replace characters in a very large string using java causing large memory?

2 Answers2