RegEx in Java: how to deal with newline

Question

I am currently trying to learn how to use regular expressions so please bear with my simple question. For example, say I have an input file containing a bunch of links separated by a newline:

www.foo.com/Archives/monkeys.htm
Description of Monkey's website.

www.foo.com/Archives/pigs.txt
Description of Pig's website.

www.foo.com/Archives/kitty.txt
Description of Kitty's website.

www.foo.com/Archives/apple.htm
Description of Apple's website.

If I wanted to get one website along with its description, this regex seems to work on a testing tool: .*www.*\\s.*Pig.*

However, when I try running it within my code it doesn't seem to work. Is this expression correct? I tried replacing "\s" with "\n" and it doesn't seem to work still.

Just to remind of potentially simpler solutions: For my own case with explicit `\n`'s, even with the suggestions of `Pattern.DOTALL` / `(?s)` and double-escaping (\\\) as noted below, I found this fiddly enough to just fall back to the non-regexp string methods. `str.contains("\n")` worked fine. `str.replaceAll("\n", replacement)` worked as well. I couldn't find variant of `String.matches` or `Pattern.compile` that returned true, though, in Java 11. (Unlike solutions below, this won't help if you need to catch various kinds of newlines.) — Joshua Goldberg, Sep 22 '20 at 17:45

Alan Moore · Answer 1 · 2017-02-12T06:03:56.120

The lines are probably separated by \r\n in your file. Both \r (carriage return) and \n (linefeed) are considered line-separator characters in Java regexes, and the . metacharacter won't match either of them. \s will match those characters, so it consumes the \r, but that leaves .* to match the \n, which fails. Your tester probably used just \n to separate the lines, which was consumed by \s.

If I'm right, changing the \s to \s+ or [\r\n]+ should get it to work. That's probably all you need to do in this case, but sometimes you have to match exactly one line separator, or at least keep track of how many you're matching. In that case you need a regex that matches exactly one of any of the three most common line separator types: \r\n (Windows/DOS), \n (Unix/Linus/OSX) and \r (older Macs). Either of these will do:

\r\n|[\r\n]

\r\n|\n|\r

Update: As of Java 8 we have another option, \R. It matches any line separator, including not just \r\n, but several others as defined by the Unicode standard. It's equivalent to this:

\r\n|[\n\x0B\x0C\r\u0085\u2028\u2029]

Here's how you might use it:

(?im)^.*www.*\R.*Pig.*$

The i option makes it case-insensitive, and the m puts it in multiline mode, allowing ^ and $ to match at line boundaries.

@Davinder Singh's answer has double backslashes to compensate against java compiler's decoding of string literals. Perhaps, Joe's observation relates to attempts at using a single backslash followed by the new regexp letter. This would probably turn into an illegal Java string literal at compile time. Following Davinder's example, I guess using double backslashes should work for Joe. — eel ghEEz, Dec 26 '20 at 22:17

javaPhobic · Answer 2 · 2016-07-21T01:46:59.723

17

For future reference, one can also use the Pattern.DOTALL flag for "." to match even \r or \n.

Example:

Say the we are parsing a single string of http header lines like this (each line ended with \r\n)

HTTP/1.1 302 Found
Server: Apache-Coyote/1.1
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: SAMEORIGIN
Location: http://localhost:8080/blah.htm
Content-Length: 0

This pattern:

final static Pattern PATTERN_LOCATION = Pattern.compile(".*?Location\\: (.*?)\\r.*?", Pattern.DOTALL);

Can parse the location value using "matcher.group(1)".

The "." in the above pattern will match \r and \n, so the above pattern can actually parse the 'Location' from the http header lines, where there might be other headers before or after the target line (not that this is a recommended way to parse http headers).

Also, you can use "?s" inside the pattern to achieve the same effect.

If you are doing this, you might be better off using Matcher.find().

edited Jul 21 '16 at 01:46

answered May 19 '15 at 02:25

javaPhobic

476
4
12

DOTALL isn't really useful in thiss case. The OP needs to know when the regex consumes the line separator so he can be sure he's matching only one of them. And it's even less useful in your example, where all of the content of interest is contained in one line. I hardly ever use DOTALL mode; it seems to cause more problems than it solves. – Alan Moore May 19 '15 at 05:20
You're probably right, but It is useful in my example though, my single string to parse actually had all the lines. – javaPhobic May 19 '15 at 05:28
The thing about DOTALL mode is that it expands the scope for mischief enormously. For example, when I apply your regex to your sample data, the first `.*?` consumes all headers that are listed above the `Location` header. I know you only care about the URL you're capturing in group #1, but you'll still get it with DOTALL mode off, and you'll save a lot of unnecessary work for the regex. – Alan Moore May 19 '15 at 08:36
Nope, without DOTALL, it won't be able to match "." with \r or \n. Hence the location cannot be parsed. If I split the string based on line boundaries and feed just the location line into the regex without DOTALL it will work. – javaPhobic May 19 '15 at 23:07
No, what I'm saying is you don't have to match *any* line separators. `"Location: (.*)"` will probably work fine, but I would use anchors to be safe: `"(?m)^Location: (.*)$"` – Alan Moore May 19 '15 at 23:46
You are talking about when you have already broken up the header into lines, I am talking about when the entire header comes in as one String, both using matches(). On the other hand, using find() is different story. – javaPhobic Jan 12 '17 at 05:52

score 4 · Answer 3 · edited Jun 02 '20 at 12:44

4

String str="I am  a   "+"\n  Man    of  Peace"+"\t"+"   .";

str=str.replaceAll("[\\s|\\t|\\r\\n]+"," ").trim();
System.out.println(str);

This above example works for tabSpaces, newLines, and normal spaces. And I have used the trim method of java.lang.String to remove all the additional spaces in 'str'. I hope this helps you and other amazing people here.

edited Jun 02 '20 at 12:44

Matt Ke

3,599
12
30
49

answered Jun 02 '20 at 12:23

davinder singh

41
3

Best solution. You can keep 2nd argument of replaceAll as "\s" instead of " " – Satish Patro Feb 18 '22 at 10:21

score 3 · Answer 4 · answered Aug 10 '10 at 02:27

3

try this

([^\r]+\r[^\r])+

answered Aug 10 '10 at 02:27

user414661

1,042
9
4

score 2 · Answer 5 · answered Aug 10 '10 at 01:19

Works for me:

import java.util.regex.Pattern;
import java.util.regex.Matcher;
public class Foo {
  public static void main(String args[]) {
    Pattern p = Pattern.compile(".*www.*\\s.*Pig.*");
    String s = "www.foo.com/Archives/monkeys.htm\n"
             + "Description of Monkey's website.\n"
             + "\n"
             + "www.foo.com/Archives/pigs.txt\n"
             + "Description of Pig's website.\n"
             + "\n"
             + "www.foo.com/Archives/kitty.txt\n"
             + "Description of Kitty's website.\n"
             + "\n"
             + "www.foo.com/Archives/apple.htm\n"
             + "Description of Apple's website.\n";
    Matcher m = p.matcher(s);
    if (m.find()) {
      System.out.println(m.group());
    } else {
      System.out.println("ERR: no match");
    }
  }
}

Perhaps the problem was with the way you were using the Pattern and Matcher objects?

This only works if lines are always formatted with \n, as in unix — Gary, Aug 10 '10 at 05:21

score 0 · Answer 6 · answered Aug 10 '10 at 05:22

This version matches newlines that may be either Windows (\r\n) or Unix (\n)

Pattern p = Pattern.compile("(www.*)((\r\n)|(\n))(.*Pig.*)");
String s = "www.foo.com/Archives/monkeys.htm\n"
           + "Description of Monkey's website.\n"
           + "\r\n"
           + "www.foo.com/Archives/pigs.txt\r\n"
           + "Description of Pig's website.\n"
           + "\n"
           + "www.foo.com/Archives/kitty.txt\n"
           + "Description of Kitty's website.\n"
           + "\n"
           + "www.foo.com/Archives/apple.htm\n"
           + "Description of Apple's website.\n";
Matcher m = p.matcher(s);
if (m.find()) {
  System.out.println("found: "+m.group());
  System.out.println("website: "+m.group(1));
  System.out.println("description: "+m.group(5));
}
System.out.println("done");

RegEx in Java: how to deal with newline

6 Answers6

Linked

Related