I try to use a series of delimiter for an input. It's for a homework. They said that we should use backslash () too. If I use it like this (it's at the end):
scanner.useDelimiter("\\;|\\:|\\?|\\~|/|\\.|,|\\<|\\>|\\`|\\[|\\]|\\{|\\}|\\(|\\)|\\!|\\#|\\#|\\$|\\%|\\^|\\&|\\-|\\_|\\+|\\'|\\=|\\*|\"|\\||\n|\t|\r|\\");
It won't work. It says unsupported escape sequence. If I add another backslash it says Illegal line end in string literal. If I add another it will escape to double backslash and that's not what I need.
I couldn't find any solution for this and that's why I'm asking. I already finished the homework and I used Scanner and right now changing it it's not a solution (a lot to re-implement).
Thank you.
You should use four backslashes at the end, like:
scanner.useDelimiter("\\;|\\:| ... |\r|\\\\");
This is the way it should work. You said if you tried it would match double backslashes. Have you tried it? If you did, and it still matches double backslashes, I suspect your input is escaped too somewhere. (maybe it is a string literal somewhere in your code?)
The reason behind this is that your string is de-escaped twice. Once at compile time as every other string literal in the Java language, and once compiling the regex. That means, after the first step it is escaped once, so the regex compiler gets two backslashes \\. The regex compiler will de-escape that too (just like \r), and will match a single \ character.
If you would like to match two backslashes this way, then you have to use eight backslash (\\\\\\\\ or \\\\{2}) in your literal. Yeah, pretty ugly.
You are using the delimiter in wrong way i think.
There is a related topic.
Check this first
How do I use a delimiter in Java Scanner?
Related
take these strings for example:
"hello world\n" (correct - regex should match this)
"I'm happy \ here" (this is incorrect as the escape character is not
used correctly - regex should not match this one)
I've tried searching on google but didn't find anything helpful.
I want this one to be used in a parser which only parses string literals from a java code file.
Here is the the regex I used:
"\\\"(\\[tbnrf\'\"\\])*[a-zA-Z0-9\\`\\~\\!\\#\\#\\$\\%\\^\\&\\*\\(\\)\\_\\-\\+\\=\\|\\{\\[\\}\\]\\;\\:\\'\\/\\?\\>\\.\\<\\,]\\\""
what am I doing wrong?
I guess you gave us the regex in Java String literal form, like
String regex = \"(\[tbnrf'"\])*[a-zA-Z0-9\`\~\!\#\#\$\%\^\&\*\(\)\_\-\+\=\|\{\[\}\]\;\:\'\/\?\>\.\<\,]\";
Unpacking that from Java's String escaping syntax gives the raw regex:
\"(\[tbnrf'"\])*[a-zA-Z0-9\`\~\!\#\#\$\%\^\&\*\(\)\_\-\+\=\|\{\[\}\]\;\:\'\/\?\>\.\<\,]\"
That consists of:
\" matching a double-quote character (Java String literal begins here). Escaping the double quotes with backslash isn't necessary: " on its own is ok as well.
(\[tbnrf'"\])*: a group, repeated 0...n times. I guess you want that to match against the various Java backslash escapes, but that should read (\\[tbnrf'"\\])* with a double backslash in front and inside the character class. And maybe you want to cover the Java octal escapes as well (see the language specification), giving (\\[tbnrf01234567'"\\])*
[a-zA-Z0-9\``\~\!\#\#\$\%\^\&\*\(\)\_\-\+\=\|\{\[\}\]\;\:\'\/\?\>\.\<\,]: a character class matching one character from a selected list of alphabetic and punctuation characters. I'd replace that with [^"\\], meaning anything but double quote or backslash.
\" matching a double-quote character (string literal ends here). Once again, no need to escape the double quote.
Besides the individual elements, the overall structure of the regex probably isn't what you want: You allow only strings beginning with any number of backslash escapes, followed by exactly one non-escape character, and this enclosed in a pair of double quotes.
The overall structure should instead be "(backslash_escape|simple_character)*"
So, the complete regex would be:
"(\\[tbnrf01234567'"\\]|[^"\\])*"
or, expressed in a Java literal:
String regex = "\"(\\\\[tbnrf01234567'\"\\\\]|[^\"\\\\])*\"";
And, although this is shorter than your original attempt, I'd still not call it readable and opt for a different implementation, not using regular expressions.
P.S. Although I did some testing with my regex, I'm not at all sure that it covers all relevant cases correctly.
P.P.S. There are the \uxxxx escapes, not yet covered by the regex.
I was wondering about regex in Java and stumbled upon the use of backslashes. For instance, if I wanted to look for occurences of the words "this regex" in a text, I would do something like this:
Pattern.compile("this regex");
Nonetheless, I could also do something like this:
Pattern.compile("this\\sregex");
My question is: what is the difference between the two of them? And why do I have to type the backslash twice, I mean, why isn't \s an escape sequence in Java? Thanks in advance!
\s means any whitespace character, including tab, line feed and carriage return.
Java string literals already use \ to escape special characters. To put the character \ in a string literal, you need to write "\\". However regex patterns also use \ as their escape character, and the way to put that into a string literal is to use two, because it goes through two separate escaping processes. If you read your regex pattern from a plain text file for example, you won't need double escaping.
The reason you need two backslashes is that when you enter a regex string in Java code you are actually dealing with two parsers:
The first is the Java compiler, which is converting your string literal to a Java String.
The second is the regex parser, which is interpreting your regex, after it has been converted to a Java string and then passed to the regex parse when you call Pattern.compile.
So when you input "this\\sregex", it will be converted to the Java string "this\sregex" by the Java compiler. Then when you call Pattern.compile with the string, the backslash will be interpreted by the regex compiler as a special character.
The difference is that \s denotes a whitespace character, which can be more than just a blank space. It can be a tab, newline, line feed, to name a few.
I recently noticed that, String.replaceAll(regex,replacement) behaves very weirdly when it comes to the escape-character "\"(slash)
For example consider there is a string with filepath - String text = "E:\\dummypath"
and we want to replace the "\\" with "/".
text.replace("\\","/") gives the output "E:/dummypath" whereas text.replaceAll("\\","/") raises the exception java.util.regex.PatternSyntaxException.
If we want to implement the same functionality with replaceAll() we need to write it as,
text.replaceAll("\\\\","/")
One notable difference is replaceAll() has its arguments as reg-ex whereas replace() has arguments character-sequence!
But text.replaceAll("\n","/") works exactly the same as its char-sequence equivalent text.replace("\n","/")
Digging Deeper:
Even more weird behaviors can be observed when we try some other inputs.
Lets assign text="Hello\nWorld\n"
Now,
text.replaceAll("\n","/"), text.replaceAll("\\n","/"), text.replaceAll("\\\n","/") all these three gives the same output Hello/World/
Java had really messed up with the reg-ex in its best possible way I feel! No other language seems to have these playful behaviors in reg-ex. Any specific reason, why Java messed up like this?
You need to esacpe twice, once for Java, once for the regex.
Java code is
"\\\\"
makes a regex string of
"\\" - two chars
but the regex needs an escape too so it turns into
\ - one symbol
#Peter Lawrey's answer describes the mechanics. The "problem" is that backslash is an escape character in both Java string literals, and in the mini-language of regexes. So when you use a string literal to represent a regex, there are two sets of escaping to consider ... depending on what you want the regex to mean.
But why is it like that?
It is a historical thing. Java originally didn't have regexes at all. The syntax rules for Java String literals were borrowed from C / C++, which also didn't have built-in regex support. Awkwardness of double escaping didn't become apparent in Java until they added regex support in the form of the Pattern class ... in Java 1.4.
So how do other languages manage to avoid this?
They do it by providing direct or indirect syntactic support for regexes in the programming language itself. For instance, in Perl, Ruby, Javascript and many other languages, there is a syntax for patterns / regexs (e.g. '/pattern/') where string literal escaping rules do not apply. In C# and Python, they provide an alternative "raw" string literal syntax in which backslashes are not escapes. (But note that if you use the normal C# / Python string syntax, you have the Java problem of double escaping.)
Why do text.replaceAll("\n","/"), text.replaceAll("\\n","/"), and text.replaceAll("\\\n","/") all give the same output?
The first case is a newline character at the String level. The Java regex language treats all non-special characters as matching themselves.
The second case is a backslash followed by an "n" at the String level. The Java regex language interprets a backslash followed by an "n" as a newline.
The final case is a backslash followed by a newline character at the String level. The Java regex language doesn't recognize this as a specific (regex) escape sequence. However in the regex language, a backslash followed by any non-alphabetic character means the latter character. So, a backslash followed by a newline character ... means the same thing as a newline.
1) Let's say you want to replace a single \ using Java's replaceAll method:
\
˪--- 1) the final backslash
2) Java's replaceAll method takes a regex as first argument. In a regex literal, \ has a special meaning, e.g. in \d which is a shortcut for [0-9] (any digit). The way to escape a metachar in a regex literal is to precede it with a \, which leads to:
\ \
| ˪--- 1) the final backslash
|
˪----- 2) the backslash needed to escape 1) in a regex literal
3) In Java, there is no regex literal: you write a regex in a string literal (unlike JavaScript for example, where you can write /\d+/). But in a string literal, \ also has a special meaning, e.g. in \n (a new line) or \t (a tab). The way to escape a metachar in a string literal is to precede it with a \, which leads to:
\\\\
|||˪--- 1) the final backslash
||˪---- 3) the backslash needed to escape 1) in a string literal
|˪----- 2) the backslash needed to escape 1) in a regex literal
˪------ 3) the backslash needed to escape 2) in a string literal
This is because Java tries to give \ a special meaning in the replacement string, so that \$ will be a literal $ sign, but in the process they seem to have removed the actual special meaning of \
While text.replaceAll("\\\\","/"), at least can be considered to be okay in some sense (though it itself is not absolutely right), all the three executions, text.replaceAll("\n","/"), text.replaceAll("\\n","/"), text.replaceAll("\\\n","/") giving same output seem even more funny. It is just contradicting as to why they have restricted the functioning of text.replaceAll("\\","/") for the same reason.
Java didn't mess up with regular expressions. It is because, Java likes to mess up with coders by trying to do something unique and different, when it is not at all required.
One way around this problem is to replace backslash with another character, use that stand-in character for intermediate replacements, then convert it back into backslash at the end. For example, to convert "\r\n" to "\n":
String out = in.replace('\\','#').replaceAll("#r#n","#n").replace('#','\\');
Of course, that won't work very well if you choose a replacement character that can occur in the input string.
I think java really messed with regular expression in String.replaceAll();
Other than java I have never seen a language parse regular expression this way. You will be confused if you have used regex in some other languages.
In case of using the "\\" in replacement string, you can use java.util.regex.Matcher.quoteReplacement(String)
String.replaceAll("/", Matcher.quoteReplacement("\\"));
By using this Matcher class you can get the expected result.
What is the best approach if for instance a question mark is expected in a String.
...[?]...
or
...\?...
Example:
The text bla?bla will match both with the pattern bla[?]bla and bla\?bla (bot not bla?bla obviously) but is there any reason to use one over the other?
There is no technical reason to prefer one over the other: They are equivalent expressions. The character class is only used to avoid entering a backslash, so IMHO the escaped version is "cleaner"
However the reason may be to avoid double-escaping the slash on input. In languages like java, the literal version of the escaped version would look like this:
// in java you need to escape a backslash with another backslash :(
String regex = "...\\?...";
It could be that wherever the regexes are coming from has a similar issue and it's easier to read [?] than \\?
This is a "what the heck is going on here" question. I don't actually need a solution.
I had to replace all single backslashes in a String with double backslashes . This is what I ended up doing...
strRootDirectory = strRootDirectory.replaceAll("\\\\", "\\\\\\\\");
...where strRootDirectory is a java.lang.String above.
Now, I understand the four backslashes for the first argument: regex expects two backslashes in order to indicate a single literal backslash, and java wants them doubled up. That's fine.
BUT, what the heck is going on with the eight backslashes for the second argument? Isn't the replacement string supposed to be a literal (non-regex, I mean) string? I expected to need four backslashes in the second argument, in order to represent two backslashes.
The second argument isn't a regex-string, but a regex-replacement-string, in which the backslash also has a special meaning (it is used to escape the special character $ used for variable interpolation and is also used to escape itself).
From The API:
Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string; see Matcher.replaceAll. Use Matcher.quoteReplacement(java.lang.String) to suppress the special meaning of these characters, if desired.
-- http://download.oracle.com/javase/6/docs/api/java/lang/String.html#replaceAll(...)
It's easier if you use replace("\\","\\\\") (String.replace takes literal strings and is more efficient when it's all literal)
or you can ensure correctness through the Pattern.quote and Matcher.quoteReplacement functions
"\\\\\\\\" leads to an in memory representation of a string with 4 backslashes: \\\\. Although the second string isn't a regex string, backslashes and dollar signs are still special characters in it, so they need to be escaped.
According to Java reference material, the replaceAll method interprets backslashes in the replacement string as escape characters too. They could be used to escape the dollar sign character, which could refer to matched expressions to re-use in the replacement string. so naturally, if you want to double the number of backslashes, and both parameters treat backslash as an escape character, you need twice as many backslashes in the replacement string.
Yep, it gets hairy when you need to do this sort of thing, doesn't it.
The reason you need so many backslashes is that you need to take into account that backslash is used for both escaping a string and for escaping a regex.
Take 1 backslash.
Double it for string escaping.
Double it again for regex escaping.
Double it again because you need to match two consecutive backslashes in your original string.
That makes 8.
As a fan of not getting into super detailed explanations of regex... I figured out from the major answer post by Bart Kiers above:
System.out.println( "line1: "+"hello\\\\world" );
System.out.println( "line2: "+"hello\\\\world".replaceAll("\\\\\\\\", Matcher.quoteReplacement("\\") ) );
prints out
line1: hello\\world
line2: hello\world
I hope it helps...