Is there a way to find of most precise regex for a string?
For e.g.
Lets say, I have 2 regex:
1) .*bourne
2) .*ne
If I try to match Melbourne with the above regex, it will match with both regex.
But more precise match will be the first regex. Similarly, there can be very complex regex.
Is there a way to find the most precise match?
Is there a way to find the most precise match?
The most "precise" match is the the one where the regex needs to process less data until it finds a match, in this case, .*bourne.
Wouldn't sorting the patterns in descending order of length solve the problem ?
For example, if Java is the language being used something like the following should be fine right (just sort the pattern in descending order of length and then return for first match)?
public class TestPattern {
public static void main(String args[]){
String text ="Melbourne";
System.out.println("Mtaching regex --> "+getMatchingRegex(text));
}
public static String getMatchingRegex(String text) {
ArrayList<String> patterns = new ArrayList<String>();
patterns.add(".*ne") ;
patterns.add(".*urne") ;
patterns.add(".*bourne") ;
patterns.add(".*rne") ;
Collections.sort(patterns, new StringComparator());
for(String pattern:patterns) {
if(Pattern.matches(pattern, text))
return pattern;
}
return "No Regex matched";
}
public static class StringComparator implements Comparator<String>
{
#Override
public int compare(String s1, String s2)
{
return s2.length()-s1.length();
}
}
}
Related
I have a utility class to resolve a string input with certain patterns as shown in the example below. All variables are surrounded by { and }. If my string is something like Language is {lang} and version 2 is {version}. Home located at {java.home} the output is Language is java and version 2 is 1.8. Home located at C:/java and if my string is like Language is {lang} and version 2 is {version}. Home located at {{lang}.home} the output is Language is java and version 2 is 1.8. Home located at {java.home}. All I am trying to find is a way to resolve nested properties recursively but ran into several issues. Can any logic be inserted into the code so that resolving of inner properties happen dynamically?
import java.util.*;
import java.util.regex.*;
public class MyClass {
public static void main(String args[]) {
System.setProperty("lang" , "java");
System.setProperty("version" , "1.8");
System.setProperty("java.home" , "C:/java");
System.out.println(resolve("Language is {lang} and version 2 is {version}. Home located at {java.home}"));
System.out.println(resolve("Language is {lang} and version 2 is {version}. Home located at {{lang}.home}"));
}
public static String resolve(String input) {
List<String> tokens = matchers("[{]\\S+[}]", input);
String value;
for(String token : tokens) {
value = getProperty(token);
if (null != value) {
input = input.replace(token, value);
}
value = "";
}
return input;
}
private static String getProperty(String key) {
key = key.substring(1, key.length()-1);
return System.getProperty(key);
}
public static List<String> matchers(String regex, String text) {
List<String> matches = new ArrayList<String>();
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(text);
while (matcher.find()) {
matches.add(matcher.group());
}
return matches;
}
public static boolean contains(String regex, String text) {
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(text);
return matcher.find();
}
}
You just have to ask for the pattern to get only the value without an inner { or } with [^{}]. No "curly bracket" means no inner values. So you can safely do the replace.
First, we create a Pattern, we need to escape those {}... and we add a capture group for later.
Pattern p = Pattern.compile("\\{([^{}]+)\\}");
Then we check with the current value:
Matcher m = p.matcher(s);
Now, we just have to check if there is a match and loop on it.
while( m.find() ){
...
}
In there, we will need the value captured, so we get the first group and get its value (let assume it will always be present) :
String key = m.group(1);
String value = properties.get(key); //add some fail safe.
Using the Matcher.replaceFirst, we will safely replace only the current match (the one we get the value from). If you use replaceAll, it will replace every pattern with the same value.
s = m.replaceFirst(properties.get(key));
Now, since we have updated the String, we need to call check the regex again :
m = p.matcher(s);
Here is a full example:
Map<String, String> properties = new HashMap<>();
properties.put("lang", "java");
properties.put("java.version", "1.8");
String s = "This is {{lang}.version}.";
Pattern p = Pattern.compile("\\{([^{}]+)\\}");
Matcher m = p.matcher(s);
while(m.find()){
String key = m.group(1);
s = m.replaceFirst(properties.get(key));
System.out.println(s);
m = p.matcher(s); //Reset the matcher
}
This is {java.version}.
This is 1.8.
This has one problem, it will required to a lot of Matcher initialisation, so it might not be optimal. Of course, it is most likely not optimized (not the point here)
FYI : Using the Matcher.replaceFirst instead of the String.replaceFirst prevent a new Pattern compilation to be done. Here is the String.replaceFirst code :
public String replaceFirst(String regex, String replacement) {
return Pattern.compile(regex).matcher(this).replaceFirst(replacement);
}
We already have a Matcher to do that, so use it.
There are lots of ways you could achieve this.
You need some way to communicate to the caller either whether a replacement is necessary, or whether one was made.
A simple option:
public boolean hasPlaceholder(String s) {
// return true if s contains a {} placeholder, else false
}
Using this you can repeatedly replace until done:
while(hasPlaceholder(s)) {
s = replacePlaceholders(s);
}
This does scan through the string more times than is strictly necessary, but you shouldn't optimise prematurely.
A more sophisticated option is for the replacePlaceholders() method to report back whether it succeeded. For that you'll need a response class that wraps the result String and the wasReplaced() boolean:
ReplacementResult replacePlaceholders(String s) {
// process string into newString, counting placeholders replaced
return new ReplacementResult(count > 0, newString);
}
(Implementation of ReplacementResult left as an exercise)
Using this you can do:
ReplacementResult result = replacePlaceholders(s);
while(result.wasReplaced()) {
result = replacePlaceholders(result.string());
}
So, each time you call replacePlaceholders() it will either make at least one replacement, or it will report false having verified that there are no more replacements to make.
You mention recursion in the question. This can of course be done, and it would mean avoiding scanning through the whole string each time -- as you can look at just the replacement fragment. This is untested Java-like pseudocode:
String replaceRecursively(String s) {
StringBuilder result = new StringBuilder();
while(Token token = takeTokenFrom(s)) {
if(token.isPlaceholder()) {
String rawReplacement = lookupReplacement(token);
String processedReplacement = replaceRecursively(rawReplacement);
result.append(processedReplacement);
} else {
result.append(token.text());
}
}
return result.toString();
}
For all of these solutions, you should beware of infinite loops or stack-blowing recursion. What if you replace "{foo}" with "{foo}"? (or worse, what if you replace "{foo}" with "{foo}{foo}"!?).
Of course the simplest way is to be in control of the configuration, and simply not trigger that problem. Detecting the problem programatically is entirely possible, but complex enough that it would warrant another SO question if you want it.
I am using guava 21.0 and trying to split a String by providing a regex(\\d).
However,I am not sure why is not working.
If I change regex to anything which is not regex (eg "a") then it works fine.
Here is the code :
public class SplitWithRegex {
public static Iterable<String> splitByRegex(String string, String regex){
return Splitter.on(regex).trimResults().omitEmptyStrings().split(string);
}
public static void main(String[] args) {
Iterable<String> itr = splitByRegex("abc243gca87asas**78sassnb32snb1ss22220220", "\\d");
for(String s : itr){
System.out.println(s);
}
}
}
Result when regex is applied :
abc243gca87asas**78sassnb32snb1ss22220220
Any help would be appreciated.
You must use Splitter.onPattern("\\d+") and not Splitter.on("\\d+").
Here's the javadoc for Splitter's on method, this is what it says:
Returns a splitter that uses the given fixed string as a separator.
For example, Splitter.on(", ").split("foo, bar,baz") returns an
iterable containing ["foo", "bar,baz"].
So, separator is a treated as String literal and not regex and hence, it does not split the String as expected. If you want regex based splitting then you can use String's split method or Splitter's onPattern method, e.g.:
String[] tokens = "abc243gca87asas**78sassnb32snb1ss22220220".split("\\d+");
for(String token : tokens){
System.out.println(token);
}
public static Iterable<String> splitByRegex(String string, String regex){
return Splitter.onPattern(regex).trimResults().omitEmptyStrings().split(string);
}
Firstly, I'm aware of similar questions that have been asked such as here:
How to split a string, but also keep the delimiters?
However, I'm having issue implementing a split of a string using Pattern.split() where the pattern is based on a list of delimiters, but where they can sometimes appear to overlap. Here is the example:
The goal is to split a string based on a set of known codewords which are surrounded by slashes, where I need to keep both the delimiter (codeword) itself and the value after it (which may be empty string).
For this example, the codewords are:
/ABC/
/DEF/
/GHI/
Based on the thread referenced above, the pattern is built as follows using look-ahead and look-behind to tokenise the string into codewords AND values:
((?<=/ABC/)|(?=/ABC/))|((?<=/DEF/)|(?=/DEF/))|((?<=/GHI/)|(?=/GHI/))
Working string:
"123/ABC//DEF/456/GHI/789"
Using split, this tokenises nicely to:
"123","/ABC/","/DEF/","456","/GHI/","789"
Problem string (note single slash between "ABC" and "DEF"):
"123/ABC/DEF/456/GHI/789"
Here the expectation is that "DEF/456" is the value after "/ABC/" codeword because the "DEF/" bit is not actually a codeword, but just happens to look like one!
Desired outcome is:
"123","/ABC/","DEF/456","/GHI/","789"
Actual outcome is:
"123","/ABC","/","DEF/","456","/GHI/","789"
As you can see, the slash between "ABC" and "DEF" is getting isolated as a token itself.
I've tried solutions as per the other thread using only look-ahead OR look-behind, but they all seem to suffer from the same issue. Any help appreciated!
If you are OK with find rather than split, using some non-greedy matches, try this:
public class SampleJava {
static final String[] CODEWORDS = {
"ABC",
"DEF",
"GHI"};
static public void main(String[] args) {
String input = "/ABC/DEF/456/GHI/789";
String codewords = Arrays.stream(CODEWORDS)
.collect(Collectors.joining("|", "/(", ")/"));
// codewords = "/(ABC|DEF|GHI)/";
Pattern p = Pattern.compile(
/* codewords */ ("(DELIM)"
/* pre-delim */ + "|(.+?(?=DELIM))"
/* final bit */ + "|(.+?$)").replace("DELIM", codewords));
Matcher m = p.matcher(input);
while(m.find()) {
System.out.print(m.group(0));
if(m.group(1) != null) {
System.out.print(" ← code word");
}
System.out.println();
}
}
}
Output:
/ABC/ ← code word
DEF/456
/GHI/ ← code word
789
Use a combination of positive and negative look arounds:
String[] parts = s.split("(?<=/(ABC|DEF|GHI)/)(?<!/(ABC|DEF|GHI)/....)|(?=/(ABC|DEF|GHI)/)(?<!/(ABC|DEF|GHI))");
There's also a considerable simplification by using alternations inside single look ahead/behind.
See live demo.
Following some TDD principles (Red-Green-Refactor), here is how I would implement such behaviour:
Write specs (Red)
I defined a set of unit tests that explain how I understood your "tokenization process". If any test is not correct according to what you expect, feel free to tell me and I'll edit my answer accordingly.
import static org.assertj.core.api.Assertions.assertThat;
import java.util.List;
import org.junit.Test;
public class TokenizerSpec {
Tokenizer tokenizer = new Tokenizer("/ABC/", "/DEF/", "/GHI/");
#Test
public void itShouldTokenizeTwoConsecutiveCodewords() {
String input = "123/ABC//DEF/456";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("123", "/ABC/", "/DEF/", "456");
}
#Test
public void itShouldTokenizeMisleadingCodeword() {
String input = "123/ABC/DEF/456/GHI/789";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("123", "/ABC/", "DEF/456", "/GHI/", "789");
}
#Test
public void itShouldTokenizeWhenValueContainsSlash() {
String input = "1/23/ABC/456";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("1/23", "/ABC/", "456");
}
#Test
public void itShouldTokenizeWithoutCodewords() {
String input = "123/456/789";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("123/456/789");
}
#Test
public void itShouldTokenizeWhenEndingWithCodeword() {
String input = "123/ABC/";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("123", "/ABC/");
}
#Test
public void itShouldTokenizeWhenStartingWithCodeword() {
String input = "/ABC/123";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("/ABC/", "123");
}
#Test
public void itShouldTokenizeWhenOnlyCodeword() {
String input = "/ABC//DEF//GHI/";
List<String> tokens = tokenizer.splitPreservingCodewords(input);
assertThat(tokens).containsExactly("/ABC/", "/DEF/", "/GHI/");
}
}
Implement according to the specs (Green)
This class make all the tests above pass
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
public final class Tokenizer {
private final List<String> codewords;
public Tokenizer(String... codewords) {
this.codewords = Arrays.asList(codewords);
}
public List<String> splitPreservingCodewords(String input) {
List<String> tokens = new ArrayList<>();
int lastIndex = 0;
int i = 0;
while (i < input.length()) {
final int idx = i;
Optional<String> codeword = codewords.stream()
.filter(cw -> input.substring(idx).indexOf(cw) == 0)
.findFirst();
if (codeword.isPresent()) {
if (i > lastIndex) {
tokens.add(input.substring(lastIndex, i));
}
tokens.add(codeword.get());
i += codeword.get().length();
lastIndex = i;
} else {
i++;
}
}
if (i > lastIndex) {
tokens.add(input.substring(lastIndex, i));
}
return tokens;
}
}
Improve implementation (Refactor)
Not done at the moment (not enough time that I can spend on that answer now). I'll do some refactor on Tokenizer with pleasure if you request me to (but later). :-) Or you can do it yourself quite securely since you have the unit tests to avoid regressions.
I would like to do some simple String replace with a regular expression in Java, but the replace value is not static and I would like it to be dynamic like it happens on JavaScript.
I know I can make:
"some string".replaceAll("some regex", "new value");
But i would like something like:
"some string".replaceAll("some regex", new SomeThinkIDontKnow() {
public String handle(String group) {
return "my super dynamic string group " + group;
}
});
Maybe there is a Java way to do this but i am not aware of it...
You need to use the Java regex API directly.
Create a Pattern object for your regex (this is reusable), then call the matcher() method to run it against your string.
You can then call find() repeatedly to loop through each match in your string, and assemble a replacement string as you like.
Here is how such a replacement can be implemented.
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class RegExCustomReplacementExample
{
public static void main(String[] args)
{
System.out.println(
new ReplaceFunction() {
public String handle(String group)
{
return "«"+group.substring(1, group.length()-1)+"»";
}
}.replace("A simple *test* string", "\\*.*?\\*"));
}
}
abstract class ReplaceFunction
{
public String replace(String source, String regex)
{
final Pattern pattern = Pattern.compile(regex);
final Matcher m = pattern.matcher(source);
boolean result = m.find();
if(result) {
StringBuilder sb = new StringBuilder(source.length());
int p=0;
do {
sb.append(source, p, m.start());
sb.append(handle(m.group()));
p=m.end();
} while (m.find());
sb.append(source, p, source.length());
return sb.toString();
}
return source;
}
public abstract String handle(String group);
}
Might look a bit complicated at the first time but that doesn’t matter as you need it only once. The subclasses implementing the handle method look simpler. An alternative is to pass the Matcher instead of the match String (group 0) to the handle method as it offers access to all groups matched by the pattern (if the pattern created groups).
I have a string which contains an underscore as shown below:
123445_Lisick
I want to remove all the characters from the String after the underscore. I have tried the code below, it's working, but is there any other way to do this, as I need to put this logic inside a for loop to extract elements from an ArrayList.
public class Test {
public static void main(String args[]) throws Exception {
String str = "123445_Lisick";
int a = str.indexOf("_");
String modfiedstr = str.substring(0, a);
System.out.println(modfiedstr);
}
}
Another way is to use the split method.
String str = "123445_Lisick";
String[] parts = string.split("_");
String modfiedstr = parts[0];
I don't think that really buys you anything though. There's really nothing wrong with the method you're using.
Your method is fine. Though not explicitly stated in the API documentation, I feel it's safe to assume that indexOf(char) will run in O(n) time. Since your string is unordered and you don't know the location of the underscore apriori, you cannot avoid this linear search time. Once you have completed the search, extraction of the substring will be needed for future processing. It's generally safe to assume the for simple operations like this in a language which is reasonably well refined the library functions will have been optimized.
Note however, that you are making an implicit assumption that
an underscore will exist within the String
if there are more than one underscore in the string, all but the first should be included in the output
If either of these assumptions will not always hold, you will need to make adjustments to handle those situations. In either case, you should at least defensively check for a -1 returned from indexAt(char) indicating that '_' is not in the string. Assuming in this situation the entire String is desired, you could use something like this:
public static String stringAfter(String source, char delim) {
if(source == null) return null;
int index = source.indexOf(delim);
return (index >= 0)?source.substring(index):source;
}
You could also use something like that:
public class Main {
public static void main(String[] args) {
String str = "123445_Lisick";
Pattern pattern = Pattern.compile("^([^_]*).*");
Matcher matcher = pattern.matcher(str);
String modfiedstr = null;
if (matcher.find()) {
modfiedstr = matcher.group(1);
}
System.out.println(modfiedstr);
}
}
The regex groups a pattern from the start of the input string until a character that is not _ is found.
However as #Bill the lizard wrote, i don't think that there is anything wrong with the method you do it now. I would do it the same way you did it.