We Keep Coding

Search substring in a string using regex

I'm trying to search for a set of words, contained within an ArrayList(terms_1pers), inside a string and, since the precondition is that before and after the search word there should be no letters, I thought of using expression regular.
I just don't know what I'm doing wrong using the matches operator. In the code reported, if the matching is not verified, it writes to an external file.
String url = csvRecord.get("url");
String text = csvRecord.get("review");
String var = null;
for(String term : terms_1pers)
{
if(!text.matches("[^a-z]"+term+"[^a-z]"))
{
var="true";
}
}
if(!var.equals("true"))
{
bw.write(url+";"+text+"\n");
}

In order to find regex matches, you should use the regex classes. Pattern and Matcher.
String term = "term";
ArrayList<String> a = new ArrayList<String>();
a.add("123term456"); //true
a.add("A123Term5"); //false
a.add("term456"); //true
a.add("123term"); //true
Pattern p = Pattern.compile("^[^A-Za-z]*(" + term + ")[^A-Za-z]*$");
for(String text : a) {
Matcher m = p.matcher(text);
if (m.find()) {
System.out.println("Found: " + m.group(1) );
//since the term you are adding is the second matchable portion, you're looking for group(1)
}
else System.out.println("No match for: " + term);
}
}
In the example there, we create an instance of a https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html to find matches in the text you are matching against.
Note that I adjusted the regex a bit. The choice in this code excludes all letters A-Z and the lowercase versions from the initial matching part. It will also allow for situations where there are no characters at all before or after the match term. If you need to have something there, use + instead of *. I also limited the regex to force the match to only contain matches for these three groups by using ^ and $ to verify end the end of the matching text. If this doesn't fit your use case, you may need to adjust.
To demonstrate using this with a variety of different terms:
ArrayList<String> terms = new ArrayList<String>();
terms.add("term");
terms.add("the book is on the table");
terms.add("1981 was the best year ever!");
ArrayList<String> a = new ArrayList<String>();
a.add("123term456");
a.add("A123Term5");
a.add("the book is on the table456");
a.add("1##!231981 was the best year ever!9#");
for (String term: terms) {
Pattern p = Pattern.compile("^[^A-Za-z]*(" + term + ")[^A-Za-z]*$");
for(String text : a) {
Matcher m = p.matcher(text);
if (m.find()) {
System.out.println("Found: " + m.group(1) + " in " + text);
//since the term you are adding is the second matchable portion, you're looking for group(1)
}
else System.out.println("No match for: " + term + " in " + text);
}
}
Output for this is:
Found: term in 123term456
No match for: term in A123Term5
No match for: term in the book is on the table456....
In response to the question about having String term being case insensitive, here's a way that we can build a string by taking advantage of java.lang.Character to options for upper and lower case letters.
String term = "This iS the teRm.";
String matchText = "123This is the term.";
StringBuilder str = new StringBuilder();
str.append("^[^A-Za-z]*(");
for (int i = 0; i < term.length(); i++) {
char c = term.charAt(i);
if (Character.isLetter(c))
str.append("(" + Character.toLowerCase(c) + "|" + Character.toUpperCase(c) + ")");
else str.append(c);
}
str.append(")[^A-Za-z]*$");
System.out.println(str.toString());
Pattern p = Pattern.compile(str.toString());
Matcher m = p.matcher(matchText);
if (m.find()) System.out.println("Found!");
else System.out.println("Not Found!");
This code outputs two lines, the first line is the regex string that's being compiled in the Pattern. "^[^A-Za-z]*((t|T)(h|H)(i|I)(s|S) (i|I)(s|S) (t|T)(h|H)(e|E) (t|T)(e|E)(r|R)(m|M).)[^A-Za-z]*$" This adjusted regex allows for letters in the term to be matched regardless of case. The second output line is "Found!" because the mixed case term is found within matchText.

There are several things to note:
matches requires a full string match, so [^a-z]term[^a-z] will only match a string like :term.. You need to use .find() to find partial matches
If you pass a literal string to a regex, you need to Pattern.quote it, or if it contains special chars, it will not get matched
To check if a word has some pattern before or after or at the start/end, you should either use alternations with anchors (like (?:^|[^a-z]) or (?:$|[^a-z])) or lookarounds, (?<![a-z]) and (?![a-z]).
To match any letter just use \p{Alpha} or - if you plan to match any Unicode letter - \p{L}.
The var variable is more logical to set to Boolean type.
Fixed code:
String url = csvRecord.get("url");
String text = csvRecord.get("review");
Boolean var = false;
for(String term : terms_1pers)
{
Matcher m = Pattern.compile("(?<!\\p{L})" + Pattern.quote(term) + "(?!\\p{L})").matcher(text);
// If the search must be case insensitive use
// Matcher m = Pattern.compile("(?i)(?<!\\p{L})" + Pattern.quote(term) + "(?!\\p{L})").matcher(text);
if(!m.find())
{
var = true;
}
}
if (!var) {
bw.write(url+";"+text+"\n");
}

you did not consider the case where the start and end may contain letters
so adding .* at the front and end should solve your problem.
for(String term : terms_1pers)
{
if( text.matches(".*[^a-zA-Z]+" + term + "[^a-zA-Z]+.*)" )
{
var="true";
break; //exit the loop
}
}
if(!var.equals("true"))
{
bw.write(url+";"+text+"\n");
}

How to replace excessive SQL wildcard by single regex pattern?

I am creating a function that strips the illegal wildcard patterns from the input string. The ideal solution should use a single regex expression, if at all possible.
The illegal wildcard patterns are: %% and %_%. Each instance of those should be replaced with %.
Here's the rub... I'm trying to perform some fuzz testing by running the function against various inputs to try to make it and break it.
It works for the most part; however, with complicated inputs, it doesn't.
The rest of this question has been updated:
The following inputs should return empty string (not an exhaustive list):
The following inputs should return % (not an exhaustive list).
%_%
%%
%%_%%
%_%%%
%%_%_%
%%_%%%_%%%_%
There will be cases where there are other characters with the input... like:
Foo123%_%
Should return "Foo123%"
B4r$%_%
Should return "B4r$%"
B4rs%%_%
Should return "B4rs%"
%%Lorem_%%
Should return "%Lorem_%"
I have tried using several different patterns and my tests are failing.
String input = "%_%%%%_%%%_%";
// old method:
public static String ancientMethod1(String input){
if (input == null)
return "";
return input.replaceAll("%_%", "").replaceAll("%%", ""); // Output: ""
}
// Attempt 1:
// Doesn't quite work right.
// "A%%" is returned as "A%%" instead of "A%"
public static String newMethod1(String input) {
String result = input;
while (result.contains("%%") || result.contains("%_%"))
result = result.replaceAll("%%","%").replaceAll("%_%","%");
if (result.equals("%"))
return "";
return input;
}
// Attempt 2:
// Succeeds, but I would like to simplify this:
public static String newMethod2(String input) {
if (input == null)
return "";
String illegalPattern1 = "%%";
String illegalPattern2 = "%_%";
String result = input;
while (result.contains(illegalPattern1) || result.contains(illegalPattern2)) {
result = result.replace(illegalPattern1, "%");
result = result.replace(illegalPattern2, "%");
}
if (result.equals("%") || result.equals("_"))
return "";
return result;
}
Here's a more complete defined example of how I'm using this: https://gist.github.com/sometowngeek/697c839a1bf1c9ee58be283b1396cf2e

This regular expression string matches all your examples:
"%(?:_?%)+"
It matches strings consisting of a '%' character followed by one or more sequences consisting of zero or one '_' character and one '%' character (close to literal translation), which is another way of saying what I did in comments: "a sequence of '%' and '_' characters, beginning and ending with '%', and not containing two consecutive '_' characters".

I'm not quite sure, if the listed inputs might have other instances, if not, maybe an expression with start and end anchor would be much applicable here, either one by one, or with something similar to:
^%{1,3}(_%{1,3})?(_%{1,3})?(_%)?$
Demo
Test
import java.util.regex.Matcher;
import java.util.regex.Pattern;
final String regex = "^%{1,3}(_%{1,3})?(_%{1,3})?(_%)?$";
final String string = "%_%\n"
+ "%%\n"
+ "%%_%%\n"
+ "%%%_%%%\n"
+ "%_%%%\n"
+ "%%%_%\n"
+ "%%_%_%\n"
+ "%%_%%%_%%%_%";
final Pattern pattern = Pattern.compile(regex, Pattern.MULTILINE);
final Matcher matcher = pattern.matcher(string);
while (matcher.find()) {
System.out.println("Full match: " + matcher.group(0));
for (int i = 1; i <= matcher.groupCount(); i++) {
System.out.println("Group " + i + ": " + matcher.group(i));
}
}
RegEx Circuit
jex.im visualizes regular expressions:

Your newMethod1 actually works, except you have a typo - you're returning the input parmeter, not the result of your processing!
Change:
return input; // oops!
to:
return result;
Also, because you're not using regex, you should use replace() rather than replaceAll(), ie:
result = result.replace("%%","%").replace("%_%","%"); // still replaces all occurrences
replace() still replaces all occurrences.
BTW, although not as strict, this works for all of your (currently) posted examples:
public static String myMethod(String input) {
return input.replaceAll("%[%_]*", "%");
}

It looks like all the patterns start with %, then have 0+ % or _ chars and end with %.
Use a mere
input = input.replaceAll("%[%_]*%", "%");
See the regex demo and the regex graph:
Details
% - a % char
[%_]* - 0 or more % or _ chars
% - a % char.

Parsing a string with [3:0] substring in it

I want to store two numbers from a string into two distinct variables - for example, var1 = 3 and var2 = 0 from "[3:0]". I have the following code snippet:
String myStr = "[3:0]";
if (myStr.trim().matches("\\[(\\d+)\\]")) {
// Do something.
// If it enter the here, here I want to store 3 and 0 in different variables or an array
}
Is it possible doing this with split and regular expressions?

Don't call trim(). Enhance you regex instead.
Your regex is missing the pattern for : and the second number, and you don't need to escape the ].
To capture the matched numbers, you need the Matcher:
String myStr = " [3:0] ";
Matcher m = Pattern.compile("\\s*\\[(\\d+):(\\d+)]\\s*").matcher(myStr);
if (m.matches())
System.out.println(m.group(1) + ", " + m.group(2));
Output
3, 0

You can use replaceAll and split
String myStr = "[3:0]";
if(myStr.trim().matches("\\[\\d+:\\d+\\]") {
String[] numbers = myStr.replaceAll("[\\[\\]]","").split(":");
}
Moreover, your regExp to match String should be \\[\\d+:\\d+\\], if you want to avoid trim you can add \\s+ at start and end to match the spaces.But trim is not bad.
EDIT
As suggested by Andreas in comments,
String myStr = "[3:0]";
String regExp = "\\[(\\d+):(\\d+)\\]";
Pattern pattern = Pattern.compile(regExp);
Matcher matcher = pattern.matcher(myStr.trim());
if(matcher.find()) {
int a = Integer.parseInt(matcher.group(1));
int b = Integer.parseInt(matcher.group(2));
System.out.println(a + " : " + b);
}
OUTPUT
3 : 0

Without any regular expressions you could do this:
// this will remove the braces [ and ] and just leave "3:0"
String numberString= myString.trim().replace("[", "").replace("]","");
// this will split the string in everything before the : and everything after the : (so two values as an array)
String[] numbers = numberString.split(":");
// get the first value and parse it as a number "3" will become a simple 3
int firstNumber = Integer.parseInt(numbers[0]) ;
// get the second value and parse it from "0" to a plain 0
int secondNumber = Integer.parseInt(numbers[1]);
be carefull when parsing numbers, depending on your input string and what other possibilities there might be (e.g. "3:12" is ok, but "3:02" might throw an error).

In case you don't need to validate input and you want to simply get numbers from it, you could simply find indexOf(":") and substring parts which you are interested, in which are:
from [ (which is at position 0) till :
and from index of : till ] (which is at position equal to length of string -1)
Your code can look like
String text = "[3:0]";
int colonIndex = text.indexOf(':');
String first = text.substring(1, colonIndex);
String second = text.substring(colonIndex + 1, text.length() - 1);

Java Regex finding operators

I'm trying to use regex to get numbers and operators from a string containing an expression. It finds the numbers but i doesn't find the operators. After every match (number or operator) at the beginning of the string it truncates the expression in order to find the next one.
String expression = "23*12+11";
Pattern intPattern;
Pattern opPattern;
Matcher intMatch;
Matcher opMatch;
intPattern = Pattern.compile("^\\d+");
intMatch = intPattern.matcher(expression);
opPattern = Pattern.compile("^[-+*/()]+");
opMatch = opPattern.matcher(expression);
while ( ! expression.isEmpty()) {
System.out.println("New expression: " + expression);
if (intMatch.find()) {
String inputInt = intMatch.group();
System.out.println(inputInt);
System.out.println("Found at index: " + intMatch.start());
expression = expression.substring(intMatch.end());
intMatch = intPattern.matcher(expression);
System.out.println("Truncated expression: " + expression);
} else if (opMatch.find()) {
String nextOp = opMatch.group();
System.out.println(nextOp);
System.out.println("Found at index: " + opMatch.start());
System.out.println("End index: " + opMatch.end());
expression = expression.substring(opMatch.end());
opMatch = opPattern.matcher(expression);
System.out.println("Truncated expression: " + expression);
} else {
System.out.println("Last item: " + expression);
break;
}
}
The output is
New expression: 23*12+11
23
Found at index: 0
Truncated expression: *12+11
New expression: *12+11
Last item: *12+11
As far as I have been able to investigate there is no need to escape the special characters *, + since they are inside a character class. What's the problem here?

First, your debugging output is confusing, because it's exactly the same in both branches. Add something to distinguish them, such as an a and b prefix:
System.out.println("a.Found at index: " + intMatch.start());
Your problem is that you're not resetting both matchers to the updated string. At the end of both branches in your if-else (or just once, after the entire if-else block), you need to do this:
intMatch = intPattern.matcher(expression);
opMatch = opPattern.matcher(expression);
One last thing: Since you're creating a new matcher over and over again via Pattern.matcher(s), you might want to consider creating the matcher only once, with a dummy-string, at the top of your code
//"": Unused string so matcher object can be reused
intMatch = Pattern.compile(...).matcher("");
and then resetting it in each loop iteration
intMatch.reset(expression);
You can implement the reusable Matchers like this:
//"": Unused to-search strings, so the matcher objects can be reused.
Matcher intMatch = Pattern.compile("^\\d+").matcher("");
Matcher opMatch = Pattern.compile("^[-+*/()]+").matcher("");
String expression = "23*12+11";
while ( ! expression.isEmpty()) {
System.out.println("New expression: " + expression);
intMatch.reset(expression);
opMatch.reset(expression);
if(intMatch.find()) {
...
The
Pattern *Pattern = ...
lines can be removed from the top, and the
*Match = *Pattern.matcher(expression)
lines can be removed from both if-else branches.

Your main problem is that when you found int you or operator you are reassigning only intMatch or opMatch. So if you find int operator is still try to find match on old version of expression. So you need to place this lines in both your positive cases
intMatch = intPattern.matcher(expression);
opMatch = opPattern.matcher(expression);
But maybe instead of your approach with two Patterns and recreating expression just use one regex which will find ints or operators and place them in different group categories? I mean something like
String expression = "23*12+11";
Pattern p = Pattern.compile("(\\d+)|([-+*/()]+)");
Matcher m = p.matcher(expression);
while (m.find()){
if (m.group(1)==null){//group 1 is null so match must come from group 2
System.out.println("opperator found: "+m.group(2));
}else{
System.out.println("integer found: "+m.group(1));
}
}
Also if you don't need to separately handle integers and operators you can just split on places before and after operators using look-around mechanisms
String expression = "23*12+11";
for (String s : expression.split("(?<=[-+*/()])|(?=[-+*/()])"))
System.out.println(s);
Output:
23
*
12
+
11

Try this one
Note:You have missed modulus % operator
String expression = "2/3*1%(2+11)";
Pattern pt = Pattern.compile("[-+*/()%]");
Matcher mt = pt.matcher(expression);
int lastStart = 0;
while (mt.find()) {
if (lastStart != mt.start()) {
System.out.println("number:" + expression.substring(lastStart, mt.start()));
}
lastStart = mt.start() + 1;
System.out.println("operator:" + mt.group());
}
if (lastStart != expression.length()) {
System.out.println("number:" + expression.substring(lastStart));
}
output
number:2
operator:/
number:3
operator:*
number:1
operator:%
operator:(
number:2
operator:+
number:11
operator:)

regular expression for file name

I have files in the format *C:\Temp\myfile_124.txt*
I need a regular expression which will give me just the number "124" that is whatever is there after the underscore and before the extension.
I tried a number of ways, latest is
(.+[0-9]{18,})(_[0-9]+)?\\.txt$
I am not getting the desired output. Can someone tell me what is wrong?
Matcher matcher = FILE_NAME_PATTERN.matcher(filename);
if (matcher.matches() && matcher.groupCount() == 2) {
try {
String index = matcher.group(2);
if (index != null) {
return Integer.parseInt(index.substring(1));
}
}
catch (NumberFormatException e) {
}

The first part [0-9]{18,} states you have atleast 18 digits which you don't have.
Usually with regex its a good idea to make the expression as simple as possible. I suggest trying
_([0-9]+)?\\.txt$
Note: you have to call find() to make it perform the lookup, otherwise it says "No match found"
This example
String s = "C:\\Temp\\myfile_124.txt";
Pattern p = Pattern.compile("_(\\d+)\\.txt$");
Matcher matcher = p.matcher(s);
if (matcher.find())
for (int i = 0; i <= matcher.groupCount(); i++)
System.out.println(i + ": " + matcher.group(i));
prints
0: _124.txt
1: 124

This may work for you: (?:.*_)(\d+)\.txt
The result is in the match group.
This one uses positive lookahead and will only match the number: \d+(?=\.txt)

.*_([1-9]+)\.[a-zA-Z0-9]+
The group 1 will contain the desired output.
Demo

You can do this
^.*_\([^\.]*\)\..*$

.*_([0-9]+)\.txt
This should work too. Of course you should double escape for Java.