I'm having trouble getting list of all parameters in SQL query using Regex.
Example of the query:
SELECT ... WHERE col1 = :user AND col2 = 'HELLO' OR col3 = :language
To obtain parameters, I use following regex pattern:
Pattern.compile(":([\\w.$]+|\"[^\"]+\"|'[^']+')", Pattern.MULTILINE)
The pattern returns list of parameters correctly:
:user
:language
The problem is with another type of query, where literals might contain character ':'
WHERE col1 = :user AND some_date > '2022-09-26T10:22:55'
The list of parameters for this case is:
:user
:22
:55
Is there any better approach that will not consider contents of literals as parameters?
You could simplify your problem by assuming that a named param in sql is just a word with prefix : and always follows after a space (this is actually not a requirement or always true but might be just good enough to get you acceptable results with as simple of regex as possible)
Pattern.compile(" :\\w+", Pattern.MULTILINE)
--
summary of the comments:
had to match
- foo = :param AND :param = bar AND foo=:param AND :param=bar
- AND FUNC(:param) OR FUNC(0, :param) OR FUNC(:param, 0)
finally this regex with fixed length lookahead and variable length lookbehind was helpful:
Pattern.compile("(?<=[=(])\\s*:[\\w_.]+|:[\\w_.]+(?=\s*[=)])", Pattern.MULTILINE)
Related
I am using jdbi3 with StringTemplate 4 templating engine, I have this test query:
#SqlQuery("select * from test "
+ "where field1 = 5"
+ "<if(cond1)> or field2 \\<= :value1<endif>"
+ "<if(cond2)> or field2 >= :value2<endif>"
+ "<if(cond3)> or field2 in (<values>)<endif>")
#RegisterBeanMapper(Test.class)
#UseStringTemplateEngine
public List<Test> selectTest(
#Define("cond1") boolean cond1, #Bind("value1") int value2,
#Define("cond2") boolean cond2, #Bind("value2") int value3,
#Define("cond3") boolean cond3,
#BindList(value="values", onEmpty=BindList.EmptyHandling.NULL_STRING ) List<Integer> values);
Using StringTemplate engine when I have to escape with \ the characters < or > in the query?
Testing I found that I have to escape <= in the query like I did.
In the IN clause using #BindList I have to use the <values> but in this case I was expecting to escape it like \\<values> otherwise it will be used as attribute by StringTemplate but if I do this the query doesn't work.
About >= escaping it or not seems the same in the query.
Introduction
Let's consider:
The 3.27.1 Jdbi version as the current Jdbi version.
The 4.3.1 StringTemplate version as the current StringTemplate version, since the current Jdbi version uses it. Please, see: jdbi/pom.xml at v3.27.1 · jdbi/jdbi.
Answer
Jdbi
Documentation: Which characters to escape
Please, note the warning on which characters to escape:
Since StringTemplate by default uses the < character to mark ST expressions, you might need to escape some SQL: String datePredSql = "<if(datePredicate)> <dateColumn> \\< :dateFilter <endif>"
— jdbi/index.adoc at v3.27.1 · jdbi/jdbi.
Unit-test: Do not escape #BindList variable name
Please, see the BindListTest.ifValueGivenWithNullValueOptionThenResultIsTruthy() test method: jdbi/BindListTest.java at v3.27.1 · jdbi/jdbi.
Please, note that the test covers a very similar annotated method:
#SqlQuery("select name from something <if(name)> where name in (<name>) <endif>")
#UseStringTemplateEngine
List<String> getForValue(#Nonnull #BindList(value = "name", onEmpty = NULL_VALUE) List<String> name);
Please, note that the #BindList variable name is not escaped:
in (<name>)
StringTemplate
Documentation: Which characters to escape
Please, note which characters to escape:
A template is a sequence of text and expression elements, optionally interspersed with comments. At the coarsest level, the basic elements are:
text
<expr>
<! comment !>
Escape delimiters with a backslash character: \< or \>.
— stringtemplate4/templates.md at 4.3.1 · antlr/stringtemplate4.
I have a query that when given a word that starts with a one-letter word followed by space character and then another word (ex: "T Distribution"), does not return results. While given "Distribution" alone returns results including the results for "T Distribution". It is the same behavior with all search terms beginning with a one-letter word followed by space character and then another word.
The problem appears when the search term is of this pattern:
"[one-letter][space][letter/word]". example: "o ring".
What would be the problem that the LIKE operator not working correctly in this case?
Here is my query:
#Cacheable(value = "filteredConcept")
#Query("SELECT NEW sina.backend.data.model.ConceptSummaryVer04(s.id, s.arabicGloss, s.englishGloss, s.example, s.dataSourceId,
s.synsetFrequnecy, s.arabicWordsCache, s.englishWordsCache, s.superId, s.categoryId, s.dataSourceCacheAr, s.dataSourceCacheEn,
s.superTypeCasheAr, s.superTypeCasheEn, s.area, s.era, s.rank, s.undiacritizedArabicWordsCache, s.normalizedEnglishWordsCache,
s.isTranslation, s.isGloss, s.arabicSynonymsCount, s.englishSynonymsCount) FROM Concept s
where s.undiacritizedArabicWordsCache LIKE %:searchTerm% AND data_source_id != 200 AND data_source_id != 31")
List<ConceptSummaryVer04> findByArabicWordsCacheAndNotConcept(#Param("searchTerm") String searchTerm, Sort sort);
the result of the query on the database itself:
link to screenshot
results on the database are returned no matter the letters case:
link to screenshot
I solved this problem.
It was due to the default configuration of the Full-text index on mysql database which is by default set to 2 (ft_min_word_len = 2).
I changed that and rebuilt the index. Then, one-letter words were returned by the query.
12.9.6 Fine-Tuning MySQL Full-Text Search
Use some quotes:
LIKE '%:searchTerm%';
Set searchTerm="%your_word%" and use it on query like this :
... s.undiacritizedArabicWordsCache LIKE :searchTerm ...
I'm trying to find a way to match user search queries with a database records in a search engine, using Spring, but I'm having trouble when the search query includes special characters such as vowels with accent.
Eg: search query = 'cafe'. Database record = 'café'
I'm using the stem of words to the query with the database records.
Which would be the most straight forward way of matching the query including a special character 'café' with the string that doesn't contain this special character 'cafe' and viceversa?
UPDATE
All the information I need is already cached so the approach of creating a new column in the db is not so appealing. I'm looking for a solution more spring based.
You could use java.text.Normalizer, like follow:
import java.text.Normalizer;
import java.text.Normalizer.Form;
public static String removeAccents(String text) {
return text == null ? null :
Normalizer.normalize(text, Form.NFD)
.replaceAll("\\p{InCombiningDiacriticalMarks}+", "");
}
The Normalizer splits the original characters into a set of two character (letter and accent).
For example the character á (U+00E1) will be split in a (U+0061) and acute accent U+0301
The \p{InCombiningDiacriticalMarks}+ regular expression will match all such diacritic codes and we will replace them with an empty string.
And your query could be like:
SQL SERVER
SELECT * FROM Table
WHERE Column Like '%stringwithoutaccents%' COLLATE Latin1_general_CI_AI
ORACLE (from 10g)
SELECT * FROM Table
WHERE NLSSORT(Column, 'NLS_SORT = Latin_AI')
Like NLSSORT('%stringwithoutaccents%', 'NLS_SORT = Latin_AI')
The CI stands for "Case Insensitive" and AI for "Accent Insensitive".
I hope it helps you.
I have following hql query,
from Channe where ip='1.11.6.0';
But in the db the IP is saving as 1.11.6.0:8080 .
So I need to modify the query in a way that, split the ip with a delimiter ':' and take the firstcome value. I do not wish to modify the search with value 1.11.6.0:8080.
See this page in the Hibernate docs. On the page below there is a section called 14.10. Expressions
http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/queryhql.html
It says, among other things:
string concatenation ...||... or concat(...,...) current_date(),
...
Any function or operator defined by EJB-QL 3.0: substring(), trim(), lower(), upper(),
length(), locate(), abs(), sqrt(), bit_length(), mod()
But you are actually better off doing as #Hansraj suggests in the comments and appending a wildcard to your search term
String query = "from Channe where ip like :term";
entityManager.createQuery(query).setParameter("term",ipString + "%");
This assumes that your data type is string, of course.
Try the following:
Say variable ip had the address
ip = "10.131.56.40:8080";
var ipSplit = ip.Split(':');
var ipStart = ipSplit[0];
ipStart will store only 10.131.56.40
This could solve your problem
Try this:
SPLIT(".", FIELDNAME)
I have string like
order by o desc,b asc
Here I want to replace o and b columns of this clause by table_o and table_b and output
order by table_o desc, table_b asc
I am using replace function for that but output becomes like
table_order table_by table_o desc,table_b asc
How to solve this problem using regular expression?
One more example
"order by orders desc, bye asc"
should be replaced as
"order by table_orders desc, table_bye asc"
Here is one possible solution. [You might have to tweak spaces around desc asc and , based on your actual SQL]
String str = "select a,b,c * from Table order by o desc,b asc,c,d";
System.out.println(str.replaceAll(
"(.*order by )?(\\w+)( desc| asc)?(,|$)", "$1table_$2$3$4"));
Result
select a,b,c * from Table order by table_o desc,table_b asc,table_c,table_d
Visual Regex
Regex details
(.*order by)? => will match select a,b,c * from Table order by =>back ref $1
(\\w+) => will match column name =>back ref $2
( desc| asc)? => will match desc or asc => back ref $3
(,|$) => will match trailing comma or endof line => back ref $4
Please Note : this solution only works with simple sql queries, and would produce wrong result if the order byclause is part of inner query of a complex SQL. Moreover Regex is not can not ideal tool to parse SQL syntax
See this link Regular expression to match common SQL syntax?
If full-fledged SQL parsing is required, Its better to use either SQL parsers or Parser generators like ANTLR to parse SQL. See this link for list of available ANTLR SQL grammer
If you just want to replace text like that just use these regexes:
" o "
" b "
Probably you are looking for this? Regular Expressions in Java SE & EE Have a look at Regular Expressions chapter that will do the work most of the times.
Simply use a space in the replace function (you do not need a regex).
Pseudo-code:
string = string_replace(string, " o ", " table_o ")
Edit:
After your example, you can but every valid boundary between [ and ]. The regex will then match is. To get back the origional boundary put it between ( and ) and replace it back.
E.g.:
string = regex_replace(string, "([ \t])o([ \t,])", "\1o\2")
\1 and \2 might be different in your regex implementation.
Also I'd suggest clarifying your case so that it is clear what you really want to replace and also take a look at Truth's suggestion of the XY problem.
You can use code like this to convert your text:
String sql = "select o, b, c,d form Table order by orders ,b asc, c desc,d desc, e";
String text = sql.toLowerCase();
String orderBy = "order by ";
int start = text.indexOf(orderBy);
if (start >= 0) {
String subtext = text.substring(start+orderBy.length());
System.out.printf("Replaceed: [%s%s%s]%n", text.substring(0, start), orderBy, subtext.replaceAll("(\\w+)(\\s+(?:asc|desc)?,?\\s*)?", "table_$1$2"));
}
OUTPUT:
Replaceed: [select o, b, c,d form table order by table_orders ,table_b asc, table_c desc,table_d desc, table_e]