Java Regex capture any (.*) with DOTALL ignores zero-width positive lookahead - java

Not a regex expert, but I know enough to be dangerous, need some help with an expression I am working on. Long story short, a recent database upgrade has invalidated thousands of queries within string literals of a legacy application that I support. I am writing a few expressions to capture the majority of these and hopefully fix them programatically.
Consider the following:
Query query = session
.createSQLQuery("SELECT distinct p.userid, p.name, f.hsid, "
+ "p.vid, p.vname, p.paymentdate, p.amount "
+ "FROM vk.payment p, (select * from vs.fuser) fu, (select * from vs.fac) f "
+ "WHERE p.description = 'Check' AND "
+ "p.paymentdate >= :startDate and p.paymentdate <= :endDate AND "
+ "fu.userid = p.userid AND fu.facid = f.facid "
+ "ORDER BY p.userid");
query.setParameter("startDate", startDate);
query.setParameter("endDate", endDate);
I have the following DOTALL expression to attempt and capture simply the ugly contents of the method argument.
(?s)(?<=\.createSQLQuery\(")(.*)(?="\)\;)
I specify the DOTALL flag with (?s) a non-capturing look behind to get \.createSQLQuery\(", capture everything including line breaks with (.*), and finally a non capturing positive lookahead to stop the capture at "\)\;.
I am expecting to capture the following:
SELECT distinct p.userid, p.name, f.hsid, "
+ "p.vid, p.vname, p.paymentdate, p.amount "
+ "FROM vk.payment p, (select * from vs.fuser) fu, (select * from vs.fac) f "
+ "WHERE p.description = 'Check' AND "
+ "p.paymentdate >= :startDate and p.paymentdate <= :endDate AND "
+ "fu.userid = p.userid AND fu.facid = f.facid "
+ "ORDER BY p.userid
Instead the expression is a lot greedier than I anticipated and is capturing this:
SELECT distinct p.userid, p.name, f.hsid, "
+ "p.vid, p.vname, p.paymentdate, p.amount "
+ "FROM vk.payment p, (select * from vs.fuser) fu, (select * from vs.fac) f "
+ "WHERE p.description = 'Check' AND "
+ "p.paymentdate >= :startDate and p.paymentdate <= :endDate AND "
+ "fu.userid = p.userid AND fu.facid = f.facid "
+ "ORDER BY p.userid");
query.setParameter("startDate", startDate);
query.setParameter("endDate", endDate);
... to EOF
The thing is that without the DOTALL the expression works as expected on a single line:
Query query = session.createSQLQuery("SELECT .... ");
and captures without the remaining characters on the end...
SELECT ....
Is there some aspect of DOTALL that every regex guru seems to know that does not seem to be documented anywhere? Does DOTALL not work with positive lookahead?
I appreciate any help!


Make the * quantifier non-greedy by adding a ? after it, like so: .*?
Also why are you even using lookarounds? It can lead to undesired behavior in some cases to use them without thought like this. (And it always irritates me. (-; )
You could just use:
(?s)\.createSQLQuery\("(.*?)"\);

Related

JDBI query fails when <> is used

I have a JDBI query that is as simple as it can be
#Override
#SqlQuery("SELECT COUNT(*) FROM " + TABLE_NAME + " WHERE (" + COLUMN_MODERATOR_CATEGORY_ID
+ " in (<categories>) OR " + COLUMN_EXPERT_CATEGORY_ID
+ " in (<categories>)) AND (" + COLUMN_STATUS + " <> 0)")
long getIdeasCountInCategories(#BindIn("categories") List<Long> categories);
The <> 0 fails with a syntax error at the end of input.... It works as soon as I change it to > 0 (which also serves the purpose).
Using Java 1.8.0 and Postgres 9.6. Please let me know if any more info is needed.

If you absolutely can't change the SQL query (e.g. switch operator to != or >) then you need to escape < with preceding \\.

JPQL Create "Dynamic" Query to execute in repository

Edit-
I'll add the use case to clear up the function of this.
The user will select two dates - a start date and an end date - these are then passed on and used to select the tables (each year has its own table). In one use case where the two given dates lie in the same year it's a simple query on that table alone.
However, if the two dates are different years I will need to join all tables (so 2011-2013 will be three tables connected, to search through) and thus, I want a dynamic fix to this. I know building up a query like below is against security - just thought something similar would work. As the system will get new tables each year I also dont want to have to manually add however many new queries for each case (2011-2016, 2014-2018, 2011-2019.. etc)
I have a question about whether it is possible to create a dynamic query as a String like below and then pass that through to service -> repository, and use that as a query?
for (int i = 0; i < yearCondition; i++) {
if (i == 0) {
query += "SELECT md.Device_ID, l.locationRef, " + reportRunForm.getStartDate() + " as 'From Date', "
+ reportRunForm.getEndDate() + " as 'To Date' "
+ "from mData.meterdata" + iDateStart.substring(0, 4)
+ " join MOL2.meters m on device_ID = m.meterUI "
+ "join MOL2.locations l on m.locationID = l.locationID "
+ "join MOL2.meterreg mr on m.meterID = mr.meterID "
+ "where mr.userID = ?1";
}
query += "UNION SELECT md.Device_ID, l.locationRef, " + reportRunForm.getStartDate() + " as 'From Date', "
+ reportRunForm.getEndDate() + " as 'To Date' "
+ "from mData.meterdata" + (Integer.parseInt(iDateStart.substring(0, 4))+i)
+ " join MOL2.meters m on device_ID = m.meterUI "
+ "join MOL2.locations l on m.locationID = l.locationID "
+ "join MOL2.meterreg mr on m.meterID = mr.meterID "
+ "where mr.userID = ?1";
}
I may have the wrong idea with how this works, and I know I could create and persist a query through entitymanager, but wanted to know whether doing it through the repository would be possible?
My thought was I'd build up the query like above, pass it through to service and then to repository, and bind it as value in #Query annotation but this doesn't seem possible. I'm likely approaching this wrong so any advice would be appreciated.
Thanks.
Edit -
Had a goof. Understand doing it at all like that is stupid, an approach to build up something similar is what I'm looking for that is still secure.

Maybe this annotations before your POJO can help
#org.hibernate.annotations.Entity(dynamicInsert = true)

for example two tables district and constituency ...
Dynamic query
query += "select *from constituency c where 1=1";
if(constituencyNumber!=null)
query +=" and c.constituency_number like '"+constituencyNumber+"%'";
query += " group by c.district_id";
OR
select *from constituency c where (c.constituency_number is null or c.constituency_number like '1%') group by c.district_id;

The expression is not a valid conditional expression

I'm trying to run this JPQL request but I'm getting this error: The expression is not a valid conditional expression.
"SELECT c "
+ "FROM CoursJoursDeviseBb c "
+ "WHERE codeDevise = :codedevise "
+ "AND dateCours = (SELECT MAX(dateCours) FROM CoursJoursDeviseBb "
+ "WHERE codeDevise = :codedevise) "

he expression is not a valid conditional expression
Thanks to JB Nizet we found missing space.
An identification variable must be provided for a range variable declaration
Try to qualify your table in subquery such as
+ "AND dateCours = (SELECT MAX(dateCours) FROM CoursJoursDeviseBb c2 "
+ " WHERE c2.codeDevise = :codedevise) "
to distinguish between two uses of table.

SQL syntax error when concatenating query string

try(Connection dbConnection = DBConnectionManager.getIntakeConnection();
PreparedStatement preparedStmtSetMaxStrikeId = dbConnection.prepareStatement(
"SELECT MAX(strike_id) FROM strike WHERE 'SELECT p.party_type_id,"
+ "p.csa_score,p.party_tn,p.rec_create_date,"
+ "s.strike_id, s.strike_date, s.strike_level, s.strike_status,
s.appealable,s.appeal_status,s.rec_change_date,s.event_id,
s.is_email_processed,s.policy_id"
+ "FROM strike s "
+ "INNER JOIN parties p"
+ "ON p.party_id = s.party_id"
+ "WHERE p.account ='"+appealStatus.getSubscriberId()
+"'AND strike_status = '"+OCIRISConstants.STRIKE_STATUS_ACTIVE+"' ");)
The error is below.
Integers in the error are subscriber ids.
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '0957936101205'AND strike_status = 'ACTIVE'' at line 1

The error message says:
near '0957936101205'AND strike_status = 'ACTIVE'' at line 1
It's actually identifying the 0 following the ', which ends the text literal started in the first line, because this is bad SQL:
SELECT MAX(strike_id) FROM strike WHERE '...'0957936101205'AND strike_status = 'ACTIVE'
-- ^^ BAD
Here is the code annotated with comments:
try(Connection dbConnection = DBConnectionManager.getIntakeConnection();
PreparedStatement preparedStmtSetMaxStrikeId = dbConnection.prepareStatement(
"SELECT MAX(strike_id) FROM strike WHERE 'SELECT p.party_type_id,"
// ^ What is this? Even without ' it makes no sense
// ^ But it STARTS A TEXT LITERAL
+ "p.csa_score,p.party_tn,p.rec_create_date,"
+ "s.strike_id, s.strike_date, s.strike_level, s.strike_status, s.appealable,s.appeal_status,s.rec_change_date,s.event_id,s.is_email_processed,s.policy_id"
+ "FROM strike s "
// ^ Missing space, but it's in a text literal so doesn't matter
+ "INNER JOIN parties p"
+ "ON p.party_id = s.party_id"
// ^ Missing space, but it's in a text literal so doesn't matter
+ "WHERE p.account ='"+appealStatus.getSubscriberId()
// ^ Missing space, but it's in a text literal so doesn't matter
// ^ END TEXT LITERAL from first line
// ^ error complains about inserted value 0957936101205
+"'AND strike_status = '"+OCIRISConstants.STRIKE_STATUS_ACTIVE+"' ");)
// ^ Starts a new text literal
// ^ Missing space, but it's in a text literal so doesn't matter
// ^ end text literal
// ^ would complain about inserted value ACTIVE
// ^ Dangling '
Also, you shouldn't be using string concatenation to build the SQL, since it'll cause syntax errors and leave you susceptible to SQL Injection attacks, allowing hackers to steal your data and delete your tables.
Assuming the initial SELECT MAX( ... WHERE ' is in error, here is a cleaned up version, formatted for clarity:
String sql = "SELECT p.party_type_id, p.csa_score, p.party_tn, p.rec_create_date" +
", s.strike_id, s.strike_date, s.strike_level, s.strike_status" +
", s.appealable, s.appeal_status, s.rec_change_date, s.event_id" +
", s.is_email_processed, s.policy_id" +
" FROM strike s" +
" INNER JOIN parties p ON p.party_id = s.party_id" +
" WHERE p.account = ?" +
" AND strike_status = ?";
try (Connection conn = DBConnectionManager.getIntakeConnection();
PreparedStatement stmt = conn.prepareStatement(sql)) {
stmt.setString(1, appealStatus.getSubscriberId());
stmt.setString(2, OCIRISConstants.STRIKE_STATUS_ACTIVE);

The specific syntax error there is saying that there should be a space before the AND on the last line of the query:
+"' AND strike_status // ... etc
^ Insert a space here
However, you have several other problems there, e.g. you have not got spaces around line breaks, e.g.
+ "INNER JOIN parties p"
+ "ON p.party_id = s.party_id"
would become
+ "INNER JOIN parties pON p.party_id = s.party_id"
You should insert more spaces appropriately, either at the start or end of each line.
There are other syntax errors like WHERE 'SELECT. You need to check all of your syntax very carefully.
Also: concatenating values into prepared statements somewhat defeats the point of prepared statements. See the Javadoc for examples of how to use them correctly.

Java query for postgresql's interval method not working

I'm writing postgresql query. When I run the query
"reservatio0_.DATE_ >(NOW() - '60 MINUTES'::INTERVAL)" on pgAdmin it works fine, but in java I get
QuerySyntaxException, unexpected token: : bla bla
if I run this code
List<Reservation> list = em.createQuery(
"select r " +
"from Reservation r " +
"where r.group.id=:groupName " +
" and r.date >(NOW() - '60 MINUTES'::INTERVAL) " +
"order by r.date asc")
.setParameter("groupName", groupName)
.setParameter("number", number)
.setMaxResults(1)
.getResultList();

try to replace your colon operator (:) with \\:
that would escape this special character..

IIrc you have to put the interval in front of the amount, like this:
and r.date >(NOW() - interval '60 minutes')

Categories