I would like to log sql statements execution time. To do this I'm using interceptor plugin. However it only "catches" the outer statement and prints combined time needed to execute the whole statement including subqueries contained in #Result. Why does it not log all the statements separetely and is there any solution for this?
Here is the code reproducing my scenario:
ADao:
#Select({
"select * from a",
"where id = #{id,jdbcType=INTEGER}"
})
#Results({
#Result(column="id", property="id", jdbcType=JdbcType.INTEGER, id=true),
#Result(column="id", property="bs", javaType=List.class, many=#Many(fetchType=FetchType.EAGER, select = "org.BDao.selectByAId")),
#Result(column="id", property="cs", javaType=List.class, many=#Many(fetchType=FetchType.EAGER, select = "org.CDao.selectByAId"))
})
A selectByPrimaryKey(Integer id);
BDao:
#Select({
"select * from b",
"where a_id = #{aId,jdbcType=INTEGER}"
})
#Results({
...
})
List<B> selectByAId(Integer aId);
CDao:
#Select({
"select * from c",
"where a_id = #{aId,jdbcType=INTEGER}"
})
#Results({
...
})
List<C> selectByAId(Integer aId);
Interceptor:
#Intercepts({ #Signature(type = Executor.class, method = "query", args = { MappedStatement.class, Object.class, RowBounds.class, ResultHandler.class }) })
public class LogTimeQueryExecutePlugin implements Interceptor {
static int MAPPED_STATEMENT_INDEX = 0;
static int PARAMETER_INDEX = 1;
static int ROWBOUNDS_INDEX = 2;
static int RESULT_HANDLER_INDEX = 3;
static Logger logger = LoggerFactory.getLogger(LogTimeQueryExecutePlugin.class);
public Object intercept(Invocation invocation) throws Throwable {
long start = System.currentTimeMillis();
Object proceed = invocation.proceed();
MappedStatement ms = (MappedStatement) invocation.getArgs()[MAPPED_STATEMENT_INDEX];
logger.info("Execution time "+ms.getId()+" : "+(System.currentTimeMillis() - start)+" ms");
return proceed;
}
public Object plugin(Object target) {
return Plugin.wrap(target, this);
}
public void setProperties(Properties properties) {
}
}
Log:
Execution time org.ADao.selectByPrimaryKey : 59033 ms
Try StatementHandler.class instead of Executor.class.
#Intercepts({
#Signature(type = StatementHandler.class, method = "query", args = { Statement.class, ResultHandler.class }),
#Signature(type = StatementHandler.class, method = "update", args = { Statement.class })
})
public class StatementHandlerPlugin implements Interceptor {
#Override
public Object intercept(final Invocation invocation) throws Throwable {
// just for simplification, use safer way
val stmt = (StatementHandler) invocation.getTarget();
val boundSql = stmt.getBoundSql(); // contains sql, parameterObject, parameterMappings ...
// rest of implementation
}
// omitted
}
Related
I'm facing a problem in which I don't get results from my query in Flink-SQL.
I have some informations stored in two Kafka Topics, I want to store them in two tables and perform a join between them in a streaming way.
These are my flink instructions :
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
StreamTableEnvironment tableEnv = TableEnvironment.getTableEnvironment(env);
// configure Kafka consumer
Properties props = new Properties();
props.setProperty("bootstrap.servers", "localhost:9092"); // Broker default host:port
props.setProperty("group.id", "flink-consumer"); // Consumer group ID
FlinkKafkaConsumer011<Blocks> flinkBlocksConsumer = new FlinkKafkaConsumer011<>(args[0], new BlocksSchema(), props);
flinkBlocksConsumer.setStartFromEarliest();
FlinkKafkaConsumer011<Transactions> flinkTransactionsConsumer = new FlinkKafkaConsumer011<>(args[1], new TransactionsSchema(), props);
flinkTransactionsConsumer.setStartFromEarliest();
DataStream<Blocks> blocks = env.addSource(flinkBlocksConsumer);
DataStream<Transactions> transactions = env.addSource(flinkTransactionsConsumer);
tableEnv.registerDataStream("blocksTable", blocks);
tableEnv.registerDataStream("transactionsTable", transactions);
Here is my SQL query :
Table sqlResult
= tableEnv.sqlQuery(
"SELECT block_timestamp,count(tx_hash) " +
"FROM blocksTable " +
"JOIN transactionsTable " +
"ON blocksTable.block_hash=transactionsTable.tx_hash " +
"GROUP BY blocksTable.block_timestamp");
DataStream<Test> resultStream = tableEnv
.toRetractStream(sqlResult,Row.class)
.map(t -> {
Row r = t.f1;
String field2 = r.getField(0).toString();
long count = Long.valueOf(r.getField(1).toString());
return new Test(field2,count);
})
.returns(Test.class);
Then, I print the results :
resultStream.print();
But I don't get any answers, my program is stuck...
For the schema used for serialization and deserialization, here is my test class which stores the result of my query (two fields a string and a long for respectively the block_timestamp and the count) :
public class TestSchema implements DeserializationSchema<Test>, SerializationSchema<Test> {
#Override
public Test deserialize(byte[] message) throws IOException {
return Test.fromString(new String(message));
}
#Override
public boolean isEndOfStream(Test nextElement) {
return false;
}
#Override
public byte[] serialize(Test element) {
return element.toString().getBytes();
}
#Override
public TypeInformation<Test> getProducedType() {
return TypeInformation.of(Test.class);
}
}
This is the same principle for BlockSchema and TransactionsSchema classes.
Do you know why I can't get the result of my query ? Should I test with BatchExecutionEnvironment ?
In my controller, I receive a string parameter on the basis of which I need to decide which service to call, how can I do the same in my Spring Boot application using Spring annotations?
For example: we have different types of cars. Now, on the basis of parameter in the request I should be able to decide which particular car service I should call.
How can I have a factory using annotations in Spring Boot, and objects should be returned from that factory on the basis of input.
I remember implementing support for this approach a few years ago, I believe inspired and using https://www.captechconsulting.com/blogs/combining-strategy-pattern-and-spring as the entry point to my utility library, use the following code snippets at your convenience:
Strategy.java
package ...
#Documented
#Target({ ElementType.TYPE })
#Retention(RetentionPolicy.RUNTIME)
public #interface Strategy {
Class<?> type();
String[] profiles() default {};
}
StrategyFactory.java
package ...
public class StrategyFactory {
private static final Logger LOG = Logger.getLogger( StrategyFactory.class );
private Map<Class<?>, Strategy> strategiesCache = new HashMap<Class<?>, Strategy>();
private String[] packages;
#PostConstruct
public void init() {
if (this.packages != null) {
Set<Class<?>> annotatedClasses = new HashSet<Class<?>>();
for (String pack : this.packages) {
Reflections reflections = new Reflections( pack );
annotatedClasses.addAll( reflections.getTypesAnnotatedWith( Strategy.class ) );
}
this.sanityCheck( annotatedClasses );
}
}
public <T> T getStrategy(Class<T> strategyClass) {
return this.getStrategy( strategyClass, null );
}
#SuppressWarnings("unchecked")
public <T> T getStrategy(Class<T> strategyClass, String currentProfile) {
Class<T> clazz = (Class<T>) this.findStrategyMatchingProfile( strategyClass, currentProfile );
if (clazz == null) {
throw new StrategyNotFoundException( String.format( "No strategies found of type '%s', are the strategies marked with #Strategy?", strategyClass.getName() ) );
}
try {
return (T) clazz.newInstance();
} catch (Exception e) {
throw ExceptionUtils.rethrowAs( e, StrategyException.class );
}
}
/**
* Checks to make sure there is only one strategy of each type(Interface) annotated for each profile Will throw an exception on startup if multiple strategies are mapped to the same profile.
* #param annotatedClasses a list of classes
*/
private void sanityCheck(Set<Class<?>> annotatedClasses) {
Set<String> usedStrategies = new HashSet<String>();
for (Class<?> annotatedClass : annotatedClasses) {
Strategy strategyAnnotation = AnnotationUtils.findAnnotation( annotatedClass, Strategy.class );
if (!strategyAnnotation.type().isAssignableFrom( annotatedClass )) {
throw new StrategyProfileViolationException( String.format( "'%s' should be assignable from '%s'", strategyAnnotation.type(), annotatedClass ) );
}
this.strategiesCache.put( annotatedClass, strategyAnnotation );
if (this.isDefault( strategyAnnotation )) {
this.ifNotExistAdd( strategyAnnotation.type(), "default", usedStrategies );
} else {
for (String profile : strategyAnnotation.profiles()) {
this.ifNotExistAdd( strategyAnnotation.type(), profile, usedStrategies );
}
}
}
}
private void ifNotExistAdd(Class<?> type, String profile, Set<String> usedStrategies) {
String key = this.createKey( type, profile );
if (usedStrategies.contains( key )) {
throw new StrategyProfileViolationException( String.format( "There can only be a single strategy for each type, found multiple for type '%s' and profile '%s'", type, profile ) );
}
usedStrategies.add( key );
}
private String createKey(Class<?> type, String profile) {
return String.format( "%s_%s", type, profile ).toLowerCase();
}
private boolean isDefault(Strategy strategyAnnotation) {
return (strategyAnnotation.profiles().length == 0);
}
private Class<?> findStrategyMatchingProfile(Class<?> strategyClass, String currentProfile) {
for (Map.Entry<Class<?>, Strategy> strategyCacheEntry : this.strategiesCache.entrySet()) {
Strategy strategyCacheEntryValue = strategyCacheEntry.getValue();
if (strategyCacheEntryValue.type().equals( strategyClass )) {
if (currentProfile != null) {
for (String profile : strategyCacheEntryValue.profiles()) {
if (currentProfile.equals( profile )) {
Class<?> result = strategyCacheEntry.getKey();
if (LOG.isDebugEnabled()) {
LOG.debug( String.format( "Found strategy [strategy=%s, profile=%s, strategyImpl=%s]", strategyClass, currentProfile, result ) );
}
return result;
}
}
} else if (this.isDefault( strategyCacheEntryValue )) {
Class<?> defaultClass = strategyCacheEntry.getKey();
if (LOG.isDebugEnabled()) {
LOG.debug( String.format( "Found default strategy [strategy=%s, profile=%s, strategyImpl=%s]", strategyClass, currentProfile, defaultClass ) );
}
return defaultClass;
}
}
}
return null;
}
public void setPackages(String[] packages) {
this.packages = packages;
}
}
StrategyException.java
package ...
public class StrategyException extends RuntimeException {
...
}
StrategyNotFoundException.java
package ...
public class StrategyNotFoundException extends StrategyException {
...
}
StrategyProfileViolationException.java
package ...
public class StrategyProfileViolationException extends StrategyException {
...
}
Usage without Spring:
NavigationStrategy.java
package com.asimio.core.test.strategy.strategies.navigation;
public interface NavigationStrategy {
public String naviateTo();
}
FreeNavigationStrategy.java
package com.asimio.core.test.strategy.strategies.navigation;
#Strategy(type = NavigationStrategy.class)
public class FreeNavigationStrategy implements NavigationStrategy {
public String naviateTo() {
return "free";
}
}
LimitedPremiumNavigationStrategy.java
package com.asimio.core.test.strategy.strategies.navigation;
#Strategy(type = NavigationStrategy.class, profiles = { "limited", "premium" })
public class LimitedPremiumNavigationStrategy implements NavigationStrategy {
public String naviateTo() {
return "limited+premium";
}
}
Then
...
StrategyFactory factory = new StrategyFactory();
factory.setPackages( new String[] { "com.asimio.core.test.strategy.strategies.navigation" } );
this.factory.init();
NavigationStrategy ns = this.factory.getStrategy( NavigationStrategy.class );
String result = ns.naviateTo();
Assert.assertThat( "free", Matchers.is( result ) );
...
Or
...
String result = factory.getStrategy( NavigationStrategy.class, "limited" ).naviateTo();
Assert.assertThat( "limited+premium", Matchers.is( result ) );
...
Usage with Spring:
Spring context file:
<bean id="strategyFactory" class="com.asimio.core.strategy.StrategyFactory">
<property name="packages">
<list>
<value>com.asimio.jobs.feed.impl</value>
</list>
</property>
</bean>
IFeedProcessor.java
package ...
public interface IFeedProcessor {
void runBatch(String file);
}
CsvRentalsFeedProcessor.java
package ...
#Configurable(dependencyCheck = true)
#Strategy(type = IFeedProcessor.class, profiles = { "csv" })
public class CsvRentalsFeedProcessor implements IFeedProcessor, Serializable {
#Autowired
private CsvRentalsBatchReporter batchReporter;
...
}
Then
...
IFeedProcessor feedProcessor = this.strategyFactory.getStrategy( IFeedProcessor.class, feedFileExt );
feedProcessor.runBatch( unzippedFeedDir.getAbsolutePath() + File.separatorChar + feedFileName );
...
Notice CsvRentalsBatchReporter is "injected" in CsvRentalsFeedProcessor bean (a Strategy implementation) but StrategyFactory instantiates the strategy implementation using return (T) clazz.newInstance();, so what's needed to make this object Spring-aware?
First CsvRentalsFeedProcessor to be annotated with #Configurable(dependencyCheck = true) and when running the Java application this argument is needed in the java command: -javaagent:<path to spring-agent-${spring.version}.jar>
I am trying to execute a sql query with IN clause which takes in multiple project_ids. This query works fine when I run it directly in Oracle SQL Developer, but does not work from a groovy class.
Below is my code
class Test {
private static final String PROJECT_INFO_FOR_USER ="select PROJECT_ID as PROJECT_NO, NAME as PROJECT_NAME from PROJECT_DIM where PROJECT_ID IN (?)"
private def getProjectList(def caseResult) {
def projectList = ""
caseResult.each { projno ->
if (projectList.length() == 0) {
projectList = "'${projno.project_no}',"
} else {
if (projectList.indexOf(projno.project_no) == -1)
projectList+="'${projno.project_no}',"
}
}
projectList = projectList.substring(0, projectList.length() - 1)
return projectList
}
private process() {
def db = [url: "jdbc:oracle:thin:#x.xx.xx.xx:1521:ORCL",
user: 'xxxx', password: 'xxxx', driver: 'oracle.jdbc.pool.OracleDataSource']
def sql = Sql.newInstance(db.url, db.user, db.password, db.driver)
println "DB connection ready"
def caseResult = [['project_no':'x-xxxx', 'case_nos':['12344'], 'updated_on':'1485335172'], ['project_no':'y-yyyy', 'case_nos':['56789'], 'updated_on':1490359241]]
def projectList = "x-xxxx"
def params = getProjectList(caseResult)
def result = sql.rows(PROJECT_INFO_FOR_USER, params).collect { // If I replace params with projectList then 'result' is assigned a row from oracle database
it as Map
}
println result
}
public static void main(String[] args) {
Test t = new Test()
t.process()
}
}
Result
DB connection ready
[]
Result when I replace params with projectList
DB connection ready
[[PROJECT_NO:x-xxxx, PROJECT_NAME:Xonions, Inc.]]
Below are the 2 questions from above problem
How to bind a list of string as parameters in groovy sql.rows query?
Is there a way to see final query executed by sql.rows with parameters?
I think you need a generic method of creating an in statement and collection of binds based on the number of project numbers but as a start...
import groovy.sql.Sql
class Test {
private static final String PROJECT_INFO_FOR_USER ="select PROJECT_ID as PROJECT_NO, NAME as PROJECT_NAME from PROJECT_DIM where PROJECT_ID IN ( "
private getProjectList( caseResult ) {
def query = new StringBuffer( PROJECT_INFO_FOR_USER )
def binds = []
caseResult.each { projno ->
if (query.length() == 0) {
query.append '?,'
binds << projno.project_no
} else {
if (query.indexOf(projno.project_no) == -1) {
query.append '?,'
binds << projno.project_no
}
}
}
query.deleteCharAt( query.length() - 1 )
query.append( ' )' )
[query: query.toString(), binds: binds]
}
private process() {
def db = [url: "jdbc:oracle:thin:#x.xx.xx.xx:1521:ORCL",
user: 'xxxx',
password: 'xxxx',
driver: 'oracle.jdbc.pool.OracleDataSource']
def sql = Sql.newInstance(db.url, db.user, db.password, db.driver)
println "DB connection ready"
def caseResult = [['project_no':'x-xxxx', 'case_nos':['12344'], 'updated_on':'1485335172'], ['project_no':'y-yyyy', 'case_nos':['56789'], 'updated_on':1490359241]]
def projectList = "x-xxxx"
def params = getProjectList(caseResult)
def result = sql.rows( params.query, params.binds ).collect { // If I replace params with projectList then 'result' is assigned a row from oracle database
it as Map
}
println result
}
public static void main(String[] args) {
Test t = new Test()
t.process()
}
}
When my query is executed i receive stack
Caused by: org.hibernate.QueryException: Not all named parameters have been set: [param0] [select count(generatedAlias0.id)
from Position as generatedAlias0 where generatedAlias0.rank=:param0]
Method where is executed
public Long CountPosition(Rank aRank, List<Long> Status, List<Long> aStatusInternal) {
QueryBuilder<PozycjaWRankingu> aQuery = queryCountLarge();
aQuery.whereEquals("rank", aRank);
aQuery.whereInNotEmpty("status", Status);
aQuery.whereInNotEmptyAndNotEquals("internal.state", aStatusInternal);
return aQuery.countLarge();
}
And finally Query Builder
public static <T> QueryBuilder<T> createCountLarge(EntityManager aEntityManager, Class<T> aClass) {
QueryBuilder r = new QueryBuilder();
r.em = aEntityManager;
r.criteriaBuilder = r.em.getCriteriaBuilder();
r.criteria = ret.criteriaBuilder.createQuery(Long.class);
r.root = ret.criteria.from(aClass);
r.criteria.select(ret.criteriaBuilder.count(ret.root.get("id")));
return r;
}
Any idea where is might be a problem ?
Edit
queryCountLarge() returns
QueryBuilder.createCountLarge(em, myClass);
I have selection parameters on flat data, only don't know how to either omit a parameter entirely, or make it a complete wildcard. The search might use one or all parameters. How is this done? With ANY or ALL? Or, is there another way?
I would like to use one general query with all the paremeters, and pass in "all" or "any", something along those lines, for some of those parameters.
existing code:
package legacy.database;
import java.sql.Timestamp;
import java.util.List;
import java.util.logging.Logger;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Query;
public class MyQueries {
private static final Logger log = Logger.getLogger(MyQueries.class.getName());
private final EntityManagerFactory emf = Persistence.createEntityManagerFactory("LegacyDatabasePU");
private final EntityManager em = emf.createEntityManager();
public MyQueries() {
}
public List<Clients> findAll() {
Query q = em.createQuery("select c from Clients c");
List<Clients> clients = q.getResultList();
return clients;
}
public List<Clients> selectWithParameters(Criteria c) {
log.info(c.toString());
String opener = c.getOpener();
String closer1 = c.getCloser1();
String status = c.getStatus();
Query q = em.createQuery(
"SELECT c FROM Clients c "
+ "WHERE c.status like :status "
+ "and c.closer1 like :closer1 "
+ "and c.opener like :opener");
q.setParameter("opener", opener);
q.setParameter("closer1", closer1);
q.setParameter("status", status);
log.info(q.toString());
List<Clients> clients = q.getResultList();
log.fine(clients.toString());
return clients;
}
public Clients findById(int id) {
Clients client = em.find(Clients.class, id);
return client;
}
public void send(int id) {
Clients c = em.find(Clients.class, id);
java.util.Date date = new java.util.Date();
Timestamp t = new Timestamp(date.getTime());
em.getTransaction().begin();
c.setDateUpdated(t.toString());
em.getTransaction().commit();
}
}
In case the parameters are optional, the criteria API provides some more flexibility.
If the selectWithParameters is called often, consider using parameters, since the DB can cache the parametrized query then.
selectWithParameters with optional parameters reads like this:
public List<Clients> selectWithParameters(Criteria criteria) {
log.info(criteria.toString());
String opener = criteria.getOpener();
String closer1 = criteria.getCloser1();
String status = criteria.getStatus();
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<Clients> query = criteriaBuilder.createQuery(Clients.class);
Root<Clients> c = query.from(Clients.class);
List<Predicate> wherePredicates = new LinkedList<Predicate>();
if (null != status) {
wherePredicates.add(criteriaBuilder.like(c.get("status"), status));
}
if (null != closer1) {
wherePredicates.add(criteriaBuilder.like(c.get("closer1"), closer1));
}
if (null != opener) {
wherePredicates.add(criteriaBuilder.like(c.get("opener"), opener));
}
query.where(wherePredicates.toArray(new Predicate[0]));
List<Clients> clients = em.createQuery(query).getResultList();
log.fine(clients.toString());
return clients;
}
Thank you, Heiner. This worked, not sure why I had trouble with Heiner's code, but his sample put me in the right direction:
public List<Clients> selectByCriteria(Criteria criteria) {
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<Clients> clientCriteriaQuery = criteriaBuilder.createQuery(Clients.class);
Root<Clients> clientRoot = clientCriteriaQuery.from(Clients.class);
clientCriteriaQuery.select(clientRoot);
List<Predicate> predicates = new ArrayList<>();
predicates.add(criteriaBuilder.like(clientRoot.get(Clients_.phone1), "%" + criteria.getPhone1() + "%"));
if (!criteria.getOpener().equalsIgnoreCase("all")) {
predicates.add(criteriaBuilder.like(clientRoot.get(Clients_.opener), "%" + criteria.getOpener() + "%"));
}
if (!criteria.getCloser1().equalsIgnoreCase("all")) {
predicates.add(criteriaBuilder.like(clientRoot.get(Clients_.closer1), "%" + criteria.getCloser1() + "%"));
}
if (!criteria.getStatus().equalsIgnoreCase("all")) {
predicates.add(criteriaBuilder.like(clientRoot.get(Clients_.status), "%" + criteria.getStatus() + "%"));
}
clientCriteriaQuery.where(predicates.toArray(new Predicate[0]));
List<Clients> clients = em.createQuery(clientCriteriaQuery).getResultList();
return clients;
}
There's probably no substantive difference (?) from what Heiner answered. JPA and JPQL are a bit murky. I can't believe it, but I almost prefer SQL! I'll have to adjust.