We Keep Coding

Apache flink 1.52 Rowtime timestamp is null

I am doing some query with the following code:
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
DataStream<Row> ds = SourceHelp.builder().env(env).consumer010(MyKafka.builder().build().kafkaWithWaterMark2())
.rowTypeInfo(MyRowType.builder().build().typeInfo())
.build().source4();
//,proctime.proctime,rowtime.rowtime
String sql1 = "select a,b,max(rowtime)as rowtime from user_device group by a,b";
DataStream<Row> ds2 = TableHelp.builder().tableEnv(tableEnv).tableName("user_device").fields("a,b,rowtime.rowtime")
.rowTypeInfo(MyRowType.builder().build().typeInfo13())
.sql(sql1).in(ds).build().result();
ds2.print();
// String sql2 = "select a,count(b) as b from user_device2 group by a";
String sql2 = "select a,count(b) as b,HOP_END(rowtime,INTERVAL '5' SECOND,INTERVAL '30' SECOND) as c from user_device2 group by HOP(rowtime, INTERVAL '5' SECOND, INTERVAL '30' SECOND),a";
DataStream<Row> ds3 = TableHelp.builder().tableEnv(tableEnv).tableName("user_device2").fields("a,b,rowtime.rowtime")
.rowTypeInfo(MyRowType.builder().build().typeInfo14())
.sql(sql2).in(ds2).build().result();
ds3.print();
env.execute("test");
note: For sql1, I use max function with rowtime, it is not working, and following Exception is thrown:
Exception in thread "main"
org.apache.flink.runtime.client.JobExecutionException:
java.lang.RuntimeException: Rowtime timestamp is null. Please make
sure that a proper TimestampAssigner is defined and the stream
environment uses the EventTime time characteristic. at
org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:625)
at
org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
at
com.aicaigroup.water.WaterTest.testRowtimeWithMoreSqls5(WaterTest.java:158)
at com.aicaigroup.water.WaterTest.main(WaterTest.java:20) Caused by:
java.lang.RuntimeException: Rowtime timestamp is null. Please make
sure that a proper TimestampAssigner is defined and the stream
environment uses the EventTime time characteristic. at
DataStreamSourceConversion$24.processElement(Unknown Source) at
org.apache.flink.table.runtime.CRowOutputProcessRunner.processElement(CRowOutputProcessRunner.scala:67)
at
org.apache.flink.streaming.api.operators.ProcessOperator.processElement(ProcessOperator.java:66)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:558)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:533)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:513)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:628)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.collect(OperatorChain.java:581)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:679)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:657)
at
org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
at com.aicaigroup.TableHelp$1.processElement(TableHelp.java:42) at
com.aicaigroup.TableHelp$1.processElement(TableHelp.java:39) at
org.apache.flink.streaming.api.operators.ProcessOperator.processElement(ProcessOperator.java:66)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:558)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:533)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:513)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:679)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:657)
at
org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:558)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:533)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:513)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:679)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:657)
at
org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)
at
org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:151)
at
org.apache.flink.table.runtime.aggregate.GroupAggProcessFunction.processElement(GroupAggProcessFunction.scala:39)
at
org.apache.flink.streaming.api.operators.LegacyKeyedProcessOperator.processElement(LegacyKeyedProcessOperator.java:88)
at
org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
at
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:104)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) at
java.lang.Thread.run(Thread.java:748) 2018-09-17 09:51:53.679 [Kafka
0.10 Fetcher for Source: Custom Source -> Map -> from: (a, b, rowtime) -> select: (a, b, CAST(rowtime) AS rowtime) (2/8)] INFO o.a.kafka.clients.consumer.internals.AbstractCoordinator - Discovered
coordinator 172.16.11.91:9092 (id: 2147483647 rack: null) for group
test.
then I tried to update sql1 like this "select a,b,rowtime from user_device", and it works.
So how to fix the error? First sql should use group by, and second sql should use rowtime by
timeWindow. 3QS

I started flink from 1.6 , meet the similar question like yours.
Solved by the those steps :
using assignTimestampsAndWatermarks , just use the default and normal implement BoundedOutOfOrdernessTimestampExtractor. You need write the extractTimestamp function to extract timestamp value and declare window interval in the constructor.
append ,proctime.proctime,rowtime.rowtime at the end of fields (i'm using fromDataStream(Flink 1.6) to convert stream as table)
if you want use the exist field as rowtime. for example, data source fields is "a,clicktime,c" , you can declare "a,clicktime.rowtime,c"
Wish it can help you.

Data truncation error while setting INOUT parameter in Stored Procedure

I'm calling a stored procedure which has INOUT parameters. Database is AS400 DB2. Type is CHARACTER. I'm getting Data Truncation error while registering and setting the variable. If I set the string directly to the column, it does not have an issue. If I set the same string in a variable and use it to set the column, it throws Data truncation error. Could you please let me know where I'm going wrong and how I can set the value of the column using a variable without causing exception? Please see the end of error stack trace for more information.
try{
String cstmt_str = "CALL " + storedProcName + "(?)";
String status="REJ";
cstmt = conn.prepareCall(cstmt_str);
cstmt.registerOutParameter("5506STS", Types.CHAR);
//cstmt.setString("5506STS","REJ"); does not give a problem
cstmt.setString("5506STS",status); //Data truncation Exception occurs here.
cstmt.execute();
}catch (DataTruncation de) {
logger.error("DatatruncationException error:", de );
displayError(de);
de.printStackTrace();
}
public static void displayError(DataTruncation dataTruncation) {
logger.info("Data truncation error: ");
logger.info("dataTruncation.getDataSize():"+dataTruncation.getDataSize() + " number of bytes of data that should have been transferred.");
if (!dataTruncation.getRead())
logger.info("dataTruncation.getRead() is False. Its Written (Error:). ");
logger.info("dataTruncation.getTransferSize():"+dataTruncation.getTransferSize()
+ " number of bytes of data actually transferred.");
}
Stored Procedure Signature:
Number Mode Name DataType Length
1 INOUT 5506STS CHARACTER 3
I tried to find out the length of the string status:
int len = status.getBytes().length; // Output: 5
I'm out of options. Please let me know how to successfully set the value in the variable "status" into "5506STS".
ERROR TRACE:
DatatruncationException error:
java.sql.DataTruncation: Data truncation
at com.ibm.as400.access.AS400JDBCPreparedStatement.testDataTruncation(AS400JDBCPreparedStatement.java:3450)
at com.ibm.as400.access.AS400JDBCPreparedStatement.setValue(AS400JDBCPreparedStatement.java:3361)
at com.ibm.as400.access.AS400JDBCPreparedStatement.setString(AS400JDBCPreparedStatement.java:2999)
at com.ibm.as400.access.AS400JDBCCallableStatement.setString(AS400JDBCCallableStatement.java:3082)
at org.jboss.jca.adapters.jdbc.WrappedCallableStatement.setString(WrappedCallableStatement.java:1563)
at com.ssss.ssjtracdbws.dao.SSJTracDBWSDAO.submitNewRequestJSON(SSJTracDBWSDAO.java:699)
at com.ssss.ssjtracdbws.webservices.SSJTracDBService.submitNewRequestJSON(SSJTracDBService.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:167)
at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269)
at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227)
at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216)
at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:542)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:524)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:126)
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:559)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:340)
at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:353)
at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:911)
at org.apache.tomcat.util.net.NioEndpoint$ChannelProcessor.run(NioEndpoint.java:920)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Data truncation error:
dataTruncation.getDataSize():5 number of bytes of data that should have been transferred.
dataTruncation.getRead() is False. Its Written (Error:).
dataTruncation.getTransferSize(): 3 number of bytes of data actually transferred.

Well, I found it out myself. The "status" that was passed to the function had a single quote surrounding it.('APP') this made the length 5 instead of 3. Now I removed the single quotes.Its working fine.

Illegal utf-8 exception while querying apache jena with tdbquery

I use Apache Jena to query RDF Data from the Billion Triple Challange 2014 Dataset. I loaded the dataset into Jena with tdbloader. I especially use queries which contain property paths with tdbquery. When I start such a query I often get the Exception:
Exception
org.apache.jena.atlas.RuntimeIOException: java.io.IOException: Illegal UTF-8: 0xFFFFFF80
at org.apache.jena.atlas.io.IO.exception(IO.java:216)
at org.apache.jena.atlas.io.BlockUTF8.exception(BlockUTF8.java:279)
at org.apache.jena.atlas.io.BlockUTF8.toCharsBuffer(BlockUTF8.java:152)
at org.apache.jena.atlas.io.BlockUTF8.toChars(BlockUTF8.java:75)
at org.apache.jena.atlas.io.BlockUTF8.toString(BlockUTF8.java:97)
at org.apache.jena.tdb.store.nodetable.NodecSSE.decode(NodecSSE.java:101)
at org.apache.jena.tdb.lib.NodeLib.decode(NodeLib.java:105)
at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:81)
at org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:186)
at org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:111)
at org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:70)
at org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128)
at org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82)
at org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:54)
at org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
at org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:308)
at org.apache.jena.sparql.engine.main.iterator.QueryIterGraph$QueryIterGraphInner.nextIterator(QueryIterGraph.java:152)
at org.apache.jena.sparql.engine.main.iterator.QueryIterGraph$QueryIterGraphInner.hasNextBinding(QueryIterGraph.java:126)
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:111)
at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:74)
at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:111)
at org.apache.jena.tdb.solver.OpExecutorTDB1.optimizeExecuteQuads(OpExecutorTDB1.java:212)
at org.apache.jena.tdb.solver.OpExecutorTDB1.execute(OpExecutorTDB1.java:148)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:66)
at org.apache.jena.sparql.algebra.op.OpQuadPattern.visit(OpQuadPattern.java:95)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:232)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:130)
at org.apache.jena.sparql.algebra.op.OpSequence.visit(OpSequence.java:75)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:393)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:267)
at org.apache.jena.sparql.algebra.op.OpProject.visit(OpProject.java:47)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:415)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:275)
at org.apache.jena.sparql.algebra.op.OpDistinct.visit(OpDistinct.java:47)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:403)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:307)
at org.apache.jena.sparql.algebra.op.OpSlice.visit(OpSlice.java:50)
at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:89)
at org.apache.jena.sparql.engine.main.QC.execute(QC.java:52)
at org.apache.jena.sparql.engine.main.QueryEngineMain.eval(QueryEngineMain.java:53)
at org.apache.jena.tdb.solver.QueryEngineTDB.eval(QueryEngineTDB.java:112)
at org.apache.jena.sparql.engine.QueryEngineBase.evaluate(QueryEngineBase.java:136)
at org.apache.jena.sparql.engine.QueryEngineBase.createPlan(QueryEngineBase.java:106)
at org.apache.jena.sparql.engine.QueryEngineBase.getPlan(QueryEngineBase.java:87)
at org.apache.jena.tdb.solver.QueryEngineTDB$QueryEngineFactoryTDB.create(QueryEngineTDB.java:169)
at org.apache.jena.sparql.engine.QueryExecutionBase.getPlan(QueryExecutionBase.java:582)
at org.apache.jena.sparql.engine.QueryExecutionBase.startQueryIterator(QueryExecutionBase.java:526)
at org.apache.jena.sparql.engine.QueryExecutionBase.execResultSet(QueryExecutionBase.java:567)
at org.apache.jena.sparql.engine.QueryExecutionBase.execSelect(QueryExecutionBase.java:184)
at org.apache.jena.sparql.util.QueryExecUtils.doSelectQuery(QueryExecUtils.java:196)
at org.apache.jena.sparql.util.QueryExecUtils.executeQuery(QueryExecUtils.java:78)
at arq.query.queryExec(query.java:218)
at arq.query.exec(query.java:160)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at tdb.tdbquery.main(tdbquery.java:33)
Caused by: java.io.IOException: Illegal UTF-8: 0xFFFFFF80
... 71 more
Is my query wrongly encoded or the dataset?
For instance is used a query: SELECT DISTINCT ?o { GRAPH ?g {A B* ?o}} LIMIT 3
where A and B are valid IRIs and it returned a valid result. But I want all results for that query so I deleted the LIMIT 3 and got the exception.
I also used the query directly:
sudo ./tdbquery -time -loc=/path/to/database "SELECT DISTINCT ?o {GRAPH ?g {A B* ?o}}"
and from a file
sudo ./tdbquery -time -loc=/path/to/database --query=/path/to/query.sparql
Sorry if any important information is missing to the question.
Any Idea why I get the exception and how I can handle it?

How to return distance and score in spatial search using spring-data-solr

I'm writing a Spring MVC based Search API using spring-data-solr 1.0.0.RELEASE with Solr 4.4.0 and Spring 3.2.4.RELEASE.
I'm able to run basic queries but unable to find any good example how can I return score and distance:geodist() in the results.
I know I can get results from Solr using query like
http://localhost:8983/solr/events/select?q=*:*&spatial=true&pt=51.435872%2C-0.427529&sfield=position&d=20&facet=true&facet.mincount=1&facet.limit=-1&facet.field=categoryIds&fl=score,*,distance:geodist()&sort=geodist()+asc
but I don't know how to get score and distance properties returned from Solr using spring-data-solr. I have tried following code
FacetQuery search = new SimpleFacetQuery();
Criteria conditions;// = new Criteria();
if(StringUtils.isNotBlank(searchCriteria.getSearchString()))
conditions = new Criteria(EventDocument.FIELD_EVENT).contains(searchCriteria.getSearchString());
if(searchCriteria.isLocationKnown()){
conditions.and(new Criteria(EventDocument.FIELD_POSITION).within(new GeoLocation(searchCriteria.getLatitude(), searchCriteria.getLongitude()), new Distance(searchCriteria.getDistance(), Distance.Unit.MILES)));
conditions.and(new Criteria(CommonParams.FL).is("score,distance:geodist(),*"));
} else{
conditions.and(new Criteria(CommonParams.FL).is("score,*"));
}
search.addCriteria(conditions);
FacetPage page = solrTemplate.queryForFacetPage(search, EventDocument.class);
but getting following Exception:
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.data.solr.UncategorizedSolrException: undefined field fl; nested exception is org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: undefined field fl
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:948)
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)
javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)
javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
root cause
org.springframework.data.solr.UncategorizedSolrException: undefined field fl; nested exception is org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: undefined field fl
org.springframework.data.solr.core.SolrTemplate.execute(SolrTemplate.java:122)
org.springframework.data.solr.core.SolrTemplate.executeSolrQuery(SolrTemplate.java:330)
org.springframework.data.solr.core.SolrTemplate.query(SolrTemplate.java:326)
org.springframework.data.solr.core.SolrTemplate.queryForFacetPage(SolrTemplate.java:285)
com.johnstonpress.api.repository.EventDocumentRepositoryImpl.search(EventDocumentRepositoryImpl.java:72)
com.johnstonpress.api.service.SearchEventServiceImpl.search(SearchEventServiceImpl.java:106)
com.johnstonpress.api.controller.EventsSearchController.searchEvents(EventsSearchController.java:124)
com.johnstonpress.api.controller.EventsSearchController.search(EventsSearchController.java:68)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:601)
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)
javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)
javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
Thanks for any attention and help!!!

To set fl you have to use query.addProjectionOnField(String fieldname).
SimpleQuery query = new SimpleQuery(conditions);
query.addProjectionOnField("*");
query.addProjectionOnField("score");
For mapping score into EventDocument you'll have to add the property as follows.
#Indexed(value = "score", readonly = true)
private Float score;
Unfortunately there seems to be an issue with geodist() that might be caused by the way spring data solr creates spatial query.
Opended DATASOLR-130 for that.
Distance can be requested using SolrCallback along with SolrTemplate by setting spatial paramters yourself.
SimpleQuery query = new SimpleQuery(conditions);
query.addProjectionOnField("*");
query.addProjectionOnField("distance:geodist()");
DefaultQueryParser qp = new DefaultQueryParser();
final SolrQuery solrQuery = qp.constructSolrQuery(query);
solrQuery.add("sfield", "store");
solrQuery.add("pt", GeoConverters.GeoLocationToStringConverter.INSTANCE.convert(new GeoLocation(45.15, -93.85)));
solrQuery.add("d", GeoConverters.DistanceToStringConverter.INSTANCE.convert(new Distance(5)));
List<EventDocument> result = template.execute(new SolrCallback<List<EventDocument>>() {
#Override
public List<EventDocument> doInSolr(SolrServer solrServer) throws SolrServerException, IOException {
return template.getConverter().read(solrServer.query(solrQuery).getResults(), EventDocument.class);
}
});
A litte bit easier would be to provide required parameters for geodist().
SimpleQuery query = new SimpleQuery(conditions);
query.addProjectionOnField("*");
query.addProjectionOnField("distance:geodist(store," + GeoConverters.GeoLocationToStringConverter.INSTANCE.convert(new GeoLocation(45.15, -93.85)) + ")");
Page<EventDocument> result = template.queryForPage(query, EventDocument.class);
hope that helps!

Delete data HQL with setParameter or setParametrList strangeness

I have a simple table and try to delete data by array of ids.
Integer[] arrayIDs = {1,2,3}; //External ids as parameteres
Query deleteItemsQuery = session.createQuery("DELETE Items WHERE id IN (:idsDeletingItems)"); //(in this row exception)
deleteItemsQuery.setParameterList("idsDeletingItems", arrayIDs);
deleteItemsQuery.executeUpdate();
And my app throw exception
I also try with deleting 1 parameter. For example
Query deleteItemsQuery = session.createQuery("DELETE Items WHERE id = '1'");
deleteItemsQuery.executeUpdate();
And with that variant no problem.
But when I do next I have also exception
Integer myID = 1;
Query deleteItemsQuery = session.createQuery("DELETE Items WHERE id = (:myID)");
deleteItemsQuery.setParameter("myID", myID);
deleteItemsQuery.executeUpdate();
I use this dialect org.hibernate.dialect.Oracle10gDialect. Any ideas?
Exception
exception
javax.servlet.ServletException: java.lang.NoSuchMethodError: antlr.collections.AST.getLine()I
org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:295)
org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
org.springframework.web.struts.DelegatingActionProxy.execute(DelegatingActionProxy.java:113)
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
com.epam.testapp.presentation.filter.CharsetFilter.doFilter(CharsetFilter.java:44)
com.epam.testapp.presentation.filter.CommandFilter.doFilter(CommandFilter.java:58)
root cause
java.lang.NoSuchMethodError: antlr.collections.AST.getLine()I
org.hibernate.hql.internal.ast.HqlSqlWalker.generateNamedParameter(HqlSqlWalker.java:956)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.parameter(HqlSqlBaseWalker.java:4821)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.expr(HqlSqlBaseWalker.java:1347)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.exprOrSubquery(HqlSqlBaseWalker.java:4297)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.comparisonExpr(HqlSqlBaseWalker.java:3772)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.logicalExpr(HqlSqlBaseWalker.java:1947)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.whereClause(HqlSqlBaseWalker.java:794)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.deleteStatement(HqlSqlBaseWalker.java:443)
org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.statement(HqlSqlBaseWalker.java:263)
org.hibernate.hql.internal.ast.QueryTranslatorImpl.analyze(QueryTranslatorImpl.java:248)
org.hibernate.hql.internal.ast.QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:183)
org.hibernate.hql.internal.ast.QueryTranslatorImpl.compile(QueryTranslatorImpl.java:136)
org.hibernate.engine.query.spi.HQLQueryPlan.<init>(HQLQueryPlan.java:105)
org.hibernate.engine.query.spi.HQLQueryPlan.<init>(HQLQueryPlan.java:80)
org.hibernate.engine.query.spi.QueryPlanCache.getHQLQueryPlan(QueryPlanCache.java:168)
org.hibernate.internal.AbstractSessionImpl.getHQLQueryPlan(AbstractSessionImpl.java:219)
org.hibernate.internal.AbstractSessionImpl.createQuery(AbstractSessionImpl.java:197)
org.hibernate.internal.SessionImpl.createQuery(SessionImpl.java:1736)
com.epam.testapp.database.NewsHibernateDAO.remove(NewsHibernateDAO.java:89)
com.epam.testapp.presentation.action.NewsAction.delete(NewsAction.java:150)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:269)
org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
org.springframework.web.struts.DelegatingActionProxy.execute(DelegatingActionProxy.java:113)
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:425)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:228)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)

Make it like this
Integer myID = 1;
Query deleteItemsQuery = session.createQuery("DELETE Items WHERE id =:myID");
deleteItemsQuery.setParameter("myID", myID);
deleteItemsQuery.executeUpdate();
and make sure no space between ":" and "myID" else it will another error.
Hope it helps ;-)

I also ran into this problem once and worked around it by adding the opening and closing brace to my parameter in stead of placing it directly in the HQL.