I am getting a strange exception thrown every time I call CFSEARCH on my collection
Code:
<cfsearch
name="qCollectionSearch_docs"
collection="mydocuments"
criteria="#LCase(Trim(LOCAL.Criteria))#"
contextPassages="1"
contextHighlightBegin=""
contextHighlightEnd="" />
Exception:
Detail: Error executing query : Unexpected EOF; was expecting a close tag for element
Message: There was a problem while attempting to perform a search.
StackTrace: coldfusion.tagext.search.SearchTag$SearchProblemException: There was a problem while attempting to perform a search.
at coldfusion.tagext.search.SolrUtils.formatSearchResult(SolrUtils.java:2022)
at coldfusion.tagext.search.SolrUtils.getSearchResult(SolrUtils.java:2516)
at coldfusion.tagext.search.SearchTag.doSolrSearch(SearchTag.java:394)
at coldfusion.tagext.search.SearchTag.doSearch(SearchTag.java:243)
at coldfusion.tagext.search.SearchTag.doStartTag(SearchTag.java:185)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2991)
This code has worked without incident in the past, but suddenly started throwing this exception on every request.
The collection is built with the following code:
<cfindex
query="LOCAL.qDocuments"
collection="mydocuments"
action="refresh"
type="file"
key="DocumentFile"
custom1="SKU"
custom2="Status" />
...where LOCAL.qDocuments is a SQL query that retrieves the SKU and Status values from a database and DocumentFile is a path to a PDF document (i.e. C:\PDFs\document1.pdf).
The server is a virtual server running:
OS: Windows Server 2012 with all of the latest updates
IIS: 8.5
ColdFusion: 11 with all of the latest updates
Java: JRE 8 v111
I have tried restarting the ColdFusion 11 Application Server service and the ColdFusion 11 Add-on Services service, but this did not help.
Pass the value of the key, rather than the word 'documentFile':
<cfindex
key="#documentFile#"
/>
Related
I'm launching 3 elastic nodes using elastic operator and i tried to set up automated snapshots for these instances.
I followed this doc
I minified the json of the service account key and created a file called gcs.client.default.credentials_file with no file extension and added this file to kubernetes secrets.
And added the secureSettings.secretName field to the spec of the elastic cluster and added the secret name to it which was gcs-credentials
But i get this error on the logs
{"#timestamp":"2022-12-26T18:45:40.037Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elasticsearch-cluster-es-node-1","elasticsearch.cluster.name":"elasticsearch-cluster","error.type":"java.lang.IllegalStateException","error.message":"failed to load plugin class [org.elasticsearch.repositories.gcs.GoogleCloudStoragePlugin]","error.stack_trace":"java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.repositories.gcs.GoogleCloudStoragePlugin]\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:607)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:482)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:290)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:159)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$14(PluginsService.java:634)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.node.Node.<init>(Node.java:406)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.node.Node.<init>(Node.java:316)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:214)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:214)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)\nCaused by: java.lang.reflect.InvocationTargetException\n\tat java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79)\n\tat java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)\n\tat java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)\n\tat org.elasticsearch.server#8.5.0/org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:600)\n\t... 9 more\nCaused by: java.lang.IllegalArgumentException: failed to load GCS client credentials from [gcs.client.default.credentials_file]\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStorageClientSettings.loadCredential(GoogleCloudStorageClientSettings.java:265)\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStorageClientSettings.getClientSettings(GoogleCloudStorageClientSettings.java:221)\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStorageClientSettings.load(GoogleCloudStorageClientSettings.java:209)\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStoragePlugin.reload(GoogleCloudStoragePlugin.java:88)\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStoragePlugin.<init>(GoogleCloudStoragePlugin.java:36)\n\tat java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)\n\t... 12 more\nCaused by: java.io.IOException: Invalid PKCS#8 data.\n\tat com.google.auth.oauth2.ServiceAccountCredentials.privateKeyFromPkcs8(ServiceAccountCredentials.java:496)\n\tat com.google.auth.oauth2.ServiceAccountCredentials.fromPkcs8(ServiceAccountCredentials.java:474)\n\tat com.google.auth.oauth2.ServiceAccountCredentials.fromJson(ServiceAccountCredentials.java:212)\n\tat com.google.auth.oauth2.ServiceAccountCredentials.fromStream(ServiceAccountCredentials.java:548)\n\tat com.google.auth.oauth2.ServiceAccountCredentials.fromStream(ServiceAccountCredentials.java:520)\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStorageClientSettings.lambda$loadCredential$13(GoogleCloudStorageClientSettings.java:257)\n\tat java.base/java.security.AccessController.doPrivileged(AccessController.java:569)\n\tat org.elasticsearch.repositories.gcs.SocketAccess.doPrivilegedIOException(SocketAccess.java:33)\n\tat org.elasticsearch.repositories.gcs.GoogleCloudStorageClientSettings.loadCredential(GoogleCloudStorageClientSettings.java:256)\n\t... 17 more\n"}
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/elasticsearch-cluster.log
Try adding the following lines to your configuration (on each Elasticsearch):
elasticsearch01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
...
ulimits:
memlock:
soft: -1
hard: -1
Also check this link on Elasticsearch for more detailed information.
I am migrating from a self-hosted Elasticsearch FusionAuth search to an AWS Elasticsearch Service solution.
I have a new FusionAuth app EC2 instance reading from the in-use database that is configured to use the new Elasticsearch service.
On triggering a reindex from the new app instance I see that only around 60k or 62.5k documents are being written to the new index when I am expecting roughly 6mil.
I see no errors from AWS's Elasticsearch Service and in the app's logs I can see: (endpoint intentionally omitted)
Feb 13, 2020 10:18:46.116 AM INFO io.fusionauth.api.service.search.ElasticSearchClientProvider - Connecting to FusionAuth Search Engine at [https://vpc-<<omitted>>.eu-west-1.es.amazonaws.com]
13-Feb-2020 11:19:55.176 INFO [http-nio-9011-exec-3] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:430)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:684)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
"/usr/local/fusionauth/logs/fusionauth-app.log" [readonly] 43708L, 4308629C 42183,1 96%
at io.fusionauth.api.service.search.client.domain.documents.IndexUser.<init>(IndexUser.java:79)
at io.fusionauth.api.service.search.ElasticsearchSearchEngine.lambda$index$1(ElasticsearchSearchEngine.java:140)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at io.fusionauth.api.service.search.ElasticsearchSearchEngine.index(ElasticsearchSearchEngine.java:140)
at io.fusionauth.api.service.user.ReindexRunner$ReindexWorker.run(ReindexRunner.java:101)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-14" java.lang.NullPointerException
at io.fusionauth.api.service.search.client.domain.documents.IndexUser.<init>(IndexUser.java:79)
at io.fusionauth.api.service.search.ElasticsearchSearchEngine.lambda$index$1(ElasticsearchSearchEngine.java:140)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at io.fusionauth.api.service.search.ElasticsearchSearchEngine.index(ElasticsearchSearchEngine.java:140)
at io.fusionauth.api.service.user.ReindexRunner$ReindexWorker.run(ReindexRunner.java:101)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "Thread-13" java.lang.NullPointerException
Exception in thread "Thread-11" java.lang.NullPointerException
Exception in thread "Thread-12" java.lang.NullPointerException
Feb 18, 2020 10:23:29.064 AM INFO io.fusionauth.api.service.user.ReindexRunner - Reindex completed in [86797] ms or [86] seconds.
Although there are some exceptions there is also an "Reindex completed" INFO log at the end.
Not knowing the ins-and-outs of Elasticsearch I'm also not sure where to start in investigating a NullPointerException.
It looks like the re-index operation is taking an exception which is likely the cause of the truncated index.
Exception in thread "Thread-14" java.lang.NullPointerException
at io.fusionauth.api.service.search.client.domain.documents.IndexUser.<init>(IndexUser.java:79)
This code makes an assumption that you have a username or email address. This should be enforced by the FusionAuth APIs. But in this case, for this exception to occur the email and username are both NULL.
How did you get users into the db, using the Import API, User API, or something else?
In theory you should find at least one user with a NULL value for the email and username.
This query - or similar - should find the offending user, then we need to identify how this user was added to FusionAuth.
SELECT email, username from identities WHERE email IS NULL OR username IS NULL
let's say my Java chaincode (running on Fabric 1.4.4) wants to throw an exception to show that the new asset to be created already exists. I am throwing a RunTimeException with the problem or error (In this case, "Contract LL00001 already registered") which is logged in the Peer node executing the transaction:
2019-11-29 20:15:37.807 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.1.4] func2 -> INFO 16a8ec Contract LL00001 already registered
2019-11-29 20:15:37.807 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.1.4] func2 -> INFO 16a8ed java.lang.RuntimeException: Contract LL00001 already registered
But then, after the stack trace I see that the peer node is returning it as 500 error without includeing my error description or any reference to the error Exception in java (this makes sense as that error is language agnostic):
2019-11-29 20:15:37.807 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.1.4] func2 -> INFO 16a8ff 20:15:37:804 SEVERE org.hyperledger.fabric.shim.impl.ChaincodeInnvocationTask call [1f56a053] Invoke failed with error code 500. Sending ERROR
Which is logged in my client java application (which uses fabrik-java-sdk):
org.hyperledger.fabric.sdk.exception.InvalidArgumentException: Proposal response is invalid.
at org.hyperledger.fabric.sdk.ProposalResponse.getChaincodeActionResponsePayload(ProposalResponse.java:272)
at ...
So I just know that there was a problem from the chaincode, but I can't know what the problem is. How can I get the error type and description so I can show the problem to the user? Now I need to go to the peer node an check logs there to see what problem is.
Note: I am extending the new org.hyperledger.fabric.contract.ContractInterface in my chaincode class.
Update: peer node logs the error exception (org.hyperledger.fabric.shim.ChaincodeException) and seems to return correctly the error message ("The document was not found") in the 500 response as shown in log, but this message does not get to Java SDK
2019-12-23 22:11:09.178 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.9.7] func2 -> INFO 5aa7 22:11:09:176 SEVERE org.hyperledger.fabric.shim.impl.ChaincodeInnvocationTask call [12cc4ad0] Invoke failed with error code 500. Sending ERROR
2019-12-23 22:11:09.179 UTC [peer.chaincode.nid1-blockchain-hapeer1-mrrc-0.9.7] func2 -> INFO 5aa8 22:11:09:177 FINE org.hyperledger.fabric.shim.impl.ChaincodeSupportClient$2 accept > sendToPeer 12cc4ad09a1feb7fc1246ac04bf69509204ca74368be2c7e4bbf0a503e90417f
2019-12-23 22:11:09.181 UTC [endorser] callChaincode -> INFO 5aa9 [mrrc][12cc4ad0] Exit chaincode: name:"mrrc" (36ms)
2019-12-23 22:11:09.181 UTC [endorser] SimulateProposal -> ERRO 5aaa [mrrc][12cc4ad0] failed to invoke chaincode name:"mrrc" , error: transaction returned with failure: The document was not found
Edit: It seems to be a error in Java SDK. I have created a JIRA issue in Fabric's JIRA:
https://jira.hyperledger.org/browse/FABJ-508
To throw this error back to your fabric java sdk client, one way would be to make your chaincode class extend the ChaincodeBase class (which can be imported in your java program by importing org.hyperledger.fabric.shim ) and then you can use its newErrorResponse method in each of your chaincode methods to throw your custom errors wherein you can provide the error string as it's first (or only) parameter. You can possibly have a look at this example from the fabric samples repo:
https://github.com/hyperledger/fabric-samples/blob/master/chaincode/abstore/java/src/main/java/org/hyperledger/fabric-samples/ABstore.java#L28
To have a look at the overloaded implementations of the newErrorResponse method, so that you can see what other possible params you can pass to it, please follow:
https://jar-download.com/artifacts/org.hyperledger.fabric-chaincode-java/fabric-chaincode-shim/1.4.0/source-code/org/hyperledger/fabric/shim/ChaincodeBase.java
UPDATE : If you are using the newer chaincodeInterface (as suggested by icordoba) for your chaincode implementations instead, then you should throw an instance of ChaincodeException class which can be imported by importing org.hyperledger.fabric.shim.ChaincodeException, to achieve the same, you can have a look at a sample chaincode here
I am getting below exception while doing EntityManager.find(). We are using DB2 database and WAS 8.0 server for our application. Any help greatly appreciated.
Caused by: <openjpa-2.1.2-SNAPSHOT-r422266:1709309 nonfatal store error> org.apache.openjpa.persistence.OptimisticLockException: Unable to obtain an object lock on "null".
at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4930)
at org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4908)
at org.apache.openjpa.jdbc.sql.DB2Dictionary.newStoreException(DB2Dictionary.java:603)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:136)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:110)
at org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:62)
at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:139)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:1012)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:870)
at org.apache.openjpa.kernel.QueryImpl.execute(QueryImpl.java:801)
at org.apache.openjpa.kernel.DelegatingQuery.execute(DelegatingQuery.java:542)
at org.apache.openjpa.persistence.QueryImpl.execute(QueryImpl.java:315)
at org.apache.openjpa.persistence.QueryImpl.getResultList(QueryImpl.java:331)
... 116 more
Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: DB2 SQL Error: SQLCODE=-913, SQLSTATE=57033, SQLERRMC=00C90088;00000304;ODNC001 .SNCPC145.X'200D65' '.X'43', DRIVER=4.15.134 {prepstmnt -1803801027
SELECT a.column1
FROM table_test a
WHERE (a.column2 = ? AND a.column3 = ?)
[params=(String) 00000, (String) 000011]} [code=-913, state=57033]
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:281)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:265)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$700(LoggingConnectionDecorator.java:72)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingPreparedStatement.executeQuery(LoggingConnectionDecorator.java:1183)
at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:284)
at org.apache.openjpa.jdbc.kernel.JDBCStoreManager$CancelPreparedStatement.executeQuery(JDBCStoreManager.java:1787)
at org.apache.openjpa.lib.jdbc.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:274)
at org.apache.openjpa.jdbc.kernel.PreparedSQLStoreQuery$PreparedSQLExecutor.executeQuery(PreparedSQLStoreQuery.java:118)
... 123 more
sqlode -913 with SQLERRMC=00C90088 means that your connection experienced a DEADLOCK.
If your Db2-server is running on Z/OS, then ask your DBA for help to find the other Db2-connection and the SQL-statements running in both transactions.The access-plans and isolation levels used by both connections are also relevant, as are any applicable lock timeouts. The Db2-server DBA has access to diagnostic tools to help you.
There are many hits online giving advice on how to reduce the likelihood of Db2 deadlocks, so do your research.
You will need to know the isolation level being used by the Websphere connection (or package, or SQL-statement(s)), and all the statements in the Db2-transaction for your connection.
The other tokens in the message are also relevant i.e. ODNC001.SNCPC145 may be the involved table.
The version of the jdbc type4 driver being used by your Websphere is out of date (looks like it is from a Db2 v10.1 fixpack 5 build) so consider getting that upgraded to a current version.
We're debugging an error that causing a crash in a Tomcat web application.
The application uses 2 3rd-party apps over jni, one of the 3rd-parties using SmartHeap (it is a memory management library for c/c++ applications), the other don't (it is webMethods broker version 5).
The strange thing is I see in the crash log that webMethods calls its native methods to initiate a connection to the broker server, but if I print the call trace of the thread where the crash happened using WinDbg (loading the minidump file created when the JVM crashed), it contains calls to SmartHeap functions. Now i feel I'm a bit lost... because I've checked, and found no references to this dll from the webMethods binaries.
(actually a memory allocation is called)
My question is how is it possible?
I mean anybody could describe how this part is working? Because I thought that the interpreted/compiled and native frames are called in a fixed order (it is logical).
maybe the call stack is invalid? (now we have many dump files with almost the same call trace)
or the call trace (the calling order of the native functions) is valid, only some of the functions have been reordered before calling (like a lazy object has to be generated before sending it to the webMethods broker, but i don't see any sign of this)
I'm querying the call trace on the dump file by calling ".ecxr" and "kv", the output is:
0:060> .ecxr
eax=4d330554 ebx=4d350010 ecx=4d330010 edx=00000000 esi=4d350010 edi=00000000
eip=4c912f15 esp=4bf1dad0 ebp=3574884d iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206
shsmp!shi_allocSmall2+0x195:
4c912f15 8b4d00 mov ecx,dword ptr [ebp] ss:0023:3574884d=????????
0:060> k
*** Stack trace for last set context - .thread/.cxr resets it
ChildEBP RetAddr
4bf1daec 4c912bbd shsmp!shi_allocSmall2+0x195
4bf1dafc 4c91b973 shsmp!MemAllocPtr+0x5d
*** WARNING: Unable to verify checksum for awssl50jn.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for awssl50jn.dll -
4bf1db14 49abc38d shsmp!shi_malloc_dbg+0x23
WARNING: Stack unwind information not available. Following frames may be wrong.
4bf1db3c 49abeca2 awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0xa1cd
4bf1db48 49ab5e66 awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0xcae2
4bf1db4c 49ab5e55 awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0x3ca6
4bf1db60 49ab667d awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0x3c95
4bf1db80 49abdbbc awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0x44bd
4bf1dc20 4c912f4f awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0xb9fc
4bf1dc78 49abd607 shsmp!shi_allocSmall2+0x1cf
00000000 00000000 awssl50jn!Java_COM_activesw_api_client_ssl_AwSSLNative_getSecurityInfo+0xb447`
Any help would be appreciated!