We Keep Coding

Java Refection : NoSuchMethodException on high load

I am getting below exception ONLY when high load is running like 25 calls (same scenario) per second and it is not coming for every call, it is coming for few times only.However, when I run few calls at a time i am not getting this exception. I checked that method public execute method exists in com.abc.block.Rules class and this is the reason exception is not coming when i run few calls.
02 Oct 2019 02:00:01,021 [Worker[23]] ERROR [SNode] 80]
NoSuchMethodException during reflective call on class
com.abc.block.Rules java.lang.NoSuchMethodException:
com.abc.block.Rules.execute(com.abc.common.cdata) at
java.lang.Class.getMethod(Class.java:1786)
While running load reflection is not working correctly. Any inputs please ?
Code :
Object port = service.getClass()
.getMethod(xmlSNode.getPortMethodName()).invoke(service);
outResult = port
.getClass()
.getMethod(xmlSNode.getOperation().getName(),
inputs.getInputTypes())
.invoke(port, data);
Rule call :
public Object[] execute(cdata c) throws Exception{
...
}
Any input please

HystrixRunTimeException: <operation> timed-out and fallback failed

I'm using hystrix 1.3.7 and my hystrix command has a fallback method defined as well. So it is setup as follows:
public final Optional<ImageData> run() throws Exception {
// does api call to get resized image from a service
}
#Override
public final Optional<ImageData> getFallback() {
// falls back to processing the image locally.
}
However, I have realized that some times (not all times) when the timeout occurs for Hystrix, it seems like it does not execute the logic in getFallback method and throws HsytrixRuntimeException. Here is the stacktrace:
com.netflix.hystrix.exception.HystrixRuntimeException: imageResize timed-out and fallback failed.
at com.netflix.hystrix.AbstractCommand.handleTimeoutViaFallback(AbstractCommand.java:980)
at com.netflix.hystrix.AbstractCommand.access$500(AbstractCommand.java:59)
at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:595)
at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:587)
at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$1.onError(OperatorOnErrorResumeNextViaFunction.java:77)
at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:70)
at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:70)
at com.netflix.hystrix.AbstractCommand$HystrixObservableTimeoutOperator$1.run(AbstractCommand.java:1121)
at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:41)
at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable$1.call(HystrixContextRunnable.java:37)
at com.netflix.hystrix.strategy.concurrency.HystrixContextRunnable.run(HystrixContextRunnable.java:57)
at com.netflix.hystrix.AbstractCommand$HystrixObservableTimeoutOperator$2.tick(AbstractCommand.java:1138)
at com.netflix.hystrix.util.HystrixTimer$1.run(HystrixTimer.java:99)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Could this be because hystrix is not configured properly? Any help would be appreciated.

From the error logs I understand that your run() method got failed and the fall back is invoked. The logic you have inside the fallback also seems to get failed. Recommendation is that, fallback should be coded in such a way that it never gets failed. Please check your fallback code if that is failing. Either handle the failure inside fallback or simply move the logic somewhere out of fallback.

Hibernate connecting to DB2 at application startup on WAS Liberty on CICS

We're running a simple webapp on WebSphere Liberty, that uses Hibernate as persistence provider (included as a library in the WAR file).
When application is starting up Hibernate is initialized and it will open a connection to DB2 and issue some SQL statements. However, this fails when running on CICS and using JDBC Type 2 Driver DataSource. The following messages are logged (some extra line breaks for readability):
WARN org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator -
HHH000342: Could not obtain connection to query metadata : [jcc][50053][12310][4.19.56]
T2zOS exception: [jcc][T2zos]T2zosCicsApi.checkApiStatus:
Thread is not CICS-DB2 compatible: CICS_REGION_BUT_API_DISALLOWED ERRORCODE=-4228, SQLSTATE=null
...
ERROR org.hibernate.hql.spi.id.IdTableHelper - Unable obtain JDBC Connection
com.ibm.db2.jcc.am.SqlException: [jcc][50053][12310][4.19.56] T2zOS exception: [jcc][T2zos]T2zosCicsApi.checkApiStatus:
Thread is not CICS-DB2 compatible: CICS_REGION_BUT_API_DISALLOWED ERRORCODE=-4228, SQLSTATE=null
at com.ibm.db2.jcc.am.kd.a(Unknown Source) ~[db2jcc4.jar:?]
...
at com.ibm.db2.jcc.t2zos.T2zosConnection.a(Unknown Source) ~[db2jcc4.jar:?]
...
at com.ibm.db2.jcc.DB2SimpleDataSource.getConnection(Unknown Source) ~[db2jcc4.jar:?]
at com.ibm.cics.wlp.jdbc.internal.CICSDataSource.getConnection(CICSDataSource.java:176) ~[?:?]
at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) ~[our-app.war:5.1.0.Final]
at org.hibernate.internal.SessionFactoryImpl$3.obtainConnection(SessionFactoryImpl.java:643) ~[our-app.war:5.1.0.Final]
at org.hibernate.hql.spi.id.IdTableHelper.executeIdTableCreationStatements(IdTableHelper.java:67) [our-app.war:5.1.0.Final]
at org.hibernate.hql.spi.id.global.GlobalTemporaryTableBulkIdStrategy.finishPreparation(GlobalTemporaryTableBulkIdStrategy.java:125) [our-app.war:5.1.0.Final]
at org.hibernate.hql.spi.id.global.GlobalTemporaryTableBulkIdStrategy.finishPreparation(GlobalTemporaryTableBulkIdStrategy.java:42) [our-app.war:5.1.0.Final]
at org.hibernate.hql.spi.id.AbstractMultiTableBulkIdStrategyImpl.prepare(AbstractMultiTableBulkIdStrategyImpl.java:88) [our-app.war:5.1.0.Final]
at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:451) [our-app.war:5.1.0.Final]
My current understanding is that when running on CICS and using JDBC Type 2 Drivers only some threads are capable of opening a DB2 connection. That would be the application threads (the ones processing HTTP requests) as well as worker threads servicing CICSExecutorService.
The current solution is to:
Disable JDBC metadata lookup in JdbcEnvironmentInitiator by
setting hibernate.temp.use_jdbc_metadata_defaults property to
false
Wrap execution of IdTableHelper#executeIdTableCreationStatements in a Runnable and submit it to CICSExecutorService.
Would you consider this solution to be sufficient and suitable for production? Or maybe you use some different approach?
Versions used:
CICS Transaction Server for z/OS 5.3.0
WebSphere Application Server 8.5.5.8
Hibernate 5.1.0
Update: Just to clarify, once our application is started, it can query DB2 with no problems (when servicing HTTP requests). The problem is only related to startup.

CICS TS v5.3 support for the JPA feature in Liberty was recently made available in a service-refresh (July 2016). Prior to that update, attempting to run JPA in applications would result in very similar problems to those you describe.
Although you are running hibernate and you are on a CICS-enabled thread, it does not have the API environment (which will allow the type 2 JDBC call to succeed). New detection logic was developed specifically (but not exclusively) for use with the DB2 JDBC type 2 driver and JPA. This update was shipped in a recent service refresh and might cure the issues you are seeing.
Try applying:
http://www-01.ibm.com/support/docview.wss?crawler=1&uid=swg1PI58375
The description says it is for 'Standard-mode Liberty' support, but it contains other developments as outlined above.

The following solution was tested to work ok.
The idea is to execute the SQL/DDL statements using CICSExecutorService#runAsCICS. The following extension is registered via hibernate.hql.bulk_id_strategy property.
package org.hibernate.hql.spi.id.global;
import java.util.concurrent.*;
import org.hibernate.boot.spi.MetadataImplementor;
import org.hibernate.engine.jdbc.connections.spi.JdbcConnectionAccess;
import org.hibernate.engine.jdbc.spi.JdbcServices;
import org.springframework.util.ClassUtils;
import com.ibm.cics.server.*;
public class CicsAwareGlobalTemporaryTableBulkIdStrategy extends GlobalTemporaryTableBulkIdStrategy {
#Override
protected void finishPreparation(JdbcServices jdbcServices, JdbcConnectionAccess connectionAccess, MetadataImplementor metadata, PreparationContextImpl context) {
execute(() -> super.finishPreparation(jdbcServices, connectionAccess, metadata, context));
}
#Override
public void release(JdbcServices jdbcServices, JdbcConnectionAccess connectionAccess) {
execute(() -> super.release(jdbcServices, connectionAccess));
}
private void execute(Runnable runnable) {
if (isCics() && IsCICS.getApiStatus() == IsCICS.CICS_REGION_BUT_API_DISALLOWED) {
RunnableFuture<Void> task = new FutureTask<>(runnable, null);
CICSExecutorService.runAsCICS(task);
try {
task.get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException("Failed to execute in a CICS API-enabled thread. " + e.getMessage(), e);
}
} else {
runnable.run();
}
}
private boolean isCics() {
return ClassUtils.isPresent("com.ibm.cics.server.CICSExecutorService", null);
}
}
Note that the newer JCICS API version has an overlaod for runAsCics method accepting a Callable, which might be useful to simplify the CICS branch of the execute method to something like this:
CICSExecutorService.runAsCICS(() -> { runnable.run(); return null; }).get();
A few alternatives tried:
Wrapping just the connection acquisition action (org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl#getConnection) did not work as the connection was closed already when it was used in the main thread.
Wrapping the whole application startup (org.springframework.web.context.ContextLoaderListener#contextInitialized) led to classloading issues.
Edit: Eventually went with a custom Hibernate's MultiTableBulkIdStrategy implementation that does not run any SQL/DDL on startup (see project page on GitHub).

Android error randomly java.lang.NoClassDefFoundError: com.facebook.internal.Utility

I am using latest Facebook Android SDK and getting that error from dozens of users in my remote crash control app in my latest released apk. I have looked here for such error, but most of the answers are too outdated for last FB SDK, and in this case there are two weird circumstances:
a) The error seems to happen randomly. I have been unable to reproduce it at all on none of my devices.
b) There were no changes in FB logic at all between that release and the previous one, and in the previous release I have never had such error.
Since I couldn't find any relevant difference in the code between such versions, I thought the problem was something wrong could have happened with Android Tools while generating the last apk, but giving the fact that very same apk is the one I am using and have been unable to reproduce the problem and, despite dozens or users are affected, hundreds using the same apk aren't, I discarded also such hypothesis.
Any ideas on how to solve or just debug this thing are welcome.
More info that could be relevant:
All crashes happened in Android 4.0.3 or older. The highest percentage is for 2.3.6 with 48% of all the crashes.
That class is actually exported in the APK. I have checked it by unzipping the apk and using dexdump to see what is inside classes.dex. I couldn't expected other thing, since it works perfectly in all my devices and if the class weren't there, it would not.
$ ~/android-sdks/build-tools/21.1.1/dexdump classes.dex | grep 'com.facebook.internal.Utility$1'
Class descriptor : 'Lcom/facebook/internal/Utility$1;'
#0 : (in Lcom/facebook/internal/Utility$1;)
#1 : (in Lcom/facebook/internal/Utility$1;)
#2 : (in Lcom/facebook/internal/Utility$1;)
#0 : (in Lcom/facebook/internal/Utility$1;)
#0 : (in Lcom/facebook/internal/Utility$1;)
#1 : (in Lcom/facebook/internal/Utility$1;)
#2 : (in Lcom/facebook/internal/Utility$1;)
#3 : (in Lcom/facebook/internal/Utility$1;)
It seems to fail after an invocation of a Utility's static method from loadAppSettingsAsync, that happens to be a static method within the same class. So, if the com.facebook.internal.Utility class does not exist or couldn't be loaded, how it is possible that com.facebook.internal.Utility.loadAppSettingsAsync is executed in first place? and if it exists and it is loaded, why is NoClassDefFoundError thrown on com.facebook.internal.Utility? I am so f* lost...
Here the stack from splunk mint (formerly known as bugsense), I just changed the name of the app. I retraced it with the proguard map file, but it seems it missed some line numbers anyway:
java.lang.NoClassDefFoundError: com.facebook.internal.Utility$1
at com.facebook.internal.Utility.void loadAppSettingsAsync(android.content.Context,java.lang.String)(Unknown Source)
at com.facebook.Settings.void sdkInitialize(android.content.Context)(Unknown Source)
at com.facebook.UiLifecycleHelper.<init>(Unknown Source)
at net.iberdroid.androidgames.framework.impl.AndroidGame.void onCreate(android.os.Bundle)(Unknown Source)
at com.marzoa.ruletafree.xmas2012.RuletaAfortunadaGame.void onCreate(android.os.Bundle)(Unknown Source)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1050)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1623)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1675)
at android.app.ActivityThread.access$1500(ActivityThread.java:121)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:943)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:130)
at android.app.ActivityThread.main(ActivityThread.java:3770)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:507)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:912)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:670)
at dalvik.system.NativeStart.main(Native Method)
Caused by: java.lang.ClassNotFoundException: com.facebook.internal.Utility$1 in loader dalvik.system.PathClassLoader[/data/app/com.marzoa.ruletafree.xmas2012-2.apk]
at dalvik.system.PathClassLoader.findClass(PathClassLoader.java:240)
at java.lang.ClassLoader.loadClass(ClassLoader.java:551)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
... 18 more

As the NoClassDefFoundError occurs in loadAppSettingsAsync at an anonymous implementation of AsyncTask it seems to be the same problem like this NoClassDefFoundError - Android 2.3.X
Its a bug in google play services. (https://code.google.com/p/android/issues/detail?id=81083)
Try to add following into your Application#onCreate() method as described in the answer of the referred issue.
try {
Class.forName("android.os.AsyncTask");
}
catch(Throwable ignore) {
// ignored
}

Firstly, be sure that you are checking for the right class. According to the error message, the "missing" class is com.facebook.internal.loadAppSettingsAsync$1. Note the $1!! That means we are talking about an anonymous inner class that is declared within the Utility.loadAppSettingsAsync.
Second, java.lang.NoClassDefFoundError can have a number of causes:
The class may be missing.
The class could be present but unloadable for some reason.
The class could be loadable, but a previous attempt to initialize it ... or some dependent class ... has failed.
The last case typically happens if a static initialization or static initializer block throws an unchecked exception the first time it was run. (You will typically a log message for this exception.) Once the class initialization has failed once, the class is marked as broken. This stops the class and any other classes that depends on it from being initialized.

Cloudant j2se prototype throwing HTTPRequest exception

I'm trying to follow the replication guide on github for cloudant to prototype a basic one way connection between a local couchdb and and a locally running j2se app utilizing the cloudant sync library.
The local datastore file is being created, however almost immediately after running my application, I'm getting a runtime error - which at a glance I'm a little unsure of how to resolve: Asside from the cloudant code and dependancies;
This is the entire code for my app - I have apache couchdb installed on my machine and the database called 'baseball' exists already:
import com.cloudant.sync.datastore.DatastoreManager;
import com.cloudant.sync.datastore.Datastore;
import com.cloudant.sync.replication.*;
import com.cloudant.sync.notifications.*;
import com.google.common.eventbus.Subscribe;
import java.util.concurrent.CountDownLatch;
import java.io.*;
import java.net.*;
//https://github.com/cloudant/sync-android/blob/master/doc/replication.md
public class CloudApp{
public static void main(String [] args) throws Exception{
File path = new File("datastores");
System.out.println(path.getAbsolutePath());
DatastoreManager manager = new DatastoreManager(path.getAbsolutePath());
URI uri = new URI("http://127.0.0.1:5984/baseball");
Datastore ds = manager.openDatastore("my_datastore");
// Create a replictor that replicates changes from the remote
// database to the local datastore.
Replicator replicator = ReplicatorFactory.oneway(uri, ds);
// Use a CountDownLatch to provide a lightweight way to wait for completion
CountDownLatch latch = new CountDownLatch(1);
Listener listener = new Listener(latch);
replicator.getEventBus().register(listener);
replicator.start();
latch.await();
replicator.getEventBus().unregister(listener);
if (replicator.getState() != Replicator.State.COMPLETE) {
System.out.println("Error replicating FROM remote");
System.out.println(listener.error);
}
}
}
class Listener {
private final CountDownLatch latch;
public ErrorInfo error = null;
Listener(CountDownLatch latch) {
this.latch = latch;
}
#Subscribe
public void complete(ReplicationCompleted event) {
latch.countDown();
}
#Subscribe
public void error(ReplicationErrored event) {
this.error = event.errorInfo;
latch.countDown();
}
}
And this is the runtime error as well as log outputs:
[I]DatastoreManager: path: /Users/reecegriffin/Documents/workspace/Cloudant 2/datastores
[I]DatastoreManager: dbDirectory: /Users/reecegriffin/Documents/workspace/Cloudant 2/datastores/my_datastore
Mar 10, 2014 9:29:02 AM com.almworks.sqlite4java.Internal log
INFO: [sqlite] DB[1]: instantiated [/Users/reecegriffin/Documents/workspace/Cloudant 2/datastores/my_datastore/db.sync]
Mar 10, 2014 9:29:02 AM com.almworks.sqlite4java.Internal log
INFO: [sqlite] Internal: loaded sqlite4java-osx from /Users/reecegriffin/Documents/workspace/Cloudant 2/src/libsqlite4java-osx.jnilib
Mar 10, 2014 9:29:02 AM com.almworks.sqlite4java.Internal log
INFO: [sqlite] Internal: loaded sqlite 3.7.10, wrapper 0.2
Mar 10, 2014 9:29:02 AM com.almworks.sqlite4java.Internal log
INFO: [sqlite] DB[1]: opened
Exception in thread "main" java.lang.IllegalStateException: java.lang.RuntimeException: Stub!
at com.cloudant.mazha.HttpRequests.createHttpClient(HttpRequests.java:185)
at com.cloudant.mazha.HttpRequests.<init>(HttpRequests.java:69)
at com.cloudant.mazha.CouchClient.<init>(CouchClient.java:47)
at com.cloudant.sync.replication.CouchClientWrapper.<init>(CouchClientWrapper.java:45)
at com.cloudant.sync.replication.BasicPullStrategy.<init>(BasicPullStrategy.java:90)
at com.cloudant.sync.replication.PullReplication.createReplicationStrategy(PullReplication.java:40)
at com.cloudant.sync.replication.BasicReplicator.getReplicationStrategy(BasicReplicator.java:41)
at com.cloudant.sync.replication.BasicReplicator.start(BasicReplicator.java:61)
at CloudApp.main(CloudApp.java:27)
Caused by: java.lang.RuntimeException: Stub!
at org.apache.http.params.AbstractHttpParams.<init>(AbstractHttpParams.java:5)
at org.apache.http.params.BasicHttpParams.<init>(BasicHttpParams.java:6)
at com.cloudant.mazha.HttpRequests.getHttpConnectionParams(HttpRequests.java:199)
at com.clou
dant.mazha.HttpRequests.createHttpClient(HttpRequests.java:173)
... 8 more
Many thanks for any help.
Regards,
Reece.

I noticed in some other threads regarding Stub related apache commons HTTPclient related stuff, that the android library build path order can have an effect. I was trying to run from within eclipse & sure enough - after I opened up the build path editor and moved android.jar to be last on my build path, the run time error went away.

There's some more information on the "stub!" issue in Using android.jar in Java project - RuntimeException Stub?.
We'll take a look into whether we can mitigate the effects of the build path order in code, but I'm not sure whether we'll be able to fix the issue.
This problem hasn't come up for us in tests. Perhaps this is because the Android dependency is specified last in our build.gradle file, meaning it ends up in a position in the path ordering that works.