Need to parse a large json file - java

Currently i am getting into "java.lang.OutOfMemoryError: Java heap space"
below is the code written, here i am passing the input-stream to get the
JsonReader object to proceed further. Please let me know what should the
best approach. Thanks in Advance.
JsonReader reader = new JsonReader(new InputStreamReader(inputStream));
JsonObject element = new JsonParser().parse(reader).getAsJsonObject();
//stacktrace
com.domo.connector.util.MagicException: Failed parsing JSON source: JsonReader at line 1 column 80720228 to Json
at com.domo.connector.adaptv.api.Client.get(Client.java:125)
at com.domo.connector.adaptv.api.Client.get(Client.java:86)
at com.domo.connector.adaptv.api.AdhocReportClient.getFirstPage(AdhocReportClient.java:24)
at com.domo.connector.adaptv.ProcessRecords.getClientWithFirstPageOfData(ProcessRecords.java:116)
at com.domo.connector.adaptv.ProcessRecords.readData(ProcessRecords.java:58)
at com.domo.connector.adaptv.ProcessRecordsFunctionalTest.testCustomer(ProcessRecordsFunctionalTest.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
Caused by: com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 1 column 80720228 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88)
at com.domo.connector.adaptv.parser.AdapTVParser.parse(AdapTVParser.java:45)
at com.domo.connector.adaptv.api.Client.get(Client.java:102)
... 29 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.ArrayList.<init>(ArrayList.java:132)
at java.util.ArrayList.<init>(ArrayList.java:139)
at com.google.gson.JsonArray.<init>(JsonArray.java:40)
at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:656)
at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:667)
at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:659)
at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:667)
at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:642)
at com.google.gson.internal.Streams.parse(Streams.java:44)
at com.google.gson.JsonParser.parse(JsonParser.java:84)
at com.domo.connector.adaptv.parser.AdapTVParser.parse(AdapTVParser.java:45)
at com.domo.connector.adaptv.api.Client.get(Client.java:102)
at com.domo.connector.adaptv.api.Client.get(Client.java:86)
at com.domo.connector.adaptv.api.AdhocReportClient.getFirstPage(AdhocReportClient.java:24)
at com.domo.connector.adaptv.ProcessRecords.getClientWithFirstPageOfData(ProcessRecords.java:116)
at com.domo.connector.adaptv.ProcessRecords.readData(ProcessRecords.java:58)
at com.domo.connector.adaptv.ProcessRecordsFunctionalTest.testCustomer(ProcessRecordsFunctionalTest.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)

If your document is very big, you can not build a DOM tree / JSON object in memory. You have to use a pull or SAX parser and compute the document step by step. That will force you to rewrite the code that handles the JSON objects.
See JSON simple or Jackson
If you are parsing some kind of stream, the event-based approach would be better, if it is just a file, a pull parser will suffice.

Related

Implementing test runner for testng cucumber appium tests

Hi I am trying to setup our automation tests to run in AWS Device Farm but when trying to add a test runner it fails miserably on device farm. I am quite new to setting this up so this may be an obvious fix that I am missing.
To get started I forked the AWS example project found here and it all worked fine when running locally and in device farm.
I modified that codebase to test our apps login screen and again it worked fine both locally and on device farm. But when I tried adding a testrunner so that I can add tests for more screens it suddenly stopped working. I have read around online and from what I can see I have set it up correctly.
A branch for the working version of our tests can be found here
Branch with the broken version trying to add a test runner is here
These are the errors i am getting from device farms appium logs:
Background: A Login Page [90m# Features/LoginTest.feature:5[0m
[31mGiven [0m[31mI navigate to the login page[0m [90m# LoginTest.setUpPage()[0m
[31mjava.lang.NullPointerException
at io.appium.java_client.pagefactory.AppiumFieldDecorator.getTypeForProxy(AppiumFieldDecorator.java:132)
at io.appium.java_client.pagefactory.AppiumFieldDecorator.proxyForLocator(AppiumFieldDecorator.java:146)
at io.appium.java_client.pagefactory.AppiumFieldDecorator.decorate(AppiumFieldDecorator.java:92)
at org.openqa.selenium.support.PageFactory.proxyFields(PageFactory.java:112)
at org.openqa.selenium.support.PageFactory.initElements(PageFactory.java:104)
at Pages.BasePage.<init>(BasePage.java:51)
at Pages.IntroPage.<init>(IntroPage.java:24)
at Tests.LoginTest.setUpPage(LoginTest.java:43)
at ✽.Given I navigate to the login page(Features/LoginTest.feature:6)
[0m
[31mFailure in after hook:[0m[31mLoginTest.restartApp()[0m
[31mMessage: [0m[31mjava.lang.NullPointerException
at Tests.AbstractBaseTests.TestBase.afterScenario(TestBase.java:28)
at Tests.LoginTest.restartApp(LoginTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.java.JavaHookDefinition.execute(JavaHookDefinition.java:60)
at cucumber.runtime.Runtime.runHookIfTagsMatch(Runtime.java:224)
at cucumber.runtime.Runtime.runHooks(Runtime.java:212)
at cucumber.runtime.Runtime.runAfterHooks(Runtime.java:206)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:46)
at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
at cucumber.api.testng.TestNGCucumberRunner.runCucumber(TestNGCucumberRunner.java:63)
at TestRunner.feature(TestRunner.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
at org.testng.SuiteRunner.run(SuiteRunner.java:254)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.TestNG.privateMain(TestNG.java:1364)
at org.testng.TestNG.main(TestNG.java:1333)
[0m
[31mjava.lang.NullPointerException
at Tests.AbstractBaseTests.TestBase.afterScenario(TestBase.java:28)
at Tests.LoginTest.restartApp(LoginTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.java.JavaHookDefinition.execute(JavaHookDefinition.java:60)
at cucumber.runtime.Runtime.runHookIfTagsMatch(Runtime.java:224)
at cucumber.runtime.Runtime.runHooks(Runtime.java:212)
at cucumber.runtime.Runtime.runAfterHooks(Runtime.java:206)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:46)
at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
at cucumber.api.testng.TestNGCucumberRunner.runCucumber(TestNGCucumberRunner.java:63)
at TestRunner.feature(TestRunner.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
at org.testng.SuiteRunner.run(SuiteRunner.java:254)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.TestNG.privateMain(TestNG.java:1364)
at org.testng.TestNG.main(TestNG.java:1333)
[0m
Not sure what I have missed but any help will be appreciated.
Thanks

SOAP using Apache Axis 1...ColdFusion Error

So I am trying to make a SOAP call, and I need to use Apache Axis 1 because the endpoint uses RPC Encoding. I am having issues because on the response, I am getting this error:
Could not find deserializer for type
{http://rpc.xml.coldfusion}QueryBean
I assume web service is trying to convert the response into a QueryBean, but cannot, since I do not have that object in my app (I also tried printing out the response, but I got the same error).
My code is below. Not sure what to do, so if anyone has any ideas on how to solve this, or what exactly a QueryBean is, I would love to hear them. Thanks!
String endpoint = "[endpoint hidden]";
Service service = new Service();
Call call = (Call) service.createCall();
call.setTargetEndpointAddress(new java.net.URL(endpoint));
call.setOperationName(new QName("http://rpc.xml.coldfusion", "getReportContent"));
String ret = (String) call.invoke( new Object[] { "142", "5088721" });
System.out.println("Response: " + ret + "'");
} catch (Exception e) {
System.err.println(e.toString());
}
Update:
I found a jar that contained QueryBean (axis-client.jar), and I figured that would solve my problem. I still get the same error though. I tried debugging, and got a bigger stack trace, so I figured I would post it here:
2016-08-12 11:20:22,266 ERROR [Call:2469] Exception:
org.xml.sax.SAXException: Deserializing parameter 'getReportContentReturn': could not find deserializer for type {http://rpc.xml.coldfusion}QueryBean
at org.apache.axis.message.RPCHandler.onStartChild(RPCHandler.java:277)
at org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)
at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:165)
at org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
at org.apache.axis.message.RPCElement.deserialize(RPCElement.java:345)
at org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
at org.apache.axis.client.Call.invoke(Call.java:2467)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.elderscan.ncoa.service.NCOAWebServiceTest.axisTest(NCOAWebServiceTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:128)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1203)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1128)
at org.testng.TestNG.run(TestNG.java:1036)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:152)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:57)
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: org.xml.sax.SAXException: Deserializing parameter 'getReportContentReturn': could not find deserializer for type {http://rpc.xml.coldfusion}QueryBean
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:org.xml.sax.SAXException: Deserializing parameter 'getReportContentReturn': could not find deserializer for type {http://rpc.xml.coldfusion}QueryBean
at org.apache.axis.message.RPCHandler.onStartChild(RPCHandler.java:277)
at org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)
at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:165)
at org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
at org.apache.axis.message.RPCElement.deserialize(RPCElement.java:345)
at org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
at org.apache.axis.client.Call.invoke(Call.java:2467)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.elderscan.ncoa.service.NCOAWebServiceTest.axisTest(NCOAWebServiceTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:128)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1203)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1128)
at org.testng.TestNG.run(TestNG.java:1036)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:152)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:57)
{http://xml.apache.org/axis/}hostname:xxx
org.xml.sax.SAXException: Deserializing parameter 'getReportContentReturn': could not find deserializer for type {http://rpc.xml.coldfusion}QueryBean
at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
at org.apache.axis.client.Call.invoke(Call.java:2470)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.elderscan.ncoa.service.NCOAWebServiceTest.axisTest(NCOAWebServiceTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:128)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1203)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1128)
at org.testng.TestNG.run(TestNG.java:1036)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:152)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:57)
Caused by: org.xml.sax.SAXException: Deserializing parameter 'getReportContentReturn': could not find deserializer for type {http://rpc.xml.coldfusion}QueryBean
at org.apache.axis.message.RPCHandler.onStartChild(RPCHandler.java:277)
at org.apache.axis.encoding.DeserializationContext.startElement(DeserializationContext.java:1035)
at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:165)
at org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
at org.apache.axis.message.RPCElement.deserialize(RPCElement.java:345)
at org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
at org.apache.axis.client.Call.invoke(Call.java:2467)
... 27 more
Problem Solved. With the help of this very helpful StackOverflow answer (Could not find deserializer for type : Error) I was able to resolve the error. The REST call worked fine after doing the following:
Added the jar containing QueryBean (axis-client.jar) to my class path
Added the following to my code:
Service service = new Service();
Call call = (Call) service.createCall();
QName q = new QName("http://rpc.xml.coldfusion", "QueryBean");
BeanSerializerFactory bsf = new BeanSerializerFactory(QueryBean.class, q); // step 2
BeanDeserializerFactory bdf = new BeanDeserializerFactory(QueryBean.class, q); // step 3
call.registerTypeMapping(QueryBean.class, q, bsf, bdf); //step 4
call.setTargetEndpointAddress(new java.net.URL(endpoint));
call.setOperationName(new QName(endpoint, "getReportContent"));
QueryBean bean = (QueryBean) call.invoke( new Object[] { "142", "5088721" });

Nullpointer Exception in testNG data provider while executing through Jenkins job

I'm trying to execute my testng test which has data provider defined through Jenkins Job. I'm facing below null pointer exception. But, same test case executing without any errors in my local machine (from Eclipse)
Exception java.lang.RuntimeException
Message: java.lang.NullPointerException
Stacktrace:
at org.testng.internal.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:161)
at org.testng.internal.Parameters.handleParameters(Parameters.java:429)
at org.testng.internal.Invoker.handleParameters(Invoker.java:1383)
at org.testng.internal.Invoker.createParameters(Invoker.java:1075)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1180)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:295)
at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:84)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:90)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by:
at com.MyTest.getDetails(GetDetailsTest.java:207)
at com.MyTest.sendValidParamters(GetDetailsTest.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:135)
... 23 more
My data provider is something like this :
#DataProvider(name = "valid_parameters")
public Object[][] sendValidParamters() throws Exception {
return new Object[][] {
{ getDetailsFromDB("getFromDB", true, "images/backgrnd.jpeg")},
getDetailsFromDB("getFromDB", true, "images/backgrnd.jpeg")}};
In which, I'm trying to fetch some rows from table based on parameters.

Error while using POI API

I am using selenium webdriver. I am writing to excel file using POI API.
I am getting following error:
org.apache.xmlbeans.impl.values.XmlValueDisconnectedException
at org.apache.xmlbeans.impl.values.XmlObjectBase.check_orphaned(XmlObjectBase.java:1258)
at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTXfImpl.getXfId(Unknown Source)
at org.apache.poi.xssf.model.StylesTable.getStyleAt(StylesTable.java:183)
at org.apache.poi.xssf.usermodel.XSSFCell.getCellStyle(XSSFCell.java:427)
at org.apache.poi.xssf.usermodel.XSSFCell.getCellStyle(XSSFCell.java:63)
at org.apache.poi.ss.usermodel.DateUtil.isCellDateFormatted(DateUtil.java:280)
at com.bcatman.util.Xls_Reader.getCellData(Xls_Reader.java:149)
at com.bcatman.base.TestBase.readExcelData(TestBase.java:159)
at com.bcatman.suite.CatalogImportExport.ImportSimpleExcel.testcaseImportSimpleExcel(ImportSimpleExcel.java:89)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)

Tess4j on Windows 64-bit: exception on multiple threads

I am using tesseract 3 with Java 8 on Windows 64-bit to OCR scanned PDFs. I have followed the instructions on the Tess4j page and have used the 64-bit versions of the required DLLs, and have installed 64-bit Ghostscript.
When I run my unit test with the normal #Test (no arguments), the code runs correctly, so I guess I have installed everything correctly.
When I run it with 2 threads in parallel (see below) I get an exception.
I have read the relevant thread here, but there it is suggested to use Tesseract1, which I am using (I have tried both).
Any ideas?
This is the code:
// #Test // works
#Test(invocationCount = 2, threadPoolSize = 2)
public void testOcr() throws OcrException, TesseractException {
File scannedPdf = new File(this.getClass().getClassLoader().getResource("scanned.pdf").getFile());
// Tesseract instance = Tesseract.getInstance(); // JNA Interface Mapping
Tesseract1 instance = new Tesseract1(); // JNA Direct Mapping
String str = instance.doOCR(scannedPdf);
System.out.println("OCR Result: " + str);
}
This is the exception:
log4j:WARN No appenders could be found for logger (org.ghost4j.Ghostscript).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Ιουλ 16, 2014 6:22:23 ΜΜ net.sourceforge.vietocr.PdfUtilities convertPdf2Png
SEVERE: Cannot initialize Ghostscript interpreter. Error code is -21
org.ghost4j.GhostscriptException: Cannot initialize Ghostscript interpreter. Error code is -21
at org.ghost4j.Ghostscript.initialize(Ghostscript.java:365)
at net.sourceforge.vietocr.PdfUtilities.convertPdf2Png(Unknown Source)
at net.sourceforge.vietocr.PdfUtilities.convertPdf2Tiff(Unknown Source)
at net.sourceforge.vietocr.ImageIOHelper.getIIOImageList(Unknown Source)
at net.sourceforge.tess4j.Tesseract1.doOCR(Unknown Source)
at net.sourceforge.tess4j.Tesseract1.doOCR(Unknown Source)
at OcrUtilsTest.testOcr(OcrUtilsTest.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
java.lang.Error: Invalid memory access
at com.sun.jna.Native.invokeInt(Native Method)
at com.sun.jna.Function.invoke(Function.java:383)
at com.sun.jna.Function.invoke(Function.java:315)
at com.sun.jna.Library$Handler.invoke(Library.java:212)
at com.sun.proxy.$Proxy3.gsapi_init_with_args(Unknown Source)
at org.ghost4j.Ghostscript.initialize(Ghostscript.java:350)
at net.sourceforge.vietocr.PdfUtilities.convertPdf2Png(Unknown Source)
at net.sourceforge.vietocr.PdfUtilities.convertPdf2Tiff(Unknown Source)
at net.sourceforge.vietocr.ImageIOHelper.getIIOImageList(Unknown Source)
at net.sourceforge.tess4j.Tesseract1.doOCR(Unknown Source)
at net.sourceforge.tess4j.Tesseract1.doOCR(Unknown Source)
at OcrUtilsTest.testOcr(OcrUtilsTest.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
net.sourceforge.tess4j.TesseractException: javax.imageio.IIOException: I/O error reading header!
at net.sourceforge.tess4j.Tesseract1.doOCR(Unknown Source)
at net.sourceforge.tess4j.Tesseract1.doOCR(Unknown Source)
at OcrUtilsTest.testOcr(OcrUtilsTest.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.imageio.IIOException: I/O error reading header!
at com.sun.media.imageioimpl.plugins.tiff.TIFFImageReader.readHeader(TIFFImageReader.java:224)
at com.sun.media.imageioimpl.plugins.tiff.TIFFImageReader.locateImage(TIFFImageReader.java:231)
at com.sun.media.imageioimpl.plugins.tiff.TIFFImageReader.getNumImages(TIFFImageReader.java:279)
at net.sourceforge.vietocr.ImageIOHelper.getIIOImageList(Unknown Source)
... 18 more
Caused by: java.io.EOFException
at javax.imageio.stream.ImageInputStreamImpl.readShort(ImageInputStreamImpl.java:229)
at javax.imageio.stream.ImageInputStreamImpl.readUnsignedShort(ImageInputStreamImpl.java:242)
at com.sun.media.imageioimpl.plugins.tiff.TIFFImageReader.readHeader(TIFFImageReader.java:199)
... 21 more
UPDATE: It seems related to this.
Tesseract on its own can only convert images to text, and not PDFs, even if the PDFs are scanned.
Under the hood, Tess4j uses Ghostscript (through ghost4j) to convert each page to a single image file, which it then feeds to Tesseract for OCR. It concatenates the resulting strings into a single string, which it returns.
The reason for the exception is that Tess4j uses Ghost4j in a way that does not support multithreading. As described here, ghost4j does provide multithreading support from its high-level API (actually it runs different instances of Ghostscript separately each invoked from a different JVM). Tess4j, however, uses its low-level API, where a single Ghostscript instance may be used.

Categories