Our enterprise datawarehouse (EDW) sources data from many interanal & external sources. We use Oracle data integrator (ODI) as teh ETL tool to extract data from the sources. ETL jobs are executed using an ODI agent which is nothing but a Java process. One of the external sources for the warehouse is a webservice. Data is extracted from this webserice using Jython (wrapped within ODI). Jython as you might know is a Python variant which runs within JVM. The Jython job runs under the ODI agent (java) process and extracts data from the webservice.
Coming to the problem we are facing now- The external webservice provider has indicated that soon they will support only TLS 1.2 and will remove support for older TLS connections. The ODI agent is currently on JDK1.6 Update 33 and support TLS 1.1. We are planning to upgrade the ODI agent to Java 7 (which supports TLS 1.2) so that we will be able to connect to this webservice. The Java upgrade involves lot of work mainly to ensure that it does not break existing ETL jobs. If there are any impacts, we might not be able to upgrade. An alternate option is it create a separate Java process (outside of ODI) running under its own JDK (which can be even Java 8) to connect to this web service.
Are there any risks in the above options? It is going to take a while for the provider to give the webservice which supports only TLS 1.2. Are there any public webservices which support only TLS 1.2 that can be used by for our testing? Can you please share them?
Maybe not entirely what you're looking for, but it might be wise to have two seperate development projects for both the old and the future ODI agent to keep things transparent. Especially when testing the new version. It's also advisable to set up a dedicated testing/QA environment for this.
Be sure to keep your production ODI repository safe, you could use a testing repository with some exported projects for this. Maybe this link below might be useful: https://www.redbridgesoftware.com/products/life-cycle-management-for-odi.html
Related
I know how to talk to MaprDB from Java program with Mapr client installed. But, want to know how can I connect to MaprDB without Mapr client installed? Is there a pure Java API (i.e. without needing to install any other software, just using some jars) for my requirement?
Please note - these are binary tables (and not JSON tables).
There is an approach that you can use.
The so-called DAG (data access gateway) is a gRPC-based gateway that you can use. There is a pure Java client for this. Performance impact is significant, but not massive and the API is the same OJAI API as when you access the database using the thick client.
See the following links for more information:
https://mapr.com/docs/61/MapROverview/MapRDataAccessGateway.html
https://mapr.com/docs/61/AdvancedInstallation/InstallingDataAccessGateway.html
https://mapr.com/docs/61/AdministratorGuide/AdminMapRDataAccessGateway.html
https://mapr.com/docs/61/EcosystemRN/MapR_Data_Access_Gateway_2.0_release_notes.html
Depends upon your definition of "without needing to install any other software".
You don't need to install the mapr-client package to access the MapR Cluster from a Java application - the MapR jars can be installed to your preferred location on the classpath or even shaded into an "uber" jar.
But you do need to be running on a supported client platform (Linux x86-64, MacOS x86-64, Windows i386 or x86-64), because the maprfs jar does contain JNI libraries packaged within it that it unpacks at initialisation time.
I have a Google AppEngine (Java) project in Eclipse. I want to debug my local code in Eclipse but use the deployed database on AppEngine. Until now I use Remote API with username/password (old way)
This method will be deprecated and I want to use OAuth but when I try to use it, it throws an exception:
java.lang.IllegalStateException: OAuth-based authorization not supported for clients running on App Engine
at com.google.appengine.tools.remoteapi.RemoteApiOptions.getOrCreateHttpTransportForOAuth(RemoteApiOptions.java:359)
at com.google.appengine.tools.remoteapi.RemoteApiOptions.useApplicationDefaultCredential(RemoteApiOptions.java:162)
Everthing is fine when I run a simple Java client app that uses remote api in Eclipse. But if the client is AppEngine development environment in Eclipse, it doesn't work.
How can I debug the server code in eclipse using appengine database?
Bug report: https://code.google.com/p/googleappengine/issues/detail?id=12556
This is expected, as value returned by ApiProxy.getCurrentEnvironment() will not be null, but com.google.appengine.tools.development.LocalHttpRequestEnvironment object. For development server you'll have to use old style (username/password) for now.
According to the documentation, you need to add appengine-remote-api.jar from ${SDK_ROOT}/lib/appengine-remote-api.jar to your WEB-INF/lib directory (or add it to your Maven dependencies) before this will work.
Update: Indeed, it looks like OAuth from App Engine with Remote API has not yet been implemented. I would assume this will change before ClientLogin is fully deprecated, but for now I would recommend opening a feature request on the public issue tracker.
One possible workaround would be to create a regular console application that connects to the Remote API (as per the example in the docs) and can act as a proxy for your App Engine application running on the development server.
In addition to what my colleagues Adam and Nikita previously said, I can officially confirm that the Google Cloud Engineering Team is determined to provide a solution to all reasonable use-cases which are affected by the ClientLogin shutdown before its deadline.
Presumably, the Remote API solution will be available in the upcoming releases 1.9.31/32, although this is just an assumption and there's no ETA for it yet.
As an alternative, you can access Cloud Datastore using Protobufs remotely using service account credentials, which might fit your needs for the moment.
UPDATE 2016/01/21:
The team has extended the ClientLogin shutdown deadline to April 12, 2016.
UPDATE 2016/04/12:
As mentioned before, the fix was properly applied and available in the latest versions of the SDK and gcloud. Remote API can now be used again with OAuth for app-to-app (or devserver-to-app) connection.
I just read the CloudBees whitepaper "CloudBees Advantages: A Guide for Java Developers". In it there is a sentence that reads:
The database is simply available as a JNDI resource as soon as it is deployed – there is no need to chase down JDBC drivers, connection strings and configure JNDI.
So I assume that means I don't need to include a MySQL JDBC driver in my WAR's WEB-INF/lib directory, as apparently CloudBees provides it to the classpath at run time, yes.
This also prompts a few other questions:
What version of the MySQL JDBC driver (and which type) is provided by CloudBees?
Does the same apply for the Cloudant CouchDB service, or drivers/dependencies used by other service partners?
Are there any other libs that CloudBees makes available to the runtime classpath for its clients?
Thanks in advance!
Cloudbees currently provided version 5.1.16 of the MySQL JDBC driver with the jboss stack, and 5.1.15 with tomcat6 and any other stacks. Unfortunately, this information isn't documented anywhere publicly. Caleb Tennis, one of their developers, informed me of this via their IRC channel, #cloudbees.
Regarding Cloudant, it's more or less just CouchDB, so to connect you can use these libraries from the CouchDB wiki.
Alternatively, CouchDB amounts to a REST API, so it might be easier to use a straight HTTP library to make requests and parse responses. Whatever you're most comfortable with, really :D
As for what other libraries Cloudbees makes available to the runtime path, more from Caleb Tennis:
Other than mysql, there are no "guaranteed" runtime libraries - you will need to put them into your war or other package for deployment.
Can any one tell me how to public the wsdl file ie., that web service is created in java using eclipse.I am self learner to java/eclipse and web service too,so can any one please make me clear with steps.
Thanks for your time!...
Apache Axis has some cool tools that I frequently use - to generate a wsdl, use java2wsdl. It has a command line version, but also a Maven plugin (which I highly recommend) that will re-generate the wsdl for you automatically each time you package your web service.
What's more, the axis stuff also provides auto-generation of client stubs by which you can use your web service in consumer apps or services. Alternatively, JAX-WS tools can be used for this as well. In either case I would recommend automating the process with Maven.
If you package your wsdl with your app in this way (rather than letting a server do this for you automatically), you can be certain that your contract will look the same on any platform.
I'm using a set of APIs developed internally by my company to communicate with some common central services in the organization. The APIs can be configured dynamically by runtime configuration to use several transport protocols as needed by the system.
The collection of internal APIs are coupled to the IBM WebService thinclient.jar to configure and call all the necessary web services. I got the standalone prototype working smoothly, but need to integrate the functionality into several other services that are being developed in Grails.
This is where things fell apart. In the code that I've written, we just call a factory method and use that to get a client session and then proceed with our business logic. Simple. Using the debugger and digging into the API getClient() call, I can see that this gets a generic transport configuration and then binds it to a SOAP transport configuration. From here, the path differs whether it is the pure standalone Java service or the service running in the Grails app.
In the pure Java standalone, this then is bound to a
com.ibm.ws.webservice.engine.client.Service where the
initService() method is called and things work as expected.
In the Grails app, with the same Java code included, the same place
in the code calls to
com.springsource.loaded.ri.ReflectiveIntercepter and then after a
lot of back and forth in the spring-loaded API, it finally throws a
java.lang.reflect.InvocationTargetException.
Does any one have any tips or ideas on how to get the reflection in Grails to behave the same as in the straight Java?
I've tried a lot of variations to get to this point and I'm nearing the end of my rope. Ideally, it would be easiest to manage the Grails service that manages our business logic and the Java code that talks to these internal systems together, so I would prefer to get everything (Grails and my Java service code) working together. I briefly tried building a standalone JAR of my service code and all it's dependencies, but had chained dependency conflicts when trying to use that in Grails. My final option will be to stand my Java service up separately from the business logic in the Grails service and just make the calls from the Grails service to the Java service. This is less than ideal.
It's easy when you stumble into the answer... ;-)
The Grails service runs as expected if I set the run configuration in IDEA to use -noreloading option.
grails -noreloading run-app
This stops Grails/IDEA from leaving in the hooks to reload classes on the fly.
Are there any thoughts on whether this is a bug in Grails or the SpringSource Loader classes?