Spark Model to use in Java Application - java

For analysis.
I know we can use the Save function and load the Model in Spark application. But it works only in Spark application (Java, Scala, Python).
We can also use the PMML and export the model to other type of application.
Is there any way to use a Spark model in a Java application?

I am one of the creators of MLeap. Check us out, it is meant for exactly your use case. If there is a transformer you need that is not currently supported, get in touch with me and we will get it in there.
Our serialization format is solely JSON/Protobuf right now, so it is very portable and supports large models like RandomForest. You can serialize your model to a zip file then load it up wherever.
Take a look at our demo to get a use case:
https://github.com/TrueCar/mleap-demo

Currently no, your options are to use PMML for those models that support it, or write your own framework for using models outside of Spark.
There is movement towards enabling this (see this issue). You could also check out Mleap.

Related

know how to use a language model in tensorflow Lite?

Does anyone know how to use a language model in TensorFlow Lite? I have a generated language model with an LSTM structure in TensorFlow. I have already converted it to .tflite for use on android. Now my question is how can I use it? My intention is to use that model to predict the next word in a sentence. The original model works perfectly in Python, now what I need is for it to work and make predictions in Java or Kotlin.
You can achieve this in two ways:
Add the .tflite model in assets folders, implement it to build Gradle file and then finally import that model to carry out requirements in java/kotlin. as shown here
By using firebase, firebase provides custom machine learning model deployment options.
you can add the model to firebase and use it as a remote service. the link to docs.
I'd recommend the first method if your model size is small & if you want to run your app offline.

Integrating ETL with Java using intellij

I want to start with ETL on java. I am using Intellij. I wanted to know how the integration can be done or which tool is compatible with intellij.
Also if there is any tutorials on the basics of ETL with java.
Exactly what and all I will need if I want to do the transformation of data
It can be basic like just taking in random input from a file and transforming
the data based on particular logic
Creating code to extract (query different sources like DB, XML, web service, etc) to transform (you know make everything compatible, removing dup's, creating Dims and Facts) to load them to targets (databases and more)...
All this is not new. Java is great but creating ETLs with it is creating a non-standar app... And is going to become a legacy and then you need to build a scheduler to run the loads and to integrate with several components.
So. I strongly recommend instead of create a Java app, take a look to products like Informatica PowerCenter and/or Oracle Data Integrator.
This solutions are business wide standard for ETL worldwide, provides objects and methods to avoid apps that needs to be hardly mantein and are on top of any applications... Also are used for integration, migration, B2B, BI... Named...
Good luck!
You would be re- inventing the wheel if you are trying to create a Java based etl product .
Talend is a java based open source ETL tool which gives the features of an ETL tool and lets one write Java code to integrate ..
Pentaho is another Java based ETL tool..
Both of them are popular and have good UI...

Java library to assist in XSD creation?

Is anyone aware of a library that makes the process of creating XSDs in Java a little easier than using something like a DocumentBuilder? Something like a helper or utility class. I came across the org.eclipse.xsd maven jar but I'm having ClassNotFoundException issues when working with it in Eclipse and I'm not entirely sure it's meant to be used as a standalone kind of thing. This is a bit difficult to Google for as well since there are lot of search results around automatic generation/translation from Java to XSD and vice versa.
Essentially what I need to do is to programmatically create an XSD from a certain source of data -- not Java classes.
Apache XMLSchema is a lightweight Java object model that can be used to manipulate and generate XML schema representations. You can use it to read XML Schema (xsd) files into memory and analyze or modify them, or to create entirely new schemas from scratch.
The fact that with this API one can create an XSD from scratch, it sounds as a starting point to achieve the ask; as to the fitness, it depends on what that "certain source of data" is.

How do I use Blender models in Java?

Just a general question really?
Let's say I am making a game and have made a character model in Blender. How would I use this model in Java?
Would I import it somehow?
Thanks.
Generally when making models in blender you export the model in a format which allows you to later import it in the game engine of your choice, which format you use differ in requirements.
The export-import cycle is often referred to as the "Asset Pipeline", and you generally want to keep it as simple and automated as possible since it is something you or your artists will perform on a regular basis.
So if we look at a few specific graphics engines and platforms;
OGRE3D (or Ogre4J) supports it's own plain-text format (.scene, .mesh.xml, .material.xml) in order to load scenes, models and materials. It also has support for armature animations among other things, there is also some support for loading .blend-files directly. See their documentation for blender.
JmonkeyEngine has support for loading both OGRE3D .scene's and .blend's directly. It also has it's own binary j3o format which these can be converted into when you want to package the game. For specific examples, see their tutorials.
There are multiple formats you can take into consideration when deciding how you want to use your model. When it is imported however, the game engine of choice represents it in an internal structure which usually allows you to be decoupled from the exact format of choice.
Picking which to use is and should not be written in stone since requirements might change and if done properly it should not have a considerable effect on the project. This is also something you should take into consideration if you are writing your own engine.
There are input/output scripts available for Blender that will help you.
Blend2Java, for example, is a set of Python scripts for use with Blender that will export to Java XML, which can be decoded with the standard java.beans.XMLDecoder class.
There's a good overview of how to do this at http://blend2java.sourceforge.net/blend2java-howto.html
Here's a better idea: Use an existing Java 3D library (dzzd I highly recommend) and load in your model using the library's built in functions. Then, instead of just working with the data, you can actually display it. From Blender, it's a simple matter of exporting as 3DS.
Yet another solution: Java .Blend provides you with a type-safe Java API to all data in a Blender file. It supports even creating new Blender files from within Java ;)

Integrating Pentaho/Talend/etc. with an OR Mapper

We have an application (Java) with an own OR mapper. Within this system we have what can be compared to Hibernate's interceptors (we call it triggers): Do specific actions just before saving data in the database, after it's deleted and so on. The underlying database is MySQL.
Now we would like to use tools such as Pentaho Data Integration or Talend to convert data to put it into our system. It's no problem to do that directly on the SQL level, but by doing so we loose the built-in power of our triggers.
Is there a way to somehow integrate any of the Data Integration solutions into our existing application? It would be great if there was a way to write into instances of our classes instead of writing into the database directly.
Any hints welcome :-)
I'd prefer Talend which is a Java code generator tool. (You can se my blog post at http://www.robertomarchetto.com/www/talend_studio_vs_kettle_pentao_pdi_comparison)
You could use a tJavaRow so you can write Java code for each processed row. In tJavaRow you can call Hibernate code, for example using a custom class defined in a new routine.
2 ways with Pentaho data integration I can think of straight off:
Simply create a plugin which adds/deletes data - you could copy the existing salesforce insert/update plugins, they would be a good start - rip out all the salesforce code and replace with yours.
Perhaps harder; But maybe more satisfying - Write a jdbc driver which uses your code!

Categories