I am using CoreNlp to get the information extraction from a large text. However, its using the "triple" approach where a single sentence produce many output which is good, but there are some sentences that doesn't make sense. I tried to eliminate this by running another unsupervised NLP and try to utilize function in CoreNlp, yet I stuck at getting word vector form CoreNlp. Can anyone point where do I need to start searching for codes that do the word embedding in CoreNlp? Also I am newbie in java and IT.
There are some open libraries like glove, word2vec, text2vec, but I noticed glove already been used in CoreNlp (correct me if wrong).
since training your own model from scratch might turn out to be a time-consuming task, you could just download pretrained vectors from:
https://nlp.stanford.edu/projects/glove/
however, there is an example with dl4j here that might do to trick:
https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/nlp/glove/GloVeExample.java
I'm trying to create a program which takes the coordinates from a csv-file and turns it into a shapefile. I downloaded the GeoTools libraries but I can't get the code to work...
In order to do this, you'll want to create a SimpleFeatureType and SimpleFeatures from the rows in the CSV, and then use the ShapefileDataStore to write those features out to disk.
This tutorial [1] in the documentation does just that. Are you using that tutorial or having trouble with something else?
http://docs.geotools.org/latest/userguide/tutorial/feature/csv2shp.html
As an aside, if this is a one-time or otherwise infrequent operation which can be done manually, I'd recommend trying out QGIS. QGIS is a desktop GIS application and for turning a CSV in a shapefile, it'd require no programming.
The Tensorflow Android demo provides a decent base for building an Android app that uses a TensorFlow graph, but I've been getting stuck on how to repurpose it for an app that does not do image classification. As it is, it loads in the Inception graph from a .pb file and uses that to run inferences (and the code assumes as such), but what I'd like to do is load my own graph in (from a .pb file), and do a custom implementation of how to handle the input/output of the graph.
The graph in question is from Assignment 6 of Udacity's deep learning course, an RNN that uses LSTMs to generate text. (I've already frozen it into a .pb file.) However, the Android demo's code is based on the assumption that they're dealing with an image classifier. So far I've figured out that I'll need to change the values of the parameters passed into tensorflow.initializeTensorflow (called in TensorFlowImageListener), but several of the parameters represent properties of image inputs (e.g. IMAGE_SIZE), which the graph I'm looking to load in doesn't have. Does this mean I'll have to change the native code? More generally, how can I approach this entire issue?
Look at TensorFlow Serving for a generic way to load and serve tensorflow models.
Good news: it recently became a lot easier to embed a pre-trained TensorFlow model in your Android app. Check out my blog posts here:
https://medium.com/#daj/using-a-pre-trained-tensorflow-model-on-android-e747831a3d6 (part 1)
https://medium.com/#daj/using-a-pre-trained-tensorflow-model-on-android-part-2-153ebdd4c465 (part 2)
My blog post goes into a lot more detail, but in summary, all you need to do is:
Include the compile org.tensorflow:tensorflow-android:+ dependency in your build.gradle.
Use the Java TensorFlowInferenceInterface class to interface with your model (no need to modify any of the native code).
The TensorFlow Android demo app has been updated to use this new approach. See TensorFlowImageClassifier.recognizeImage for where it uses the TensorFlowInferenceInterface.
You'll still need to specify some configuration, like the names of the input and output nodes in the graph, and the size of the input, but you should be able to figure that information out from using TensorBoard, or inspecting the training script.
For analysis.
I know we can use the Save function and load the Model in Spark application. But it works only in Spark application (Java, Scala, Python).
We can also use the PMML and export the model to other type of application.
Is there any way to use a Spark model in a Java application?
I am one of the creators of MLeap. Check us out, it is meant for exactly your use case. If there is a transformer you need that is not currently supported, get in touch with me and we will get it in there.
Our serialization format is solely JSON/Protobuf right now, so it is very portable and supports large models like RandomForest. You can serialize your model to a zip file then load it up wherever.
Take a look at our demo to get a use case:
https://github.com/TrueCar/mleap-demo
Currently no, your options are to use PMML for those models that support it, or write your own framework for using models outside of Spark.
There is movement towards enabling this (see this issue). You could also check out Mleap.
Having seen some suggestions for graphs, I wonder what's the optimum for my problem.
I want to render a directed graph to a servlet/picture that is displayed in the browser. There should be some kind of optimization of position. No dependency to Swing would be preferred. Algorithms are not important, since the structure of the graph is determined by business logic. It would be desired to be able add labels to edges as well.
it would be optimal if i can serve this as png/svg.
Which library/service would you recommend?
clarifications:
1) The question is all about Graphs - like Directed Acyclic Graph - NOT - Charts.
2) flot, Google Charts - cannot plot graphs, only charts, or have i missed something?
3) no i do not need interactivity
4) graphviz would be nice, but the grappa java library is quite outdated and is built upon swing/awt. while it may be theoretically possible to render swing to images, it would not be my favorite way to to so in a server-app.
5) it would be fine to use an online service where the images are not hosted locally.
edit: added links to Wikipedia to clarify graph/chart term
Take a look at graphviz
yFiles might be useful for this.
How about the dot component of Graphviz? It produces graphs (not charts), outputs to PNG and SVG, and supports labeling edges. You can shell out to dot to generate the image you need, and return an img tag that references that. Alternatively, you can return an img tag that references a URL that will generate the requisite graph (or retrieve a cached copy). Here's the dot info:
http://www.graphviz.org/pdf/dotguide.pdf
You might also take a look at WebDot, which is apparently designed for this purpose:
http://www.graphviz.org/webdot/
As well as waiting weeks to hear about the Magic Framework that's going to solve all your problems in one line of code, there is also the other option of just Writing Some Code yourself to do exactly what you want... (I'm not saying it's 10 minutes' work, but it's probably one or two days, and you posted your question over two weeks ago...)
Have you had a look, for example, at the Wikipedia entry on Force-based algorithms-- it has pseudocode and a few links that might be helpful.
I'm assuming it is the layout algorithm that's the issue, and not the matter of creating a BufferedImage, drawing to its graphics context, PNG-encoding it and sending it down the socket. You really don't need a framework for that bit, I don't think.
Try aiSee. It is used by all kinds of web-based applications for data mining, static program analysis, matrix visualization, network analysis, and whatnot. It is also used by some MediaWikis as their graph-layout backend.
They have a huge database of sample graphs over at aiSee.com. Check it out. It supports edge labels, export to mapped SVG and HTML, and is not dependent on Swing.
We create mxGraph for such requirements. We did actually release it in 2006, but took a while to notice this question...
For serverside, try JUNG, you can run it against Batik and produce beautiful SVG or PNG files. JUNG has a nice design and very powerful layout algorithms...
Also, since you mention that "it would be fine to use an online service", graphviz provide a service called webdot to render graphs.
There are others along this line as well... e.g. http://graph.gafol.net/ (seems to be down)
Client side:
Try http://arborjs.org/ for a minimal(ish) library it is dedicated to layout, use this is you like to prefer your own rendering routines (div, canvas, svg, paper.js, processing... etc).
I also like http://sigmajs.org/ for a more complete approach, build in touch support, plugins, file formats, etc.
Interestingly, the Eclipse project has an SWT/JFace component/framework capable of displaying and generating (import/export) Graphviz's 'DOT' format, in pure Java:
ZEST (home page & download links)
See http://wiki.eclipse.org/Graphviz_DOT_as_a_DSL_for_Zest for usage examples.
Although ZEST is touted as an Eclipse plugin, it does seem that the DOT-manipulation API's can be used standalone and external to an Eclipse installation.
Cheers
Rich
You may try sigma.js: http://sigmajs.org/
It is a lightweight, open source library in Javascript to display large graphs on the Web.
JPGD is a Graphviz parser in Java. It's a little abandoned, but the code is nice and clear, and if you find bugs I'm sure the author would accept contributed fixes.
Although advertised as a parser, it is also a generator. You can build Graphs as collections of Node and Edge objects, then get .dot using Graph.toString(). Getting this as a graphic would be a simple shell out to the Graphviz dot executable.
Alternatively, dot is very easy to generate yourself. In the simplest case, it's just a matter of writing a potted header
digraph myGraph {
... followed by one edge definition per edge
node1 -> node2 ;
... followed by a closing brace
}
so i took alook at all the given answers and links, it looks like Prefuse/Flare will by the optimal choice.
they have very appealing visialisations, plus they have built in support for graphs.
Maybe check out Google Charts?
You can use SVG in combination with Batik. I have used this several times for displaying graphics. Batik with Java 1.5 is very fast. With this solution you can program your graph in Java with no dependency on Swing. You can add labels where you want, host it as a Servlet and display it as png or svg.
You can create the graphs in SVG (this is an XML document).
You use Batik to transform the SVG-document to a PNG/JPG image.
You can use a Servlet to stream this image back to the browser.
In java you build an SVG(=XML) document. Samples for SVG graphs can be found here:
Directed graph and here: Simple directed graph
See this question, especially Stephan's answer about prefuse. I read that you do not need interactivity, but prefuse still may be useful.
I can whole-heartedly recommend flot - excellent!
See examples here.
JFreeChart might be the way you want to go, but you make a distinction between Charts and Graphs. Maybe you can explain what you mean by that. I've usually used these terms synonymously. :)
JFreeChart has good scatter, bar and line graphs as well as fun ones like Pie and Dial so maybe it will work for you.