We Keep Coding

Bayesian networks in Scala [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I'm looking for a library to create Bayes nets and perform learning and inference on them in Scala (or Java, in case of lack of a better solution). The library should be actively maintained, performant, preferably easy, definitely well-documented unless the usage is really straightforward. Free, open-source and commercial alternatives are ok, but for commercial solutions a free trial is required.
An ideal solution would be the equivalent of what in the .NET world is Infer.NET by Microsoft Research, but more documented.
Thanks in advance!

FACTORIE is a young project, but it fits the bill and is implemented in Scala:
FACTORIE is a toolkit for deployable probabilistic modeling,
implemented as a software library in Scala. It provides its users with
a succinct language for creating relational factor graphs, estimating
parameters and performing inference.
It's developed by Andrew McCallum and his lab at UMass, who are also responsible for the hugely useful MALLET machine learning toolkit.

You might want to look into SMILE. It is free and has Java API. Other free options in Java are UnBBayes and SamIam.
SMILE
SMILE (Structural Modeling, Inference, and Learning Engine) is a fully
portable library of C++ classes implementing graphical
decision-theoretic methods, such as Bayesian net-works and influence
diagrams, directly amenable to inclusion in intelligent systems.
UnBBayes
UnBBayes is a probabilistic network framework written in Java. It has
both a GUI and an API with inference, sampling, learning and
evaluation. It supports BN, ID, MSBN, OOBN, HBN, MEBN/PR-OWL, PRM,
structure, parameter and incremental learning.
SamIam
Samiam includes two main components: a graphical user interface and a
reasoning engine. The graphical interface lets users develop Bayesian
network models and save them in a variety of formats. The reasoning
engine supports many tasks including: classical inference; parameter
estimation; time-space tradeoffs; sensitivity analysis; and
explanation-generation based on MAP and MPE.
Pure Scala and free options are FACTORIE (already mentioned) and Figaro. But Figaro currently lacks learning part.
Figaro - Probabilistic Modeling
Figaro models are data structures in the Scala programming language,
which is interoperable with Java, and can be constructed, manipulated,
and used directly within any Scala or Java program.

Perhaps Banjo fits the bill? I'm not sure how actively it is developed, but I know it has been around for at least a few years ... (never used it myself).
Banjo: Bayesian Network Inference with Java Objects

Some Java alternatives to Infer.NET were presented as answers to this question. So, I think basically you're asking about either a follow up to that question (it was asked during the second half of 2010) with respect to Java or fully Scala-based solution.

There is a Scala lib out there by now:
https://github.com/danielkorzekwa/bayes-scala

Is there any statistical library for javascript? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I need to implement some statistical tests like: T-test, Anova and Wilcoxon on javascript.
Similar to Java's - Apache Commons Math Library, is there any statistical tests library or codes for javascript?

jStat : a JavaScript statistical library
https://github.com/jstat/jstat

OpenEpi is a Javascript stats library, is open source, and has ANOVA and t-tests. I've not tried it (it's a bit too focused on epidemiology for my needs) but it might be useful.
jStat is a javascript statistical library project, and it looks like it's got a great future, but it might not have all you need right now. Edit: as of Dec 2012 it looks like the jStat project page is no longer maintained but the project is continuing to be developed. There's more up to date documentation on github. It now does have anova tests and varieties of t-test. No sign of Wilcoxon signed-rank though.
If you need very specific statistical processing in javascript urgently, you might have most success by browsing Omegahat who have various little tools that bridge the established stats language R with others including javascript.
It'll depend on the details of exactly what you want to do, but you might have some success with packages such as RJavascript - a code translator which aims to help turn existing R features into Javascript (just don't expect quality results first time). Also, SpiderMonkey builds on R for browsers, so it might be useful for internal or personal uses (but it's unlikely to be suitable for public publishing).

Some years ago I ported https://code.google.com/p/statistics-distributions-js/ so that I could use it in http://elem.com/~btilly/effective-ab-testing/ - it may have the functionality you need if you only need simple things.

If you're looking for a simple library for descriptive statistics, you could use javascriptstats.com
It does:
Mean
Median
Mode
Range
Variance
Standard Deviation
Best!

Leveraging a related answer:
The following blog post lists some recent packages: http://jgoodall.me/posts/2012/02/01/javascript-statistical-libraries/
As mentioned by others, native JS is a far cry from R, which web-wise has progressed from RApache (http://rapache.net/) to shiny (http://www.rstudio.com/shiny/). The latter uses node.js server-side, so this is quite promising. Of course both approaches will require you to code stats in R server-side, instead of using JS either on client or server.
Marc

High-End 2D Java (SE) Graphics Library [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I am looking for a high-end graphics library for Java Standard Edition. I know some fairly low-level libraries: AWT/Swing, JOGL, SDL. Are/is there an alternative?
My requirements are (atleast):
Anti-aliased
Fullscreen support
Alpha channel
Blend modes and Z-depth
Rasterized Effects: (motion) blur, glow, gloom, etc.
And, optionally:
Hardware acceleration
Vector graphics (scale/rotate/translate in floating point precision)
Well documentated, easy to get started.
I understand if there are no libraries/frameworks matching all requirements, if so, can you comment on how well it's extendible?
Edit:
Are there any other alternatives besides Processing? Is Processing usable (and easy?) as a library?

All of the effects you ask for can be done with Java 2D fairly simply and Java2D is to some degree hardware accelerated. The book Filthy Rich Clients shows how to implement all of these effects and many others. Painters in the SwingX has also implemented several of these effects.
Another possibility is JavaFx. It has a lot of effects built in. A year and a half ago when I tried it, it was still a little slow, but at least one version has been released since then. It may be more performant now. It's built on top of Java2D, so it should be able to take advantage of the hardware acceleration in Java2D.
As far as full screen support here's more information.

Maybe Processing (homepage) can be a good tradeoff.
it has 4 kinds of backing, including OpenGL
it has AA
it is 2d or 3d
it supports hw acceleration
it supports transformations
its syntax is quite easy to learn
it's easily embeddable
everything you need to learn is in this page

Not really a graphics library but rather an advanced applet AND graphics library:
PulpCore
Be sure to grab a quite recent version (maybe even fetch the source via mercurial and build it yourself) because the filter effects you mentioned (blur etc.) aren't available in earlier versions.
I know it might not be exactly what you're looking for; it is not a processing alternative, but it might be the better choice, depending on what you're planning to do.

Take a look at http://processing.org

Starting out NLP - Python + large data set [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I've been wanting to learn python and do some NLP, so have finally gotten round to starting. Downloaded the english wikipedia mirror for a nice chunky dataset to start on, and have been playing around a bit, at this stage just getting some of it into a sqlite db (havent worked with dbs in the past unfort).
But I'm guessing sqlite is not the way to go for a full blown nlp project(/experiment :) - what would be the sort of things I should look at ? HBase (.. and hadoop) seem interesting, i guess i could run then im java, prototype in python and maybe migrate the really slow bits to java... alternatively just run Mysql.. but the dataset is 12gb, i wonder if that will be a problem? Also looked at lucene, but not sure how (other than breaking the wiki articles into chunks) i'd get that to work..
What comes to mind for a really flexible NLP platform (i dont really know at this stage WHAT i want to do.. just want to learn large scale lang analysis tbh) ?
Many thanks.

NLTK is where you should start from (it's Python-based -- not sure why you're already thinking about parallelizing your processing at such an early stage... start with a more flexible experimental setup, is my advice). sqlite should be fine for a few GB -- if you need more advanced and standard SQL power you could consider postgresql.

There is a related talk on PyCon 2010 "The Python and the Elephant: Large Scale Natural Language Processing with NLTK and Dumbo".
The link has introductory information, slides and video.
I think sqlite is still a good choice for 12G size data. I have a text classification training set which has the similar size, both sqlite and plain text is fine as long as just iterator it line by line.

It is most likely that you are going to use Vector Space Model to represent the text while doing the anlaysis.
In which case, you should look at platforms that can help you store term vectors with term frequencies. It makes your life so much easier.
Have a look at Apache Lucene which has a python library to access Java Lucene. Elasticsearch is also a good alternative, which uses Apache Lucene underneath and has a really good python package. Elasticsearch also exposes a REST API.
Postgresql is also really good at storing tokens. Check out this article to learn more.
I have worked with sizable language data before and I personally prefer Lucene/Elasticsearch for analysis projects.
Cheers.

Summary from the internet:
Spacy is a natural language processing (NLP) library for Python designed to have fast performance, and with word embedding models built in, it’s perfect for a quick and easy start.
Gensim is a topic modelling library for Python that provides access to Word2Vec and other word embedding algorithms for training, and it also allows pre-trained word embeddings that you can download from the internet to be loaded.
NLTK details already given above.
Standford NLP has recently launched 50+ langauge supported python framework. You should check it out for sure.
There are many others but the above 4 are most usable in the sense of community support and latest features
I personally prefer Spacy.
Spacy is one of fastest of all and can use gensim/other APIs integrated into its model.
Moreover, Spacy models has a lots of languages in its alpha stage making it a perfect choice for multilingual apps.
Scaling is whole different thing[you can use alot of tools].But lets stick to scaling in NLP: Spacy gives so much control over different pipelines that you can disable unwanted pipelines making it faster.
Look into it try yourself and explore.

Choose a language to write technical engineering maths (can I use Java?) [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I have to write a prototype app for an engineering company. Most of the work is calculating various engineering properties (I'm talking pipes and real things here, not software engineering).
However, there will also have to be a GUI for:
parameter entry
displaying results
some basic diagramming
The calculation work at the moment doesn't involve complex math elements (no matrices at the moment), just logs, square roots, relatively simple formulae. Later I will have to do some:
curve fitting
numerical approximation
I was wondering if Java has been used for real world engineering apps?
Are there libraries available for this sort of thing?
Or am I better off writing in MatLab and then connecting to the code through Java?
Also open to other languages (although we are a Java shop).

I have some experience of both Matlab and of Java for scientific/engineering type codes. Yes, Java is used for real-world scientific and engineering codes, and yes there are libraries available. You can certainly do what you want using either so I'm not sure that you could sensibly distinguish between the two on the sole basis of your current requirements. I'd ask myself the following questions:
How good am I at programming advanced mathematical operations? Operations such as function minimisation, differential equation solvers, matrix algebra. If the answer is not very then lean towards Matlab which will provide all of these out-of-the-box (though you may need additional toolboxes). If you opt for Java, make sure you are very comfortable with floating-point arithmetic and dealing with the sorts of errors which occur when you use it.
Do I want to code everything in Java, everything in Matlab, or am I happy to use both and to wrestle with, say, a Java GUI on a Matlab engine ? I think you can do much better (in a vague sense) GUIs with Java than with Matlab, but Matlab's GUI facilities are good enough for most of its users that the added complexity of integrating Matlab with Java is not worth tackling. But then many Matlab users are not software engineers.
What speed of development do I need for the prototype work ? If you were equally skilled in Java and in Matlab then I'd guess that you could do it quicker in Matlab, because the numeric stuff is already provided, you could concentrate on the GUI. But if you are a skilled Java programmer coming newly to Matlab you might decide to stick with what you know.
How will I develop and deploy the production app if the prototype is successful ? If Matlab doesn't fit your deployment ideas then learning it and forgetting it may not be rewarding.
Finally, since you solicit other language recommendations: forget Java, forget Matlab, forget Python, forget R, use Mathematica, it's way more fun and very powerful.

This sounds like a job for Matlab: you don't give any reason not to use it. There's some code for evaluating Matlab expressions from Java: http://www.cs.virginia.edu/~whitehouse/matlab/JavaMatlab.html

I have done some work where I had to reimplement Matlab code in Java so it is certainly possible. The Java code can end up being quite verbose compared to the Matlab original due to Matlab being able to operate directly on matrices/arrays etc.
Some math libraries that you might want to look at to see if they support the functionality you are looking for:
Commons Math
Colt
JSci

I guess that Java would be a good choice, even though it is not considered a typical language for rapid application development.
Pros:
versatile GUI toolkit for desktop applications in standard library (Swing),
(relatively) cross-platform,
great libraries, e.g. from Apache; a great math library to look at would be colt; for charts and diagrams, you may like jfreechart ..
Cons:
"not so rapid" prototyping capabilities
Further reading:
Technical Java: Applications for Science and Engineering

Python has several decent GUI toolkits as well as NumPy, and is easy and fun to write in.

This depends mainly on how easy it will be in your environment to include the mathlab or other math engine in your product. If this is easy, I would suggest to use mathlab, but if not, e.g. you have licensing issues or deployment issues, you are probably better of just using plain Java code.

You may also want to consider R language.

I would do a search for software that's written to do piping calculations. This problem has been done. (As you've noted, the calculations need not be difficult.) At a minimum I'd recommend that you know what's available to you, how much it would cost, and where the break even point was for development costs.
A commercial product will have one huge advantage over anything you'll write: It will have a larger user community that's been banging on it and finding bugs for a longer period of time than your prototype. That's worth something as well.
What's your opportunity cost? What else could you have done with the development time that would drive more revenue?

Don't forget numpy or scipy. Both allow you to call fast matrix libraries from Python.