Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'd like to compute power spectral density of time series; do some bandpass, lowpass, and highpass filtering; maybe some other basic stuff.
Is there a nice open-source Java library to do this?
I've hunted a bit without success (e.g., Googling "power spectral density java" or "signal processing java" and clicking through links, looking in Apache Commons, Sourceforge, java.net, etc.).
There are lots of applets, books, tutorials, commercial products, etc., that don't meet my needs.
Update: I found org.apache.commons.math.transform for Fourier transforms. This doesn't implement power spectral density, bandpass, etc., but it is something.
My first suggestion is to not do your DSP implementation in Java. My second suggestion would be to roll your own simple DSP implementations yourself in Java.
Why not to use Java:
I have lots of experience writing DSP code over the last 10+ years... and almost none of the DSP code is in Java... so forgive me when I am hesitant to read about someone who wants to implement DSP in Java.
If you are going to be doing non-trivial DSP then you shouldn't be using Java. The reason that DSP is so painful to implement in Java is because all the good DSP implementations use low level memory management tricks, pointers (crazy amounts of pointers), large raw data arrays, etc.
Why to use Java:
If you are doing simple DSP stuff roll your own Java implementation. Simple DSP things like PSD and filtering are both relatively easy to implement (easy implementation but they won't be fast) because there is soo many implementation examples and well documented theory online.
In my case I implemented a PSD function in Java once because I was graphing the PSD in a Java GUI so it was easiest to just take the performance hit in Java and have the PSD computed in the java GUI and then plot it.
How to implement a PSD:
The PSD is usually just the magnitude of the FFT displayed in dB. There are many examples from academic, commercial and open-source showing how to compute the magnitude of the FFT in dB. For example Apache has a Java implementation that gives you the FFT output and then you just need to convert to magnitude and dB. Anything after the FFT should be tailored to what you need/want.
How to implement lowpass, bandpass filtering:
The easiest implementation (not the most computationally efficient) would in my opinion be using an FIR filter and doing time domain convolution.
Convolution is very easy to implement it is two nested for loops and there are literally millions of example code on the net.
The FIR filter will be the tricky part if you don't know anything about filter design. The easiest method would be to use Matlab to generate your FIR filter and then copy the coefficents into java. I suggest using firpmord() and firpm() from Matlab. Shoot for -30 to -50 dB attenuation in the stopband and 3 dB ripple in the passband.
I found the book Java Digital Signal Processing and its example source code. You might look through the code to see if it fits your needs.
You can also check out DSP Laboratory.
As duffymo and basszero mentioned in the comments, there have been changes to Java since the publication of Java DSP that may impact some of the code examples. In particular, the (relatively) new Concurrency Utilties package might prove useful.
I have written a collection of some Java DSP classes, e.g. IIR filters:
Java DSP collection
It looks pretty sparse. Try Signalgo or jein or the Intel Signal Processing Library, although I think the last one is just a JNI wrapper.
I saw a lot of those applets you were talking about. I think you may be able to get the JARs for them and use the class APIs inside. May have to use eclipse and jad to decompile and figure out what they do, though, due to lack of documentation. Try the source on this page for example.
I found another resource, although it's not a library: http://www.dickbaldwin.com/tocdsp.htm. It's just a basic discussion of signal processing and Fourier transform, with some Java examples. See for example tutorials 1478, 1482, 1486. Not sure what the license on the code is.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
The company I work at has a piece of code they would like to rewrite. It uses Java/Java server faces. The problems with it are that it is old and uses depreciated code, was written in an IDE that is no longer available and doesn't work well with netbeans, and that it is kind of sloppy coding in the first place. No one really knows its structure and there is limited documentation.
Before beginning on the rewrite, we would like to find the structure of the old program and get a decent UML diagram. What tools would work the best in this situation? So far we have looked at one called Agilej.
Sorry if this is a little vague, I'm just a lowly intern an haven't been filled in on everything yet =p
You can use JArchitect, a pretty complete java static analysis tool
I'm not giving clickable hyperlinks as you should be able to find them easily through a web search and there may be more than 1 link that is useful for you
Sparx Systems Enterprise Architect has very powerful reverse engineering capabilities. Java (also compiled binary JAR) is in the list of supported languages.
I have used Enterprise Architect's reverse engineering features several years ago to help us understand and design modifications of legacy C++, QT code.
Enterprise Architect was able to automatically retrieve the class model. I then used the raw class model to drag/drop draw several other diagrams including only classes of my interest with level of detail I needed etc.
In the range of other static code analysis tools (e.g. the code flow visualization) I don't know which tool particularly supports Java well enough. Quick Google points e.g. to Coder Gears JArchitect. In the past project I have mentioned we used Doxygen's automatically generated documentation. Part of that were also some automatically generated graphviz dependency diagrams.
To clarify the design and visualize the flow (especially big legacy functions) I have found useful the Rapid Quality Systems Code Rocket flow visualizer
Once you get the basic facts at your hands through the various reverse engineering tools next step would be to go through them, annotate what is the unknown, what is doing what, basically apply some formal code review method (e.g. Fagan inspection or some of its derivatives).
(using evaluation versions of the various tools may be enough if you think ahead what are the required deliverables for your follow up actions. I guess the company is not planing to give you a >0$ budget)
The Agile Modeling site may have some good refactoring tips and some minimal UML mapping guidelines, start e.g. at Agile Legacy System Analysis and Integration Modeling
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I've been wanting to learn python and do some NLP, so have finally gotten round to starting. Downloaded the english wikipedia mirror for a nice chunky dataset to start on, and have been playing around a bit, at this stage just getting some of it into a sqlite db (havent worked with dbs in the past unfort).
But I'm guessing sqlite is not the way to go for a full blown nlp project(/experiment :) - what would be the sort of things I should look at ? HBase (.. and hadoop) seem interesting, i guess i could run then im java, prototype in python and maybe migrate the really slow bits to java... alternatively just run Mysql.. but the dataset is 12gb, i wonder if that will be a problem? Also looked at lucene, but not sure how (other than breaking the wiki articles into chunks) i'd get that to work..
What comes to mind for a really flexible NLP platform (i dont really know at this stage WHAT i want to do.. just want to learn large scale lang analysis tbh) ?
Many thanks.
NLTK is where you should start from (it's Python-based -- not sure why you're already thinking about parallelizing your processing at such an early stage... start with a more flexible experimental setup, is my advice). sqlite should be fine for a few GB -- if you need more advanced and standard SQL power you could consider postgresql.
There is a related talk on PyCon 2010 "The Python and the Elephant: Large Scale Natural Language Processing with NLTK and Dumbo".
The link has introductory information, slides and video.
I think sqlite is still a good choice for 12G size data. I have a text classification training set which has the similar size, both sqlite and plain text is fine as long as just iterator it line by line.
It is most likely that you are going to use Vector Space Model to represent the text while doing the anlaysis.
In which case, you should look at platforms that can help you store term vectors with term frequencies. It makes your life so much easier.
Have a look at Apache Lucene which has a python library to access Java Lucene. Elasticsearch is also a good alternative, which uses Apache Lucene underneath and has a really good python package. Elasticsearch also exposes a REST API.
Postgresql is also really good at storing tokens. Check out this article to learn more.
I have worked with sizable language data before and I personally prefer Lucene/Elasticsearch for analysis projects.
Cheers.
Summary from the internet:
Spacy is a natural language processing (NLP) library for Python designed to have fast performance, and with word embedding models built in, it’s perfect for a quick and easy start.
Gensim is a topic modelling library for Python that provides access to Word2Vec and other word embedding algorithms for training, and it also allows pre-trained word embeddings that you can download from the internet to be loaded.
NLTK details already given above.
Standford NLP has recently launched 50+ langauge supported python framework. You should check it out for sure.
There are many others but the above 4 are most usable in the sense of community support and latest features
I personally prefer Spacy.
Spacy is one of fastest of all and can use gensim/other APIs integrated into its model.
Moreover, Spacy models has a lots of languages in its alpha stage making it a perfect choice for multilingual apps.
Scaling is whole different thing[you can use alot of tools].But lets stick to scaling in NLP: Spacy gives so much control over different pipelines that you can disable unwanted pipelines making it faster.
Look into it try yourself and explore.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I would like to simulate some scenarios using the multiagent
paradigm, and it seems NetLogo and Repast are the most popular tools for that.
I'd like to know if anyone has had any experience with either one and could tell me more about them? For example, I've noticed that there is a fluxogram-like modeling option for Repast, but I believe it is rather limited. I've looked around the tutorials and documentation in the official site, and the documentation seems to be lacking. While there are some examples with it, I'd say extending it to simulate an ambient which it has not been specifically prepared to seems like an unreachable goal at the moment, despite Repast obviously being very robust and apparently able to handle it, given enough familiarity with it.
On the other hand, NetLogo has more examples and overall I've liked it more for its simplicity, but it seems to be more focused on the simulating propagation of diseases or similar models. I've found a programming book teaching Logo, so I figure it'd be easier to get started with it too.
Currently, I am thinking of simulating botnets and IDSes as multiagents. The problem, however, is that I would have to abstract the network and transport layers to an extent to be able to do it, as well as generate traffic between the nodes. Repast is apparently more fitting for this, but given its complexity and lack of documentation I'm thinking of using NetLogo. While there are some examples of NetLogo with traditional applications (ex: Tetris or Pac-Man), I'm not sure about how appropriate it'd be for that.
I have a webpage with a couple dozed netlogo multiagent simulations. I use netlogo for teaching and I have found that, once you get past the learning curve, you can develop simulations amazingly fast. Stuff that would take you 80 man-hours in other so-called agent environments (Jade, Repast, which are really mostly just programming libraries) can be done in 2 hours.
On the other hand, netlogo is not really good for simulations that require immense amount of details, like say simulating a network all the way from TCP/IP to HTTP. That would just require large amounts of code, regardless of programming language, and netlogo currently sucks if your program ends up being more that 10 pages long. Having said that, most people would be amazed at what you can get done in 10 pages of netlogo code.
Short answer: it depends on the programming paradigm or language you want to use, and the design you want for your agents:
If you want a low-entry-high-ceiling language allowing quick prototyping but sophisticated simulations, and are willing to learn a new paradigm (avoiding loops) use NetLogo. Good documentation.
If you want to make a real application to use on highly-parallelized clusters or just want to use Java Groovy or need a specific Java library for your purpose, use Repast or better Repast for High Performance Computing (but avoid ReLogo which is very slow). Mild documentation.
If you want to model cognitive agents (instead of reactive) with FIPA communications, better use Jason or better JaCaMo which supports AgentSpeak + Java (so you can also use your favourite Java libraries), and there's no Groovy required. Bad documentation (a lot of non detailed features and commands and bad too-complex-not-commented examples).
Long answer:
Disclaimer: I am more experienced with NetLogo but I also used Repast and a few others like Jason.
Basically, the difference between NetLogo and Repast is that with NetLogo you will have a simpler framework but you'll need to learn how to program in a turtle-and-patch-oriented paradigm, while in Repast you will have to learn that + the mechanisms behind Java Groovy but you will eventually get more flexibility. Speed isn't really a criteria here (see below).
To be more clear, you can program efficiently in NetLogo if you use to a maximum the turtles and the patchs native functions. For example, if you want to implement A*, instead of implementing a list of nodes, you should directly use the patchs and filter them using stuffs like this:
ask patchs with [criteria1 = value and criteria2 = value2] [do-some-stuff]
ask patchs with-min [criteria][do]
let var [somevalue] of min-one-of patches [criteria]
Also if you can't find a way to efficiently do what you want, be sure to check if maybe an extension exists (check also here under Libraries and Tools) for your purpose, like the now native matrix extension which allowed me to make an efficient neural network in NetLogo.
On the other hand, Repast is potentially more flexible than NetLogo (since you have access to the whole range of Java libraries), but a bit more complex since you have to know how to handle Groovy.
If you are solely interested in speed, do NOT use ReLogo (NetLogo-like syntax for Repast) which has been shown to be a whole lot slower than NetLogo (see the 2012 paper below). In any cases, your best bet would either to try an implementation with NetLogo using the tricks above, or if you want to use your application for real later, there is also a distribution called Repast for High Performance Computing which removes most of the overload that come with turtles and patchs objects, and thus it can be used for real applications. A similar extension exists for NetLogo to compute in clusters with parallelization but it's not an official distribution.
If you want more infos about the diverse platforms, here is a nice review of 2006:
Railsback, S. F., Lytinen, S. L., & Jackson, S. K. (2006). Agent-based Simulation Platforms: Review and Development Recommendations. SIMULATION, 82(9), 609-623.
And an updated version of this paper in 2012 dealing with NetLogo vs ReLogo:
Lytinen, S. L., & Railsback, S. F. (2012, April). The evolution of agent-based simulation platforms: A review of netlogo 5.0 and relogo. In Proceedings of the Fourth International Symposium on Agent-Based Modeling and Simulation.
/EDIT: I cited Jason but didn't give any more details. If you want to model cognitive agents (instead of reactive agents), you can do that in NetLogo using the unofficial BDI extension which works well but is a bit limited (but it's easily extensible since it's pure NetLogo), but your best bet is to use a framework specifically designed to model cognitive agent with full support of AgentSpeak.
Jason is very nice since you have access to a full AgentSpeak language + JAVA to implement the technical side. In fact, you can do whole projects using only AgentSpeak (which I did), but you can also make more Java-oriented versions, it's up to you how you want to design your program, the result will be more or less the same. This offers you a lot of flexibility in your design workflow.
Tip: search for "Jason internal actions" in the documentation to get a good description of the available AgentSpeak commands.
Also if you are interested in Jason, you might be interested in JaCaMo (= Jason + Cartago + Moise) which is the result of a cooperation of three projects authors to make a full-fledged cognitive agents framework which also can model complex environments (with artifacts theory) and multi-agents organisations (roles, groups, missions, etc.).
A last framework I know of but didn't have a chance to try is Mason which supports 2D and 3D environments. Never had a chance to try this one so I don't know how this compares with the others but you can try it out.
Here's a generic comparison.
http://www.duncanrobertson.com/research/AMLE.pdf
I had more or less the same problem a few months ago when I had to choose a framework for my simulation. I look at Repast, NetLogo, Swarm and Jade.
NetLogo was nice and I tried to write some simple test applications but since I wanted to use Java as my programming language, NetLogo wasn't the best candidate. Repast has pretty much everything you need to write larger simulations and there are many projects (especially in social sciences) where Repast is used. My problems with Repasts were: bad API documentation, parameters that are passed to methods or constructers that are never used and don't make any sense at all (have a look at the source code) and a lot of boilerplate code.
I'm using Jade (http://jade.tilab.com/) now and I'm really happy with it. The community is good and their mailing list is VERY active. Okay, Jade is just a library and a framework for agent-based modelling. You don't get anything like those visual editor in Repast and you'll have to write your own tool for visualising the results.
Cheers
You could simulate the traffic using a agent type called "packet" that will be spawned and send from a agent called "bot" to another agent called "bot" or "server". Instead of sending the packets to a IP address, you would be sending them to a pair of X and Y coordinates.
Netlogo has an example of how a virus spreads in a network, this might be a good starting point.
I have never tried NetLogo, but have I tried Repast-J and Simphony. It seems Simphony is good, but at the moment I am stuck at changing the Edge type from straight line to curved one. There is not enough documentation and examples available.
Once I tried Mason which is based on java, too. It is similar to Repast-J, yet it was faster. But recently there is not much development in Mason.
I would like to try out Jade later.
If you can already code in Java, you can also look at the following paper for a comparison between RePast, Swarm, Quicksilver, and VSEit, different freely available programming libraries for support of social scientific agent based computer simulation
Tobias, Robert, and Carole Hofmann. "Evaluation of free Java-libraries for social-scientific agent based simulation." Journal of Artificial Societies and Social Simulation 7.1 (2004).
Repast is definitely more flexible than NetLogo but the documentation is not very detailed for RePast Symphony
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I have to write a prototype app for an engineering company. Most of the work is calculating various engineering properties (I'm talking pipes and real things here, not software engineering).
However, there will also have to be a GUI for:
parameter entry
displaying results
some basic diagramming
The calculation work at the moment doesn't involve complex math elements (no matrices at the moment), just logs, square roots, relatively simple formulae. Later I will have to do some:
curve fitting
numerical approximation
I was wondering if Java has been used for real world engineering apps?
Are there libraries available for this sort of thing?
Or am I better off writing in MatLab and then connecting to the code through Java?
Also open to other languages (although we are a Java shop).
I have some experience of both Matlab and of Java for scientific/engineering type codes. Yes, Java is used for real-world scientific and engineering codes, and yes there are libraries available. You can certainly do what you want using either so I'm not sure that you could sensibly distinguish between the two on the sole basis of your current requirements. I'd ask myself the following questions:
How good am I at programming advanced mathematical operations? Operations such as function minimisation, differential equation solvers, matrix algebra. If the answer is not very then lean towards Matlab which will provide all of these out-of-the-box (though you may need additional toolboxes). If you opt for Java, make sure you are very comfortable with floating-point arithmetic and dealing with the sorts of errors which occur when you use it.
Do I want to code everything in Java, everything in Matlab, or am I happy to use both and to wrestle with, say, a Java GUI on a Matlab engine ? I think you can do much better (in a vague sense) GUIs with Java than with Matlab, but Matlab's GUI facilities are good enough for most of its users that the added complexity of integrating Matlab with Java is not worth tackling. But then many Matlab users are not software engineers.
What speed of development do I need for the prototype work ? If you were equally skilled in Java and in Matlab then I'd guess that you could do it quicker in Matlab, because the numeric stuff is already provided, you could concentrate on the GUI. But if you are a skilled Java programmer coming newly to Matlab you might decide to stick with what you know.
How will I develop and deploy the production app if the prototype is successful ? If Matlab doesn't fit your deployment ideas then learning it and forgetting it may not be rewarding.
Finally, since you solicit other language recommendations: forget Java, forget Matlab, forget Python, forget R, use Mathematica, it's way more fun and very powerful.
This sounds like a job for Matlab: you don't give any reason not to use it. There's some code for evaluating Matlab expressions from Java: http://www.cs.virginia.edu/~whitehouse/matlab/JavaMatlab.html
I have done some work where I had to reimplement Matlab code in Java so it is certainly possible. The Java code can end up being quite verbose compared to the Matlab original due to Matlab being able to operate directly on matrices/arrays etc.
Some math libraries that you might want to look at to see if they support the functionality you are looking for:
Commons Math
Colt
JSci
I guess that Java would be a good choice, even though it is not considered a typical language for rapid application development.
Pros:
versatile GUI toolkit for desktop applications in standard library (Swing),
(relatively) cross-platform,
great libraries, e.g. from Apache; a great math library to look at would be colt; for charts and diagrams, you may like jfreechart ..
Cons:
"not so rapid" prototyping capabilities
Further reading:
Technical Java: Applications for Science and Engineering
Python has several decent GUI toolkits as well as NumPy, and is easy and fun to write in.
This depends mainly on how easy it will be in your environment to include the mathlab or other math engine in your product. If this is easy, I would suggest to use mathlab, but if not, e.g. you have licensing issues or deployment issues, you are probably better of just using plain Java code.
You may also want to consider R language.
I would do a search for software that's written to do piping calculations. This problem has been done. (As you've noted, the calculations need not be difficult.) At a minimum I'd recommend that you know what's available to you, how much it would cost, and where the break even point was for development costs.
A commercial product will have one huge advantage over anything you'll write: It will have a larger user community that's been banging on it and finding bugs for a longer period of time than your prototype. That's worth something as well.
What's your opportunity cost? What else could you have done with the development time that would drive more revenue?
Don't forget numpy or scipy. Both allow you to call fast matrix libraries from Python.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I have learned that the best way to learn a language is by finding a good project to start on, and working out how you would create it. I've been programming for 4 years as a hobby, and it's only been in PHP, so I decided that I should learn a "real" programming language. I have tried C++ in the past, however I was never able to find a project that I could apply it to, and therefore I never really learned much.
Now, though, I'm checking out Java, and so far I really like it. My question for you is: What would be a good project or projects for a beginner to learn the main features of Java? I understand basic OOP practices and such. What I really want to learn is all the little libraries and functions inside of it. For example, in PHP you learn to use many array manipulations and other functions built-in to do work for you. I would like to learn what these are in Java.
As I said, I don't really want just documentation, I'd like a project that I can apply Java to.
Whenever I learn a new language I start with a Mandelbrot. Just to get the hang of loops, variables and drawing.
Many other simple learning progs are possible. For basic screen/key/mouse i/o and timers, try a Tetris, Breakout, or Clock/alarm clock. To learn audio interface make an Electric piano/synth. To learn maths/recursion/string manipulation i often write a quick expression parser + graphing prog. To learn sockets, you could try a simple p2p chat.
Java is a good choice, good luck!
My advice, take a project based course some university puts up. Something past COMP 101, but not too theory heavy.
This one's how my university breaks sophomores. Its fun, and requires you to think about design and (a little) about data structures. Good "salt of the earth" programming: consoles, file IO, parsing, collections, etc. No GUI, though there's another project for that and threading.
dont rush urself, get a good java book and study from the chapter 1st and make sure you solve all the exercises of each chapter. First get ur basic / foundation strong, then u can think about how to apply that knowledge in project. And make sure u also get a good book about algorithm. If you had an experiences in php for 4 years, you must developed something with php i can assume, so try to develop same application with Java now, it will give you good insight about Java. And one thing... try to focus on how you can write good quality code! Anybody can write code but good quality code is rare, focus there.
while general, I would say find a web application that interests you. If you are familiar with PHP, you are probably familiar with many web technologies. Java on the server is a nice alternative to PHP, and deploying a Java web application is pretty simple.