I have a task to implement a program in Java (pure Java without 3rd party libraries) that reads a history of any git repository and puts the commits into tree data structure.
Could you give me any hints? How to read git log in Java without 3rd party libraries?
You might want to take a look at Processes and Threads and how to execute a command in the runtime. It does have some details and need fundamental understanding of java.lang.Runtime, java.io and some other relevant topics, so that I'd refrain to write a whole method here and recommend you to search for a good tutorial and also get the first idea from other questions here, like → getting output from executing a command line program
Related
So I have been tasked with integrating a program called "lightSIDE" into a hadoop job, and I'm having some trouble figuring out how to go about this.
So essentially, rather than a single JAR, lightSIDE comes as an entire directory, including xml files that are crucial to its running.
Up until now, the way the data scientists on my team have been using this program is by running a python script that actually runs an executable, but this seems extremely inefficient as it would be spinning up a new JVM every time it gets called. That being said, I have no idea how else to handle this.
If you are writing your own MapReduce jobs then it is possible to include all the jar files as as libraries and xml files as resources.
I'm one of the maintainers for the LightSide Researcher's Workbench. LightSide also includes a tiny PredictionServer class to handle predictions on new instances over HTTP - you can see it here on BitBucket.
If you want to train new models instead, you could modify this server to do what you want, drawing clues from the side.recipe.Chef class.
I have .exe file (I don't have source files so I won't be able to edit the program) taking as parameter path to file which be processing and on the end giving results. For example in console I run this program as follow : program.exe -file file_to_process [other_parametrs]. I have also jar executable file which take two parameters file_to_process and second file and [others_parameters]. In both cases I would like to split input file into smallest part and run programs in parallel. Is there any way to do it efficient with Apache Spark Java framework. I'm new with parallel computations and I read about RDD and pipe operator but I don't know if it would be good in my case because I have path to file.
I will be very grateful for some help or tips.
I have run into similar issues recently, and I have a working code with spark 2.1.0. The basic idea is that, you put your exe with its dependencies such as dll into HDFS or your local and use addFiles to add them into driver, which will also copy them into work executors. Then you can load your file as a RDD, and use mapPartitionsWithIndex function to save each partition into local and execute the exe (use SparkFiles.get to get the path from the work executor) to that partition using Process.
Hope that helps.
I think the general answer is "no". Spark is a framework and in general it administers very specific mechanisms for cluster configuration, shuffling its own data, read big inputs (based on HDFS), monitoring task completion and retries and performing efficient computation. It is not well suited for a case where you have a program you can't touch and that expects a file from the local filesystem.
I guess you could put your inputs on HDFS, then, since Spark accepts arbitrary java/Scala code, you could use whatever language facilities you have to dump to a local file, launch a process (i.e.this), then build some complex logic to monitor for completion (maybe based on the content of the output). the mapPartitions() Spark method would be the one best suited for this.
That said, I would not recommend it. It will be ugly, complex, require you to mess with permissions on the nodes and things like that and would not take good advantage of Spark's strengths.
Spark is well suited for you problem though, especially if each line of your file can be processed independently. I would look to see if there is a way to get the program's code, a library that does the same or if the algorithm is trivial enough to re-implement.
Probably not the answer you were looking for though :-(
I want to find a library that I can use from my Java application that will allow me to access specific Javadoc in the scope of my project (I specify where Javadocs are located). Just like in Netbeans, I want to potentially access the Javadoc from html files locally and remotely, and from source.
I expect that I could use code from Netbeans to achieve this, but I don't know how, and I can't easily digest their documentation.
Today I started thinking about the same thing.
From CI point of view, I could use #author annotation to send e-mail to someone, who wrote a test that is failing with error, not with a failure.
Google didn't help me (or I didn't google deep enough), so I started wondering how to do it on my own.
First thing that came to my mind is writing a little tool that will check all *.java files specified in a directory, bound file name to annotations and allow user to perform some actions on them.
Is that reasonable?
I need to create a Java command line to that will be invoked remotely on a server. It will need to:
Read "lines" of text from the user.
Recognise if the user presses the "tab" key to facilitate auto-complete.
Recognise if the user presses the "up/down" keys for history.
Before I go off and roll my own, Is anyone aware of a Java library that might facilitate all or part of this?
i.e. From the command line in ssh it might look like this:
bob> java -jar MyTool.jar
MyTool Started.
Please enter command:
> server1 startup
server starting...
server started
> server2 load accounts
Done
> server3 shutdown
Complete
>quit
Check out JReadline and jline2.
Update: picocli-shell-jline2 and picocli-shell-jline3 combine the strength of JLine 2 and 3 with picocli.
Picocli allows you to write commands (and subcommands) with options and positional parameters using very little code, and the library will generate JLine 2 and 3 command completers for all commands.
It uses ANSI colors and styles in the usage help message, and has many other unique features like negatable options, repeatable nested argument groups, variable interpolation and more.
Disclaimer: I am the author and therefore biased.
it seems like you're trying to ask 3 different questions at once and don't know what you really want an answer to.
accepting user input and providing auto-complete is trivial and i highly doubt you will find a generalized library for such a task
parsing complex bash-like statements sounds like something cool to have and a library may exist to do that, but i don't think it would give you much headroom to create your own set of bash-like instructions. (especially considering you say it needs to be more sophisticated than anything you could do as a bash script - which is a tall order)
parsing simple user input as if it was a command-line input or command is also rather trivial, and if this is what you are looking for, you should look at this possible duplicate: How to parse command line arguments in Java?
i recommend restructuring your question to be more specific in exactly what you are looking for and to avoid putting emphasis on the trivial task of "auto-complete" and simply accepting the users input in a text box.
Have you taken a look at the BeanShell? It doesn't act like a shell proper (like bash or csh) but it'll let you type java commands like an interpreter and you can use tab to autocomplete.
I've only used the 1.X versions of bean shell but they always open a window for you so it's not something you can run inside an existing shell.
I assume you mean something similar to the python interpreter. The reason there is not equivalent in Java is because Java needs to be compiled to bytecode before it can be executed.
If you are looking for something with good auto-complete capabilities. I would recommend eclipse or netbeans. They also compile your application automatically, allowing you to quickly run your code once you are done writing it.
Hope that helps.
Bit late to the party on this one, but I'd add Crash and Cliche to the mix.
Apache Karaf has a command shell which can be used as a library to build a command shell for other applications. You create classes to represent commands, and use annotations to specify the options to the commands. The Karaf library provides tab completion, history and line editing, and the ability to run interactively or read in files or command line arguments to execute as a script.
I found out about it here and have used it in my own project; it works quite well. I can't compare it to any of the others as I haven't used them.
I am invoking the Git executable from my Java program using the process builder class and running various git commands. Java program is being run from the command prompt.
However, I'm not finding a way to extract only a particular part of the output.
I would assume you use a StreamGobbler thread to get the input.
Just put some logic in there that filters the output.
like the others I would recommend the use of a java library, JGit allows many low-level operations if you need something very precise.
But maybe the Git command you are looking for is : "git diff commit_parent commit -U0" giving only the changes and no context.
You can also read the manual: "git help diff", you will find what Git offers.
But once again, I do not think it is a better idea to use a java Git library: what if the Git developers decide to change the output format tomorrow? Will you need to start over your program?