We Keep Coding

How to load a .java file into a variable? [duplicate]

This question already has answers here:
What is a Java ClassLoader?
(8 answers)
Closed 5 months ago.
I'm still fairly new to java and currently working on a text-based adventure game as a practice project.
The engine loads scenes to play that are all child-classes from a "Scene" superclass and appear as for eg. "dungeon.java". Now I want the game to be expandable.
The ideas is that a user can drop new scenes as .java-files into a "Scenes" folder and everytime the game is launched it reads all files in that folder and safes them into a "Scene"-class array.
My problem is that I don't know how to code this. I googled a lot and tried various phrasings but all I can find are tutorials for reading lines from txt-files or similar.
Is it even possible to read a complete file into a variable without serialzation?
I already tried the following code, but didn't get around to test it yet.
private static void buildScenePool() {
File scenesFolder = new File("/scenes");
\\ setup filter for .java files
FilenameFilter javafilter = (dir, name) -> name.endsWith(".java");
File[] sceneList = scenesFolder.listFiles(javafilter);
\\ create new arry large enough for all scenes
allScenes = new Scene[sceneList.length];
try{
FileInputStream fileIn = new FileInputStream(scenesFolder);
ObjectInputStream objectIn = new ObjectInputStream(fileIn);
\\ iterate trough the list array and safe files to array
for (int x = 0; x < allScenes.length; x++) {
allScenes[x] = objectIn.readObject( (Scene)sceneList[x] );
}
objectIn.close;
} catch (IOException e) {
System.err.println(e.toString());
}
}

This is rather very complicated. java code is normally compiled, and thus, that means you'd have to scan for a new java file, compile it (which is its own complicated ordeal, as that also means setting up the classpath and the like properly so that the compiler knows what to do), then load in the class file the compiler produced, and then call the appropriate methods in there.
You can do that. It's just, quite complicated. Your standard JVM doesn't necessarily even ship with a compiler; this is solvable too (either demand that this runs only on one that does, and the modern deployment rules for java involve you getting a JVM on your user's machines, so you thus pick one that does include a compiler – or you can just ship the compiler as dependency with your app, javac is itself a java app and runs on any JVM).
However, the more usual approach is to not actually use java for this. Instead, use something java-like, but not java: Scripting languages, like groovy, or javascript (okay, that is not particularly java-like perhaps).
That is its own sort of complication. There are no easy answers to any of this.
I think it's time to first think broad strokes and determine how you want the user's experience (that is, a user that wants to add a scene) should be, and then ask a new SO question about your specific choice.
You write em, they install em
In this model, users simply download or pick a 'scene' impl that someone else wrote (a 'real' programmer with a full fork of the entire source code, an IDE, a build tool, the works). Don't knock it - programming is hard, and saying: "Oh, look, anybody can customize a scene, it's easy, just open notepad.exe, write something like this (example java file here), and dump it in the Scene folder and off you go!", but this is not at all easy. To us programmers that seems familiar at least, but to your average user you're asking them to just rattle off a poem in ancient sumerian - normal people don't program. And if they do program, they're programmers. They would much rather get instructions about how to fork a project and set it up in an IDE than some bizarreness about tossing raw java files someplace.
Scripting
This is what Processing (for programming arduinos) does, more or less what webbrowsers did (which explains why javascript is so weird), and most 'plugin systems' for pseudo-smart editors like TextMate and Emacs do: You script them. As in, you fully buy into the idea that the custom stuff written by the user is extremely simple and highly targeted (really only makes sense to run within the confines of your app, not as standalone stuff - and dependencies don't generally come up), and pick a language that fits that model, and java certainly is not that.
Obvious options are javascript and groovy. This is certainly possible but not at all easy. Still, if you're interested, search the web for tutorials on how to run javascript or groovy inside a JVM, and you'll get plenty of hits.
Java code, and you compile it
That's what your question is positing as only option. I don't recommend it, but you can do this if you must. Be aware that it seems to me, based on the way you worded your question and your example code which makes various newbie mistakes (such as print-and-continue exception handling, which is always wrong, using obsolete APIs, and messing with built-in serialization) that this is a few ballparks beyond your current skillset. A challenge is always cool, so, if you want to go for it, you should! Just be aware it'll be the most difficult thing you've ever written and it'll take a few fully dedicated weeks, with a lot of reading and experimenting.
Just definitions, really
The central tenet so far has been that you can actually program. Instructions that make the machine act in certain ways. Possibly you don't need any of that. If a Scene is really just a background colour and a few widgets displayed here and there, should it even be code at all? Maybe you just want a declarative setup: The plugin/scene writer just declares certain properties and that's all they get to do, you just 'run' such a declarative definition. In which case the 'language' of the declaration can be in JSON, XML, YAML, TOML, or any other format designed for configuration files and such, and you can forego the hairy business of attempting to compile/run user-provided code in the first place.

In order to load the Java classes into your application, you need to compile them. You could do this from Java by invoking the javac executable. See Starting a process in Java? for instructions on how you could do that. Once compiled, you'd then need to load the classes into the JVM using a class loader, e.g. by invoking ClassLoader.defineClass. You probably want to configure a protection domain as well, to prevent user provided classes from misbehaving.
However, Java might not be the best approach for extending your application. You could consider using a scripting language instead, like JavaScript. See Nashorn (an open source script engine that was included in previous versions of Java, and that can now be downloaded separately) and the Java Scripting Programmer's Guide for more information.

When do new things get added to the Java API

I have been wondering about this for a while and I can’t really find a clear answer. You see the standard Java API is really big and it includes a lot of different libraries and classes for you to use from GUI design to sending data over the Internet to basic things like sending a String to the console.
It also includes things like reading MIDI generating secure random Strings, things that seem really specific. But at the same time there doesn’t seem to be any standard JSON libraries available while JSON is an universal way of sending data between systems.
So what I want to know is: When does something get added to the Java API? What does something need to be considered to be added to the API?

There is a "framework" that drives how new features "get" into java; to manifest themselves later on as new language elements or libraries.
Enter ... the Java Community Process!
Meaning: this is a forum where people make suggestions; which then get discussed; and at some point are either "added to Java somehow"; or rejected.
And for starters: the JSON-P project about a JSON processing API was/is driven by the jcp, see entry 374.
Finally: but you are correct, not everything that shows up in the "standard library" should be there; whereas other important parts take way too long before people can agree on a proposal. And of course, there is also a long history of evolution.
So: when you could restart Java from scratch; you would organize things in a different way (and to a certain degree, that is what Java9 is trying to enable with the new module concept).

why Preproccessor is not a part of java programs like C & C++?

As in C/C++ the program is first given to the preprocessor to include files & perform macro expansions etc... then given to the compiler to convert the code into assembly format and the process goes on.But in Java I do not see the use of preprocessor.Why so and then who does all the task that normally the preprocessor handles?

The pre-processor is not a necessary step of the compilation process in Java.
In C/C++, functions stored in different files are "included" in other files, which essentially means they are copied and pasted in their entirety into the document. This was a pretty good idea at the time, given the hardware capabilities at the time, but nowadays more modern languages use something called "symbolic imports".
Symbolic imports involve looking for symbols in another file rather than using text directly. In Java, this can involve importing constants or classes. These imports act as references to code in other files. Thus, rather than having to go through the trouble of having the pre-processor copy and paste code around and eventually figuring out what code belongs to which file, Java allows doing these imports on a semantic level directly.
This makes a pre-processor unnecessary to the compilation process of the language, and has therefore, along with other reasons been left out.

Is it recommended to separate Scala and Java source files for a Maven project?

I've been working on a Maven project consisting entirely of Java, and lately started to mix Scala code into it.
I'm amazed by the great expressiveness Scala offers, the easy use of scala-maven-plugin, and especially the incredible interoperability between Java and Scala.
However, I hit one inconvenience; according to the Maven's convention, Java's source code goes into src/main/java, whereas Scala's into src/main/scala. I found it quite cumbersome because I have to frequently go back and forth Java and Scala source files and every time I have to traverse the deep hierarchy of package directories (I often close tabs to keep my editor from cluttered).
So the question is: Is it recommended to maintain separate directories src/main/java and src/main/scala? If so, why?
To add more background, I've been working on the web application framework Wicket, whose convention is to put the HTML files alongside with their corresponding Java files. If we keep the directories separated, naturally the HTML files are separated as well (I don't think putting Scala files and corresponding HTML files in different directories makes sense). And then it goes "why I can't find Foo.html? Oh, I was looking for the wrong directory."
The source files themselves are very easy to distinguish both by humans and by machines by inspecting their extensions. I configured pom.xml to handle both Java and Scala put together in src/main/java and it worked (compiles and runs). On the other hand, separating directories poses a risk of defining conflicting classes in Java and in Scala, a careless mistake.
Well, I don't want to name a directory java if it contained not only Java's but also Scala's. But this is the only point I can come up with for separating directories.
(Edit: I've come up with a workaround-interpretation; let us think java stands for Java Virtual Machine. In this way, having src/main/c doesn't contradict if we ever decided to use JNI because C doesn't run on JVM.)
Oh, and one more point; my project is not intended as an open-source project; development convenience is preferred than trying hard to follow conventions.
This question is inspired by this answer.

I'd say yes, re-use code as much as possible. Maybe in future you can use this Java piece somewhere else...
As you probably know, you can use Java in Scala projects but not Scala in Java projects. So in this specific example it will help you with (future?) Java projects. If you want to re-use a piece of your Java code you can do that in either Java projects as well as Scala projects.
So i.m.h.o. it doesn't stop at the src/main/... but you should really put them even in different components.
Btw, little side note: if I'm correct, Wicket allows you to put the html somewhere else too, even in a different project... I saw it being handy (only) once, where we had to create different frontend for different clients of us. The java code stayed the same, the wicket-id's as well, but the html changed everywhere. Though it did give us some problems as well using the Qwicky plugin, as it could not find the html files in our IDE anymore.

Automatically generating Java source code

I'm looking for a way to automatically generate source code for new methods within an existing Java source code file, based on the fields defined within the class.
In essence, I'm looking to execute the following steps:
Read and parse SomeClass.java
Iterate through all fields defined in the source code
Add source code method someMethod()
Save SomeClass.java (Ideally, preserving the formatting of the existing code)
What tools and techniques are best suited to accomplish this?
EDIT
I don't want to generate code at runtime; I want to augment existing Java source code

What you want is a Program Transformation system.
Good ones have parsers for the language you care about, build ASTs representing the program for the parsed code, provide you with access to the AST for analaysis and modification, and can regenerate source text from the AST. Your remark about "scanning the fields" is just a kind of traversal of the AST representing the program. For each interesting analysis result you produce, you want to make a change to the AST, perhaps somewhere else, but nonetheless in the AST.
And after all the chagnes are made, you want to regenerate text with comments (as originally entered, or as you have constructed in your new code).
There are several tools that do this specifically for Java.
Jackpot provides a parser, builds ASTs, and lets you code Java procedures to do what you want with the trees. Upside: easy conceptually. Downside: you write a lot more Java code to climb around/hack at trees than you'd expect. Jackpot only works with Java.
Stratego and TXL parse your code, build ASTs, and let you write "surce-to-source" transformations (using the syntax of the target language, e.g., Java in this case) to express patterns and fixes. Additional good news: you can define any programming language you like, as the target language to be processed, and both of these have Java definitions.
But they are weak on analysis: often you need symbol tables, and data flow analysis, to really make analyses and changes you need. And they insist that everything is a rewrite rule, whether that helps you or not; this is a little like insisting you only need a hammer in toolbox; after all, everything can be treated like a nail, right?
Our DMS Software Reengineering Toolkit allows the definition of an abitrary target language (and has many predefined langauges including Java), includes all the source-to-source transformation capabilities of Stratego, TXL, the procedural capability of Jackpot,
and additionally provides symbol tables, control and data flow analysis information. The compiler guys taught us these things were necessary to build strong compilers (= "analysis + optimizations + refinement") and it is true of code generation systems too, for exactly the same reasons. Using this approach you can generate code and optimize it to the extent you have the knowledge to do so. One example, similar to your serialization ideas, is to generate fast XML readers and writers for specified XML DTDs; we've done that with DMS for Java and COBOL.
DMS has been used to read/modify/write many kinds of source files. A nice example that will make the ideas clear can be found in this technical paper, which shows how to modify code to insert instrumentation probes: Branch Coverage Made Easy.
A simpler, but more complete example of defining an arbitrary lanauges and transformations to apply to it can be found at How to transform Algebra using the same ideas.

Have a look at Java Emitter Templates. They allow you to create java source files by using a mark up language. It is similar to how you can use a scripting language to spit out HTML except you spit out compilable source code. The syntax for JET is very similar to JSP and so isn't too tricky to pick up. However this may be an overkill for what you're trying to accomplish. Here are some resources if you decide to go down that path:
http://www.eclipse.org/articles/Article-JET/jet_tutorial1.html
http://www.ibm.com/developerworks/library/os-ecemf2
http://www.vogella.de/articles/EclipseJET/article.html

Modifying the same java source file with auto-generated code is maintenance nightmare. Consider generating a new class that extends you current class and adds the desired method. Use reflection to read from user-defined class and create velocity templates for the auto-generating classes. Then for each user-defined class generate its extending class. Integrate the code generation phase in your build lifecycle.
Or you may use 'bytecode enhancement' techniques to enhance the classes without having to modify the source code.
Updates:
mixing auto-generated code always pose a risk of someone modifying it in future to just to tweak a small behavior. It's just the matter of next build, when this changes will be lost.
you will have to solely rely on the comments on top of auto-generated source to prevent developers from doing so.
version-controlling - Lets say you update the template of someMethod(), now all of your source file's version will be updated, even if the source updates is auto-generated. you will see redundant history.

You can use cglib to generate code at runtime.

Iterating through the fields and defining someMethod is a pretty vague problem statement, so it's hard to give you a very useful answer, but Eclipse's refactoring support provides some excellent tools. It'll give you constructors which initialize a selected set of the defined members, and it'll also define a toString method for you.
I don't know what other someMethod()'s you'd want to consider, but there's a start for you.

I'd be very wary of injecting generated code into files containing hand-written code. Hand-written code should be checked into revision control, but generated code should not be; the code generation should be done as part of the build process. You'd have to structure your build process so that for each file you make a temporary copy, inject the generated source code into it, and compile the result, without touching the original source file that the developers work on.

Antlr is really a great tool that can be used very easily for transforming Java source code to Java source code.