cross-platform automatic build of non-C projects - java

Actually, I need to build binaries for different languages - Java, Pascal (Delphi/FPC), C++, C#, etc under Windows / Linux platforms.
How this could be done? I am using CMake for C projects for now like this:
cmake --build . with CMakeLists.txt providen.

Every *make system is not usefull for Java project because make basically assumes one of "One file in, one file out" (e.g. .c to .o) or "N files in, one file out" (e.g. *.o to one of .a, .dll, .exe). With these "primitives" a build is performed.
But the Java compiler requires inherently "complete sourcetree in, comlete class-tree out".
Those two paradigmata don't match and trying to do will introduce more little nasty problems than its worth sooner or later. Been there, done that.
The best you can do is to call Ant or Maven (or whatever) from within a toplevel Makefile.
Edit: Not to forget two more things:
Compiling Java stuff with make is a dark, arcane art. You will not find many people who know both Java and make and who can solve problems with that combination. Ant/Maven and Java is a standard combination. There is simply no problem finding help or advises with that combination.
Many frameworks in Java require special compilation steps. In almost all cases there are plugins for Ant and Maven available. Using make you will have to reinvent the wheel every time again.

Free Pascal provides its own build systems. See e.g.
http://www.stack.nl/~marcov/buildfaq.pdf
the default makefiles (based on mostly generated gmake makefiles) are already cross- capable.
I don't know anybody who uses cmake with Free Pascal, and I've been on their maillists for over a decade.

Related

Does compiling Java to native code protect the source code?

Now, I know that...
Anything can be reverse engineered, given enough time and resources.
However, would compiling your Java code to native code with a tool like GCJ make it more difficult to decompile? I mean, given a few minutes, I can decompile a .jar using JD-GUI, and it is relatively accurate. Most of the "Java to EXE" converters are just .exe launchers for the JVM, and while there are many benefits to the JVM, I have been led to believe that security of the source code is not one of them.
Bottom line: Can you use something like GCJ to compile your Java source (or .class) files to native machine code, and if so, will that protect it from decompiling?
EDIT: Ideally, it would be something more than just obfuscation. The specific project is a commercial game, so what we are looking for is a way to make it more difficult to get to the source code to begin with, not just understand it. Also, I'm not sure that Steam accepts .jars, and we are planning on submitting it to the new Green Light project.
I wouldn't choose that approach just for source-security.
Check out some Obfuscator tools out there like ProGuard
If you want to see what those tools do to your source code, just try read the decompiled Minecraft jar if you have one on hand.
A downside to using this is, that if your code depends on using reflection, you'll have to configure the tools to ignore those functions/classes/whatever, as those will not be found at runtime otherwise.
Technically, yes. Using something like GCJ will make it harder to decompile, however keep in mind that you are losing some major benefits of using Java if you do this. Namely, you lose the ability to write cross-platform applications.
You could use an obfuscator to make the code harder to decompile AND still keep the benefits of using Java.
a Source code obfuscator like
this , this and this
makes your variables, functions, etc... unreadable by other(has no logical meaning). You should read here too!

Libraries for building projects? Can I compile C++ using C++, or java with python, or java with C++, etc?

Are there any libraries or frameworks out there that are designed to facilitate the building of projects from within another full fledged programming language?
It is very easy to specify logic, sets, and complicated rules in programming languages like C++, java, etc, but it seems like an uphill battle to do these things in a Makefile. I haven't dug into Ant or Maven or any of the other building tools yet, but if I could just write all my build logic into a C++ program it would be much easier! (Assuming I had some helpful tools.)
Maybe you'd like to look at this project: http://www.scons.org/
But, what would you like to do besides specifying modules dependencies and build rules? I think putting too much logic into the build system is just creating another extra problem; try to stick to well known configuration tools and you'll have one thing less to worry about.
About Maven and Ant, they are a little bit java-centric, but can be used for any kind of projects and have loads of plugins to perform almost any task you could imagine. If you prefer to use a more unix-oriented environment, but want a higher level layer on top make you can use the Autotools.
Dig into Ant and/or Maven. These can handle all the different tasks needed for creating complicated builds. This will save you reinventing the wheel by trying to replicate all those features in C++. I think the issue you are struggling with is that you need to spend the time to learn to use a new build tool.

Why is no one using make for Java?

Just about every Java project that I've seen either uses Maven or Ant. They are fine tools and I think just about any project can use them. But what ever happened to make? It's used for a variety of non-Java projects and can easily handle Java. Sure you have to download make.exe if you use Windows, but Ant and Maven also don't come with the JDK.
Is there some fundamental flaw with make when used with Java? Is it just because Ant and Maven are written in Java?
The fundamental issue with Make and Java is that Make works on the premise that you have specify a dependency, and then a rule to resolve that dependency.
With basic C, that typically "to convert a main.c file to a main.o file, run "cc main.c".
You can do that in java, but you quickly learn something.
Mostly that the javac compiler is slow to start up.
The difference between:
javac Main.java
javac This.java
javac That.java
javac Other.java
and
javac Main.java This.java That.java Other.java
is night and day.
Exacerbate that with hundreds of classes, and it just becomes untenable.
Then you combine that with the fact that java tends to be organized as groups of files in directories, vs C and others which tend towards a flatter structure. Make doesn't have much direct support to working with hierarchies of files.
Make also isn't very good at determining what files are out of date, at a collection level.
With Ant, it will go through and sum up all of the files that are out of date, and then compile them in one go. Make will simply call the java compiler on each individual file. Having make NOT do this requires enough external tooling to really show that Make is not quite up to the task.
That's why alternatives like Ant and Maven rose up.
Actually, make can handle the recompilation in one command of all outdated java files. Change the first line if you don't want to compile all files in the directory or want a specific order...
JAVA_FILES:=$(wildcard *.java)
#
# the rest is independent of the directory
#
JAVA_CLASSES:=$(patsubst %.java,%.class,$(JAVA_FILES))
.PHONY: classes
LIST:=
classes: $(JAVA_CLASSES)
if [ ! -z "$(LIST)" ] ; then \
javac $(LIST) ; \
fi
$(JAVA_CLASSES) : %.class : %.java
$(eval LIST+=$$<)
The venerable make program handles separately compiled languages like C and C++ reasonably well. You compile a module, it uses #include to pull in the text of other include files, and writes a single object file as output. The compiler is very much a one-at-a-time system, with a separate linking step to bind the object files into an executable binary.
However, in Java, the compiler has to actually compile other classes that you import with import. Although it would be possible to write something that generated all the necessary dependencies from Java source code, so that make would build classes in the correct order one at a time, this still wouldn't handle cases such as circular dependencies.
The Java compiler can also be more efficient by caching the compiled results of other classes while compiling further classes that depend on the results of ones already compiled. This sort of automatic dependency evaluation is not really possible with make alone.
The question is based on an incorrect assumption: a non-trivial number of developers do use make. See Java Build Tools: Ant vs. Maven. As for why a developer wouldn't use make: many developers either have never used make, or used it and hated it with a fire that burns hotter than a thousand suns. As such, they use alternative tools.
All the other answers about the technical merits of each are true. Ant and Maven may be better suited to Java than make, or as Hank Gay points out, they may not :)
However, you asked if it matters that Ant and Maven are written in Java. Although on StackOverflow we don't consider such thoughts (closed! not-programming-related! etc.), OF COURSE THAT'S PART OF THE THING. On rails we use Rake, C dudes use make, and in Java we use Ant and Maven. While it's true that the Ant or Maven developers will look after the Java developer perhaps better than others, there's also another question: what do you write Ant tasks in? Java. If you're a Java developer, that's an easy fit.
So yeah, part of it is to use tools written in the language you are tooling.
Ant and later Maven were designed to solve some headaches caused by Make ( while creating new ones in the process ) It is just evolution.
...Soon thereafter, several open source Java projects realized that Ant could solve the problems they had with Makefiles....
From http://ant.apache.org/faq.html#history
Whether they solve anything or just create an extra format to learn is a subjective topic. The truth is that's pretty much the history of every new invention: The creator says it solves a lot of problems and the original users say those are virtues.
The main advantage it has, is the possibility to integrate with java.
I guess a similar history would be with rake for instance.
One of the major issues solved by Maven (and Ivy-enabled Ant setups) over make is automated dependency resolution and downloading of your dependency jars.
Short answer: Because make isn't good. Even on the C front you see many alternatives popping up.
Long answer: make has several flaws that make it barely suitable for compiling C, and unsuitable at all for compiling Java. You can force it to compile Java, if you want, but expect running into issues, some of which do not have a suitable solution or workaround. Here are a few:
Dependency resolution
make inherently expects files to have a tree-like dependency on each other, in which one file is the output of building several others. This already backfires in C when dealing with header files. make requires a make-specific include file to be generated to represent the dependency of a C file on its header files, so a change to the latter would cause the prior to be rebuilt. However, since the C file itself isn't recreated (merely rebuilt), make often requires specifying the target as .PHONY. Fortunately, GCC supports generating those files automatically.
In Java, dependency can be circular, and there's no tool for auto-generating class dependencies in make format. ant's Depend task can, instead, read the class file directly, determine which classes it imports, and delete the class file if any of them are out of date. Without this, any non-trivial dependency may result in you being forced to use repeated clean builds, removing any advantage of using a build tool.
Spaces in filenames
While neither Java nor C encourage using spaces in your source code filenames, in make this can be problem even if the spaces are in the file path. Consider, for example, if your source code exists in C:\My Documents\My Code\program\src. This would be enough to break make. This is because make treats filenames as strings. ant treats paths as special objects.
Scanning files for build
make requires explicitly setting which files are to be built for each target. ant allows specifying a folder which is to be auto-scanned for source files. It may seem like a minor convenience, but consider that in Java each new class requires a new file. Adding files to the project can become a big hassle fast.
And the biggest problem with make:
make is POSIX-dependent
Java's motto is "compile once run everywhere". But restricting that compilation to POSIX-based systems, in which Java support is actually the worst, is not the intention.
Build rules in make are essentially small bash scripts. Even though there is a port of make to Windows, for it to work properly, it has to be bundled with a port of bash, which includes a POSIX emulation layer for the file system.
This comes in two varieties:
MSYS which tries to limit the POSIX translation to file paths, and can therefore have unpleasant gotchas when running external tools not made especially for it.
cygwin which provides a complete POSIX emulation. The resulting programs, however, tend to still rely on that emulation layer.
For that reason, on Windows, the standard build tool isn't even make at all, but rather MSBuild, which is also an XML-based tool, closer in principle to ant.
By contrast, ant is built in Java, can run everywhere, and contains internal tools, called "tasks", for manipulating files and executing commands in a platform-independent way. It's sufficiently versatile that you can actually have an easier time building a C program in Windows using ant than using make.
And one last minor one:
Even C programs don't use make natively
You may not initially notice this, but C programs generally aren't shipped with a Makefile. They are shipped with a CMakeLists.txt, or a bash configuration script, which generates the actual Makefile. By contrast, the source of a Java program built using ant is shipped with an ant script pre-built. A Makefile is a product of other tools - That's how much make is unsuitable to be a build tool on its own. ant is standalone, and deals with everything you need for your Java build process, without any additional requirements or dependencies.
When you run ant on any platform, it Just Works(tm). You can't get that with make. It's incredibly platform and configuration dependent.
I think the most likely explanation is that several factors discouraged the use of make within the Java community in a critical period of time (the late 1990s):
Because Java encompasses multiple platforms, Java programmers in general were not as adept at Unix tools as were programmers generally confined to a Unix environment (e.g., C and Perl programmers). Note that this is IN GENERAL. Without a doubt there are and were gifted Java programmers with a deep understanding of Unix.
Consequently they were less adept at make and didn't know how to use make effectively.
While it is possible to write a short and simple Makefile that compiles Java efficiently, extra care is required to do so in a platform-independent way.
Consequently there was an appetite for an intrinsically platform-independent build tool.
It was in this environment that Ant and later Maven were created.
In short, while make most certainly can be used for Java projects, there was a moment of opportunity to make it the de facto Java build tool. That moment has passed.
Unless I am no one the assumption no one is (mis)using make for java is wrong.
"Managing Projects with GNU Make" (available under GFDL) contains a complete chapter dedicated to using make with java projects.
As it contains a long (and hopefully fair) list of the pros and cons of using make instead of other tools you might want to take a look there. (see: http://oreilly.com/catalog/make3/book/)
Make scripts tend to be inherently platform dependent. Java is supposed to be platform independent. Therefore having a build system that only works on one platform for a multi-platform sourcebase is kindof a problem.
Ant is an XML configuration oriented improvement over Makefiles and Maven is a dependency build tool improvement over Ant. Some projects use all three. I think the JDK projects used to use a mix of makefiles and ant.
One big reason is that both Ant and Maven (and most java targeted SCM, CI and IDE tools) are written in java by/for java developers. This makes it simpler to integrate into your development environment and allows other tools such as the IDE and CI servers to integrate portions of the ant/maven libraries within the build/deployment infrastructure.
Once upon a time I worked on a Java project that used gmake. My recollection is hazy but IIRC we had a hard time dealing with the package directory structure that javac expects. I also remember that building JAR files was a hassle unless you had something trivial.
ApacheAnt isn't anything like Make. Make is about describing dependencies between files, and how to build files. Ant is about dependencies between "tasks", and is really more of a way of gluing build scripts together.
it may helps you AntVsMake
Ant and Maven approach the build dependency graph and the management of it from a more 'modern' view... But as Oscar says, they created their own problems while attempting to address the old problems with make.
I've never used GNU Make for Java projects, but I used to use jmk. Sadly it hasn't been updated since 2002.
It had some Java-specific functionality but was small enough to include in your source tarball without significantly increasing its size.
Nowadays I just assume any Java developer I share code with has Ant installed.

Write once, run anywhere languages & Unrar for AIR

I wanted to ask if there are any other Write Once, run anywhere languages like Java or AIR. I have mostly been a Java developer up till now, and a bit of Python and recently AIR has peaked my interest as I just found out about it.
Also I wanted to know if there are any unrar libraries for AIR or someway to read RAR archives with AIR, kind of like how you can with FZip and zip files. Kinda want a solution better than including binaries for Win/Lin/OSX with my program and using those based on the system.
Also I don't mind a compiled language as long as I can write 1 set of code, and it can be compiled to all the platforms with minimal to no changes. Kinda want to not use any OS Specific code or APIs, its why I asked for runtime languages like AIR or Java. Well Python is too, but it requires a bit of OS specific coding for file management.
Perl is also write once run everywhere.
I can recommend Lua. It is very similar to Python, lightweight and very portable.
YMMV depending on what you want to do with the language. If you're looking for a language with lots of bells and whistles you're probably better off with Python. That said, there are a lot of bindings for popular libraries available at LuaForge
I may suggest this Java API to un-rar file

When learning Java, how important is it to know Unix well?

Other than the Java language itself, you have to learn the java framework. Similiar to how you have to learn the .net framework in addition to the language (C#/VB).
How important is it to know unix? Or rather, what unix areas should one focus on?
Seeing as many people run java based applications (desktop/web) on unix boxes, what sort of unix skills do you need? Are we just talking basic directory traversing, creating files, etc or is there much more to it?
The answer as read from Sun marketing material is that Java is cross platform, so you don't.
The practical answer is that you need to know enough to get your own application up and running on the platform where you plan to use it. Getting to know Apache or Tomcat configuration helps if you're working with web development, and so does knowing how to use the basic network analysis tools - the ifconfig, netstat and traceroute commands are all useful. File permission tools are also a must for getting a system working - look into chmod and chown and how those commands work.
Desktop systems have it easier, since most windowing systems are very good at working cross platform, but you still need to know a little bit about how the file system and permissions are structured.
Really, you don't need unix skills directly for writing java-based applications. However, if you want to develop java-based applications on unix boxes and deploy there, you want a good working understanding of how to operate and administer a unix box.
But for the areas you mention (directory traversing, creating files), you'll be using Java APIs that only occasionally touch on Unix-specific ("\n" vs "\r\n", directories rooted at "/", etc.) information. When they do touch, it's not something you need to know in a programming sort of way, it's something you need to know in a user/administrator sort of way.
Not important for the language it self.
You can learn java very well and never have touched a unix box at all.
If you want to deploy on a unix server however you should know just the basics.
File paths, new lines, permissions, etc. but the knowledge needed can be acquired after a few hours in the typing in the terminal.
Most of the os specifics are abstracted into the language from the beginning. Those that cannot be abstracted ( such as cron for instance ) are left behind.
You don't really need Unix skills to use Java, but if you do you'll have a good toolbox relevant for any kind of development. I certainly appreciate the ability to grep my entire source tree for files, use Perl for code generation and so forth. Even a simple matter of counting all lines of source code can be hard to do in a GUI only world. Knowing the basic Unix command line tools will make you a better developer imo.
I think your metaphor means you should learn Java library in addition to the language itself.
Certainly knowing UNIX will help a bit as it increases your general knowledge, but I don't think it's really directly related to Java at all.
As well as knowing how to traverse directories, you need to be able to edit and grep files. You need to know how do do some basic process management also. But the level of detail you need depends on what you want to do on a unix system. Eg. web development requires that you run a web container such as tomcat. You will probably want to learn the package manager of your system of choice.
It depends on your OS. You can do Java development on Windows in which case no Unix/Linux knowledge is required. That said, even on Windows GNU/Cygwin utilities can be helpful.
I assume that you are talking about learning and not developing a full scale project.
Because of the JVM, which by vision should provide a uniform API towards the programmers, regardless of the underlaying system, Java programs are meant to be written in a way that is not specific to the underlaying system (up to the point of using JNI etc').
That means that as a writer of small programs for learning,
your knowledge of the underlaying system should be minimal.
If you are not using IDE, you should at least know how to run 'javac', 'java' from the command line, and to test your program.
If you are Using an IDE like Eclipse, from the point that the IDE is running, you should expect an experience that is almost isolated from the system underneath.
However keep in mind that this is the vision, and it is always advised to know at least a bit about what's going on on your own system.
You don't need to know *NIX to develop a Java application, maybe if you're going to run it on a *NIX machine or if it uses something very attached to *NIX it'll be good if you know the basis about file names and permissions, for example.
The need of knowing *NIX comes with the need of deploying the app on a *NIX server. If this is the case it'll be good if you know something about system variables (set the JAVA_HOME, for example), the tipicall directory structure of a *NIX machine, and basic use (creating files, directories, deleting, symlinks...).
I think it's very good to know some scripting (bash, csh, sh). Depending of the complexity of the deployment this can give you extra points at the look of your boss for example.

Categories