We Keep Coding

AWS Lambda Package Native Executable with JAR

I have an app that runs on the JVM, provided as a fat jar. It needs to invoke a native Linux binary, for example ffmpeg. What directory structure would a Zip file need to contain in order to package both the jar and the executable together? I cannot find documentation, just code examples using build tools that I have not worked with.
Let's pretend the name of my lambda is blah. I am hoping to get an answer like:
Deployable jar contains:
+ blah/ # contains fat jar
+ lib/ # contains ffmpeg
Here is a bash script I wrote that does not work. It just puts the fat jar and the native executable in the dist/ directory before zipping them together.
FATJAR=blah-assembly-0.0.4.jar
mkdir -p dist/
rm -f dist/*
rm -f $DEPLOYED_ZIP
cp $FATJAR dist/
cp /usr/local/bin/ffmpeg dist/
(cd dist && zip -r $FATJAR ffmpeg && mv $FATJAR ../$DEPLOYED_ZIP)

Any structure of the zip file will do as long as the native binary included in the package appears in the PATH or is invoked through its full path. In both cases this means that you must update the Java code in one of the following ways:
modify the PATH environment variable or
call the binary by its full path.
As of this writing the path of the directory where the package contents is extracted on an AWS Lambda system can be read from the LAMBDA_TASK_ROOT environment variable.

Update a file inside a JAR in a specific directory with jar command

So I've been looking high and low for an answer to this and obviously haven't found a satisfactory answer.
The problem is that I want to update a JAR (or any file for that matter) inside a JAR, but said file is inside a folder structure, which I would like to avoid reproducing outside the JAR just so I can update it.
Example:
Foo.jar (root directory)
/
|->/folder1
|
|->/folder2
|
|->/foo
|
|-->/bar
|
|---->/baz
|
|------>file_to_update.jar
So, if I want to update this using the jar command, I would have to first create the same folder structure outside the JAR and then do
jar -uf Foo.jar -C foo/bar/baz/file_to_update.jar
However I would like to avoid creating this folder structure since for certain files it can be quite deep and also because I might need to update selected files all over the container JAR, so I would not like to create so many folders with a specific structure just so the jar command can update them.
To put it in another way, I would like to tell the jar command to update a specific destination file inside a specific destination directory, using a specific source file inside a specific source directory.
I.e. something like
jar -uf Foo.jar -sourceFile /x/y/z/filetoupdate -destFile /a/b/c/filetoupdate
(I know this syntax doesn't work, it's just to exemplify what I need).
Can this be done? It would strike me as very odd that I would absolutely need to mimic the whole folder structure when the jar command could find a match for it or something, since I could have the need to update a file inside a folder structure that is 100 folders deep. That would be overly complex to reproduce just to update the file.
Also, I know a regular ZIP utility could be used, but the one I have available in Linux is not updating the file even if it says it did (upon re-opening the jar, it has the old version intact), and right now I can't go looking for another one (company policy, no time, testing issues, you name it).
Finally, if I can do this from a command line, it means I can create a batch file to update lots of specific files without having to create the folder structure.
I forgot to say that I would also like to avoid having to unjar, update, rejar the whole thing, as this JAR can go upwards to 2GB so it takes a while to do this just to, basically, add a little resource file to my gigantic JAR.
Any help on this is greatly appreciated!

As the accepted answer already pointed out, you can't use neither the zip nor the jar command to insert a specific file on the file system to a specific directory inside the compressed file.
But there is this workaround, which in a nutshell extracts a specific file inside the compressed file first, you can update it and then put it back into the compressed file:
1. Extracting a single file :
Consider a jar with the following contents
$ jar tvf foo.jar
0 Thu Jan 10 00:05:06 IST 2013 META-INF/
68 Thu Jan 10 00:05:06 IST 2013 META-INF/MANIFEST.MF
0 Thu Jan 10 00:04:30 IST 2013 x/
0 Thu Jan 10 00:07:36 IST 2013 x/b/
9 Thu Jan 10 00:07:36 IST 2013 x/b/hello.txt
0 Thu Jan 10 00:04:30 IST 2013 x/a/
To extract only hello.txt, you have to give fully qualified path of the file-to-extract. Jar creates appropriate folders too.
Example :
$ jar xvf foo.jar x/b/hello.txt
inflated: x/b/hello.txt
$ tree x/
x
└── b
└── hello.txt
1 directory, 1 file
2. Updating a Single File
Modify the extracted file
update the jar using the 'u' flag. Provide the exploded directory root and jar shall update all the path elements under it.
Example:
$ jar vfu foo.jar x/
adding: x/(in = 0) (out= 0)(stored 0%)
adding: x/b/(in = 0) (out= 0)(stored 0%)
adding: x/b/hello.txt(in = 23) (out= 11)(deflated 52%)
3. Which version of jar I tried this out with ?
I used jar bundled with JDK 7 on Fedora 17.
I have verified all the steps mentioned here. Hope this helps.

4129445 : An API to incrementally update ZIP files in the Sun/Oracle bug database asks for this feature to be implemented in the java api.
From the Evaluation:
"It is easy to sympathize with those who want this bug fixed. It is perhaps easier to sympathize with those who have, inadvertently, overwritten JAR files that are already in-use by a running JVM."
and
"It would have been nice if the jar & zip APIs had allowed for mutable zip files from day one. But at this point adding mutability to jar & zip files from Java(or rather, increasing the ease of mutating them) seems likely to introduce more hard-to-debug problems."

This previous discussion shows some Java source code for updating a zip (or jar) file.
It's not entirely clear to me whether this will actually be any faster than using jar -u, but at least you should be able to adapt that code so that you don't have to recreate the file structure on disk.
Edit: Here is the documentation on the 'jar' utility - you can see for yourself that the particular behavior that you're looking for, while it would be useful, isn't there.
I think that the reason is that the command-line interface is basically a clone of the Unix/Linux 'tar' command. Of course, it produces zip format, rather than tar.
Edit #2 - It occurred to me that to add a file to a zip, you'd just have to strip off the central directory from the end of the file, append the new entry, and then re-write the central directory. An update would be the same thing, except instead of adding the entry to the directory, you'd update the existing directory entry to point to the new zip entry.
While exploring this possibility, I found that Java 7 includes a zip file system provider which looks like it does this, and could form the basis of a solution for what you want to do.

I think this is due to how the classes in the java.util.zip package work.
They sit on top of streams.
The ZipFile class is built on top of a ZipFileInputStream.
Writing zipfiles is done by feeding the ZipEntries one by one to a ZipOutputStream.
I assume the jar executable is built on top of these classes.

I too was trying to update a jar without reproducing the jar's file structure external to the jar, and for some reason the jar uf jarFileName -C ... command never let me copy the file to a child directory in the jar. I then tried cutting and pasting the file into the jar using ubuntu 14.04's built in archive manager (i.e. right click on jar file and chose "open with archive manager" then paste the file anywhere in the jar's file structure), and it worked. It's not a cli solution but it works. Hope this saves someone else some time.

It works , but its not the extact solution you are looking at.
Instead of manually creating folder structure , i am asking to mention the whole path ,so mkdir will create folder and sub-directories too.hope this helps....
set /p Folder=where you want to move inside jar
set /p File=Mention path of file to be moved
set /p Jarfile=Where is your jar file
mkdir %Folder%
copy %File% %Folder%
jar -uvf %Jarfile% %File%
rd /s /q %Folder%
rmdir %Folder%

How to extract .war files in java? ZIP vs JAR

I have a web program where I want the user to be able to import a .war file and I can extract certain files out of the .war file. I have found two class libraries: java.util.zip.* and java.util.jar.*. From what I understand, a WAR file is a special JAR file which is a special ZIP file. So would it be better to use java.util.jar? If ZIP and JAR files are pretty much the same why is there a need for two different libraries?

WAR file is just a JAR file, to extract it, just issue following jar command –
jar -xvf yourWARfileName.war
If the jar command is not found, which sometimes happens in the Windows command prompt, then specify full path i.e. in my case it is,
c:\java\jdk-1.7.0\bin\jar -xvf my-file.war

If you look at the JarFile API you'll see that it's a subclass of the ZipFile class.
The jar-specific classes mostly just add jar-specific functionality, like direct support for manifest file attributes and so on.
It's OOP "in action"; since jar files are zip files, the jar classes can use zip functionality and provide additional utility.

Just rename the .war into .jar and unzip it using Winrar (or any other archive manager).

If you using Linux or Ubuntu than you can directly extract data from .war file.
A war file is just a jar file, to extract it, just issue following command using the jar program:
jar -xvf yourWARfileName.war

You can use a turn-around and just deploy the application into tomcat server: just copy/paste under the webapps folder.
Once tomcat is started, it will create a folder with the app name and you can access the contents directly

For mac users: in terminal command :
unzip yourWARfileName.war

Like you said, a jar is a zip file (not a special type, but just a plain old zip), so either library could be made to work. The reasoning is that the average person, seeing a *.zip extension, tends to unzip it. Since the app server wants it unzipped, a simple rename keeps people from unzipping it simply out of habit. Likewise, *.war file also should remain uncompressed.
java.util.jar basically just adds additional functionality to java.util.zip with very little extra overhead. Let the java.util.jar be a helper in posting, etc... and use it.

Jar class/package is for specific Jar file mechanisms where there is a manifest that is used by the Jar files in some cases.
The Zip file class/package handles any compressed files that include Jar files, which is a type of compressed file.
The Jar classes thus extend the Zip package classes.

This is the way to unarchive war
mkdir mywarfile
cp -r mywarfile.war mywarfile/
cd mywarfile/
jar -xvf mywarfile.war
ls
rm -rf mywarfile.war

Dealing with Excel files

I want to read ms-excel file in java and I've also downloaded the compressed file containing 'poi' api by Apache( on the site http://jakarta.apache.org/poi) but where actually to unzip it? means in which path?? whether in org.Apache or somewhere else??

I just unpacked it in the same directory as my NetBeans project and created a library as shown here.
Addendum: Download the preferred file from Apache Poi and unzip it in a convenient place such as your project directory. I added poi-3.6-20091214.jar as a new library from my NetBeans project directory.
$ ls -1
build
build.xml
manifest.mf
nbproject
poi-3.6
src
test
test.xls
Building form the command line is straightforward, too.
$ javac \
-cp poi-3.6/poi-3.6-20091214.jar \
-d build/classes \
-sourcepath src \
src/POIExcelReader.java

Using tools that the JDK and/or JRE provide, is there a way to view the Manifest file for a given JAR?

I'm able to read the Manifest file inside of my Java code, but I would also like to know if it's possible, and if it is, how to open up a JAR file from the command line and view its Manifest.MF file, or at the very least be able to specify a property of the Manifest.MF file and view it's value.

From here:
You can extract selected entries from a jar file. For instance, if you only want to view the meta-inf/manifest.mf file, you can
C:\Sun\AppServer\lib>jar xvf j2ee.jar META-INF/MANIFEST.MF
inflated: META-INF/MANIFEST.MF
Or using a backslash instead of a forward slash:
C:\Sun\AppServer\lib>jar xvf j2ee.jar META-INF\MANIFEST.MF
inflated: META-INF/MANIFEST.MF
The entry names are case sensitive, and so the following will not extract anything:
C:\Sun\AppServer\lib>jar xvf j2ee.jar meta-inf/manifest.mf
Of course, you can always double-click the entry to view it in WinZip, fileroller, or other tools.

Something like this should work:
jar -xf <jarfile.jar> META-INF/MANIFEST.MF

it looks like the unzip command will help you -- it's available on most Un*x variants and is part of cygwin as well, if you are on Windows. unzip -qc *jar-file* META-INF/MANIFEST.MF will dump the contents of the manifest to the console.

the jar executable provided by JDK works the same way tar works on linux.
jar xvf for example .... See jar options.

Properties for runtime should not be defined in the manifest, they should be defined in separated config files that follow the Java Properties style. Assuming you are checking the manifest at runtime for whatever reason.
But if you need to:
jar xvf somejar.jar META-INF/MANIFEST.MF
will inflate the manifest for your viewing pleasure.

There is not a way with the jar command; the closest you can get is to use -tf to show the presence or absence of a META-INF/MANIFEST.MF file or -xf to extract it.
Work-arounds:
You could write your own class to do this
You could use any ZIP file viewer to extract the contents - many will write the contents of a file in an archive to stdout so that it can be piped into another command (remember, jar files are just a specific usage of zip files)

You can just use vi for that. If you want to make sure that you don't change the file, open with with -R switch (for readonly). e.g:
vi -R sample-1.0.0-SNAPSHOT.jar
You can navigate to the zip file with the up/down arrow, or search with / e.g.:
/MANIFEST.MF
To exit type the following sequence
:q <Enter>