There seems to be some debate about whether JPEGs with alpha channels are valid or not. The answer I had always understood to be correct is that in the JPEG FAQ, which is essentially "No". (This is reiterated in another question on Stack Overflow.)
However, Java's JPEGImageWriter in Sun's ImageIO library will happily write and read grayscale and RGB images with an alpha channel, even though there are virtually no applications on Linux I've tried so far that will load such JPEGs correctly. This has been reported in the past as a bug, but Sun's response is that these are valid files:
This is not an Image I/O bug, but rather a deficiency in the other applications
the submitter mentions. The IIO JPEGImageWriter is able to write images with
a color model that contains an alpha channel (referred to in the IJG native
source code as the "NIFTY" color spaces, such as RGBA, YCbCrA, etc.), but many applications are not aware of these color spaces. So even though these images
written by the IIO JPEG writer are compliant with the JPEG specification (which
is blind to the various color space possiblities), some applications may not
recognize color spaces that contain an alpha channel and may throw an
error or render a corrupted image, as the submitter describes.
Developers wishing to maintain compatibility with these other alpha-unaware
applications should write images that do not contain an alpha channel (such as
TYPE_INT_RGB). Developers who want the capability of writing/reading an image
containing an alpha channel in the JPEG format can do so using the Image I/O
API, but need to be aware that many native applications out there are not quite
compliant with the YCbCrA and RGBA formats.
For more information, see the Image I/O JPEG Metadata Format Specification and Usage Notes:
http://java.sun.com/j2se/1.4.1/docs/api/javax/imageio/metadata/doc-files/jpeg_metadata.html
Closing as "not a bug".
xxxxx#xxxxx 2003-03-24
I'm working with a Java application that creates files like these, and want to write some C code that will load these as fast as possible. (Essentially the problem is that the Java ImageIO library is remarkably slow at decompressing these files, and we would like to replace the loader with native code via JNI that improves this - it's a performance bottleneck at the moment.)
There are some example files here - apologies to anyone who's coulrophobic:
http://mythic-beasts.com/~mark/example-jpegs/
And here you can see the results of trying to view the grayscale+alpha and RGB+alpha images with various bit of Linux software that I believe use libjpeg:
grayscale image with alpha channel view with various programs http://mythic-beasts.com/~mark/all-alpha-bridges.png
(source: mark at mythic-beasts.com)
So it looks as if the color space is just being misinterpreted in each case. The only allowed values in jpeglib.h are:
/* Known color spaces. */
typedef enum {
JCS_UNKNOWN, /* error/unspecified */
JCS_GRAYSCALE, /* monochrome */
JCS_RGB, /* red/green/blue */
JCS_YCbCr, /* Y/Cb/Cr (also known as YUV) */
JCS_CMYK, /* C/M/Y/K */
JCS_YCCK /* Y/Cb/Cr/K */
} J_COLOR_SPACE;
... which doesn't look promising.
If I load these images with a slightly modified version of example.c from libjpeg, the values of cinfo.jpeg_color_space and cinfo.out_color_space for each image after reading the header are as follows:
gray-normal.jpg: jpeg_color_space is JCS_GRAYSCALE, out_color_space is JCS_GRAYSCALE
gray-alpha.jpg: jpeg_color_space is JCS_CMYK, out_color_space is JCS_CMYK
rgb-normal.jpg: jpeg_color_space is JCS_YCbCr, out_color_space is JCS_RGB
rgb-alpha.jpg: jpeg_color_space is JCS_CMYK, out_color_space is JCS_CMYK
So, my questions are:
Can libjpeg be used to correctly read these files?
If not, is there an alternative C library I can use which will cope with them?
Obviously there are at least two other solutions to the more general problem:
Change the software to output normal JPEGs + a PNG file representing the alpha channel
Somehow improve the performance of Sun's ImageIO
... but the first would involve a lot of code changes, and it's not clear how to go about the latter. In any case, I think that the question of how to use libjpeg to load such files is likely to be one of more general interest.
Any suggestions would be much appreciated.
Even if you store your images as 4 channel jpeg images, there's no standardized way I know of how to specify the color format in the jpeg file.
The JFIF standard assumes YCbCr.
Have you already tried libjpeg-turbo? It is supposed to be able to decode RGBA and there is already a Java wrapper for it.
I have tried running libjpeg-turbo on color images that have an alpha channel and which have been saved with java's ImageIO as jpeg.
This is how I compiled libjpeg-turbo for linux 64-bit:
$ autoreconf -fiv
$ mkdir build
$ cd build
$ sh ../configure --with-java CPPFLAGS="-I$JAVA_HOME/include -I$JAVA_HOME/include/linux"
$ make
$ cd ..
$ mkdir my-install
$ cd build
$ make install prefix=$PWD/../my-install libdir=$PWD/../my-install/lib64
This is how I run Fiji with the correct library path and classpath, to include libjpeg-turbo:
$ cd Programming/fiji
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/../java/libjpeg-turbo/libjpeg-turbo/my-install/lib64
$ ./fiji -cp $PWD/../java/libjpeg-turbo/libjpeg-turbo/my-install/classes/turbojpeg.jar
This is a small jython script to read such jpeg+alpha files:
######
path = "/home/albert/Desktop/t2/trakem2.1263462814399.1347985440.1111111/trakem2.mipmaps/0/17.07may04b_GridID02043_Insertion001_00013gr_00005sq_00014ex.tif.jpg"
from org.libjpegturbo.turbojpeg import TJDecompressor, TJ
from java.io import File, FileInputStream
from java.awt.image import BufferedImage
from jarray import zeros
f = File(path)
fis = FileInputStream(f)
b = zeros(fis.available(), 'b')
print len(b)
fis.read(b)
fis.close()
d = TJDecompressor(b)
print d.getWidth(), d.getHeight()
bi = d.decompress(d.getWidth(), d.getHeight(), BufferedImage.TYPE_INT_ARGB, 0)
ImagePlus("that", ColorProcessor(bi)).show()
####
The problem: no matter what flag I use (the '0' above in the call to decompress) from the TJ class (see http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/java/doc/org/libjpegturbo/turbojpeg/TJ.html), I cannot get the jpeg to load.
Here's the error message:
Started turbojpeg.py at Thu Jun 02 12:36:58 EDT 2011
Traceback (most recent call last):
File "", line 15, in
at org.libjpegturbo.turbojpeg.TJDecompressor.decompressHeader(Native Method)
at org.libjpegturbo.turbojpeg.TJDecompressor.setJPEGImage(TJDecompressor.java:89)
at org.libjpegturbo.turbojpeg.TJDecompressor.(TJDecompressor.java:58)
at sun.reflect.GeneratedConstructorAccessor10.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.python.core.PyReflectedConstructor.constructProxy(PyReflectedConstructor.java:210)
java.lang.Exception: java.lang.Exception: tjDecompressHeader2(): Could not determine subsampling type for JPEG image
So it appears that either libjpeg-turbo cannot read jpeg with alpha as saved by ImageIO, or there is a very non-obvious setting in the call to "decompress" that I am unable to grasp.
Related
I'm trying to get the length of an audio file. Sadly I run into some issues trying to retrieve that file. This is my code (which is in Kotlin):
val inputStream = AudioSystem.getAudioInputStream(URL(url))
val format = inputStream.format
val durationSeconds = inputStream.frameLength / format.frameRate
lengthTicks = (durationSeconds * 20).toDouble()
The link I use is https://cdn.bandithemepark.net/lindburgh/HyperionHal.mp3
When my code gets ran, I get "UnsupportedAudioFileException: URL of unsupported format"
I am unsure why I am getting this error, since MP3 looks like a pretty normal file format to me. I also tried using an MP4 file, but I got the same error with that. Does anybody know what is happening here?
According to the docs:
The provided reference implementation of this API supports the following features:
Audio file formats: AIFF, AU and WAV
Music file formats: MIDI Type 0, MIDI Type 1, and Rich Music Format (RMF)
So it does looks like mp3 and mp4 are not supported. You'll most likely need a library/plugin.
Deciding on which one you might need is beyond the scope of SO, as that would be an opinion-based answer and is not considered acceptable.
The following code produces files which is deterministic (shasum is the same) for two strings.
try(
FileOutputStream fos = new FileOutputStream(saveLocation);
GZIPOutputStream zip = new GZIPOutputStream(fos, GZIP_BUFFER_SIZE);
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(zip, StandardCharsets.UTF_8));
){
writer.append(str);
}
Produces:
a.gz f0200d53f7f9b35647b5dece0146d72cd1c17949
However, if I take the file on the command line and re-zip it, it produces a different result
> gunzip -n a.gz ;gzip -n a ; shasum a.gz
50f478a9ceb292a2d14f1460d7c584b7a856e4d9 a.gz
How can I get it to match the original sha using /usr/bin/gzip and gunzip ?
I think that the problem is likely to be the Gzip file header.
The Gzip format has provision for including a file name and file timestamp in the file headers. (I see you are using the -n when uncompressing and recompressing ... which is probably correct here.)
The Gzip format also includes an "operating system id" in the header. This is supposed to identify the source file system type; e.g. 0 for FAT, 3 for UNIX, and so on.
Either of these could lead to differences in the Gzip files and hence different hashes.
If I was going to solve this myself, I would start by using cmp to see where the compressed file differences start, and then od to identify what the differences are. Refer to the Gzip file format spec to figure out what the differences mean:
RFC 1952 - GZIP file format specification version 4.3
Wikipedia's gzip page.
How can I get it to match the original SHA using gzip and gunzip ?
Assuming that the difference is the OS id, I don't think there is a practical way to solve this with the gzip and gunzip commands.
I looked at the source code for GZIPOutputStream in Java 11, and it is not promising.
It is hard-wiring the timestamp to zero.
It is hard-wiring the OS identifier to zero (which is supposed to mean FAT).
The hard-wiring is in a private method and would be next to impossible to "fix" by subclassing or reflection. You could copy the code and fix it that way, but then you have to maintain your variant GZIPOutputStream class indefinitely.
(I would be looking at changing the application ... or whatever ... so that I didn't need the checksums to be identical. You haven't said why you are doing this. It is for testing purposes only, try looking for a different way to implement the tests.)
I am trying to convert an image (url below) using two libraries (thumbnailator and imgscalr. My code works on most of the images except a few which after conversion have a pink/reddish tint.
I am trying to understand the cause and would welcome any recommendation.
Note - Image type of this image is 5 i.e BufferedImage.TYPE_3BYTE_BGR and i am using Java 7
Using Thumbnailator
Thumbnails.of(fromDir.listFiles())
.size(thumbnailWidth, thumbnailHeight)
.toFiles(Rename.SUFFIX_HYPHEN_THUMBNAIL);
Using imgscalr
BufferedImage bufferedImage = ImageIO.read(file);
final BufferedImage jpgImage;
LOG.debug("image type is =[{}] ", bufferedImage.getType());
BufferedImage scaledImg = Scalr.resize(bufferedImage, Method.ULTRA_QUALITY, thumbnailWidth, thumbnailHeight, Scalr.OP_ANTIALIAS);
File thumbnailFile = new File(fromDirPath + "/" + getFileName(file.getName()) +THUMBNAIL_KEYWORD + ".png");
ImageIO.write(scaledImg, getFileExtension(file.getName()), thumbnailFile);
bufferedImage.flush();
scaledImg.flush();
I get this question a lot (author of imgscalr) -- the problem is almost always that you are reading/writing out different file formats and the ALPHA channel is causing one of your color channels (R/G/B) to be culled from the resulting file.
For example, if you read in a file that was ARGB (4 channel) and wrote it out as a JPG (3 channel) - unless you purposefully manipulate the image types yourself and render the old image to the new one directly, you will get a file with a "ARG" channels... or more specifically, just Red and Green - no Blue.
PNG supports an alpha channel and JPG does not, so be aware of that.
The way to fix this is to purposefully create appropriate BufferedImage's of the right type (RGB, ARGB, etc.) and using the destImage.getGraphics() call to render one image to the other before writing it out to disk and re-encoding it.
Sun and Oracle have NEVER made the ImageIO libraries smart enough to detect the unsupported channels when writing to differing file types, so this behavior happens all the time :(
Hope that helps!
The following piece of code resolved my issue:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
Thumbnails.of(new ByteArrayInputStream(imageByteArray))
.outputFormat("jpg")
.size(200, 200)
.toOutputStream(outputStream);
return baos.toByteArray();
I am using Thumbnailator and the code was posted here: https://github.com/coobird/thumbnailator/issues/23
So I'm making an application to store shortcuts to all the user's favorite applications, acting kind of like a hub. I can have support for actual files and I have a .lnk parser for shortcuts. I thought it would be pretty good for the application to support Internet shortcuts, too. This is what I'm doing:
Suppose I'm trying to get Google's icon (http://www.google.com/favicon.ico).
I start out by getting rid of the extra pages (e.g. www.google.com/anotherpage would become www.google.com.
Then, I use ImageIO.read(java.net.URL) to get the Image.
The problem is that ImageIO never returns an Image when I call this method:
String trimmed = getBaseURL(page); //This removes the extra pages
Image icon = null;
try {
String fullURLString = trimmed + "/favicon.ico";
URL faviconURL = new URL(fullURLString);
icon = ImageIO.read(faviconURL);
} catch (IOException e) {
e.printStackTrace();
}
return icon;
Now I have two questions:
Does Java support the ICO format even though it is from Microsoft?
Why does ImageIO fail to read from the URL?
Thank you in advance!
Try Image4J.
As this quick Scala REPL session shows (paste-able as Java code):
> net.sf.image4j.codec.ico.ICODecoder.read(new java.net.URL("http://www.google.com/favicon.ico").openStream())
res1: java.util.List[java.awt.image.BufferedImage] = [BufferedImage#65712a80: type = 2 DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=ff000000 IntegerInterleavedRaster: width = 16 height = 16 #Bands = 4 xOff = 0 yOff = 0 dataOffset[0] 0]
UPDATE
To answer your questions: Does Java support ICO? Doesn't seem like it:
> javax.imageio.ImageIO.read(new java.net.URL("http://www.google.com/favicon.ico"))
java.lang.IllegalArgumentException: Empty region!
Why does ImageIO fail to read from the URL? Well, the URL itself seems to work for me, so you may have a proxy/firewall issue, or it could be the problem above.
Old post, but for future reference:
I've written a plugin for ImageIO that adds support for .ICO (MS Windows Icon) and .CUR (MS Windows Cursor) formats.
You can get it from GitHub here: https://github.com/haraldk/TwelveMonkeys/
After you have installed the plugin, you should be able to read the icon, using the code in the original post without any modifications.
You don't need ImageIO for this. Just copy the bytes, same as for any other static resource.
There is Apache Commons Imaging for reading ico files and others: https://commons.apache.org/proper/commons-imaging/index.html
Reading an ico file works like this:
List<BufferedImage> images = org.apache.commons.imaging.Imaging.getAllBufferedImages(yourIcoFile);
In your case you have to download it first, I guess.
I'm not looking for a library to convert excel files to pdf, there are plenty of those available. I'm looking for a clean way to convert a spreadsheet with more rows than the width of a page into a pdf.
Can this even be done? I don't consider making the text smaller a valid option because it could feasibly reach an upper limit (i.e. 1 pt font), and there may be enough columns in the spreadsheet to actually reach that limit (~30).
My only idea right now is to make the pages landscape, but is there a way to have the pdf show as "two-up" with both of the pages in landscape and have the proper page ordering underneath to look like a cohesive spreadsheet?
Any other ideas? or suggestions for the idea I have?
Assuming you can read the Excel file (for instance with Apache POI), consider writing to the PDF with Apache FOP using a custom paper size that you define. It may be difficult to print without a roll paper printer, but it will display on the screen just fine.
Have you looked at JasperReports? It has a pretty strong templating engine.
I've never used JasperReports the way you do, but their specialty is dynamic reports so I'd guess they know how to handle page overflows in a nice way.
Here's what I ended up doing. It uses the QuickLook feature on MacOS to make a HTML file, then uses wkhtmltopdf to turn the HTML file into a PDF.
#!/usr/bin/python
#
# convert an excel workbook to a PDF on a Mac
#
#
from subprocess import Popen,call,PIPE
import os, os.path, sys
import xml.dom.minidom
import plistlib
if len(sys.argv)==1:
print("Usage: %s filename.xls output.pdf" % sys.argv[0])
exit(1)
if os.path.exists("xdir"):
raise RuntimeError,"xdir must not exists"
os.mkdir("xdir")
call(['qlmanage','-o','xdir','-p',sys.argv[1]])
# Now we need to find the sheets and sort them.
# This is done by reading the property list
qldir = sys.argv[1] + ".qlpreview"
propfile = open("%s/%s/%s" % ('xdir',qldir,'PreviewProperties.plist'))
plist = plistlib.readPlist(propfile)
attachments = plist['Attachments']
sheets = []
for k in attachments.keys():
if k.endswith(".html"):
basename = os.path.basename(k)
fn = attachments[k]['DumpedAttachmentFileName']
print("Found %s -> %s" % (basename,fn))
sheets.append((basename,fn))
sheets.sort()
# Use wkhtmltopdf to generate the PDF output
os.chdir("%s/%s" % ('xdir',qldir))
cmd = ['wkhtmltopdf'];
for (basename,fn) in sheets:
cmd.append(fn)
cmd.append("../../" + sys.argv[2])
try:
call(cmd)
except OSError:
print("\n\nERROR: %s is not installed\n\n" % (cmd[0]))
exit(1)
os.chdir("../..")
call(['/bin/rm','-rf','xdir'])