I need to parse a PDF file which contains tabular data. I'm using PDFBox to extract the file text to parse the result (String) later. The problem is that the text extraction doesn't work as I expected for tabular data. For example, I have a file which contains a table like this (7 columns: the first two always have data, only one Complexity column has data, only one Financing column has data):
+----------------------------------------------------------------+
| AIH | Value | Complexity | Financing |
| | | Medium | High | Not applicable | MAC/Other | FAE |
+----------------------------------------------------------------+
| xyz | 12.43 | 12.34 | | | 12.34 | |
+----------------------------------------------------------------+
| abc | 1.56 | | 1.56 | | | 1.56|
+----------------------------------------------------------------+
Then I use PDFBox:
PDDocument document = PDDocument.load(pathToFile);
PDFTextStripper s = new PDFTextStripper();
String content = s.getText(document);
Those two lines of data would be extracted like this:
xyz 12.43 12.4312.43
abc 1.56 1.561.56
There are no white spaces between the last two numbers, but this is not the biggest problem. The problem is that I don't know what the last two numbers mean: Medium, High, Not applicable? MAC/Other, FAE? I don't have the relation between the numbers and their columns.
It is not required for me to use the PDFBox library, so a solution that uses another library is fine. What I want is to be able to parse the file and know what each parsed number means.
You will need to devise an algorithm to extract the data in a usable format. Regardless of which PDF library you use, you will need to do this. Characters and graphics are drawn by a series of stateful drawing operations, i.e. move to this position on the screen and draw the glyph for character 'c'.
I suggest that you extend org.apache.pdfbox.pdfviewer.PDFPageDrawer and override the strokePath method. From there you can intercept the drawing operations for horizontal and vertical line segments and use that information to determine the column and row positions for your table. Then its a simple matter of setting up text regions and determining which numbers/letters/characters are drawn in which region. Since you know the layout of the regions, you'll be able to tell which column the extracted text belongs to.
Also, the reason you may not have spaces between text that is visually separated is that very often, a space character is not drawn by the PDF. Instead the text matrix is updated and a drawing command for 'move' is issued to draw the next character and a "space width" apart from the last one.
Good luck.
You can extract text by area in PDFBox. See the ExtractByArea.java example file, in the pdfbox-examples artifact if you're using Maven. A snippet looks like
PDFTextStripperByArea stripper = new PDFTextStripperByArea();
stripper.setSortByPosition( true );
Rectangle rect = new Rectangle( 464, 59, 55, 5);
stripper.addRegion( "class1", rect );
stripper.extractRegions( page );
String string = stripper.getTextForRegion( "class1" );
The problem is getting the coordinates in the first place. I've had success extending the normal TextStripper, overriding processTextPosition(TextPosition text) and printing out the coordinates for each character and figuring out where in the document they are.
But there's a much simpler way, at least if you're on a Mac. Open the PDF in Preview, ⌘I to show the Inspector, choose the Crop tab and make sure the units are in Points, from the Tools menu choose Rectangular selection, and select the area of interest. If you select an area, the inspector will show you the coordinates, which you can round and feed into the Rectangle constructor arguments. You just need to confirm where the origin is, using the first method.
I had used many tools to extract table from pdf file but it didn't work for me.
So i have implemented my own algorithm ( its name is traprange ) to parse tabular data in pdf files.
Following are some sample pdf files and results:
Input file: sample-1.pdf, result: sample-1.html
Input file: sample-4.pdf, result: sample-4.html
Visit my project page at traprange.
It may be too late for my answer, but I think this is not that hard. You can extend the PDFTextStripper class and override the writePage() and processTextPosition(...) methods. In your case I assume that the column headers are always the same. That means that you know the x-coordinate of each column heading and you can compare the the x-coordinate of the numbers to those of the column headings. If they are close enough (you have to test to decide how close) then you can say that that number belongs to that column.
Another approach would be to intercept the "charactersByArticle" Vector after each page is written:
#Override
public void writePage() throws IOException {
super.writePage();
final Vector<List<TextPosition>> pageText = getCharactersByArticle();
//now you have all the characters on that page
//to do what you want with them
}
Knowing your columns, you can do your comparison of the x-coordinates to decide what column every number belongs to.
The reason you don't have any spaces between numbers is because you have to set the word separator string.
I hope this is useful to you or to others who might be trying similar things.
There's PDFLayoutTextStripper that was designed to keep the format of the data.
From the README:
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import org.apache.pdfbox.pdfparser.PDFParser;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.util.PDFTextStripper;
public class Test {
public static void main(String[] args) {
String string = null;
try {
PDFParser pdfParser = new PDFParser(new FileInputStream("sample.pdf"));
pdfParser.parse();
PDDocument pdDocument = new PDDocument(pdfParser.getDocument());
PDFTextStripper pdfTextStripper = new PDFLayoutTextStripper();
string = pdfTextStripper.getText(pdDocument);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
};
System.out.println(string);
}
}
I've had decent success with parsing text files generated by the pdftotext utility (sudo apt-get install poppler-utils).
File convertPdf() throws Exception {
File pdf = new File("mypdf.pdf");
String outfile = "mytxt.txt";
String proc = "/usr/bin/pdftotext";
ProcessBuilder pb = new ProcessBuilder(proc,"-layout",pdf.getAbsolutePath(),outfile);
Process p = pb.start();
p.waitFor();
return new File(outfile);
}
Try using TabulaPDF (https://github.com/tabulapdf/tabula) . This is very good library to extract table content from the PDF file. It is very as expected.
Good luck. :)
Extracting data from PDF is bound to be fraught with problems. Are the documents created through some kind of automatic process? If so, you might consider converting the PDFs to uncompressed PostScript (try pdf2ps) and seeing if the PostScript contains some sort of regular pattern which you can exploit.
I had the same problem in reading the pdf file in which data is in tabular format. After regular parse using PDFBox each row were extracted with comma as a separator... losing the columnar position.
To resolve this I used PDFTextStripperByArea and using coordinates I extracted the data column by column for each row. This is provided that you have a fixed format pdf.
File file = new File("fileName.pdf");
PDDocument document = PDDocument.load(file);
PDFTextStripperByArea stripper = new PDFTextStripperByArea();
stripper.setSortByPosition( true );
Rectangle rect1 = new Rectangle( 50, 140, 60, 20 );
Rectangle rect2 = new Rectangle( 110, 140, 20, 20 );
stripper.addRegion( "row1column1", rect1 );
stripper.addRegion( "row1column2", rect2 );
List allPages = document.getDocumentCatalog().getAllPages();
PDPage firstPage = (PDPage)allPages.get( 2 );
stripper.extractRegions( firstPage );
System.out.println(stripper.getTextForRegion( "row1column1" ));
System.out.println(stripper.getTextForRegion( "row1column2" ));
Then row 2 and so on...
You can use PDFBox's PDFTextStripperByArea class to extract text from a specific region of a document. You can build on this by identifying the region each cell of the table. This isn't provided out of the box, but the example DrawPrintTextLocations class demonstrates how you can parse the bounding boxes of individual characters in a document (it would be great to parse bounding boxes of strings or paragraphs, but I haven't seen support in PDFBox for this - see this question). You can use this approach to group up all touching bounding boxes to identify distinct cells of a table. One way to do this is to maintain a set boxes of Rectangle2D regions and then for each parsed character find the character's bounding box as in DrawPrintTextLocations.writeString(String string, List<TextPosition> textPositions) and merge it with the existing contents.
Rectangle2D bounds = s.getBounds2D();
// Pad sides to detect almost touching boxes
Rectangle2D hitbox = bounds.getBounds2D();
final double dx = 1.0; // This value works for me, feel free to tweak (or add setter)
final double dy = 0.000; // Rows of text tend to overlap, so no need to extend
hitbox.add(bounds.getMinX() - dx , bounds.getMinY() - dy);
hitbox.add(bounds.getMaxX() + dx , bounds.getMaxY() + dy);
// Find all overlapping boxes
List<Rectangle2D> intersectList = new ArrayList<Rectangle2D>();
for(Rectangle2D box: boxes) {
if(box.intersects(hitbox)) {
intersectList.add(box);
}
}
// Combine all touching boxes and update
for(Rectangle2D box: intersectList) {
bounds.add(box);
boxes.remove(box);
}
boxes.add(bounds);
You can then pass these regions to PDFTextStripperByArea.
You can also go one further and separate out the horizontal and vertical components of these regions, and so infer regions of all the table's cells, regardless of whether then hold any content.
I have had cause to perform these steps, and eventually wrote my own PDFTableStripper class using PDFBox. I've shared my code as a gist on GitHub. The main method gives an example of how the class can be used:
try (PDDocument document = PDDocument.load(new File(args[0])))
{
final double res = 72; // PDF units are at 72 DPI
PDFTableStripper stripper = new PDFTableStripper();
stripper.setSortByPosition(true);
// Choose a region in which to extract a table (here a 6"wide, 9" high rectangle offset 1" from top left of page)
stripper.setRegion(new Rectangle(
(int) Math.round(1.0*res),
(int) Math.round(1*res),
(int) Math.round(6*res),
(int) Math.round(9.0*res)));
// Repeat for each page of PDF
for (int page = 0; page < document.getNumberOfPages(); ++page)
{
System.out.println("Page " + page);
PDPage pdPage = document.getPage(page);
stripper.extractTable(pdPage);
for(int c=0; c<stripper.getColumns(); ++c) {
System.out.println("Column " + c);
for(int r=0; r<stripper.getRows(); ++r) {
System.out.println("Row " + r);
System.out.println(stripper.getText(r, c));
}
}
}
}
It is not required for me to use the PDFBox library, so a solution that uses another library is fine
Camelot and Excalibur
You may want to try Python library Camelot, an open source library for Python. If you are not inclined to write code, you may use the web interface Excalibur created around Camelot. You "upload" the document to a localhost web server, and "download" the result from this localhost server.
Here is an example from using this python code:
import camelot
tables = camelot.read_pdf('foo.pdf', flavor="stream")
tables[0].to_csv('foo.csv')
The input is a pdf containing this table:
Sample table from the PDF-TREX set
No help is provided to camelot, it is working on its own by looking at pieces of text relative alignment. The result is returned in a csv file:
PDF table extracted from sample by camelot
"Rules" can de added to help camelot identify where are fillets in sophisticated tables:
Rule added in Excalibur. Source
GitHub:
Camelot: https://github.com/camelot-dev/camelot
Excalibur: https://github.com/camelot-dev/excalibur
The two projects are active.
Here is a comparison with other software (with test based on actual documents), Tabula, pdfplumber, pdftables, pdf-table-extract.
I want is to be able to parse the file and know what each parsed number means
You cannot do that automatically, as pdf is not semantically structured.
Book versus document
Pdf "documents" are unstructured from a semantic standpoint (it's like a notepad file), the pdf document gives instructions on where to print a text fragment, unrelated to other fragments of the same section, there is no separation between content (what to print, and whether this is a fragment of a title, a table or a footnote) and the visual representation (font, location, etc). Pdf is an extension of PostScript, which describes a Hello world! page this way:
!PS
/Courier % font
20 selectfont % size
72 500 moveto % current location to print at
(Hello world!) show % add text fragment
showpage % print all on the page
(Wikipedia).
One can imagine what a table looks like with the same instructions.
We could say html is not clearer, however there is a big difference: Html describes the content semantically (title, paragraph, list, table header, table cell, ...) and associates the css to produce a visual form, hence content is fully accessible. In this sense, html is a simplified descendant of sgml which puts constraints to allow data processing:
Markup should describe a document's structure and other attributes
rather than specify the processing that needs to be performed, because
it is less likely to conflict with future developments.
exactly the opposite of PostScript/Pdf. SGML is used in publishing. Pdf doesn't embed this semantical structure, it carries only the css-equivalent associated to plain character strings which may not be complete words or sentences. Pdf is used for closed documents and now for the so-called workflow management.
After having experimented the uncertainty and difficulty in trying to extract data from pdf, it's clear pdf is not at all a solution to preserve a document content for the future (in spite Adobe has obtained from their pairs a pdf standard).
What is actually preserved well is the printed representation, as the pdf was fully dedicated to this aspect when created. Pdf are nearly as dead as printed books.
When reusing the content matters, one must rely again on manual re-entering of data, like from a printed book (possibly trying to do some OCR on it). This is more and more true, as many pdf even prevent the use of copy-paste, introducing multiple spaces between words or produce an unordered characters gibberish when some "optimization" is done for web use.
When the content of the document, not its printed representation, is valuable, then pdf is not the correct format. Even Adobe is unable to recreate perfectly the source of a document from its pdf rendering.
So open data should never be released in pdf format, this limits their use to reading and printing (when allowed), and makes reuse harder or impossible.
ObjectExtractor oe = new ObjectExtractor(document);
SpreadsheetExtractionAlgorithm sea = new SpreadsheetExtractionAlgorithm(); // Tabula algo.
Page page = oe.extract(1); // extract only the first page
for (int y = 0; y < sea.extract(page).size(); y++) {
System.out.println("table: " + y);
Table table = sea.extract(page).get(y);
for (int i = 0; i < table.getColCount(); i++) {
for (int x = 0; x < table.getRowCount(); x++) {
System.out.println("col:" + i + "/lin:x" + x + " >>" + table.getCell(x, i).getText());
}
}
}
How about printing to image and doing OCR on that?
Sounds terribly ineffective, but it's practically the very purpose of PDF to make text inaccessible, you gotta do what you gotta do.
http://swftools.org/ these guys have a pdf2swf component. They are also able to show tables.
They are also giving the source. So you could possibly check it out.
This works fine if PDF file has "Only Rectangular table" using pdfbox 2.0.6. Won't work with any other table only Rectangular table.
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.PDFTextStripperByArea;
public class PDFTableExtractor {
public static void main(String[] args) throws IOException {
ArrayList<String[]> objTableList = readParaFromPDF("C:\\sample1.pdf", 1,1,6);
//Enter Filepath, startPage, EndPage, Number of columns in Rectangular table
}
public static ArrayList<String[]> readParaFromPDF(String pdfPath, int pageNoStart, int pageNoEnd, int noOfColumnsInTable) {
ArrayList<String[]> objArrayList = new ArrayList<>();
try {
PDDocument document = PDDocument.load(new File(pdfPath));
document.getClass();
if (!document.isEncrypted()) {
PDFTextStripperByArea stripper = new PDFTextStripperByArea();
stripper.setSortByPosition(true);
PDFTextStripper tStripper = new PDFTextStripper();
tStripper.setStartPage(pageNoStart);
tStripper.setEndPage(pageNoEnd);
String pdfFileInText = tStripper.getText(document);
// split by whitespace
String Documentlines[] = pdfFileInText.split("\\r?\\n");
for (String line : Documentlines) {
String lineArr[] = line.split("\\s+");
if (lineArr.length == noOfColumnsInTable) {
for (String linedata : lineArr) {
System.out.print(linedata + " ");
}
System.out.println("");
objArrayList.add(lineArr);
}
}
}
} catch (Exception e) {
System.out.println("Exception " +e);
}
return objArrayList;
}
}
For anyone wanting to do the same thing as OP (as I do), after days of research Amazon Textract is the best option (if your volume is low free tier might be enough).
consider using PDFTableStripper.class
The class is available on git :
https://gist.github.com/beldaz/8ed6e7473bd228fcee8d4a3e4525be11#file-pdftablestripper-java-L1
I'm not familiar with PDFBox, but you could try looking at itext. Even though the homepage says PDF generation, you can also do PDF manipulation and extraction. Have a look and see if it fits your use case.
For reading content of the table from pdf file,you have to do only just convert the pdf file into a text file by using any API(I have use PdfTextExtracter.getTextFromPage() of iText) and then read that txt file by your java program..now after reading it the major task is done.. you have to filter the data of your need. you can do it by continuously using split method of String class until you find record of your intrest.. here is my code by which I have extract part of record by an PDF file and write it into a .CSV file.. Url of PDF file is..http://www.cea.nic.in/reports/monthly/generation_rep/actual/jan13/opm_02.pdf
Code:-
public static void genrateCsvMonth_Region(String pdfpath, String csvpath) {
try {
String line = null;
// Appending Header in CSV file...
BufferedWriter writer1 = new BufferedWriter(new FileWriter(csvpath,
true));
writer1.close();
// Checking whether file is empty or not..
BufferedReader br = new BufferedReader(new FileReader(csvpath));
if ((line = br.readLine()) == null) {
BufferedWriter writer = new BufferedWriter(new FileWriter(
csvpath, true));
writer.append("REGION,");
writer.append("YEAR,");
writer.append("MONTH,");
writer.append("THERMAL,");
writer.append("NUCLEAR,");
writer.append("HYDRO,");
writer.append("TOTAL\n");
writer.close();
}
// Reading the pdf file..
PdfReader reader = new PdfReader(pdfpath);
BufferedWriter writer = new BufferedWriter(new FileWriter(csvpath,
true));
// Extracting records from page into String..
String page = PdfTextExtractor.getTextFromPage(reader, 1);
// Extracting month and Year from String..
String period1[] = page.split("PEROID");
String period2[] = period1[0].split(":");
String month[] = period2[1].split("-");
String period3[] = month[1].split("ENERGY");
String year[] = period3[0].split("VIS");
// Extracting Northen region
String northen[] = page.split("NORTHEN REGION");
String nthermal1[] = northen[0].split("THERMAL");
String nthermal2[] = nthermal1[1].split(" ");
String nnuclear1[] = northen[0].split("NUCLEAR");
String nnuclear2[] = nnuclear1[1].split(" ");
String nhydro1[] = northen[0].split("HYDRO");
String nhydro2[] = nhydro1[1].split(" ");
String ntotal1[] = northen[0].split("TOTAL");
String ntotal2[] = ntotal1[1].split(" ");
// Appending filtered data into CSV file..
writer.append("NORTHEN" + ",");
writer.append(year[0] + ",");
writer.append(month[0] + ",");
writer.append(nthermal2[4] + ",");
writer.append(nnuclear2[4] + ",");
writer.append(nhydro2[4] + ",");
writer.append(ntotal2[4] + "\n");
// Extracting Western region
String western[] = page.split("WESTERN");
String wthermal1[] = western[1].split("THERMAL");
String wthermal2[] = wthermal1[1].split(" ");
String wnuclear1[] = western[1].split("NUCLEAR");
String wnuclear2[] = wnuclear1[1].split(" ");
String whydro1[] = western[1].split("HYDRO");
String whydro2[] = whydro1[1].split(" ");
String wtotal1[] = western[1].split("TOTAL");
String wtotal2[] = wtotal1[1].split(" ");
// Appending filtered data into CSV file..
writer.append("WESTERN" + ",");
writer.append(year[0] + ",");
writer.append(month[0] + ",");
writer.append(wthermal2[4] + ",");
writer.append(wnuclear2[4] + ",");
writer.append(whydro2[4] + ",");
writer.append(wtotal2[4] + "\n");
// Extracting Southern Region
String southern[] = page.split("SOUTHERN");
String sthermal1[] = southern[1].split("THERMAL");
String sthermal2[] = sthermal1[1].split(" ");
String snuclear1[] = southern[1].split("NUCLEAR");
String snuclear2[] = snuclear1[1].split(" ");
String shydro1[] = southern[1].split("HYDRO");
String shydro2[] = shydro1[1].split(" ");
String stotal1[] = southern[1].split("TOTAL");
String stotal2[] = stotal1[1].split(" ");
// Appending filtered data into CSV file..
writer.append("SOUTHERN" + ",");
writer.append(year[0] + ",");
writer.append(month[0] + ",");
writer.append(sthermal2[4] + ",");
writer.append(snuclear2[4] + ",");
writer.append(shydro2[4] + ",");
writer.append(stotal2[4] + "\n");
// Extracting eastern region
String eastern[] = page.split("EASTERN");
String ethermal1[] = eastern[1].split("THERMAL");
String ethermal2[] = ethermal1[1].split(" ");
String ehydro1[] = eastern[1].split("HYDRO");
String ehydro2[] = ehydro1[1].split(" ");
String etotal1[] = eastern[1].split("TOTAL");
String etotal2[] = etotal1[1].split(" ");
// Appending filtered data into CSV file..
writer.append("EASTERN" + ",");
writer.append(year[0] + ",");
writer.append(month[0] + ",");
writer.append(ethermal2[4] + ",");
writer.append(" " + ",");
writer.append(ehydro2[4] + ",");
writer.append(etotal2[4] + "\n");
// Extracting northernEastern region
String neestern[] = page.split("NORTH");
String nethermal1[] = neestern[2].split("THERMAL");
String nethermal2[] = nethermal1[1].split(" ");
String nehydro1[] = neestern[2].split("HYDRO");
String nehydro2[] = nehydro1[1].split(" ");
String netotal1[] = neestern[2].split("TOTAL");
String netotal2[] = netotal1[1].split(" ");
writer.append("NORTH EASTERN" + ",");
writer.append(year[0] + ",");
writer.append(month[0] + ",");
writer.append(nethermal2[4] + ",");
writer.append(" " + ",");
writer.append(nehydro2[4] + ",");
writer.append(netotal2[4] + "\n");
writer.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
Please, suggest me some libraries that will help me print PDF files that contain JBIG2 encoded images. PDFRenderer, PDFBox don't help me. These libs can print simple PDF, but not PDF containing JBIG2 images. PDFRenderer tries to fix it (according to bug issue on PDFRedndrer's bug tracker), but some pages still (especially where barcodes exist) don't want to print.
P.S. I use javax.print API within applet
Thanks!
UPDATE: also tried ICEPdf, is too don't want to work.
I came to the conclusion that all these libraries(PDFRenderer, ICEPdf, PDFBox) use JPedals jbig2 decoder. Bug (some pages didn't print) come from this decoder library. The open source version of this decoder (which is used in PDFRenderer, ICEPdf, PDFBox) is no longer supported, but JPedal has a new commercial branch of the project, and they wrote that the bug has been fixed in new commercial release, which costs $9k.
Any ideas?
UPDATE 2: yesterday I tried to replace JPedal's free library with other open-source jbig2-imageio libraries. But yet I don't get any successful results, so I created a new topic on their project's page (google-code's forum - here ). Would be grateful for any help.
I also found some helpfull discussions on Apache PDFBox bug-tracker: here and here.
As going through your comment in yms answer ie. " but what library I can use to extract images and (more importantly) put them back in PDF?"
Here is a simple demonstration of
1 ) Extracting jbig2 or you can say all images from pdf.
2 ) Converting jbig2 image to any other format, in my case its jpeg.
3 ) Creating new pdf containing the jpeg.
Using libraries jbig2-imageio and itext.
In the below example please change the resources and the directories path as per your need.
For this I had to go through several resources that I will attach in the end. Hope this helps.
import com.itextpdf.text.Document;
import com.itextpdf.text.Image;
import com.itextpdf.text.pdf.PdfPCell;
import com.itextpdf.text.pdf.PdfPTable;
import com.itextpdf.text.pdf.PdfReader;
import com.itextpdf.text.pdf.PdfWriter;
import com.itextpdf.text.pdf.parser.*;
import com.levigo.jbig2.JBIG2ImageReader;
import com.levigo.jbig2.JBIG2ImageReaderSpi;
import com.levigo.jbig2.JBIG2ReadParam;
import com.levigo.jbig2.io.DefaultInputStreamFactory;
import java.awt.image.BufferedImage;
import java.io.*;
import javax.imageio.ImageIO;
import javax.imageio.stream.ImageInputStream;
public class JBig2Image {
private String filepath;
private int imageIndex;
public JBig2Image() {
this.filepath = "/home/blackadmin/Desktop/pdf/demo18.jbig2";
this.imageIndex = 0;
extractImgFromPdf();
convertJBig2ToJpeg();
createPDF();
}
private void extractImgFromPdf() {
try {
/////////// Extract all Images from pdf /////////////////////////
PdfReader reader = new PdfReader("/home/blackadmin/Desktop/pdf/orig.pdf");
PdfReaderContentParser parser = new PdfReaderContentParser(reader);
MyImageRenderListener listener = new MyImageRenderListener("/home/blackadmin/Desktop/pdf/demo%s.%s");
for (int i = 1; i <= reader.getNumberOfPages(); i++) {
parser.processContent(i, listener);
}
} catch (IOException ex) {
System.out.println(ex);
}
}
private void convertJBig2ToJpeg() {
InputStream inputStream = null;
try {
///////// Read jbig2 image ////////////////////////////////////////
inputStream = new FileInputStream(new File(filepath));
DefaultInputStreamFactory disf = new DefaultInputStreamFactory();
ImageInputStream imageInputStream = disf.getInputStream(inputStream);
JBIG2ImageReader imageReader = new JBIG2ImageReader(new JBIG2ImageReaderSpi());
imageReader.setInput(imageInputStream);
JBIG2ReadParam param = imageReader.getDefaultReadParam();
BufferedImage bufferedImage = imageReader.read(imageIndex, param);
////////// jbig2 to jpeg ///////////////////////////////////////////
ImageIO.write(bufferedImage, "jpeg", new File("/home/blackadmin/Desktop/pdf/demo18.jpeg"));
} catch (IOException ex) {
System.out.println(ex);
} finally {
try {
inputStream.close();
} catch (IOException ex) {
System.out.println(ex);
}
}
}
public void createPDF() {
Document document = new Document();
try {
PdfWriter.getInstance(document,
new FileOutputStream("/home/blackadmin/Desktop/pdf/output.pdf"));
document.open();
PdfPTable table = new PdfPTable(1); //1 column.
Image image = Image.getInstance("/home/blackadmin/Desktop/pdf/demo18.jpeg");
image.scaleToFit(800f, 600f);
image.scaleAbsolute(800f, 600f); // Give the size of image you want to print on pdf
PdfPCell nestedImgCell = new PdfPCell(image);
table.addCell(nestedImgCell);
document.add(table);
document.close();
System.out.println(
"======== PDF Created Successfully =========");
} catch (Exception e) {
System.out.println(e);
}
}
public static void main(String[] args) throws IOException {
new JBig2Image();
}
}
class MyImageRenderListener implements RenderListener {
/**
* The new document to which we've added a border rectangle.
*/
protected String path = "";
/**
* Creates a RenderListener that will look for images.
*/
public MyImageRenderListener(String path) {
this.path = path;
}
/**
* #see com.itextpdf.text.pdf.parser.RenderListener#beginTextBlock()
*/
public void beginTextBlock() {
}
/**
* #see com.itextpdf.text.pdf.parser.RenderListener#endTextBlock()
*/
public void endTextBlock() {
}
/**
* #see com.itextpdf.text.pdf.parser.RenderListener#renderImage(
* com.itextpdf.text.pdf.parser.ImageRenderInfo)
*/
public void renderImage(ImageRenderInfo renderInfo) {
try {
String filename;
FileOutputStream os;
PdfImageObject image = renderInfo.getImage();
if (image == null) {
return;
}
filename = String.format(path, renderInfo.getRef().getNumber(), image.getFileType());
os = new FileOutputStream(filename);
os.write(image.getImageAsBytes());
os.flush();
os.close();
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
/**
* #see com.itextpdf.text.pdf.parser.RenderListener#renderText(
* com.itextpdf.text.pdf.parser.TextRenderInfo)
*/
public void renderText(TextRenderInfo renderInfo) {
}
}
References :
1 ) Extracting jbig2 from pdf (extract images) (MyImageRenderListener).
2 ) Converting jbig2 (JBIG2ImageReaderDemo)
There is a fork of the JPedal library by Borisvl located at
https://github.com/Borisvl/JBIG2-Image-Decoder#readme
which contains speed improvements and I believe it should also fix your bug.
EDIT : The bug is related to simple range checking. Basically you need to prevent GetPixel from accessing x,y values outside of the bitmap extents.
You need to make sure the following conditions are met before calling getPixel
col >= 0 and col < bitmap.width
row >= 0 and row < bitmap.height
Here is some Delphi code with a couple of small range checks. I cannot test the Java code myself but you need to make changes to src/org/jpedal/jbig2/image/JBIG2Bitmap.java
procedure TJBIG2Bitmap.combine(bitmap: TJBIG2Bitmap; x, y: Integer; combOp: Int64);
...
...
var
begin
srcWidth := bitmap.width;
srcHeight := bitmap.height;
srcRow := 0;
srcCol := 0;
if (x < 0) then x := 0;
if (y < 0) then y := 0;
for row := y to Min(y + srcHeight - 1, Self.height - 1) do // <<<<<<<< HERE
begin
for col := x to x + srcWidth - 1 do
begin
srcPixel := bitmap.getPixel(srcCol, srcRow);
Andrew.
How about using AcrobatReader itself? It's a bit muddy getting it to work, and not a robust solution I guess. But will probably print all of it perfectly. And be free
Some info about this route;
http://vineetreynolds.blogspot.nl/2005/12/silent-print-pdf-print-pdf.html
http://www.codeproject.com/Questions/98586/Programmatically-print-PDF-documents
http://forums.adobe.com/message/2336723
You have tools as ImageMagick which handle images and convert them to a lot of formats. I used it some years ago so I can't tell you if the jbig2 format is properly handled by default or if you have to install some plugin.
You can try the following to have a list of supported formats beginning with J like the JBIG2 you are searching for:
$ convert -list format | grep -i J
It is really obvious to convert to pdf with with tool too, coupled with gs tool aka GhostScript.
If fact nothing prevent you to display a PNG/JPEG version of the image and provide a download link to the original JBIG2 file with its own metadatas.
As an alternative, you could try doing this server-side:
Approach 1:
Convert the PDF files to raster images using an external application and print that instead.
Approach 2:
Adjust your PDF files by recompressing JBIG2 images:
1- Extracting the images compressed as JBIG2 from your files.
2- Re-compress them with some other algorithm (jpeg, png, etc). In order to do this you might need to go outside of Java using either JNI or calling an external application. You can try with jbig2dec or ImageMagic for example if the GPL lincense suits your needs.
3- Put the recompressed images back in your PDF.
This approach will imply some quality loss on those images, but at least you will be able to print the files.
You can do this in Java with iText, there is a chapter about resizing images in the book iText in Action (with sample code). The idea there is to extract the image, resize it (including recompression) and put it back. You can use this as starting point. Be aware that iText is an AGPL project, hence you cannot use it for free in commercial closed-source applications.
If you are using a Windows-based server and you can afford a commercial tool, you can also achieve this with Amyuni PDF Creator either with C#/VB.Net or C++ (Usual disclaimer applies for this suggestion). You just need to go though all objects of type acObjectTypePicture and set the attribute Compression to acJPegHigh, this approach does not require any external JBIG2 decoder, (I can include some sample code here if you are interested).
If you are using an applet just to print your PDF files, you could also try generating a PDF file that shows the print dialog when opened
My application needs to use a .properties file for configuration.
In the properties files, users are allow to specify paths.
Problem
Properties files need values to be escaped, eg
dir = c:\\mydir
Needed
I need some way to accept a properties file where the values are not escaped, so that the users can specify:
dir = c:\mydir
Why not simply extend the properties class to incorporate stripping of double forward slashes. A good feature of this will be that through the rest of your program you can still use the original Properties class.
public class PropertiesEx extends Properties {
public void load(FileInputStream fis) throws IOException {
Scanner in = new Scanner(fis);
ByteArrayOutputStream out = new ByteArrayOutputStream();
while(in.hasNext()) {
out.write(in.nextLine().replace("\\","\\\\").getBytes());
out.write("\n".getBytes());
}
InputStream is = new ByteArrayInputStream(out.toByteArray());
super.load(is);
}
}
Using the new class is a simple as:
PropertiesEx p = new PropertiesEx();
p.load(new FileInputStream("C:\\temp\\demo.properties"));
p.list(System.out);
The stripping code could also be improved upon but the general principle is there.
Two options:
use the XML properties format instead
Writer your own parser for a modified .properties format without escapes
You can "preprocess" the file before loading the properties, for example:
public InputStream preprocessPropertiesFile(String myFile) throws IOException{
Scanner in = new Scanner(new FileReader(myFile));
ByteArrayOutputStream out = new ByteArrayOutputStream();
while(in.hasNext())
out.write(in.nextLine().replace("\\","\\\\").getBytes());
return new ByteArrayInputStream(out.toByteArray());
}
And your code could look this way
Properties properties = new Properties();
properties.load(preprocessPropertiesFile("path/myfile.properties"));
Doing this, your .properties file would look like you need, but you will have the properties values ready to use.
*I know there should be better ways to manipulate files, but I hope this helps.
The right way would be to provide your users with a property file editor (or a plugin for their favorite text editor) which allows them entering the text as pure text, and would save the file in the property file format.
If you don't want this, you are effectively defining a new format for the same (or a subset of the) content model as the property files have.
Go the whole way and actually specify your format, and then think about a way to either
transform the format to the canonical one, and then use this for loading the files, or
parse this format and populate a Properties object from it.
Both of these approaches will only work directly if you actually can control your property object's creation, otherwise you will have to store the transformed format with your application.
So, let's see how we can define this. The content model of normal property files is simple:
A map of string keys to string values, both allowing arbitrary Java strings.
The escaping which you want to avoid serves just to allow arbitrary Java strings, and not just a subset of these.
An often sufficient subset would be:
A map of string keys (not containing any whitespace, : or =) to string values (not containing any leading or trailing white space or line breaks).
In your example dir = c:\mydir, the key would be dir and the value c:\mydir.
If we want our keys and values to contain any Unicode character (other than the forbidden ones mentioned), we should use UTF-8 (or UTF-16) as the storage encoding - since we have no way to escape characters outside of the storage encoding. Otherwise, US-ASCII or ISO-8859-1 (as normal property files) or any other encoding supported by Java would be enough, but make sure to include this in your specification of the content model (and make sure to read it this way).
Since we restricted our content model so that all "dangerous" characters are out of the way, we can now define the file format simply as this:
<simplepropertyfile> ::= (<line> <line break> )*
<line> ::= <comment> | <empty> | <key-value>
<comment> ::= <space>* "#" < any text excluding line breaks >
<key-value> ::= <space>* <key> <space>* "=" <space>* <value> <space>*
<empty> ::= <space>*
<key> ::= < any text excluding ':', '=' and whitespace >
<value> ::= < any text starting and ending not with whitespace,
not including line breaks >
<space> ::= < any whitespace, but not a line break >
<line break> ::= < one of "\n", "\r", and "\r\n" >
Every \ occurring in either key or value now is a real backslash, not anything which escapes something else.
Thus, for transforming it into the original format, we simply need to double it, like Grekz proposed, for example in a filtering reader:
public DoubleBackslashFilter extends FilterReader {
private boolean bufferedBackslash = false;
public DoubleBackslashFilter(Reader org) {
super(org);
}
public int read() {
if(bufferedBackslash) {
bufferedBackslash = false;
return '\\';
}
int c = super.read();
if(c == '\\')
bufferedBackslash = true;
return c;
}
public int read(char[] buf, int off, int len) {
int read = 0;
if(bufferedBackslash) {
buf[off] = '\\';
read++;
off++;
len --;
bufferedBackslash = false;
}
if(len > 1) {
int step = super.read(buf, off, len/2);
for(int i = 0; i < step; i++) {
if(buf[off+i] == '\\') {
// shift everything from here one one char to the right.
System.arraycopy(buf, i, buf, i+1, step - i);
// adjust parameters
step++; i++;
}
}
read += step;
}
return read;
}
}
Then we would pass this Reader to our Properties object (or save the contents to a new file).
Instead, we could simply parse this format ourselves.
public Properties parse(Reader in) {
BufferedReader r = new BufferedReader(in);
Properties prop = new Properties();
Pattern keyValPattern = Pattern.compile("\s*=\s*");
String line;
while((line = r.readLine()) != null) {
line = line.trim(); // remove leading and trailing space
if(line.equals("") || line.startsWith("#")) {
continue; // ignore empty and comment lines
}
String[] kv = line.split(keyValPattern, 2);
// the pattern also grabs space around the separator.
if(kv.length < 2) {
// no key-value separator. TODO: Throw exception or simply ignore this line?
continue;
}
prop.setProperty(kv[0], kv[1]);
}
r.close();
return prop;
}
Again, using Properties.store() after this, we can export it in the original format.
Based on #Ian Harrigan, here is a complete solution to get Netbeans properties file (and other escaping properties file) right from and to ascii text-files :
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.io.Reader;
import java.io.Writer;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Properties;
/**
* This class allows to handle Netbeans properties file.
* It is based on the work of : http://stackoverflow.com/questions/6233532/reading-java-properties-file-without-escaping-values.
* It overrides both load methods in order to load a netbeans property file, taking into account the \ that
* were escaped by java properties original load methods.
* #author stephane
*/
public class NetbeansProperties extends Properties {
#Override
public synchronized void load(Reader reader) throws IOException {
BufferedReader bfr = new BufferedReader( reader );
ByteArrayOutputStream out = new ByteArrayOutputStream();
String readLine = null;
while( (readLine = bfr.readLine()) != null ) {
out.write(readLine.replace("\\","\\\\").getBytes());
out.write("\n".getBytes());
}//while
InputStream is = new ByteArrayInputStream(out.toByteArray());
super.load(is);
}//met
#Override
public void load(InputStream is) throws IOException {
load( new InputStreamReader( is ) );
}//met
#Override
public void store(Writer writer, String comments) throws IOException {
PrintWriter out = new PrintWriter( writer );
if( comments != null ) {
out.print( '#' );
out.println( comments );
}//if
List<String> listOrderedKey = new ArrayList<String>();
listOrderedKey.addAll( this.stringPropertyNames() );
Collections.sort(listOrderedKey );
for( String key : listOrderedKey ) {
String newValue = this.getProperty(key);
out.println( key+"="+newValue );
}//for
}//met
#Override
public void store(OutputStream out, String comments) throws IOException {
store( new OutputStreamWriter(out), comments );
}//met
}//class
You could try using guava's Splitter: split on '=' and build a map from resulting Iterable.
The disadvantage of this solution is that it does not support comments.
#pdeva: one more solution
//Reads entire file in a String
//available in java1.5
Scanner scan = new Scanner(new File("C:/workspace/Test/src/myfile.properties"));
scan.useDelimiter("\\Z");
String content = scan.next();
//Use apache StringEscapeUtils.escapeJava() method to escape java characters
ByteArrayInputStream bi=new ByteArrayInputStream(StringEscapeUtils.escapeJava(content).getBytes());
//load properties file
Properties properties = new Properties();
properties.load(bi);
It's not an exact answer to your question, but a different solution that may be appropriate to your needs. In Java, you can use / as a path separator and it'll work on both Windows, Linux, and OSX. This is specially useful for relative paths.
In your example, you could use:
dir = c:/mydir