We Keep Coding

GSON / iText: Extract Text From PDF 1.7 byte[]

I'm automating tests using Rest-Assured and GSON - and need to validate the contents of a PDF file that is returned in the response of a POST request. The content of the files vary and can contain anything from just text, to text and tables, or text and tables and graphics. Every page can, and most likely will be different as far a glyph content. I am only concerned with ALL text on the pdf page - be it just plain text, or text inside of a table, or text associated with (or is inside of) an image. Since all pdf's returned by the request are different, I cannot define search areas (as far as I know). I just need to extract all text on the page.
I extract the pdf data into a byte array like so:
Gson pdfGson = new Gson();
byte[] pdfBytes =
pdfGson.fromJson(this.response.as(JsonObject.class)
.get("pdfData").getAsJsonObject().get("data").getAsJsonArray(), byte[].class);
(I've tried other extraction methods for the byte[], but this is the only way I've found that returns valid data.) This returns a very large byte[] like so:
[37, 91, 22, 45, 23, ...]
When I parse the array I run into the same issue as This Question (except my pdf is 1.7) and I attempt to implement the accepted answer, adjusted for my purposes and as explained in the documentation for iText:
byte[] decodedPdfBytes = PdfReader.decodeBytes(pdfBytes, new PdfDictionary(), FilterHandlers.getDefaultFilterHandlers());
IRandomAccessSource source = new RandomAccessSourceFactory().createSource(decodedPdfBytes);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ReaderProperties readerProperties = new ReaderProperties();
// Ineffective:
readerProperties.setPassword(user.password.getBytes());
PdfReader pdfReader = new PdfReader(source, readerProperties);
// Ineffective:
pdfReader.setUnethicalReading(true);
PdfDocument pdfDoc = new PdfDocument(pdfReader, new PdfWriter(baos));
for(int i = 1; i < pdfDoc.getNumberOfPages(); i++) {
String text = PdfTextExtractor.getTextFromPage(pdfDoc.getPage(i));
System.out.println(text);
}
This DOES decode the pdf page, and return text, but it is only the header text. No other text is returned.
For what it's worth, on the front end, when the user clicks the button to generate the pdf, it returns a blob containing the download data, so I'm relatively sure that the metadata is GSA encoded, but I'm not sure if that matters at all. I'm not able to share an example of the pdf docs due to sensitive material.
Any point in the right direction would be greatly appreciated! I've spent 3 days trying to find a solution.

For those looking for a solution - ultimately we wound up going a different route. We never found a solution to this specific issue.

Extract textual data from a PDF

I'm using a Java program to extract textual data from a PDF.
When I use this type of PDF I have no problem :
But when I use this type the extraction is not performed :
Have you any idea to resolve this problem?

Try using iText7 and following code:
File inputFile = new File("path_to_your_pdf");
PdfDocument pdfDocument = new PdfDocument(new PdfReader(inputFile));
String text = PdfTextExtractor.getTextFromPage(pdfDocument.getPage(1));
pdfDocument.close();
And let us know what the output is. And whether the output corresponds to what you'd expect.
As #mkl points out, this may simply be the the difference between extracting form-fields or not. In any case, the links to your pdfs would be much appreciated. As well as some code.
But you can of course extract both using iText.
Reading material:
http://developers.itextpdf.com/content/itext-7-examples/itext-7-form-examples
http://developers.itextpdf.com/content/itext-7-examples/itext-7-content-extraction-and-redaction

How to export data set using pdf template with itext?

In my project, some data sets are needed to be exported in PDF format.
I learned that iText is helpful, and PdfpTable can do the work, but it needs much code to deal with styles. While using PDF template can save time and code for adjusting style, but I can only set certain fields left in the template.
Can you give me some suggestions to show the data sets using commands like foreach? Thanks in advance!
Here are my code using pdfpTable, which has done the work, but the code is a little ugly:
PdfPTable pdfTable = createNewPDFTable();
for (int i = 0; i < dataSet.size(); i++) {
MetaObject metaObject = SystemMetaObject.forObject(dataSet.get(i));
for (String field : fields) {
Phrase phrase = new Phrase(String.valueOf(metaObject.getValue(field) != null ? metaObject.getValue(field) : "")
, PDFUtil.createChineseSong(DEFAULT_CELL_FONT_SIZE));
PdfPCell fieldCell = new PdfPCell(phrase);
fieldCell.setBorder(Rectangle.NO_BORDER);
fieldCell.setFixedHeight(DEFAULT_COLUMN_HEIGHT);
fieldCell.setHorizontalAlignment(Element.ALIGN_CENTER);
fieldCell.setVerticalAlignment(Element.ALIGN_MIDDLE);
pdfTable.addCell(fieldCell);
}
}
Here are some code using pdfp template,which is copied from itext examples, the work is unfinished yet because i haven't find a proper way to show the data set.
PdfReader reader = new PdfReader(src);
PdfStamper stamper = new PdfStamper(reader, new FileOutputStream(dest));
AcroFields form = stamper.getAcroFields();
form.setField("text_1", "Bruno Lowagie");
form.setFieldProperty("text_1", "setfflags", PdfFormField.FF_READ_ONLY, null);

There is an inconsistency in your question. You write: PdfpTable can do the work, but it needs much code to deal with styles. However, in your first code snippet, you don't really create your PDFs the way one would expect. Instead of producing a high volume of finished PDFs, you create use PdfPTable to create a template. I assume you then use that template to create a high volume of finished PDFs.
If you want to use a template and populate it afterwards, you shouldn't create your form using iText. Create it manually, for instance using Open Office or Libre Office. See for instance the example in chapter 6 of my book (section 6.3.5). Create the template with a tool that has a GUI, then fill out that template many times using iText.
This approach has some down-sides: all the content has to fit the fields you define. All fields have a fixed position on a fixed page.
If "applying styles through code" is a problem, you may want to follow the approach described in the ZUGFeRD book. In that book, we create HTML first: Creating HTML invoices.
Once you have the HTML, then convert the HTML to PDF, and use CSS to apply styles: Creating PDF invoices.
This is how we create a ZUGFeRDDocument:
ZugferdDocument pdfDocument = new ZugferdDocument(
new PdfWriter(fos), ZugferdConformanceLevel.ZUGFeRDComfort,
new PdfOutputIntent("Custom", "", "http://www.color.org",
"sRGB IEC61966-2.1", new FileInputStream(INTENT)));
pdfDocument.addFileAttachment(
"ZUGFeRD invoice", dom.toXML(), "ZUGFeRD-invoice.xml",
PdfName.ApplicationXml, new PdfDictionary(), PdfName.Alternative);
pdfDocument.setTagged();
HtmlConverter.convertToPdf(
new ByteArrayInputStream(html), pdfDocument, getProperties());
The getProperties() method looks like this:
public ConverterProperties getProperties() {
if (properties == null) {
properties = new ConverterProperties()
.setBaseUri("resources/zugferd/");
}
return properties;
}
You can find other examples on how to use HTML to PDF here: pdfHTML add-on (read the introduction).
Note that you are using an old version of iText. The examples I shared are using iText 7. There's a huge difference between iText 5 and iText 7.

how can I write java code with syntax highlighting using pdfbox

I am trying to create a java project that simply prints a project source code, be it java, php, c++ and others.
I can create the PDF just fine with iText, but now I need some kind of highlighting the java code I read the same way a code editor like sublime highlights. I discovered pdfbox: a library for creating/manipulating PDF files, but I can't find how to highlight code text(like sublime does) by using this library. Any help?

Copying from another SO question : highlight text using pdfbox when it's location in the pdf is known
PDDocument doc = PDDocument.load(/*path to the file*/);
PDPage page = (PDPage)doc.getDocumentCatalog.getAllPages.get(i);
List annots = page.getAnnotations;
PDAnnotationTextMarkup markup = new PDAnnotationTextMarkup(PDAnnotationTextMarkup.Su....);
markup.setRectangle(/*your PDRectangle*/);
markup.setQuads(/*float array of size eight with all the vertices of the PDRectangle in anticlockwise order*/);
annots.add(markup);
doc.save(/*path to the output file*/);

RTF Line count in Java

Kindly let me know any API to calculate the line count for RTF document.
Apache POI or Aspose works for document, but its not able to find line count for RTF.
Thanks.

Java already has a built-in RTF-Parser: RTFEditorKit.
Take a look at its read method.
For example:
test.rtf file contents
hello
stackoverflow
users
So, it has 3 lines separated by \n.
Code:
FileInputStream stream = new FileInputStream("test.rtf");
RTFEditorKit kit = new RTFEditorKit();
Document doc = kit.createDefaultDocument();
kit.read(stream, doc, 0);
String plainText = doc.getText(0, doc.getLength());
System.out.println(plainText.split("\\n").length);
Output = 3

You can use Aspose.Words for Java to get the number of lines of an RTF document. Please do the following:
Read RTF file using document class
Get BuiltInDocumentProperties object using getBuiltInDocumentProperties method
Now, get number of lines using getLines property of BuiltInDocumentProperties object
I hope this helps. Please note that I work as developer evangelist at Aspose. If you need any help with Aspose, do let me know.