We Keep Coding

pdfbox embedding subset font for annotations

I am trying to use Apache PDFBOX v2.0.21 to modify existing PDF documents, adding signatures and annotations. That means that I am actively using incremental save mode. I am also embedding LiberationSans font to accommodate some Unicode characters. It makes sense for me to use the subsetting feature of PDF embedded fonts as embedding LiberationSans in full makes the PDF file around 200+ KB more in side.
After multiple trials and errors I finally managed to have something working - all but the font subsetting. The way I do this is to initialize the PDFont object once using
try (InputStream fs = PDFService.class.getResourceAsStream("/static/fonts/LiberationSans-Regular.ttf")) {
_font = PDType0Font.load(pddoc, fs, true);
}
And then to use custom Appearance Stream to show the text.
private void addAnnotation(String name, PDDocument doc, PDPage page, float x, float y, String text) throws IOException {
List<PDAnnotation> annotations = page.getAnnotations();
PDAnnotationRubberStamp t = new PDAnnotationRubberStamp();
t.setAnnotationName(name); // might play important role
t.setPrinted(true); // always visible
t.setReadOnly(true); // does not interact with user
t.setContents(text);
PDRectangle rect = ....;
t.setRectangle(rect);
PDAppearanceDictionary ap = new PDAppearanceDictionary();
ap.setNormalAppearance(createAppearanceStream(doc, t));
ap.getCOSObject().setNeedToBeUpdated(true);
t.setAppearance(ap);
annotations.add(t);
page.setAnnotations(annotations);
t.getCOSObject().setNeedToBeUpdated(true);
page.getResources().getCOSObject().setNeedToBeUpdated(true);
page.getCOSObject().setNeedToBeUpdated(true);
doc.getDocumentCatalog().getPages().getCOSObject().setNeedToBeUpdated(true);
doc.getDocumentCatalog().getCOSObject().setNeedToBeUpdated(true);
}
private PDAppearanceStream createAppearanceStream(final PDDocument document, PDAnnotation ann) throws IOException
{
PDAppearanceStream aps = new PDAppearanceStream(document);
PDRectangle rect = ann.getRectangle();
rect = new PDRectangle(0, 0, rect.getWidth(), rect.getHeight());
aps.setBBox(rect); // set bounding box to the dimensions of the annotation itself
// embed our unicode font (NB: yes, this needs to be done otherwise aps.getResources() == null which will cause NPE later during setFont)
PDResources res = new PDResources();
_fontName = res.add(_font).getName();
aps.setResources(res);
PDAppearanceContentStream apsContent = null;
try {
// draw directly on the XObject's content stream
apsContent = new PDAppearanceContentStream(aps);
apsContent.beginText();
apsContent.setFont(_font, _fontSize);
apsContent.showText(ann.getContents());
apsContent.endText();
}
finally {
if (apsContent != null) {
try { apsContent.close(); } catch (Exception ex) { log.error(ex.getMessage(), ex); }
}
}
aps.getResources().getCOSObject().setNeedToBeUpdated(true);
aps.getCOSObject().setNeedToBeUpdated(true);
return aps;
}
This code runs, but creates a PDF with dots instead of actual characters, which, I guess, means that the font subset has not been embedded. Moreover, I get the following warnings:
2021-04-17 12:33:31.326 WARN 20820 --- [ main]
o.a.p.pdmodel.PDAbstractContentStream : attempting to use subset
font LiberationSans without proper context
After looking through the source code, I get and I guess that I am messing something up when creating the appearance stream - somehow it's not connected with the PDDocument and the subsetting does not continue normally. Note that the above code works well when the font is embedded fully (i.e. if I call PDType0Font.load with the last parameter set to false)
Can anyone think of some hint to give to me? Thank you!

I don't know - am I lucky? It is very often that luckiness in programming points to something completely wrong or misleading. In any case, if someone can still give a hint, my ears are more than open...
Again, after looking through the code, I saw the following in PDDocument.save():
// subset designated fonts
for (PDFont font : fontsToSubset)
{
font.subset();
}
This is not happening in PDDocument.saveIncremental() which I am using. Just to mess around with the code, I went and did the following just before calling saveIncremental() on my document:
_font.subset(); // you can see in the beginning of the question how _font is created
_font.getCOSObject().setNeedToBeUpdated(true);
pddoc.saveIncremental(baos);
Believe it or not, but the document was saved correctly - at least it appears correct in Acrobat Reader DC and Chrome & Firefox PDF viewers. Note that Unicode codepoints are added to the subset for the font during showText() on appearance content stream.
UPDATE 18/04/2021: as I mentioned in the comments, I got reports from users that started seeing messages like "Cannot extract the embedded font XXXXXX+LiberationSans-Regular from ...", when they opened the modified PDF files. Strangely enough, I didn't see these messages during my tests. It turns out that my copy of Acrobat Reader DC was newer than theirs, and specifically with the continuous release version 2021.001.20149 no errors were shown, while with the continuous release version 2020.012.20043 the above message was shown.
After investigations, it turns out that the problem was with the way I was embedding the font. I am not aware if any other way exists, and I am not that familiar with the PDF specification to know otherwise. What I was doing, as you can see from the above code, was to load the font ONCE for the document, and then to use it freely in the resource dictionary of the appearance stream of EVERY annotation. This had as a result all the resource dictionaries of the annotation content streams to reference an F1 font that was defined with the SAME /BaseFont name. The PDF Reference, 3rd ed. on p.323 specifically states that:
"... the PostScript name of the font - ... - begins with a tag
followed by a plus sign (+). The tag consists of exactly six uppercase
letters; the choice of letters is arbitrary, but different subsets in
the same PDF file must have different tags..."
Once I started to call PDType0Font.load for each of my annotations and calling subset() (and of course setNeedToBeUpdated) after creating appearance stream for each of them, I saw that the BaseName attributes started to look indeed differently - and indeed, the older 2020 version of Acrobat Reader DC stopped complaining.
[edit 07/10/2021: even trying to use a single PDFont object per page (having multiple annotations with this font), and subsetting it once, after having called showText on appearances of all annotations, appears to not work - it appears that the subsetting uses the letters I passed to the first showText, and not the others, resulting in wrong rendering of the 2nd, 3rd etc. annotations that might have characters that didn't exist in the 1st annotation - so I reiterate that what worked was to use loadFont for each separate annotation and then (after modifying appearance with showText, which will mark the letters to be used during subsetting) to call subset() on each of these fonts (which will result in the change of the font name)]
Note that other than using iText RUPS for inspecting the PDF contents, one could use Foxit PDF viewer to at least ensure that the subset font names are different. Acrobat Reader DC and PDF-xChange in Properties -> Fonts just show the initial font name, like LiberationSans, without showing the 6-letter unique prefix.
UPDATE 19/04/2021 I am still working on this issue - because I still get reports about the infamous "Cannot extract the embedded font" message. It is quite possible that the original cause of that message was not (or not only) the fact that the different subsets had same BaseFont names. One thing that I am observing is that on some computers, the stamp annotations that I am using cause Acrobat Reader DC to open automatically the so called "Comments pane" - there are options to turn this automatic thing off (Preferences -> Commenting -> Show comments pane when a PDF with comments is opened). When this pane opens, either manually or automatically, the error message appears (and I was on my wits ends to see why same version of Acrobat Reader DC behaves differently for different machines). I think that Acrobat Reader tries to extract the full version of the font and fails, since it is only a subset. But, I guess, this doesn't have to do with the semantic contents of the document - the document still passes "qpdf --check". I am currently trying to find if it is possible to restrict stamps to not allow comments - i.e. some way to disable the comments pane in Acrobat Reader DC, although I have little hope.
UPDATE 20/04/2021 opened a new question here

How to reuse font from one pdf in another in iText7?

I'm trying to open PDF file in iText7, write there some new piece of text, apply font from original PDF to it and save it in another PDF document. I'm using Java 1.8
Thus, I need a set of font names used in original pdf, from where user will choose one, that will be applied to a new paragraph.
And I also need to somehow apply this font.
For now I have this piece of code, that I've taken from here:
public static void main(String[] args) throws IOException {
PdfDocument pdf = new PdfDocument(new PdfReader("example.pdf"));
Set<PdfName> fonts = listAllUsedFonts(pdf);
fonts.stream().forEach(System.out::println);
}
public static Set<PdfName> listAllUsedFonts(PdfDocument pdfDoc) throws IOException {
PdfDictionary acroForm = pdfDoc.getCatalog().getPdfObject().getAsDictionary(PdfName.AcroForm);
if (acroForm == null) {
return null;
}
PdfDictionary dr = acroForm.getAsDictionary(PdfName.DR);
if (dr == null) {
return null;
}
PdfDictionary font = dr.getAsDictionary(PdfName.Font);
if (font == null) {
return null;
}
return font.keySet();
}
It returns this output:
/Helv
/ZaDb
However, the only font example.pdf has is Verdana (it is what document properties in Adobe Acrobat Pro says). Moreover, there are Verdana in two implementations: Bold and normal.
So, I have these questions:
Why does this function returns two fonts instead of one (Verdana).
How can I generate normal well-read names of fonts to display them
to user (e.g. Helvetica instead of Helv)?
How can I apply font got from the original document to the
new paragraph?
Thank you in advance!

If you only wish to display the names of the fonts being used (which you are legally allowed to do) you can use the following code:
public void go() throws IOException {
final Set<String> usedFontNames = new HashSet<>();
IEventListener fontNameExtractionStrategy = new IEventListener() {
#Override
public void eventOccurred(IEventData iEventData, EventType eventType) {
if(iEventData instanceof TextRenderInfo)
{
TextRenderInfo tri = (TextRenderInfo) iEventData;
String fontName = tri.getFont().getFontProgram().getFontNames().getFontName();
usedFontNames.add(fontName);
}
}
#Override
public Set<EventType> getSupportedEvents() {
return null;
}
};
PdfCanvasProcessor parser = new PdfCanvasProcessor(fontNameExtractionStrategy);
File inputFile = new File("YOUR_INPUT_FILE_HERE.pdf");
PdfDocument pdfDocument = new PdfDocument(new PdfReader(inputFile));
for(int i=1;i<=pdfDocument.getNumberOfPages();i++)
{
parser.processPageContent(pdfDocument.getPage(i));
}
pdfDocument.close();
for(String fontName : usedFontNames)
{
System.out.println(fontName);
}
}

You should not reuse a font from one PDF in another PDF, and here's why: fonts are hardly ever fully embedded in a PDF document. For instance: you use the font Verdana regular (238 KB) and the font Verdana bold (207 KB), but when you create a simple PDF document saying "Hello World" in regular and bold, the file size will be much smaller than 238 + 207 KB. Why is this? Because the PDF will only consist of a subset of the font Verdana regular and a subset of the font Verdana bold.
You may have noticed that I am talking of the font Verdana regular
and the font Verdana bold. Those are two different fonts from
the same font family. Reading your question, I notice that you don't make that distinction. You talk about the font Verdana with
two implementations bold and normal. This is incorrect. You should
talk about the font family Verdana and two fonts Verdana bold and
Verdana regular.
A PDF usually contains subsets of different fonts. It can even contain two different subsets of the same font. See also What are the extra characters in the font name of my PDF?
Your goal is to take the font of one PDF and to use that font of another PDF. However, suppose that your original PDF only contains the subset that is required to write "Hello World" and that you want to create a new PDF saying "Hello Universe." That will never work, because the subset won't contain the glyphs to render the letter U, n, i, v, r, and s.
Also take into account that fonts are usually licensed. Many fonts
have a license that states that you can use to font to create a
document and embed that font in that document. However, there is
often a clause that says that other people are not allowed to
extract to font to use it in a different context. For instance: you paid for the font when you purchased a copy of MS Windows, but someone
who receives a PDF containing that font may not have a license to use
that font. See Does one need to have a license for fonts if we are using ttf files in itext?
Given the technical and legal issues related to your question, I don't think it makes sense to work on a code sample. Your design is flawed. You should work with a licensed font program instead of trying to extract a font from an existing PDF. This answers question 3: How can I apply font got from the original document to the new paragraph? You can't: it is forbidden by law (see Extra info below) and it might be technically impossible if the subset doesn't contain all the characters you need!
Furthermore, the sample you found on the official iText web site looks for the fonts defined in a form. /Helv and ZaDb refer to Helvetica and Zapfdingbats. Those are two fonts of a set of 14 known as the Standard Type 1 fonts. These fonts are never embedded in the document since every viewer is supposed to know how to render them. You don't need a full font program if you want to use these fonts; the font metrics are sufficient. For instance: iText ships with 14 AFM files (AFM = Adobe Font Metrics) that contain the font metrics.
You wonder why you don't find Verdana, since Verdana is used as font for the text in your document, but you are looking at the wrong place. You are asking iText for the fonts used for the form, not for the fonts used in the text. This answer question 1: Why does this function returns two fonts instead of one (Verdana).
As for your question 2: you are looking at the internal name of the font, and that internal name can be anything (even /F1, /F2,...). The postscript name of the font is stored in the font dictionary. That's the name you need.
Extra info:
I checked the Verdana license:
Microsoft supplied font. You may use this font to create, display, and print content as permitted by the license terms or terms of use, of the Microsoft product, service, or content in which this font was included. You may only (i) embed this font in content as permitted by the embedding restrictions included in this font; and (ii) temporarily download this font to a printer or other output device to help print content. Any other use is prohibited.
The use you want to make of the font is prohibited. If you have a license for Verdana, you can embed the font in a PDF. However, it is not permitted to extract that font and use it for another purpose. You need to use the original font program.

PDFBox 2.0.3 Set cropBox using TextPosition coordinates

I've located a region of interest in the page by tracking TextPosition objects using PDFTextStripper as shown in the example: https://github.com/apache/pdfbox/blob/trunk/examples/src/main/java/org/apache/pdfbox/examples/util/PrintTextLocations.java
As shown, the TextPosition has been retrieved from fields like
text.getXDirAdj(), text.getWidthDirAdj(), text.getYDirAdj(), text.getHeightDir() .
From this example I tried to keep everything else the same except setting the cropBox of the target page.
https://github.com/apache/pdfbox/blob/2.0.3/tools/src/main/java/org/apache/pdfbox/tools/PDFToImage.java
OLD CROPBOX: [0.0,0.0,595.276,841.89] -> NEW CROPBOX [50.0,42.0,592.0,642.0].
So how can I use the getYDirAdj and getXDirAdj in setting the cropbox correctly ?
The original pdf file I'm processing can be downloaded from here: http://downloadcenter.samsung.com/content/UM/201504/20150407095631744/ENG-US_NMATSCJ-1.103-0330.pdf

Cropping the page
In a comment the OP reduced his problem to
Ok. Given a java PDRectangle rect = new PDRectangle(40f, 680f, 510f, 100f) obtained from TextLocation how would a java code snippet, that sets the cropBox of a single page look like ? Or how would you do it? TextLocation based rect --> some transformation --> setCropBox(theRightBox).
To set the crop box of the page twelve of the given document to the given PDRectangle you can use code like this:
PDDocument pdDocument = PDDocument.load(resource);
PDPage page = pdDocument.getPage(12-1);
page.setCropBox(new PDRectangle(40f, 680f, 510f, 100f));
pdDocument.save(new File(RESULT_FOLDER, "ENG-US_NMATSCJ-1.103-0330-page12cropped.pdf"));
(SetCropBox.java test method testSetCropBoxENG_US_NMATSCJ_1_103_0330)
Adobe Reader now shows merely this part of page twelve:
Beware, though, the page in question does not only specify a media box (mandatory) and a crop box, it also defines a bleed box and an art box. Thus, application which consider those boxes more interesting than the crop box, might display the page differently. In particular the art box (being defined as "the extent of the page’s meaningful content") might by some applications be considered important.
Rendering the cropped page
In a comment to this answer the OP remarked
This is good and works. It correctly saves the page in the PDF file. I've tried to do the same in JPG and failed.
I reduced the OP's code to the essentials
PDDocument pdDocument = PDDocument.load(resource);
PDPage page = pdDocument.getPage(12-1);
page.setCropBox(new PDRectangle(40f, 680f, 510f, 100f));
PDFRenderer renderer = new PDFRenderer(pdDocument);
BufferedImage img = renderer.renderImage(12 - 1, 4f);
ImageIOUtil.writeImage(img, new File(RESULT_FOLDER, "ENG-US_NMATSCJ-1.103-0330-page12cropped.jpg").getAbsolutePath(), 300);
pdDocument.close();
(SetCropBox.java test method testSetCropBoxImgENG_US_NMATSCJ_1_103_0330)
The result:
Thus, I cannot reproduce an issue here.
Possible details to check for:
ImageIOUtil is not part of the main PDFBox artifact, instead it is located in pdfbox-tools; does the version of that artifact match the version of the core pdfbox artifact?
I run the code in an Oracle Java 8 environment; other Java environments might give rise to different results.
There are minor differences in our implementations. E.g. I load the PDF via an InputStream, you directly from file system, I have hardcoded the page number, you have it in some variable, ... None of these differences should cause your problem, but who knows...

using iText api how to stamp image to existing different size pdf in java [duplicate]

I'm using C# and iTextSharp to add a watermark to my PDF files:
Document document = new Document();
PdfReader pdfReader = new PdfReader(strFileLocation);
PdfStamper pdfStamper = new PdfStamper(pdfReader, new FileStream(strFileLocationOut, FileMode.Create, FileAccess.Write, FileShare.None));
iTextSharp.text.Image img = iTextSharp.text.Image.GetInstance(WatermarkLocation);
img.SetAbsolutePosition(100, 300);
PdfContentByte waterMark;
//
for (int pageIndex = 1; pageIndex <= pdfReader.NumberOfPages; pageIndex++)
{
waterMark = pdfStamper.GetOverContent(pageIndex);
waterMark.AddImage(img);
}
//
pdfStamper.FormFlattening = true;
pdfStamper.Close();
It works fine, but my problem is that in some PDF files no watermark is added although the file size increased, any idea?

The fact that the file size increases is a good indication that the watermark is added. The main problem is that you're adding the watermark outside the visible area of the page. See How to position text relative to page using iText?
You need something like this:
Rectangle pagesize = reader.GetCropBox(pageIndex);
if (pagesize == null)
pagesize = reader.GetMediaBox(pageIndex);
img.SetAbsolutePosition(
pagesize.GetLeft(),
pagesize.GetBottom());
That is: if you want to add the image in the lower-left corner of the page. You can add an offset, but make sure the offset in the x direction doesn't exceed the width of the page, and the offset in the y direction doesn't exceed the height of the page.

Although I don't know the specifics of iTextSharp, likely on the pages where your image is not showing, the previous PDF content has modified the current transformation matrix such that whatever you put on the page is moved off the page.
This can be fixed by emitting a gsave operator before the original page content and emitting a grestore operator after the original page content (but before yours). This, however may not fix all cases with a PDF document that modifies the CTM does a gsave and no grestore. This is not supposed to happen in theory, according to the PDF specification:
Occurrences of the q and Q operators shall be balanced within a given content stream (or within the sequence of streams specified in a page dictionary’s Contents array).
but I can tell you from experience that this is not the case in practice.

Pink/Reddish tint while resizing jpeg images using java thumbnailator or imgscalr

I am trying to convert an image (url below) using two libraries (thumbnailator and imgscalr. My code works on most of the images except a few which after conversion have a pink/reddish tint.
I am trying to understand the cause and would welcome any recommendation.
Note - Image type of this image is 5 i.e BufferedImage.TYPE_3BYTE_BGR and i am using Java 7
Using Thumbnailator
Thumbnails.of(fromDir.listFiles())
.size(thumbnailWidth, thumbnailHeight)
.toFiles(Rename.SUFFIX_HYPHEN_THUMBNAIL);
Using imgscalr
BufferedImage bufferedImage = ImageIO.read(file);
final BufferedImage jpgImage;
LOG.debug("image type is =[{}] ", bufferedImage.getType());
BufferedImage scaledImg = Scalr.resize(bufferedImage, Method.ULTRA_QUALITY, thumbnailWidth, thumbnailHeight, Scalr.OP_ANTIALIAS);
File thumbnailFile = new File(fromDirPath + "/" + getFileName(file.getName()) +THUMBNAIL_KEYWORD + ".png");
ImageIO.write(scaledImg, getFileExtension(file.getName()), thumbnailFile);
bufferedImage.flush();
scaledImg.flush();

I get this question a lot (author of imgscalr) -- the problem is almost always that you are reading/writing out different file formats and the ALPHA channel is causing one of your color channels (R/G/B) to be culled from the resulting file.
For example, if you read in a file that was ARGB (4 channel) and wrote it out as a JPG (3 channel) - unless you purposefully manipulate the image types yourself and render the old image to the new one directly, you will get a file with a "ARG" channels... or more specifically, just Red and Green - no Blue.
PNG supports an alpha channel and JPG does not, so be aware of that.
The way to fix this is to purposefully create appropriate BufferedImage's of the right type (RGB, ARGB, etc.) and using the destImage.getGraphics() call to render one image to the other before writing it out to disk and re-encoding it.
Sun and Oracle have NEVER made the ImageIO libraries smart enough to detect the unsupported channels when writing to differing file types, so this behavior happens all the time :(
Hope that helps!

The following piece of code resolved my issue:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
Thumbnails.of(new ByteArrayInputStream(imageByteArray))
.outputFormat("jpg")
.size(200, 200)
.toOutputStream(outputStream);
return baos.toByteArray();
I am using Thumbnailator and the code was posted here: https://github.com/coobird/thumbnailator/issues/23