We Keep Coding

PDFBox in Java trying to Edit Pdf with existing acroForms but values are hidden untill i press on them

I am using PDFBox to get a document that was already generated from a Nestjs using PDF-lib js via the command form.createTextField(field.id); after that i send it to java so i can but a signature box ontop of it and fill the forms now the forms are filled and everything works with pdf viewer js
i can see the fields and the values but when i try to open the pdf file in google chrome i dont see the values at all or when i try to open that in Adobe reader i dont see the values untill i click on the field
here is my java code
public void prepareForSigning(DigestAlgorithm digestAlgorithm,
SignatureType signatureType,
UserData userData, List<FieldInput> formFields) throws IOException, NoSuchAlgorithmException {
this.digestAlgorithm = digestAlgorithm;
id = Utils.generateDocumentId();
pdDocument = PDDocument.load(contentIn);
int accessPermissions = getDocumentPermissions();
if (accessPermissions == 1) {
throw new AisClientException("Cannot sign document [" + name + "]. Document contains a certification " +
"that does not allow any changes.");
}
// add fields
// get the document catalog
try {
PDAcroForm acroForm = pdDocument.getDocumentCatalog().getAcroForm();
acroForm.setSignaturesExist(true);
acroForm.setAppendOnly(true);
acroForm.getCOSObject().setDirect(true);
acroForm.getCOSObject().setNeedToBeUpdated(true);
// acroForm.setNeedAppearances(true);
COSObject pdfFields = acroForm.getCOSObject().getCOSObject(COSName.FIELDS);
if (pdfFields != null) {
pdfFields.setNeedToBeUpdated(true);
}
for (int i = 0; i < formFields.size(); i++) {
PDField field = acroForm.getField(formFields.get(i).id);
if (field != null) {
// will also set a checkbox if the value is Yes
// checking for formFields.get(i).value == "true" returns
if (field.getFieldType() == "Btn" && formFields.get(i).value.equals("true")) {
field.setValue("Yes");
} else {
field.setValue(formFields.get(i).value);
}
field.setReadOnly(true);
field.getCOSObject().setNeedToBeUpdated(true);
field.getWidgets().get(0).getAppearance().getCOSObject().setNeedToBeUpdated(true);
Log.info("set field: " + field.getFullyQualifiedName() + " to " + formFields.get(i).value);
}
}
pdDocument.getDocumentCatalog().getCOSObject().setNeedToBeUpdated(true);
} catch (Exception e) {
Log.warn(e);
}
PDSignature pdSignature = new PDSignature();
Calendar signDate = Calendar.getInstance();
if (signatureType == SignatureType.TIMESTAMP) {
// Now, according to ETSI TS 102 778-4, annex A.2, the type of a Dictionary that
// holds document timestamp should be DocTimeStamp
// However, adding this (as of Feb/17/2021), it trips the ETSI Conformance
// Checked online tool, making it say
// "There is no signature dictionary in the document". So, for now (Feb/17/2021)
// this has been removed. This makes the
// ETSI Conformance Checker happy.
// pdSignature.setType(COSName.DOC_TIME_STAMP);
pdSignature.setFilter(PDSignature.FILTER_ADOBE_PPKLITE);
pdSignature.setSubFilter(COSName.getPDFName("ETSI.RFC3161"));
} else {
pdSignature.setFilter(PDSignature.FILTER_ADOBE_PPKLITE);
pdSignature.setSubFilter(PDSignature.SUBFILTER_ETSI_CADES_DETACHED);
// Add 3 Minutes to move signing time within the OnDemand Certificate Validity
// This is only relevant in case the signature does not include a timestamp
// See section 5.8.5.1 of the Reference Guide
signDate.add(Calendar.MINUTE, 3);
}
pdSignature.setSignDate(signDate);
pdSignature.setName(userData.getSignatureName());
pdSignature.setReason(userData.getSignatureReason());
pdSignature.setLocation(userData.getSignatureLocation());
pdSignature.setContactInfo(userData.getSignatureContactInfo());
SignatureOptions options = new SignatureOptions();
options.setPreferredSignatureSize(signatureType.getEstimatedSignatureSizeInBytes());
// create a visible signature at the specified coordinates
if (signatureDefinition != null) {
Rectangle2D humanRect = new Rectangle2D.Float(signatureDefinition.getX(),
signatureDefinition.getY(),
signatureDefinition.getWidth(),
signatureDefinition.getHeight());
PDRectangle rect = createSignatureRectangle(pdDocument, humanRect);
options.setVisualSignature(
createVisualSignatureTemplate(pdDocument, signatureDefinition.getPage(),
signatureDefinition.getImage(), rect, pdSignature));
options.setPage(signatureDefinition.getPage());
}
pdDocument.addSignature(pdSignature, options);
// Set this signature's access permissions level to 0, to ensure we just sign
// the PDF, not certify it
// for more details:
// https://wwwimages2.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf
// see section 12.7.4.5
setPermissionsForSignatureOnly();
pbSigningSupport = pdDocument.saveIncrementalForExternalSigning(inMemoryStream);
MessageDigest digest = MessageDigest.getInstance(digestAlgorithm.getDigestAlgorithm());
byte[] contentToSign = IOUtils.toByteArray(pbSigningSupport.getContent());
byte[] hashToSign = digest.digest(contentToSign);
options.close();
base64HashToSign = Base64.getEncoder().encodeToString(hashToSign);
}
now the field with value 5 is appearing because i already clicked on it which is on focus() mode
adobe reader
when i use acoForm.setNeedAppearances to true i can then see the values but then the signature field is not there am i missing something in code ?
i am expecting to see the values in google chrome or Adobe Reader appearing without me pressing on them
Picture of the pdf fields without values with one field being focused on
PDF SAMLE FILE

How do I ADD bullet points to a word document using Apache POI in Java

I have a word document which is used as a template. Inside this template I have some tables that contain predefined bullet points. Now I'm trying to replace the placeholder string with a set of strings.
I'm totally stuck on this. My simplified methods looks like this.
replaceKeyValue.put("[DescriptionOfItem]", new HashSet<>(Collections.singletonList("This is the description")));
replaceKeyValue.put("[AllowedEntities]", new HashSet<>(Arrays.asList("a", "b")));
replaceKeyValue.put("[OptionalEntities]", new HashSet<>(Arrays.asList("c", "d")));
replaceKeyValue.put("[NotAllowedEntities]", new HashSet<>(Arrays.asList("e", "f")));
try (XWPFDocument template = new XWPFDocument(OPCPackage.open(file))) {
template.getTables().forEach(
xwpfTable -> xwpfTable.getRows().forEach(
xwpfTableRow -> xwpfTableRow.getTableCells().forEach(
xwpfTableCell -> replaceInCell(replaceKeyValue, xwpfTableCell)
)
));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
template.write(baos);
return new ByteArrayResource(baos.toByteArray());
} finally {
if (file.exists()) {
file.delete();
}
}
private void replaceInCell(Map<String, Set<String>> replacementsKeyValuePairs, XWPFTableCell xwpfTableCell) {
for (XWPFParagraph xwpfParagraph : xwpfTableCell.getParagraphs()) {
for (Map.Entry<String, Set<String>> replPair : replacementsKeyValuePairs.entrySet()) {
String keyToFind = replPair.getKey();
Set<String> replacementStrings = replacementsKeyValuePairs.get(keyToFind);
if (xwpfParagraph.getText().contains(keyToFind)) {
replacementStrings.forEach(replacementString -> {
XWPFParagraph paragraph = xwpfTableCell.addParagraph();
XWPFRun run = paragraph.createRun();
run.setText(replacementString);
});
}
}
}
I was expecting that some more bullet points will be added to the current cell. Am I missing something? The paragraph is the one containing the placeholder string and format.
Thanks for any help!
UPDATE: This is how part of the template looks like. I would like to automatically search for the terms and replace them. Searching works so far. But trying to replace the bullet points ends in an unlocatable NullPointer.
Would it be easier to use fields? I need to keep the bullet point style though.
UPDATE 2: added download link and updated the code. Seems I can't alter the paragraphs if I'm iterating through them. I get a null-pointer.
Download link: WordTemplate

Since Microsoft Word is very, very "strange" in how it divides text in different runs in it's storage, such questions are not possible to answer without having a complete example including all code and the Word documents in question. Having a general usable code for adding content to Word documents seems not be possible, except all the adding or replacement is only in fields (form fields or content controls or mail merge fields).
So I downloaded your WordTemplate.docx which looks like so:
Then I runned the following code:
import java.io.*;
import org.apache.poi.xwpf.usermodel.*;
import org.openxmlformats.schemas.wordprocessingml.x2006.main.CTR;
import org.apache.xmlbeans.XmlCursor;
import java.util.*;
import java.math.BigInteger;
public class WordReadAndRewrite {
static void addItems(XWPFTableCell cell, XWPFParagraph paragraph, Set<String> items) {
XmlCursor cursor = null;
XWPFRun run = null;
CTR cTR = null; // for a deep copy of the run's low level object
BigInteger numID = paragraph.getNumID();
int indentationLeft = paragraph.getIndentationLeft();
int indentationHanging = paragraph.getIndentationHanging();
boolean first = true;
for (String item : items) {
if (first) {
for (int r = paragraph.getRuns().size()-1; r > 0; r--) {
paragraph.removeRun(r);
}
run = (paragraph.getRuns().size() > 0)?paragraph.getRuns().get(0):null;
if (run == null) run = paragraph.createRun();
run.setText(item, 0);
cTR = (CTR)run.getCTR().copy(); // take a deep copy of the run's low level object
first = false;
} else {
cursor = paragraph.getCTP().newCursor();
boolean thereWasParagraphAfter = cursor.toNextSibling(); // move cursor to next paragraph
// because the new paragraph shall be **after** that paragraph
// thereWasParagraphAfter is true if there is a next paragraph, else false
if (thereWasParagraphAfter) {
paragraph = cell.insertNewParagraph(cursor); // insert new paragraph if there are next paragraphs in cell
} else {
paragraph = cell.addParagraph(); // add new paragraph if there are no other paragraphs present in cell
}
paragraph.setNumID(numID); // set template paragraph's numbering Id
paragraph.setIndentationLeft(indentationLeft); // set template paragraph's indenting from left
if (indentationHanging != -1) paragraph.setIndentationHanging(indentationHanging); // set template paragraph's hanging indenting
run = paragraph.createRun();
if (cTR != null) run.getCTR().set(cTR); // set template paragraph's run formatting
run.setText(item, 0);
}
}
}
public static void main(String[] args) throws Exception {
Map<String, Set<String>> replaceKeyValue = new HashMap<String, Set<String>>();
replaceKeyValue.put("[AllowedEntities]", new HashSet<>(Arrays.asList("allowed 1", "allowed 2", "allowed 3")));
replaceKeyValue.put("[OptionalEntities]", new HashSet<>(Arrays.asList("optional 1", "optional 2", "optional 3")));
replaceKeyValue.put("[NotAllowedEntities]", new HashSet<>(Arrays.asList("not allowed 1", "not allowed 2", "not allowed 3")));
XWPFDocument document = new XWPFDocument(new FileInputStream("WordTemplate.docx"));
List<XWPFTable> tables = document.getTables();
for (XWPFTable table : tables) {
List<XWPFTableRow> rows = table.getRows();
for (XWPFTableRow row : rows) {
List<XWPFTableCell> cells = row.getTableCells();
for (XWPFTableCell cell : cells) {
int countParagraphs = cell.getParagraphs().size();
for (int p = 0; p < countParagraphs; p++) { // do not for each since new paragraphs were added
XWPFParagraph paragraph = cell.getParagraphArray(p);
String placeholder = paragraph.getText();
placeholder = placeholder.trim(); // this is the tricky part to get really the correct placeholder
Set<String> items = replaceKeyValue.get(placeholder);
if (items != null) {
addItems(cell, paragraph, items);
}
}
}
}
}
FileOutputStream out = new FileOutputStream("Result.docx");
document.write(out);
out.close();
document.close();
}
}
The Result.docx looks like so:
The code loops trough the table cells in the Word document and looks for a paragraph which contains exactly the placeholder. This even might be the tricky part since that placeholder might be splitted into differnt text runs by Word. If found it runs a method addItems which takes the found paragraph as a template for numbering and indention (might be incomplter though). Then it sets the first new item in first text run of found paragraph and removes all other text runs which possibly are there. Then it determines wheter new paragraphs must be inserted or added to the cell. For this a XmlCursor is used. In new inserted or added paragrahs the other items are placed and the numbering and indention settings are taken from the placeholder's paragraph.
As said, this is code for showing the principles of how to do. It would must be extended very much to be general usable. In my opinion those trials using text placeholders in Word documents for text replacements are not really good. Placeholders for variable text in Word documents should be fields. This could be form fields, content controls or mail merge fields. Advantage of fields in contrast of text placeholders is that Word knows the fields being entities for variable texts. It will not split them into multiple text runs for multiple strange reasons as it often does with normal text.

Why pdf contain one field only is around 500Kb

Here you can download pdf with one acroform field and his size is exactly 427Kb
If I remove this unique field, file is 3Kb only, why this happens please ?
I tried analyse using PDF Debugger and nothing seems weird to me.

There's an embedded "Arial" font in the acroform default resources, see Root/AcroForm/DR/Font/Arial/FontDescriptor/FontFile2.
Either you or whoever created the pdf added it for no reason. The font is not used / referenced. For the acroform default resources you could check the /DA entry (default appearance) of each field whether it contains the font name.
When you removed the field somehow you also removed the font from the acroForm default resources. (You didn't write how you removed it)
Here's some code to do it (null checks mostly missing):
PDAcroForm acroForm = doc.getDocumentCatalog().getAcroForm();
PDResources defaultResources = acroForm.getDefaultResources();
COSDictionary fontDict = (COSDictionary) defaultResources.getCOSObject().getDictionaryObject(COSName.FONT);
List<String> defaultAppearances = new ArrayList<>();
List<COSName> fontDeletionList = new ArrayList<>();
for (PDField field : acroForm.getFieldTree())
{
if (field instanceof PDVariableText)
{
PDVariableText vtField = (PDVariableText) field;
defaultAppearances.add(vtField.getDefaultAppearance());
}
}
for (COSName fontName : defaultResources.getFontNames())
{
if (COSName.HELV.equals(fontName) || COSName.ZA_DB.equals(fontName))
{
// Adobe default, always keep
continue;
}
boolean found = false;
for (String da : defaultAppearances)
{
if (da != null && da.contains("/" + fontName.getName()))
{
found = true;
break;
}
}
System.out.println(fontName + ": " + found);
if (!found)
{
fontDeletionList.add(fontName);
}
}
System.out.println("deletion list: " + fontDeletionList);
for (COSName fontName : fontDeletionList)
{
fontDict.removeItem(fontName);
}
The resulting file has 5KB size now.
I haven't checked the annotations. Some of them have also a /DA string but it is unclear if the acroform default resources fonts are to be used when reconstructing a missing appearance stream.
Update:
Here's some additional code to replace Arial with Helv:
for (PDField field : acroForm.getFieldTree())
{
if (field instanceof PDVariableText)
{
PDVariableText vtField = (PDVariableText) field;
String defaultAppearance = vtField.getDefaultAppearance();
if (defaultAppearance.startsWith("/Arial"))
{
vtField.setDefaultAppearance("/Helv " + defaultAppearance.substring(7));
vtField.getWidgets().get(0).setAppearance(null); // this removes the font usage
vtField.setValue(vtField.getValueAsString());
}
defaultAppearances.add(vtField.getDefaultAppearance());
}
}
Note that this may not be a good idea, because the standard 14 fonts have only limited characters. Try
vtField.setValue("Ayşe");
and you'll get an exception.
More general code to replace font can be found in this answer.

Itext 7 Split Paragraph

How can I split a given paragraph to 2 paragraphs, due to that it fits only partial into canvas. After split, I would like to add the first part into canvas and the second to a new canvas.
public Paragraph addParagraphToPage(PdfDocument pdfDocument, int pageNum, Rectangle rectangle, Paragraph p)
{
PdfPage page = pdfDocument.getPage(pageNum);
PdfCanvas pdfCanvas = new PdfCanvas(page.newContentStreamAfter(), page.getResources(), pdfDocument);
Canvas canvas = new Canvas(pdfCanvas, pdfDocument, rectangle);
ParagraphRenderer currentRenderer = (ParagraphRenderer) p.createRendererSubTree();
currentRenderer.setParent(canvas.getRenderer());
result = currentRenderer.layout(new LayoutContext(new LayoutArea(pageNum, rectangle)));
ArrayList<Paragraph> paragraphs = new ArrayList<Paragraph>();
if (result.getStatus() != LayoutResult.FULL)
{
paragraphs = ????? // getNextParagraph(paragraphs, result, pageNum, rectangle, canvas);
if(paragraphs.size() == 2)
{
canvas.add( paragraphs.get(0));
return paragraphs.get(1);
}
}
return null;
}

Your approach is correct in general and layout in iText7 is flexible enough to allow you to do required thing in an easy manner. The only thing I see that is not very clear is that Paragraph is actually an element that cannot split itself and no classes in layout framework facilitate element splitting. You could do it manually, but there is no need to. Instead you should work with IRenderer, and ParagraphRenderer in particular, directly.
IRenderer can split itself as a result of layout operation and represents the necessary portion of data only compared to the Paragraph which contains full data.
You can add an IRenderer to the CanvasRenderer:
canvas.getRenderer().addChild(rendererToAdd.setParent(canvas.getRenderer()));
And you can access the partial renderers (the portion that fit the passed area and overflow part) from LayoutResult#getSplitRenderer() and LayoutResult#getOverflowRenderer().
In general, your code can be adapted like follows:
public ParagraphRenderer addParagraphToPage(PdfDocument pdfDocument, int pageNum, Rectangle rectangle, ParagraphRenderer renderer) {
PdfPage page = pdfDocument.getPage(pageNum);
PdfCanvas pdfCanvas = new PdfCanvas(page.newContentStreamAfter(), page.getResources(), pdfDocument);
Canvas canvas = new Canvas(pdfCanvas, pdfDocument, rectangle);
renderer.setParent(canvas.getRenderer());
LayoutResult result = renderer.layout(new LayoutContext(new LayoutArea(pageNum, rectangle)));
IRenderer rendererToAdd = result.getStatus() == LayoutResult.FULL ? renderer : result.getSplitRenderer();
canvas.getRenderer().addChild(rendererToAdd.setParent(canvas.getRenderer()));
return result.getStatus() != LayoutResult.FULL ? (ParagraphRenderer) result.getOverflowRenderer() : null;
}
And then for adding paragraph to sequential pages until all the content is placed you basically need only two lines of code:
ParagraphRenderer renderer = (ParagraphRenderer) p.createRendererSubTree();
while ((renderer = addParagraphToPage(pdfDocument, pageNum++, rectangle, renderer)) != null);

Discover titles/paragraphs in word docs

I'm trying to discover paragraphs/titles in word documents.
I use Apache POI to do this.
An example that I use is:
fs = new POIFSFileSystem(new FileInputStream(filesname));
HWPFDocument doc = new HWPFDocument(fs);
WordExtractor we = new WordExtractor(doc);
ArrayList titles = new ArrayList();
try {
for (int i = 0; i < we.getText().length() - 1; i++) {
int startIndex = i;
int endIndex = i + 1;
Range range = new Range(startIndex, endIndex, doc);
CharacterRun cr = range.getCharacterRun(0);
if (cr.isBold() || cr.isItalic() || cr.getUnderlineCode() != 0) {
while (cr.isBold() || cr.isItalic() || cr.getUnderlineCode() != 0) {
i++;
endIndex += 1;
range = new Range(endIndex, endIndex + 1, doc);
cr = range.getCharacterRun(0);
}
range = new Range(startIndex, endIndex - 1, doc);
titles.add(range.text());
}
}
}
catch (IndexOutOfBoundsException iobe) {
//sometimes this happens have to find out why.
}`enter code here`
This works for all bold, italic or underlined text.
But what I want is to discover the font that is used most often. And then to discover variations compared to that font style. Anyone an Idea?

Well, some thoughts would be to try some of the following:
cr.getFontSize() could be used at the beginning of a paragraph to see if the range changes font size. That in conjunction with bold, italic or underlined would be a good identifier.
cr.getFontName() could also be used to determine when and where the font changes in a given range.
cr.getColor() would be another possibility to help identify if the user is using different colors for a font.
I guess I would iterate over the range and create multiple CharacterRun items each time the text characteristics change. Then evaluate each item based on position in the paragraph as well as all of the afore-mentioned characteristics (size, color, name, bold, italics, etc.). Perhaps create some sort of weighting scale based on the most common values.
It might also be of value to create a Title object and store the values for each set of characteristics to help optimize searches in later character runs in the same document.

You might want to take a look at the buildParagraphTagAndStyle method in Tika's WordExtractor:
https://svn.apache.org/repos/asf/tika/trunk/tika-parsers/src/main/java/org/apache/tika/parser/microsoft/WordExtractor.java
For HWPF (.doc), to call it you'd do:
StyleDescription style =
document.getStyleSheet().getStyleDescription(p.getStyleIndex());
TagAndStyle tas = buildParagraphTagAndStyle(
style.getName(), (parentTableLevel>0)
);
For XWPF (.docx) you'd do:
XWPFStyle style = styles.getStyle(paragraph.getStyleID());
TagAndStyle tas = WordExtractor.buildParagraphTagAndStyle(
style.getName(), paragraph.getPartType() == BodyType.TABLECELL
);

It will be easier if you process the data by converting it into paragraphs.
WordExtractor we = new WordExtractor(doc);
String[] para = we.getParagraphText();
Then work paragraph wise. If your code already couldn't figure out the titles, then you can check for bold and underlines in each paragraph.
The paragraphs function as follows:
for(int i=0;i<para.length;i++)
{
System.out.println("Length of paragraph "+(i+1)+": "+ para[i].length());
System.out.println(para[i].toString());
}
A working example can be found here:
http://sanjaal.com/java/120/java-file/how-to-read-doc-file-using-java-and-apache-poi/#comments