5

I am trying to learn to use Apache's pdfBox to deal with digitaly signed documents for work. During testing, I created a completely empty pdf document.

I then signed the document through Adobe reader using the sign with certificate function.

I tried to open, save and close the signed file with pdfBox without any modifications. However once I open the file in Adobe the files are no longer valid.

Adobe tells me: "There are errors in the formatting or information contained in this signature (support information: SigDict/Contents illegal data)"

Since I have not modified the content of the file, intuitively there should not have been any problems and the signature should be still valid, however this is not the case and I don't know what the solutions are (googling yielded no results).

How I create the document:

@Test
public void createEmptyPDF() throws IOException {
    String path = "path to file";
    PDDocument document = new PDDocument();
    PDPage page = new PDPage();
    document.addPage(page);
    document.save(path);
    document.close();
}

I then sign it with adobe and pass it through this:

 @Test
public void copySignedDocument() throws IOException {
    String path = "path to file";
    File file = new File(path);
    PDDocument document = PDDocument.load(file);
    document.save(file);
    document.close();

    //just opening and saving the file invalidates the signatures
}

I am truly at a loss as to why this does not work. Any help would be great!

EDIT:

So I did some digging around and it seems that updating an existing signed document (either adding annotations or filling forms) is not yet implemented in PDFBox 2.0.1 and is scheduled to come in versions 2.1 (however no release date has been specified). More information here and here.

However it seems possible to add annotations on signed documents with IText without invalidating the signature using PDFStamper, from this question

EDIT 2: Code to add a stamp to a document and save it incrementally: 

 @Test
public void stampSignedDocument() throws IOException {
    File file = new File("path to file");
    PDDocument document = PDDocument.load(file);
    File image = new File("path to image to be added to annotation");
    PDPage page = document.getPage(0);
    List<PDAnnotation> annotations = page.getAnnotations();
    PDImageXObject ximage = PDImageXObject.createFromFileByContent(image, document);

    //stamp
    PDAnnotationRubberStamp stamp = new PDAnnotationRubberStamp();
    stamp.setName("testing rubber stamp");
    stamp.setContents("this is a test");
    stamp.setLocked(true);
    stamp.setReadOnly(true);
    stamp.setPrinted(true);

    PDRectangle rectangle = createRectangle(100, 100, 100, 100, 100, 100);
    PDFormXObject form = new PDFormXObject(document);
    form.setResources(new PDResources());
    form.setBBox(rectangle);
    form.setFormType(1);

    form.getResources().add(ximage);
    PDAppearanceStream appearanceStream = new PDAppearanceStream(form.getCOSObject());
    PDAppearanceDictionary appearance = new PDAppearanceDictionary(new COSDictionary());
    appearance.setNormalAppearance(appearanceStream);
    stamp.setAppearance(appearance);
    stamp.setRectangle(rectangle);
    PDPageContentStream stream = new PDPageContentStream(document, appearanceStream);
    Matrix matrix = new Matrix(100, 0, 0, 100, 100, 100);
    stream.drawImage(ximage, matrix);
    stream.close();
    //close and save   
    annotations.add(stamp);
    page.getCOSObject().setNeedToBeUpdated(true);
    OutputStream os = new FileOutputStream(file);
    document.saveIncremental(os);
    document.close();
    os.close();
}

The above code doesn't invalidate my signature but doesn't save the annotation that I have added.

As suggested I've set the NeedToBeUpdated flag to true for the added annotation, page and annotations list (I hope I did the last one correctly):

    stamp.getCOSObject().setNeedToBeUpdated(true);
    COSArrayList<PDAnnotation> list = (COSArrayList<PDAnnotation>) annotations;
    COSArrayList.converterToCOSArray(list).setNeedToBeUpdated(true);
    page.getCOSObject().setNeedToBeUpdated(true);
    document.getDocumentCatalog().getCOSObject().setNeedToBeUpdated(true);

The annotation is still not saved so I'm obviously missing something.

EDIT 3:

This is my current method to add an annotation:

    @Test
public void stampSignedDocument() throws IOException {
    File file = new File(
            "E:/projects/eSign/g2digitalsignature/G2DigitalSignatureParent/G2DigitalSignatureTest/src/test/resources/pdfBoxTest/empty.pdf");
    PDDocument document = PDDocument.load(file);
    File image = new File(
            "E:/projects/eSign/g2digitalsignature/G2DigitalSignatureParent/G2DigitalSignatureTest/src/test/resources/pdfBoxTest/digitalSign.png");
    PDPage page = document.getPage(0);
    List<PDAnnotation> annotations = page.getAnnotations();
    PDImageXObject ximage = PDImageXObject.createFromFileByContent(image, document);

    //stamp
    PDAnnotationRubberStamp stamp = new PDAnnotationRubberStamp();
    stamp.setName("testing rubber stamp");
    stamp.setContents("this is a test");
    stamp.setLocked(true);
    stamp.setReadOnly(true);
    stamp.setPrinted(true);

    PDRectangle rectangle = createRectangle(100, 100, 100, 100, 100, 100);
    PDFormXObject form = new PDFormXObject(document);
    form.setResources(new PDResources());
    form.setBBox(rectangle);
    form.setFormType(1);

    form.getResources().getCOSObject().setNeedToBeUpdated(true);
    form.getResources().add(ximage);
    PDAppearanceStream appearanceStream = new PDAppearanceStream(form.getCOSObject());
    PDAppearanceDictionary appearance = new PDAppearanceDictionary(new COSDictionary());
    appearance.setNormalAppearance(appearanceStream);
    stamp.setAppearance(appearance);
    stamp.setRectangle(rectangle);
    PDPageContentStream stream = new PDPageContentStream(document, appearanceStream);
    Matrix matrix = new Matrix(100, 0, 0, 100, 100, 100);
    stream.drawImage(ximage, matrix);
    stream.close();
    //close and save   
    annotations.add(stamp);

    appearanceStream.getCOSObject().setNeedToBeUpdated(true);
    appearance.getCOSObject().setNeedToBeUpdated(true);
    rectangle.getCOSArray().setNeedToBeUpdated(true);
    stamp.getCOSObject().setNeedToBeUpdated(true);
    form.getCOSObject().setNeedToBeUpdated(true);
    COSArrayList<PDAnnotation> list = (COSArrayList<PDAnnotation>) annotations;
    COSArrayList.converterToCOSArray(list).setNeedToBeUpdated(true);
    document.getPages().getCOSObject().setNeedToBeUpdated(true);
    page.getCOSObject().setNeedToBeUpdated(true);
    document.getDocumentCatalog().getCOSObject().setNeedToBeUpdated(true);

    OutputStream os = new FileOutputStream(file);
    document.saveIncremental(os);
    document.close();
    os.close();

}

When I add an annotation using it on a non signed document, the annotation gets added and is visible. However when using it on a signed document, the annotation does not appear.

I have opened the pdf file in notepad++ and have found that the annotation seems to have been added since I found this as well as the rest of the code pertaining to the annotation:

<<
/Type /Annot
/Subtype /Stamp
/Name /testing#20rubber#20stamp
/Contents (this is a test)
/F 196
/AP 29 0 R
/Rect [100.0 100.0 200.0 200.0]
>>

However it does not appear when I open the document in adobe reader. Perhaps this has more to do with the appearance streams than the annotation itself?

Community
  • 1
  • 1
Subbies
  • 246
  • 2
  • 8
  • I'm starting to think that saving the document results in the creation of a new document constructed by pdfBox, which would invalidate my signature. I'm currently trying with the saveIncremental function of PDDocument, which seems more in line with what I want, but when I try this with a document which has been signed multiple times, only the first signature is validated. – Subbies Jan 04 '17 at 16:20
  • Indeed, loading and saving creates a new document, everything reordered etc. so the signature is no longer valid. Don't do it. I'm not sure if saveIncremental works outside of signing, I believe I saw an issue. – Tilman Hausherr Jan 04 '17 at 16:37
  • 1
    My final goal is to be able to add annotations to a signed document programatically. I know that adding annotations to a signed document is possible, so there should be a way to do so (hopefully) with pdfbox. It's just that if it's impossible to save the changes, then adding anything is kind of moot. – Subbies Jan 04 '17 at 16:54
  • 1
    I suggest you create code that creates the annotation and then does incremental save. Link to your signed input PDF. See the AddAnnotations examples for how to add annotations. I suggest you rewrite the question with saveIncremental. – Tilman Hausherr Jan 04 '17 at 17:06
  • @Tilman Hausherr: I've tried with saveIncremental rather than save. Unfortunately saveIncremental does not update my added annotation. It seems that saveIncremental is only used to save signatures. I've then researched for issues and it seems that what I wish to do is not yet possible, but will coming for PDFBox 2.1. – Subbies Jan 05 '17 at 09:02
  • 2
    You need to mark the elements for update that exist and have been changed. So this would be the annotations COSArray and - if there was no annotation array - the page COSObject. That's why I asked that you bring some code. Use `object.getCOSObject().setNeedToBeUpdated(true);`. – Tilman Hausherr Jan 05 '17 at 09:45
  • I see you updated the question. Please add this to your code: `((COSArray) page.getCosObject().getItem(COSName.ANNOTS)).setNeedToBeUpdated(true);` this makes sure that the annots array is also updated. Additionally please share the original and the target PDF. The document catalog does not have to be updated. Btw to be sure that your annotation code is correct, test it the traditional way i.e. without incremental save on a non signed file. – Tilman Hausherr Jan 05 '17 at 17:24
  • @TilmanHausherr Sorry for the late response, I was out for the wk. Unfortunately I cannot cast from the `COSBase` to the `COSArray` as you suggested. However I have checked in the debugger, and my code does set the `needToBeUpdated` flag to true for the pages `COSArray` containing the annotations. So this doesn't seem to be the issue. – Subbies Jan 09 '17 at 09:01
  • @TilmanHausherr I generated the original pdf using the `createEmptyPDF` method in my initial question (I'm at work so I can't upload files). I had already tested the annotation code without digitally signing and using the `save` method. It worked fine, the problem appeared with signing and saveIncremental. – Subbies Jan 09 '17 at 09:01
  • Don't bother re annots. I tried your code and it worked fine, i.e. updating the page dictionary is enough. I did get a valid PDF with the signature and the annotation. One difference: `createRectangle(100, 100, 100, 100, 100, 100);` doesn't work as `createRectangle` is your code. I used `PDRectangle rectangle = new PDRectangle(100, 100, 100, 100);` instead. (I can create rectangles with 4 parameters :-) ) – Tilman Hausherr Jan 10 '17 at 14:27

2 Answers2

4

The problem is that using PDDocument.save() creates a new document and thus invalidates the signature.

Using PDDocument.saveIncremental(...) does not invalidate the signature, however it will not update any changes to the document (such as annotations or form filling), it is only used to save a signature.

Updating a signed PDF document with annotations or form filling is not yet possible with PDFBox 2.0 but should be possible once PDFBox 2.1 rolls out.

Information on the problem : here and here

Using IText's PDFStamper however solve the problem of adding annotations to a signed document without invalidating the signature as answered here.

Community
  • 1
  • 1
Subbies
  • 246
  • 2
  • 8
  • 3
    *"Updating a signed PDF document with annotations or form filling is not yet possible with PDFBox 2.0"* - more to the point: you have to mark all changed or added objects as `NeedToBeUpdate` including a path from the document catalog. You currently have to do that manually while issues currently are targeted for 2.1 which focus on automatizing that. – mkl Jan 05 '17 at 10:04
  • @mkl So I understand how to set the NeedToBeUpdated flag for the newly added objects as well as the modified ones, but I don't understand how I would go about updating the path from the document catalog. Would you care to expand on that? – Subbies Jan 05 '17 at 13:17
  • 2
    *"Would you care to expand on that?"* - As far as I remember, PDFBox (when saving incrementally) starts from the PDF catalog dictionary and looks for referenced objects with `NeedToBeUpdate`. It adds these objects to the update section and then in these objects looks for references to other objects with `NeedToBeUpdate` etc.. Thus, you need a chain of references from the catalog to the objects you added/changed. For annotation you will likely choose **Catalog** -> **Pages** [-> **Pages** ...]-> **Page** -> **Annots**. – mkl Jan 05 '17 at 15:58
  • do this: `COSDictionary dict = page.getCOSObject(); while (dict.containsKey(COSName.PARENT)) { COSBase parent = dict.getDictionaryObject(COSName.PARENT); if (parent instanceof COSDictionary) { dict = (COSDictionary) parent; dict.setNeedToBeUpdated(true); } }` however this will only work for files that haven't been signed. Doing a non-signature related update on such files doesn't work, see in PDFBOX-45 why. – Tilman Hausherr Mar 01 '17 at 17:22
  • Thanks @TilmanHausherr, unfortunately in my case the files may already be signed and I have been trying to add annotations, not signatures per se. I was able to make it work, albeit inconstantly with PDFbox, but also implemented a variant with Itext, which produces the intended behavior. – Subbies Mar 01 '17 at 21:56
1

From what I can gather from some of Adobe's documentation, there is a timestamp which is included as part of the signature. I would guess by saving the document (even without any content changes) would modify this timestamp value, hence invalidating the signature (which would have been created using the original timestamp).

This isn't an authoritative, mind you - this is just what I've been able glean from a quick glance at Adobe's documentation on the matter.

Nathan Crause
  • 874
  • 10
  • 7
  • Yeah, I think that saving the document actually creates a new one, invalidating the signatures in the process. Hopefully there is a simple way to update rather than save. – Subbies Jan 04 '17 at 16:17
  • 1
    Nathan: *"there is a timestamp which is included as part of the signature. I would guess by saving the document ... would modify this timestamp value"* - No, saving does not modify the time stamp in the signature. – mkl Jan 05 '17 at 10:08
  • 1
    Loading and saving creates a new document, everything reordered etc. so the signature is no longer valid. The signature verifies a specific byte sequence. By saving, the sequence changes (every pdf software has its own way to order things) so even if the PDF looks identical it is no longer the same so the signature is invalid. So please consider deleting your answer. – Tilman Hausherr Jan 10 '17 at 14:34