PDF Document Management Software, Services & Support

PDF Forms: More than Fields Alone

Monday, July 12, 2010

I was discussing a PDF forms development project the other day. I had one of those moments when the clouds part and you see straight through to the other side.

I guess for some, that might be heaven. For me, at the moment, it was the realization that annotations can also be a form. Indeed it's ludicrous to think otherwise – from the end-user point of view.

A little background

Annotations (also known variously as comments, notes or markup) are a basic feature of better PDF viewing tools.  A lot of users buy Adobe Acrobat and competing products precisely because they offer the ability to markup PDF files.

Annotation tools include notes, text boxes, arrows, redaction zones and more.  (As it so happens, regular PDF forms are also "annotations", technically speaking. More on that below.)

Problem: Dynamic Designer/XFA forms - the newfangled thing with the expandable fields - are simply incompatible with PDF annotations.

Who decides what a 'form' is?

Who ever said that annotations can't be just as much of the form's data as the form fields themselves?

I was looking at a form used by physical therapists in a major hospital. It included diagrams of the human body. The hospital's IT consultant said that the therapists wanted to markup the PDF using Acrobat's commenting tools. His problem? He couldn't figure out how to put his comments “...into a form”.

Adobe couldn't help him, he said, and Acrobat kept telling him to “print his PDF”.

This guy was confused, I thought. Clearly, PDFs can include form fields, and clearly, PDFs can be marked-up. Both are major features of Adobe Acrobat.

“It's right there in the “Submit Form” action,” I said. “Just have the form send both form-field AND annotation information to the server, and you're all set!”

"Huh?"

I asked how he was making his PDF form. “Designer” he said. And then I knew. The hair-pulling was about to start again in yet another boardroom.

Is it really PDF? That is the question

This consultant had all the skills necessary (it's not rocket-science). He knew about Acrobat forms (AcroForms), and how a PDF can submit form fields to a server. He knew more about Acrobat Designer and XFA forms. He liked XFA; it was the technology the hospital had chosen, and he was happy to be developing with it. Until now.

The problem was that Acrobat Designer didn't even acknowledge that the sort of forms he was tackling even existed.

Perhaps more to the point, no-one had bothered to tell the hospital's CIO that his new “dynamic interactive forms' technology, installed at great software and even greater integration expense, couldn't handle the therapist's form. Nor, in fact, could it handle ANY form for which the correct usage meant support for PDF markup tools.

From marginalia to lines, arrows and circles, annotations are a standard feature of PDF. The users wanted to use these annotations to indicate areas of interest in their diagrams in several different and elaborate ways.

It turned out that when questioned in detail about their needs, several departments - from surgery instruction to the aforementioned physical therapists to the dentistry - wanted to use PDF annotations for one or another reason. Even the HR folks wanted to be able to use Acrobat's highlighter on specific terms and conditions in contracts on a case-by-case basis. "Why not?" they kept asking. "It works on my other PDFs!".

Ah, but the hospital was committed to the OTHER type of PDF forms technology: XFA.  The kind that doesn't do annotations unless the PDF is static.

Of course, a static XFA form is basically the same as an AcroForm, which already does both fields and annotations, no problem.

Forms and annotations go together for a reason

Is this hospital's need unusual?  Not really.  When you think about it, there are many instances where the ability to provide a basic ad hoc diagram, or sketch on top of an existing diagram, would be considered crucial to the functioning of a form.

From police reports to product specification forms to forms accepting images captured by remote cameras.  PDF annotations provide an easy way to markup PDF content for a wide variety of needs. 

We've built forms that mix fields and annotations to graph calculations. For design, engineering and supply-chain managers alike, the ability to add ad-hoc markup to an existing form is a perennial request.

So far, no-one has ever objected to using annotations within a "form". For some, it's essential.

FDF: have your form, and annotations too

How did profoundly arbitrary distinctions as “text field” and “line annotation” fail to translate across the dynamic XFA divide?

After all, forms fields and markup annotations exist side-by-side as components of a comprehensive page and document-oriented forms solution. Known as AcroForms, the first iteration of this solution was introduced by Adobe with the PDF Reference version 1.2 in 1996. FDF is enshrined in ISO 32000-1; it's not going anywhere.

XFA is included in ISO 32000-1 largely by reference to Adobe's own XFA Specification document; it isn't part of the basic PDF model. Also unlike PDF, isn't managed by the ISO community - it still belongs to and is published by Adobe.  Formalists may argue the point, but XFA isn't really PDF, and the proof is in the implementation.

Back in the day, Adobe had the basic wisdom to design the same technical mechanism for transmitting any annotation (be it a form field, a blue circle, or whatever) to and from a server.

That mechanism goes by the humble name of FDF (Forms Data Format). Along with its XML-besotted (and sadly, less capable) cousin, XFDF, FDF is alive and well today because it is a part of ISO 32000. Tens of thousands of implementations worldwide rely on FDF for their annotations-handling needs, forms and otherwise.

At the end of the day

It doesn't matter how much you spend.  Dynamic XFA, the forms technology acquired by Adobe in 2003, doesn't do PDF annotations, because XFA technology was glued onto rather than integrated into PDF.  In today's Designer/XFA world, if it doesn't go in a XFA form field, it's not a form.  Your request does not compute.

On the other hand, good old FDF technology, 100% PDF through and through, handles form fields and annotations together just fine, always has, always will.  Need XML on the back-end?  Just bind it up on the server, and stop worrying about it. Want more dynamic authoring?  There are solutions there as well.

I told the consultant that he needed to go tell the CIO that they'd need to start supporting conventional PDF AcroForms technology, what else could I say?

"Don't shoot, I'm just the messenger!"