Comparing DocJet to Doxygen, NDoc, and JavaDoc

Comparing documentation tools is difficult to do, mainly because while you are evaluating the tool, you spend most of your time configuring the tool to produce the sort of output you want. However, once you get to actually using the tool, you spend almost all of your time writing comments. Its this business of writing the comments that makes up the vast bulk of the cost of producing your documentation - whether you spend money on your documentation tool or not is neither here nor there. This page is dedicated to helping you understand the value that you get when you lay out money for DocJet.

Support

If you have a problem with a free tool, you can toss a question out to a message board and you may get an answer; then again, you might not, because nobody is getting paid to answer you. With DocJet, you'll get a definitive answer from someone who has been helping people document source code for a very long time.

With open source software, the answer to the question "Can I make it do X?" always boils down to "yes, but can I afford to spend the time?" DocJet is not open-source, but it bristles with published API's, source code examples and other ways to customize it, making it just as configurable as any open-source tool. Further, you can ask our support staff, "How can I make it do X?" and get a good starting point.  In most all cases, we are able to offer a very simple way to get you where you need to go. If it's more complicated and something that lots of people are likely to want, we might even do the work for you. If it's not something that would be generally useful, we can offer reasonable consulting rates.

Commenting style

All of the free tools require the comment-writer to adhere to an exacting commenting standard. The reasons have less to do with the markups being good as it is that the markups make it easy for the tool. Anytime you are talking about a free tool, you'll see compromises in usability being made in favor of the tool rather than the user. The reason is simple: free tools are written by the user; commercial tools are written for the user.

Both Doxygen and JavaDoc comments can be written with HTML markups in the comment. The appeal of this is that you can directly control the formatting of your output. Writing HTML markup in your comments can lead to subtle, but very thorny problems.   For instance, suppose we have this comment:

//  A return value of <B>TRUE<B> indicates that the
//  input value was acceptable; any other value indicates
//  an error.

Did you spot the error?  (Failing to use </B> at the end of the bold block). It is an easy mistake to make, and neither Doxygen or JavaDoc will detect it. The only way to find it is to browse through the documentation, and our experience is that in a production environment, that doesn't reliably happen. In most cases, your readers will report the error, not your coders. Given what we all know about the cost of fixing a problem increasing with the time it takes to detect it, this trivial little problem can easily become quite expensive to fix in addition to being embarrassing.

The second problem with this comment revolves around the issue of why we bolded "TRUE" at all. Might it be the case that we really want to bold all instances of "TRUE" and "FALSE"?  Essentially we're creating a meta-class here, and so the correct way to denote it would be <SPAN CLASS=BoolConstants>TRUE</SPAN>.

That leads us to NDoc's commenting style, which is based on XML. Were we faced with this issue in code maintained with NDoc, we might create a new markup, <true>, and write this, instead:

//  A return value of <true/> indicates that the...

That way, every time we wanted to create a properly decorated "TRUE", we would just use that markup. This is quite an improvement in correctness over plain HTML, but it does impose a certain amount of overhead (you have to create the markup) and requires some discipline (you have to make sure that each time you want to use "TRUE", you write "<true/>" instead.

Both Doxygen and JavaDoc define their own comment markups, in conjunction with HTML markups. NDoc does the same sorts of things, but since it uses XML as a starting point, no new syntax need be invented - just more XML markups.

For compatibility's sake, DocJet can parse JavaDoc and XML comments, but that's not really the best way to use DocJet.

A Better Way

When you buy a tool, you expect it to serve your needs, and that's what DocJet is about. It has an amazingly sophisticated heuristically-driven natural-language comment parsing engine. For example, if you give it something like this:

//  Return Value:
//   0           The operation completed successfully
//   E_NOTFOUND  The requested object was not found
//   E_INVAL     The argument was invalid (null pointer)

The DocJet parser would first see "Return Value:" and know that what followed described what the function/method returns, and then see the rest and notice, based on the spacing and capitalization that the rest of the text was a table. Further, since this table was the sole contents of the return value section of the comment, that distinguishes this table from other tables that might be found elsewhere in the comment. So, in the output, this whole affair would be built into a neatly formatted HTML table complete with appropriate column headers.

This table can be expressed with markup, of course, and the adherents of markup in comments will say that the markup makes it more precise. But take a look at that comment!  The writer has clearly and unambiguously defined how he wants the information presented; any programmer could look at that comment and instantly grasp the intent. That's because we humans have implicitly written a markup language for ASCII formatting which we humans find easy to parse. Instead of symbols like "<", "/" and ">", this markup language uses spaces and newlines.

The argument that a comment composed of HTML markup is better than one written in plain text holds no water. Comments don't just serve the documentation generator - they also serve their traditional role of helping people who are maintaining the code. Thus it's just as important for comments to be readable in the code as it is for comments to be readable in the output. With markup, comments become far less readable, and thus far less valuable to the code maintainer. Moreover, writing the comment with markups requires a great deal more effort. Realistically, with this additional workload, most coders will change the above comment to "0 on success, else the result is an error code."

The bottom line is this: It's one thing to define a strong commenting convention with nice neat markups and style guidelines, but it's quite another to see it properly implemented. If you make it hard for programmers to write comments, they simply won't write them. If, on the other hand, you give them broad latitude in terms of style and content and just tell them to focus on writing a good description of what the program does, they are much more likely to give you good content.

It's that observation that drove the design of DocJet's comment-recognition system. Instead of programmers struggling to meet a standard set forth the commenting tool, DocJet adapts to the commenting style of the programmer.

Documenting the Obvious

On the surface, a comment documenter tool is a pretty simple thing.  It's just a matter of finding objects, rearranging the comment to match the output format, and generating the output. That's what the free tools do, and it works, to the extent that humans do a good job of writing comments.

It's this last part where things get interesting. Do you own a hammer?  Notice how that the hammer's head is about 1" in diameter. The physics of the tool are such that the hammer would work as well or better if the head were smaller. If you're like most of us, you bang your finger plenty often enough with the 1" diameter, so while a smaller hammer head might look good on the drawing board, it would not work well in practice. Software tools, like woodworking tools, cannot be called good or bad based on the tool by itself - you need to look at the total system, the tool plus the human that uses it.

Take a look at these two constructors:

// Default constructor
CFoo::CFoo()
{...}

CBar::CBar()
{...}

Would you say that CFoo's constructor is documented and CBar's is not?  Yes, CFoo does have a comment in front of it, but the comment doesn't say anything that isn't immediately obvious from the declaration. For all practical purposes, CFoo and CBar are both equally well commented and the author of CFoo did a bit of extra work in throwing in the comment.

DocJet understands the code it's documenting a great deal better than NDoc, Doxygen or JavaDoc, and, as a result, one of the things it's able to do is supply comments for obvious cases like what we have above. If you submit that code to DocJet, it would fill in the blank on CBar::CBar, producing documentation that is essentially the same as that for the commented CFoo.

Let's increase our sophistication a bit and move on to the copy constructor:

CFoo::CFoo( const CFoo &c )
{...}

Suppose you were a programmer on a documentation binge and came across that dude. In the absence of DocJet's automatic commenting, almost all programmers would either type in "// Copy constructor" or cut & paste the comment from another copy constructor into this spot.  Programmers know a downhill road when they see one.

However, if you were commenting along and ran across this and you knew that a reasonable default will be provided for you, you could either (1) skip the comment altogether, or (2) provide more detail. For instance, in DocJet, if you supplied:

// This makes a deep copy of c.
CFoo::CFoo( const CFoo &c )
{...}

Your comment, regarding the deep vs. shallow copy, would become the "Remarks" in the output and the default copy constructor brief description would be used as well. To put it another way, writing that, above, would be as good as writing this, in NDoc:

// <description>This creates a new instance of the object.</description>
// <remarks>This makes a deep copy of c.</remarks>
CFoo::CFoo( const CFoo &c )
{...}

Again, what DocJet is doing here goes beyond code. It's goal is to provide an environment where the easy thing for the programmer to do is also the best thing a programmer can do. When faced with the constructor, the programmer is asked to think, "Is there anything that is special about this particular copy constructor?"  If so, you can supply a comment that specifically talks about that. If not, fine, move on to another object where you do have something important to say.

The Perils of Commenting by Cut&Paste

So far, we really have been talking about how DocJet helps you document the rare cases that were pretty easy to document to begin with. So now let's take it up another notch into the world of inherited methods:

interface IFoo {
  // <extensive comment>
  void frob();
};

class CFoo implements IFoo {
  void frob();
};

DocJet employs the notion of inherited comments as well as the automatic comments we discussed before. In this case, any comment you put in front of CFoo::frob would override and complement IFoo::frob in much the same way as we discussed before for DocJet's notion of automatic commenting. For just about any reasonable case, the root method defines the purpose and interface of a method. That is, in IFoo::frob, you would talk about what the inputs are, what the effects are, and what the output is. In CFoo::frob, all that would continue to be the same; all you have to add there is any comments on how the class implements the method. In most cases, there's no need to put any comment at all on CFoo's frob method, because, in most cases, the reader of the documentation really doesn't care how CFoo implements frob.

Now that we have talked about the advantages of inherited documentation, let's talk about what happens in its absence. In that event, given the mandate to "document" all the code, how would a programmer react to seeing this bit:

void CFoo::frob();
{
...

In most cases, a programmer will "document" the above code by cutting&pasting the extensive comment from IFoo. That certainly works for the first go at documentation, but what happens later when someone updates the interface to IFoo::frob?  Only the most determined optimist would expect that the comment for CFoo::frob will get updated to reflect the change.

Organization and supporting documents

Just for a moment, let's pick on Microsoft. Consider this page from the .Net Framework SDK:

These are the comments that try men's souls. There's almost nothing on this page that you couldn't have gleaned from the declaration.  When do changes to this property actually display on the screen?  Do you need to redraw?  What do they mean by "ambient"?  What exactly constitutes the "background"?  One would hope that the root class, AmbientProperties, would tell us what ambient properties are, but no, it really doesn't. We might hope that the link to BackColor will yield answers to the issue of what constitutes "background", but that turns out not to be the case either. The reality is that we have an object with a page full of "documentation" that says absolutely nothing.

It's easy to blame the writer of this comment for its shortcomings, but put yourself in their shoes. What would you write?  It wouldn't be appropriate to discuss background colors here, would it?  Not in the code, no; the notion of "background" is wider than one or two properties. Similarly for ambient properties - you couldn't properly discuss ambient properties without contrasting them with primary properties.

People don't set out to write comments like poor BackColor here, they just end up doing it when the documentation tool they are using doesn't offer them an alternative. The real shortcomings here are those of the tool, not the author. Neither NDoc nor JavaDoc provide any meaningful way to integrate externally-written documents into the documentation-set.  Further, they have no means of controlling what the table of contents looks like - they just dump the objects out by a combination of code-structure and alphabetical order.

Doxygen, much to its credit, does allow for that, but not to the extent nor sophistication of DocJet. With Doxygen, you can create an overview topic either in the source file or in a separate file. Those methods are supported by DocJet; in addition, you can use a GUI-based tool for editing overview topics, glossary terms and other such supporting documentation.  Also, with DocJet, you can bring in documents written on a Word Processor and integrate those into the output set.

Here's a typical screenshot of DocJet's tool for creating and managing your documentation's table of contents:

Having a visual tool like this makes it far easier to see the organizational structure of your code. You can either totally control how your objects are laid out in the table of contents using the graphical interface here, or you can use a wide variety of means to organize them from the code.

Once again, the advantages to a tool like this are more sociological than technical. Producing documentation is a team effort, involving coders, writers and their managers. It's here that the manager of the documentation product merges the work of the coders with documents produced by technical writers to form the total organization of the document. With DocJet, all the members of the documentation team work with tools and methods that are familiar to them.

Summary

When selecting a code commenting tool, it's crucial to understand this fact: no matter how good the tool is, it will produce garbage unless the programmers write good comments. By "good comment", we mean that it says something meaningful about the object being described; whether it's a syntactic tour-de-force or not is irrelevant.

Complicated markup requirements discourage the writing of comments, that's why DocJet uses a freeform style of comment input. Of course, some people just feel more comfortable with markup, so DocJet can do that too.  Whatever makes the comment-writer happy is good with DocJet.

Programmers, like most people, are positively motivated by the thought of producing a top-quality product. By allowing for the integration of external documents and a table of contents, you don't have to shoehorn the subject matter into one comment or another - you can write comments that talk about just the object, and, if needed easily refer to other documents that describe the broader concepts.