There is a large body of good-practice traditions for open-source code that helps other people port, use, and cooperate with developing it. Some of these conventions are traditional in the Unix world and predate Linux; others have developed recently in response to particular new tools and technologies such as the World Wide Web.

This document will help you learn good practice. It is organized into topic sections, each containing a series of checklist items. Think of these as a pre-flight checklist for your patch or software distribution.

A translation into German is available.

This document will be posted monthly to the newsgroups comp.os.linux.answers. You can also view the latest version of this HOWTO on the World Wide Web via the URL http://tldp.org/HOWTO/Software-Release-Practice.html.

Feel free to mail any questions or comments about this HOWTO to Eric S. Raymond, <esr@snark.thyrsus.com>.

If your change includes a new file that doesn't exist in the code, then of course you have to send the whole file. But if you're modifying an already-existing file, don't send the whole file. Send a diff instead; specifically, send the output of the diff(1) command run to compare the baseline distributed version against your modified version.

The diff command and its dual, patch(1) (which automatically applies a diff to a baseline file) are the most basic tools of open-source development. Diffs are better than whole files because the developer you're sending a patch to may have changed the baseline version since you got your copy. By sending him a diff you save him the effort of separating your changes from his; you show respect for his time.

It is both counterproductive and rude to send a maintainer patches against the code as it existed several releases ago, and expect him to do all the work of determining which changes duplicate things he have since done, versus which things are actually novel in your patch.

As a patch submitter, it is your responsibility to track the state of the source and send the maintainer a minimal patch that expresses what you want done to the main-line codebase. That means sending a patch against the current version.

Nowadays effectively all open-source projects make their source code available through public anonymous access to the project's version-control repositrity. The most effective way to ensure that you're patching what's current is to check the head version of the code out of the project's repository. All version-control systems have a command that lets you make a diff between your working copy and head; use that.

Before you send your patch, walk through it and delete any patch bands for files in it that are going to be automatically regenerated once he applies the patch and remakes. The classic examples of this error are C files generated by Bison or Flex.

These days the most common mistake of this kind is sending a diff with a huge band that is nothing but changebars between your configure script and his. This file is generated by autoconf.

This is inconsiderate. It means your recipient is put to the trouble of separating the real content of the patch from a lot of bulky noise. It's a minor error, not as important as some of the things we'll get to further on — but it will count against you.

You will probably avoid this automatically if you have checked out the code from the project repo and use the version-control system's diff command to generate your patch.

Some people put special tokens in their source files that are expanded by the version-control system when the file is checked in: the $Id$ construct used by RCS, CVS, and Subversion, for example.

If you're using a local version-control system yourself, your changes may alter these tokens. This isn't really harmful, because when your recipient checks his code back in after applying your patch they'll get re-expanded based on his version-control status. But those extra patch bands are noise. They're distracting. It's more considerate not to send them.

This is another minor error. You'll be forgiven for it if you get the big things right. But you want to avoid it anyway.

The default (-e) format of diff(1) is very brittle. It doesn't include any context, so the patch tool can't cope if any lines have been inserted or deleted in the baseline code since you took the copy you modified.

Getting an -e diff is annoying, and suggests that the sender is either an extreme newbie, careless, or clueless. Most such patches get tossed out without a second thought.

This is very important. If your patch makes a user-visible addition or change to the software's features, include changes to the appropriate man pages and other documentation files in your patch. Do not assume that the recipient will be happy to document your code for you, or else to have undocumented features lurking in the code.

Documenting your changes well demonstrates some good things. First, it's considerate to the person you are trying to persuade. Second, it shows that you understand the ramifications of your change well enough to explain it to somebody who can't see the code. Third, it demonstrates that you care about the people who will ultimately use the software.

Good documentation is usually the most visible sign of what separates a solid contribution from a quick and dirty hack. If you take the time and care necessary to produce it, you'll find you're already 85% of the way to having your patch accepted with most developers.

Your patch should include cover notes explaining why you think the patch is necessary or useful. This is explanation directed not to the users of the software but to the maintainer to whom you are sending the patch.

The note can be short — in fact, some of the most effective cover notes I've ever seen just said "See the documentation updates in this patch". But it should show the right attitude.

The right attitude is helpful, respectful of the maintainer's time, quietly confident but unassuming. It's good to display understanding of the code you're patching. It's good to show that you can identify with the maintainer's problems. It's also good to be up front about any risks you perceive in applying the patch. Here are some examples of the sorts of explanatory comments that I like to see in cover notes:

“ I've seen two problems with this code, X and Y. I fixed problem X, but I didn't try addressing problem Y because I don't think I understand the part of the code that I believe is involved. ”

“ Fixed a core dump that can happen when one of the foo inputs is too long. While I was at it, I went looking for similar overflows elsewhere. I found a possible one in blarg.c, near line 666. Are you sure the sender can't generate more than 80 characters per transmission? ”

“ Have you considered using the Foonly algorithm for this problem? There is a good implementation at <http://www.somesite.com/~jsmith/foonly.html>. ”

“ This patch solves the immediate problem, but I realize it complicates the memory allocation in an unpleasant way. Works for me, but you should probably test it under heavy load before shipping. ”

“ This may be featurititis, but I'm sending it anyway. Maybe you'll know a cleaner way to implement the feature. ”

Usually as a maintainer, I will want to have strong confidence that I understand your changes before merging them in. This isn't an invariable rule; if you have a track record of good work, with me I may just run a casual eyeball over the changes before checking them in semi-automatically. But everything you can do to help me understand your code and decrease my uncertainty increases your chances that I will accept your patch.

Good comments in your code help me understand it. Bad comments don't.

Here's an example of a bad comment:

/* norman newbie fixed this 13 Aug 2009 */

This conveys no information. It's nothing but a muddy territorial bootprint you're planting in the middle of my code. If I take your patch (which you've made less likely) I'll almost certainly strip out this comment. If you want a credit, include a patch band for the project NEWS or HISTORY file. I'll probably take that.

Here's an example of a good comment:

/*
 * This conditional needs to be guarded so that crunch_data()
 * never gets passed a NULL pointer.  <norman_newbie@foosite.com>
 */

This comment shows that you understand not only my code but the kind of information that I need to have confidence in your changes. This kind of comment gives me confidence in your changes.

Don't gather various bugfixes, new features or other stuff and send them as one single diff file. Instead, build a single diff for every bugfix or feature. Every single diff should be generated using the current version of the code and should not depend on any other patch you or someone else sent before.

This helps the maintainer to read and understand your code, and he can decide whether he wants to accept or abadon this single bugfix or feature. For example, if you add a feature garanting full root access for everyone because it's useful in your special setting, the maintainer will probably not accept this part of your patch. When you send a single diff file confounding this root access feature along with some bugfixes and other useful things, you have a good chance the maintainer will ignore your complete patch.

With each bugfixes and new feature as a single diff file, the maintainer could e.g. include your bugfixes in a few minutes and take later a closer look to your new features or security issues.

Patches depending on other patches raise similar problems. If the maintainer rejects your basic patch, he won't be able to apply the others. If you can't avoid such dependencies, offer to the maintainer that you will bundle a patch with the sections or features he wants to add.

It's helpful to everybody if your archive files all have GNU-like names — all-lower-case alphanumeric stem prefix, followed by a dash, followed by a version number, extension, and other suffixes.

Let's suppose you have a project you call `foobar' at version 1, release 2, level 3. If it's got just one archive part (presumably the sources), here's what its names should look:

Please don't use these:

If you have to differentiate between source and binary archives, or between different kinds of binary, or express some kind of build option in the file name, please treat that as a file extension to go after the version number. That is, please do this:

Please don't use names like `foobar-ELF-1.2.3.tar.gz', because programs have a hard time telling type infixes (like `-ELF') from the stem.

A good general form of name has these parts in order:

Some projects and communities have well-defined conventions for names and version numbers that aren't necessarily compatible with the above advice. For instance, Apache modules are generally named like mod_foo, and have both their own version number and the version of Apache with which they work. Likewise, Perl modules have version numbers that can be treated as floating point numbers (e.g., you might see 1.303 rather than 1.3.3), and the distributions are generally named Foo-Bar-1.303.tar.gz for version 1.303 of module Foo::Bar. (Perl itself, on the other hand, switched to using the conventions described in this document in late 1999.)

Look for and respect the conventions of specialized communities and developers; for general use, follow the above guidelines.

The stem prefix should be easy to read, type, and remember. So please don't use underscores. Don't capitalize or BiCapitalize without extremely good reason — it messes up the natural human-eyeball search order, among other things.

It confuses people when two different projects have the same stem name. So try to check for collisions before your first release. Google is your friend — and if the stem you're thinking about gets lots of Google hits, that in itself may be sufficient reason to think up a different one. Another good place to check is the application indexes at Freecode and SourceForge; do a name search there.

Anything that is not public domain has a copyright, possibly more than one. Under the Berne Convention (which has been U.S. law since 1978), the copyright does not have to be explicit. That is, the authors of a work hold copyright even if there is no copyright notice.

Who counts as an author can be very complicated, especially for software that has been worked on by many hands. This is why licenses are important. By setting out the terms under which material can be used, they grant rights to the users that protect them from arbitrary actions by the copyright holders.

In proprietary software, the license terms are designed to protect the copyright. They're a way of granting a few rights to users while reserving as much legal territory is possible for the owner (the copyright holder). The copyright holder is very important, and the license logic so restrictive that the exact technicalities of the license terms are usually unimportant.

In open-source software, the situation is usually the exact opposite; the copyright exists to protect the license. The only rights the copyright holder always keeps are to enforce the license. Otherwise, only a few rights are reserved and most choices pass to the user. In particular, the copyright holder cannot change the terms on a copy you already have. Therefore, in open-source software the copyright holder is almost irrelevant — but the license terms are very important.

Normally the copyright holder of a project is the current project leader or sponsoring organization. Transfer of the project to a new leader is often signaled by changing the copyright holder. However, this is not a hard and fast rule; many open-source projects have multiple copyright holders, and there is no instance on record of this leading to legal problems.

Some projects choose to assign copyright to the Free Software Foundation, on the theory that it has an interest in defending open source and lawyers available to do it.

For licensing purposes, we can distinguish several different kinds of rights that a license may convey. Rights to copy and redistribute, rights to use, rights to modify for personal use, and rights to redistribute modified copies. A license may restrict or attach conditions to any of these rights.

The Open Source Initiative is the result of a great deal of thought about what makes software “open source” or (in older terminology) “free software”. Its constraints on licensing require that:

The guidelines prohibit restrictions on redistribution of modified binaries; this meets the needs of software distributors, who need to be able to ship working code without encumbrance. It allows authors to require that modified sources be redistributed as pristine sources plus patches, thus establishing the author's intentions and an “audit trail” of any changes by others.

The OSD is the legal definition of the `OSI Certified Open Source' certification mark, and as good a definition of “free software” as anyone has ever come up with. All of the standard licenses (MIT, BSD, Artistic, and GPL/LGPL) meet it (though some, like GPL, have other restrictions which you should understand before choosing it).

Note that licenses which allow noncommercial use only do not qualify as open-source licenses, even if they are decorated with “GPL” or some other standard license. They discriminate against particular occupations, persons, and groups. They make life too complicated for CD-ROM distributors and others trying to spread open-source software commercially.

In some cases, if you have a sponsoring organization behind you with lawyers, you might wish to give copyright to that organization.

The Open Source Definition is the community gold standard for licenses. The OSD is not a license itself; rather, it defines a minimum set of rights that a license must guarantee in order to be considered an open-source license. The OSD, and supporting materials, may be found at the web site of the Open Source Initiative.

The widely-known OSD-conformant licenses have well-established interpretive traditions. Developers (and, to the extent they care, users) know what they imply, and have a reasonable take on the risks and tradeoffs they involve. Therefore, use one of the standard licenses carried on the OSI site if at all possible.

If you must write your own license, be sure to have it certified by OSI. This will avoid a lot of argument and overhead. Unless you've been through it, you have no idea how nasty a licensing flamewar can get; people become passionate because the licenses are regarded as almost-sacred covenants touching the core values of the open-source community.

Furthermore, the presence of an established interpretive tradition may prove important if your license is ever tested in court. At time of writing (early 2002) there is no case law either supporting or invalidating any open-source license. However, it is a legal doctrine (at least in the U.S., and probably in other common-law countries such as England and the rest of the British Commonwealth) that courts are supposed to interpret licenses and contracts according to the expectations and practices of the community in which they originated.

As larger and larger volumes of open-source software are deployed, the problem of auditing these volumes for which licenses cover them become nontrivial — in fact, it becomes larger than an unaided human being can perform. Therefore, it is valuable to have conventions that support mechanized querying of the licensing information. Fortunately, existing community practise already tends in this direction.

As a beginning, the license information for your software should live in a file named either COPYING or LICENSE in the top-level directory of your source distribution. If a single license applies to the entire distribution, that file should include a copy of the license. If multiple licenses apply, that file should list the applicable licenses and indicate to which files and subdirectories they apply.

Choose a development language which minimizes the differences of the underlying environments in which it will run. C/C++ programs are likely to be more portable than Fortran. Java is preferable to C/C++, and a high-level scripting language such as Perl, Python or Ruby is the best choice of all (since scripting languages have only one cross-platform implementation).

Scripting languages that qualify include Python, Perl, Tcl, Emacs Lisp, and PHP. Historic Bourne shell (/bin/sh) does not qualify; there are too many different implementations with subtle idiosyncrasies, and the shell environment is subject to disruption by user customizations such as shell aliases.

If you choose a compiled language such as C/C++, do not use compiler extensions (such as allocating variable-length arrays on the stack, or indirecting through void pointers). Regularly build using as many different compilers and test machines as you can.

Don't rely on proprietary languages, libraries, or other code. In the open-source community this is considered rude. Open-source developers don't trust code for which they can't review the source.

A significant advantage of open source distributions is they allow each source package to adapt at compile-time to the environment it finds. One of the choices you need to make is of a "build system", the toolkit you (and other people) will rely on to transform your source into executables.

One thing your build script cannot do is ask the user for system information at compile-time. The user installing the package will not necessarily know the answers to your questions, so this approach is doomed from the start. Your software, calling the build system tools, must be able to determine for itself any information that it may need at compile- or install-time.

Community notions of best practice in build systems are now (rearly 2010) in a starte of some flux.

A good test suite allows the team to buy inexpensive hardware for testing and then easily run regression tests before releases. Create a strong, usable test framework so that you can incrementally add tests to your software without having to train developers up in the intricacies of the test suite.

Distributing the test suite allows the community of users to test their ports before contributing them back to the group.

Encourage your developers to use a wide variety of platforms as their desktop and test machines so that code is continuously being tested for portability flaws as part of normal development.

If you're writing C/C++ using GCC, test-compile with -Wall and clean up all warning messages before each release. Compile your code with every compiler you can find — different compilers often find different problems. Specifically, compile your software on true 64-bit machine. Underlying data types can change on 64-bit machines, and you will often find new problems there. Find a UNIX vendor's system and run the lint utility over your software.

Run tools that for memory leaks and other run-time errors; Electric Fence and Valgrind are two good ones available in open source.

For Python projects, the PyChecker program can be a useful check. It's not out of beta yet, but nevertheless often catches nontrivial errors.

If you're writing Perl, check your code with perl -c (and maybe -T, if applicable). Use perl -w and 'use strict' religiously. (See the Perl documentation for further discussion.)

Spell-check your documentation, README files and error messages in your software. Sloppy code, code that produces warning messages when compiled, and spelling errors in README files or error messages, leads users to believe the engineering behind it is also haphazard and sloppy.

If you are writing C, feel free to use the full ANSI features. Specifically, do use function prototypes, which will help you spot cross-module inconsistencies. The old-style K&R compilers are history.

Do not assume compiler-specific features such as the GCC "-pipe" option or nested functions are available. These will come around and bite you the second somebody ports to a non-Linux, non-GCC system.

Code required for portability should be isolated to a single area and a single set of source files (for example, an "os" subdirectory). Compiler, library and operating system interfaces with portability issues should be abstracted to files in this directory. This includes variables such as "errno", library interfaces such as "malloc", and operating system interfaces such as "mmap".

Portability layers make it easier to do new software ports. It is often the case that no member of the development team knows the porting platform (for example, there are literally hundreds of different embedded operating systems, and nobody knows any significant fraction of them). By creating a separate portability layer it is possible for someone who knows the port platform to port your software without having to understand it.

Portability layers simplify applications. Software rarely needs the full functionality of more complex system calls such as mmap or stat, and programmers commonly configure such complex interfaces incorrectly. A portability layer with abstracted interfaces ("__file_exists" instead of a call to stat) allows you to export only the limited, necessary functionality from the system, simplifying the code in your application.

Always write your portability layer to select based on a feature, never based on a platform. Trying to create a separate portability layer for each supported platform results in a multiple update problem maintenance nightmare. A "platform" is always selected on at least two axes: the compiler and the library/operating system release. In some cases there are three axes, as when Linux vendors select a C library independently of the operating system release. With M vendors, N compilers and O operating system releases, the number of "platforms" quickly scales out of reach of any but the largest development teams. By using language and systems standards such as ANSI and POSIX 1003.1, the set of features is relatively constrained.

Portability choices can be made along either lines of code or compiled files. It doesn't make a difference if you select alternate lines of code on a platform, or one of a few different files. A rule of thumb is to move portability code for different platforms into separate files when the implementations diverged significantly (shared memory mapping on UNIX vs. Windows), and leave portability code in a single file when the differences are minimal (using gettimeofday, clock_gettime, ftime or time to find out the current time-of-day).

Avoid using complex types such as "off_t" and "size_t". They vary in size from system to system, especially on 64-bit systems. Limit your usage of "off_t" to the portability layer, and your usage of "size_t" to mean only the length of a string in memory, and nothing else.

Never step on the namespace of any other part of the system, (including file names, error return values and function names). Where the namespace is shared, document the portion of the namespace that you use.

Choose a coding standard. The debate over the choice of standard can go on forever — regardless, it is too difficult and expensive to maintain software built using multiple coding standards, and so some coding standard must be chosen. Enforce your coding standard ruthlessly, as consistency and cleanliness of the code are of the highest priority; the details of the coding standard itself are a distant second.

The single most annoying mistake newbie developers make is to build tarballs that unpack the files and directories in the distribution into the current directory, potentially stepping on files already located there. Never do this!

Instead, make sure your archive files all have a common directory part named after the project, so they will unpack into a single top-level directory directly beneath the current one.

Here's a makefile trick that, assuming your distribution directory is named `foobar' and SRC contains a list of your distribution files, accomplishes this. SRC may also contain names of subdirectories to be included whole.

foobar-$(VERS).tar.gz:
	@find $(SRC) -type f | sed s:^:foobar-$(VERS)/: >MANIFEST
	@(cd ..; ln -s foobar foobar-$(VERS))
	(cd ..; tar -czvf foobar/foobar-$(VERS).tar.gz `cat foobar/MANIFEST`)
	@(cd ..; rm foobar-$(VERS))

Have a file called README or READ.ME that is a roadmap of your source distribution. By ancient convention, this is the first file intrepid explorers will read after unpacking the source.

Good things to have in the README include:

Before even looking at the README, your intrepid explorer will have scanned the filenames in the top-level directory of your unpacked distribution. Those names can themselves convey information. By adhering to certain standard naming practices, you can give the explorer valuable clues about what to look in next.

Here are some standard top-level file names and what they mean. Not every distribution needs all of these.

Note the overall convention that filenames with all-caps names are human-readable metainformation about the package, rather than build components (TAGS is an exception to the first, but not to the second).

Having a FAQ can save you a lot of grief. When a question about the project comes up often, put it in the FAQ; then direct users to read the FAQ before sending questions or bug reports. A well-nurtured FAQ can decrease the support burden on the project maintainers by an order of magnitude or more.

Having a HISTORY or NEWS file with timestamps in it for each release is valuable. Among other things, it may help establish prior art if you are ever hit with a patent-infringement lawsuit (this hasn't happened to anyone yet, but best to be prepared).

Your software will change over time as you put out new releases. Some of these changes will not be backward-compatible. Accordingly, you should give serious thought to designing your installation layouts so that multiple installed versions of your code can coexist on the same system. This is especially important for libraries — you can't count on all your client programs to upgrade in lockstep with your API changes.

The Emacs, Python, and Qt projects have a good convention for handling this; version-numbered directories. Here's how an installed Qt library hierarchy looks (${ver} is the version number):

/usr/lib/qt
/usr/lib/qt-${ver}
/usr/lib/qt-${ver}/bin          # Where you find moc
/usr/lib/qt-${ver}/lib          # Where you find .so
/usr/lib/qt-${ver}/include      # Where you find header files

With this organization, you can have multiple versions coexisting. Client programs have to specify the library version they want, but that's a small price to pay for not having the interfaces break on them.

Provide checksums with your binaries (tarballs, RPMs, etc.). This will allow people to verify that they haven't been corrupted or had Trojan-horse code inserted in them.

While there are several commands you can use for this purpose (such as sum and cksum) it is best to use a cryptographically-secure hash function. The GPG package provides this capability via the —detach-sign option; so does the GNU command md5sum.

For each binary you ship, your project web page should list the checksum and the command you used to generate it.

Here are the documentation markup formats now in widespread use among open-source developers. When we speak of "presentation" markup, we mean markup that controls the document's appearance explicitly (such as a font change). When we speak of "structural" markup, we mean markup that describes the logical structure of the document (like a section break or emphasis tag.) And when we speak of "indexing", we mean the process of extracting from a collection of documents a searchable collection of topic pointers that users can employ to reliably find material of interest across the entire collection.

Documentation practice has been changing since 2000, when some key open-source project groups (including the Linux kernel project, GNOME, KDE, the Free Software Foundation, and the Linux Documentation Project) agreed on an approach more web-friendly than traditional Unix's print-oriented tools. Today's best practice, since the XML-DocBook toolchain reached production status in mid-2001, is this:

For converting legacy documentation in troff formats to DocBook, check out doclifter. If you're unwilling to move from using man sources as a master format, at least try to clean them up so doclifter can lift them to XML automatically.

See Freecode. Distribution watch this channel to see when new releases are issuing.

If you intend try to build any substantial user or developer community around your project, it should have a website. Standard things to have on the website include:

Some project sites even have URLs for anonymous access to the master source tree.

It's standard practice to have a private development list through which project collaborators can communicate and exchange patches. You may also want to have an announcements list for people who want to be kept informed of the project's process.

If you are running a project named `foo'. your developer list might be foo-dev or foo-friends; your announcement list might be foo-announce.

Since it was launched in fall 1999, SourceForge has exploded in popularity. It is not just an archive and distribution site, though you can use it that way. It is an entire free project-hosting service that tries to offer a complete set of tools for open-source development groups — web and archive space, mailing lists, bug-tracking, chat forums, CVS repositories, and other services.

Other important locations include: