What a digital format registry records about each file format
A digital format registry, read as a file format reference, answers a deceptively simple question: what is this file, and can it still be opened in twenty years? Answering it well takes more than a list of extensions. A useful registry entry describes a format the way a museum catalog describes an object, with an identity, physical evidence, documentation and context.
Identity comes first. A format has a name, usually several (“JPEG”, “JFIF”, “JPEG File Interchange Format”), one or more versions, the extensions it travels under and the media type registered for it with IANA. Registries keep these apart on purpose: PDF 1.4 and PDF/A-1b share a signature and an extension but carry different promises about fonts, color and external content.
Evidence is the signature. Each version gets one or more byte sequences, the offset where they appear and sometimes a sequence at the end of the file as well. JPEG files, for example, end with the marker FF D9. These signatures are what make file format identification repeatable: two archives running the same tool against the same registry release reach the same answer.
Documentation means the specification itself and who controls it. A format published as an international standard (PNG as ISO/IEC 15948, PDF as ISO 32000) is in a very different position from one whose only description lives inside a vendor's software. Registries record whether the specification is fully disclosed, partially disclosed or not available at all, because that single fact predicts how hard a future migration will be.
Context covers everything around the format: which versions came before and after, which formats it is based on, what compression it uses, which programs create and render it, and which risks are known. A file format reference that keeps these relationships lets a preservation system ask useful questions, such as “which formats in the collection depend on a codec that is no longer maintained?”
Where registries come from
The idea of a shared digital format registry grew out of practical archival work in the early 2000s. PRONOM, launched by The National Archives of the United Kingdom, became the reference most identification tools rely on; its persistent identifiers (PUIDs such as fmt/11, fmt/12 and fmt/13 for successive PNG versions) appear in the technical metadata of archives worldwide. The Library of Congress publishes detailed format descriptions focused on sustainability, and Wikidata now links many formats to their PRONOM identifiers, media types and specifications. The history of format registry projects shows how these efforts tried, with mixed success, to merge into one shared resource.
Why the metadata vocabulary matters
Identifying a format is only useful if the result is written down in a way other systems understand. That is where Dublin Core and the DCMI Metadata Terms come in: dc:format carries the media type, dcterms:FileFormat types the value, and encoding schemes such as IMT state where the value comes from. Language and place need the same discipline, which is why the ISO 639-2 language codes and ISO 3166-1 country codes sit next to the format records here.
How archives use a digital format registry day to day
Most of the work happens at intake. A deposit arrives (a hard drive from a retiring professor, a network share from a closed department, a batch of email exports) and nobody can say with confidence what is on it. The first pass runs an identification tool over every file and writes the result next to the file’s checksum. Out of that pass comes a format profile of the deposit: so many TIFF files, so many PDF versions, a long tail of things nobody expected. The digital format registry is what turns raw byte matches into names, versions and identifiers that mean the same thing in every archive using the same release.
The second job is characterization. Once a file is known to be, say, a TIFF, the archive wants to know more: its dimensions, bit depth, compression, color space, whether it carries embedded metadata. That information goes into technical metadata so that later decisions do not require opening the file again.
Then there is risk watch, which is quieter and easier to forget. A collection that was safe ten years ago may now hold a format whose last rendering application is no longer sold. Because every file carries a registry identifier, a repository can run a simple query (show all objects in format X) whenever the registry or the institution’s own policy flags a format as a concern.
Migration planning closes the loop. When a format is judged at risk, the registry entry tells the planner which formats it relates to, whether a documented successor exists, and whether the conversion is lossless. A planner looking at a few thousand files in an old word processor format needs to know where they should go and what will be lost on the way, and the registry entry is where both questions start.
A small practical point: the registry release matters. Signature files are updated several times a year, and a file that came back unidentified in one release may be identified in the next. Recording the tool version and the signature version with every result is what makes a re-run worth doing.
Identification, validation and characterization
These three words get used loosely, and the difference is worth getting right, because each answers a different question and each is done by different software.
Identification: what is this file?
File format identification assigns a format, ideally a specific version, to a file based on internal evidence. DROID, developed by The National Archives of the United Kingdom, and Siegfried, an independent open-source tool, both match files against the PRONOM signature set and report a PUID. They work fast enough to run across millions of files, and they are designed to say “unknown” when the evidence is weak. That last behavior is a feature. An identification that admits uncertainty is far more useful than one that guesses from the extension.
Validation: does the file follow its specification?
Validation checks whether a file conforms to the rules of its declared format. JHOVE, now maintained under the Open Preservation Foundation, has modules for formats such as PDF, TIFF, JPEG and several audio formats, and reports whether a file is well-formed and valid. A PDF can be identified correctly as PDF 1.4 and still fail validation because its cross-reference table is broken. Identification would not catch that, and an archive that only identifies will find out years later when the file refuses to render.
Validation is also where the most argument happens. Real-world files bend rules constantly, and a strict validator can flag thousands of files that open perfectly well in common software. Archives usually treat a validation failure as a reason to look, not as an automatic rejection.
Characterization: what are this file’s properties?
Characterization extracts technical properties. MediaInfo reports codec, container, duration, frame rate and audio channels for audio and video files. ExifTool reads and writes embedded metadata in images and many other file types, from camera settings to XMP blocks. JHOVE also characterizes as part of validation. None of these tools is a registry; they produce the measurements that a registry’s vocabulary then gives names to, such as a compression lossiness type or a FourCC inside a video container.
The order is usually identification first, then validation and characterization chosen by format. Running a PDF validator on a WAV file wastes time, so the identification result decides which tool runs next. The file format identification guide walks through that sequence with real signatures and offsets.
Format risk and obsolescence
A format does not become unreadable on a particular day. It drifts: the vendor stops updating the application, the operating system that ran it is retired, the people who understood the internals move on. By the time an archive notices, the files are still there and the bytes are intact, but nothing on a current machine will open them.
The Library of Congress sustainability factors give a practical checklist for judging that drift. They ask about disclosure (is the specification public?), adoption (how widely is the format used?), transparency (can the content be read with basic tools?), self-documentation (does the file carry its own metadata?), external dependencies (does it need particular hardware, fonts or software?), the impact of patents, and technical protection mechanisms such as encryption. A format that scores poorly on several of these deserves attention even if it opens fine today.
Disclosure is the factor a digital format registry can record most precisely. A specification may be fully public, available only to members of a consortium, or reverse-engineered by third parties. A record marked as partial disclosure tells a planner that a converter exists but may not cover every feature.
Formats that became hard to read
WordStar is the example most often cited. It dominated word processing on CP/M and early DOS machines, and its files mix text with control codes and high-bit characters that later software did not interpret consistently. The text can usually be recovered; the formatting is another matter.
Proprietary multimedia authoring formats are a harder case. Interactive CD-ROMs from the 1990s were often built with Macromedia Director, and their content depended on a specific player version running on a specific operating system. Adobe ended support for Flash Player at the end of 2020, which left large amounts of web animation and games depending on emulation projects to stay playable. HyperCard stacks on the classic Macintosh fall into the same pattern.
Early computer-aided design files are frequently mentioned in preservation literature as well, since many CAD packages of the 1980s and 1990s used undocumented binary formats tied to one vendor’s release. Hard numbers on how much engineering data was lost this way are scarce, and the cases tend to surface one project at a time, as anecdotes from engineers and archivists, not as a measured trend.
What these examples share is dependency on a single implementation. The formats that aged best (plain text, TIFF, PNG, WAV) are boring in comparison: published, simple, and implemented independently many times. The preservation file formats guide covers those choices format by format.
Linked data and format records
A format record written as a spreadsheet row works inside one institution. As soon as two systems need to agree on what “PDF/A-1b” means, each record needs a stable identity of its own. That is the reason a modern digital format registry tends to publish its data as RDF.
In RDF, every format, every version and every property gets a URI. A statement such as “this format has the media type application/pdf” becomes a triple: the format’s URI, a property URI, a value. Because the property is also a URI, a second system can look it up and know exactly what was meant. The Dublin Core vocabulary pages on this site are written in that spirit: dcterms:FileFormat is a class with a URI, so a record that uses it says something unambiguous.
How identifiers connect
A single format often carries several identifiers at once: a PRONOM PUID, a media type, a Library of Congress format description identifier, a Wikidata item number. None of them replaces the others. The PUID is what identification tools report; the media type is what web servers and Dublin Core records use; the Wikidata item is where the other identifiers are gathered together.
Wikidata is the practical hub. Many file format items carry a dedicated property for the PRONOM identifier and another for the media type, plus links to specifications and to the software that reads the format. A system that holds only a PUID can follow Wikidata to find the media type, and the reverse. That cross-walk is fragile in places (some items mix several versions, others duplicate one another), so a careful repository stores the identifiers it trusts locally and uses linked data to check them.
Labels need the same care. A format may have a preferred name and several others it travels under, which is why a registry records an alias name separately and why SKOS offers skos:altLabel for alternative labels. Search depends on those alternatives; a user looking for “JFIF” should still find the JPEG record.
Where code lists come in
Format records also point outward to language and place. A text format record may state the languages a specification is written in, and a registry entry for a national standard may name the country that issued it. Those values only interoperate when they come from shared lists, which is why the language code type and country code type pages explain which codes are expected and in what form.
How to use this file format reference
A digital format registry is only as useful as the route into it, and different questions lead to different parts of the site. If you have a file in hand and want to know what it is, the identification tool near the top of this page reads its first bytes, and the identification guide linked above explains what to do when the answer is a container or an unknown.
If the question is which format to keep for the long term, start with the preservation guide mentioned in the section on risk. It compares the usual candidates for images, documents, audio, video and data.
If the task is writing a metadata record, the guide to Dublin Core records covers which elements matter, and the term pages give one definition, one example and the usual mistakes for each property. The dc:format page is the natural starting point for anything to do with file formats in Dublin Core.
For questions about the code lists themselves (whether a country code is still assigned, what a special language code means), the code list pages hold the full tables with every form of each code.
And if the interest is in how this file format reference is written and maintained, the project page describes the editorial approach. One detail is worth knowing before using any term page as a citation: each one gives the official URI of the term, and that URI, not the page address, is what belongs in a metadata record.