Skip to content
Formatory · file formats, signatures, metadata

Digital format registry: a file format reference for identification

Formatory is an independent digital format registry. It shows how a file format is identified from its first bytes, how Dublin Core describes it, which formats hold up for preservation, and the language and country codes that format records depend on.

Floating glass file objects, a layered cube and a ring lit in violet and mint
89 50 4E 47 PNG
dc:format
ISO 639-2 zxx
19byte signatures in the identifier
15Dublin Core elements explained
249ISO 3166-1 country codes
487ISO 639-2 language entries
01 · the registry at a glance

One file format reference, every layer of a file

From the signature in the first bytes to the metadata record that still describes the file decades later.

Identify a file by its bytes

Choose any file. Its first 32 bytes are read in the browser and compared with known signatures. Nothing is uploaded.

No file read yet. Only the first 32 bytes are examined and nothing leaves the device.

Format families

  • Image6 sig.
  • Document2 sig.
  • Archive3 sig.
  • Audio4 sig.
  • Video2 sig.
  • Data2 sig.
All signatures

Dublin Core and DCMI terms

dc:format

Fifteen elements, dozens of refinements, each explained with examples.

Language codes

zxx

ISO 639-2 for “no linguistic content”, used for music, images and data.

Country codes

249

ISO 3166-1 alpha-2, alpha-3 and numeric codes in one table.

02 · how file format identification works

File format identification: an extension is a guess, the bytes are evidence

A glass prism splitting a stream of small cubes into separate colored streams
Step 1

Read the first bytes

Most formats open with a fixed sequence, the magic number. A PNG always starts with 89 50 4E 47 0D 0A 1A 0A, whatever its filename says. A digital format registry stores that sequence together with its offset.

Step 2

Check the offset

Some signatures sit further in. WebP and WAVE are RIFF containers, so their type appears at byte 8, while MP4 and QuickTime show ftyp at byte 4. A signature without its offset matches the wrong files.

Step 3

Resolve the container

A ZIP signature also opens DOCX, XLSX, EPUB and ODT. Telling them apart means reading inside the package, which identification tools such as DROID and Siegfried do with container signatures from PRONOM.

Step 4

Record the result

The identification becomes metadata: a dc:format media type, a registry identifier such as a PRONOM PUID, the tool and version that made the call, and the date it was checked.

03 · what a format record holds

What a digital format registry records about each file format

A digital format registry, read as a file format reference, answers a deceptively simple question: what is this file, and can it still be opened in twenty years? Answering it well takes more than a list of extensions. A useful registry entry describes a format the way a museum catalog describes an object, with an identity, physical evidence, documentation and context.

Identity comes first. A format has a name, usually several (“JPEG”, “JFIF”, “JPEG File Interchange Format”), one or more versions, the extensions it travels under and the media type registered for it with IANA. Registries keep these apart on purpose: PDF 1.4 and PDF/A-1b share a signature and an extension but carry different promises about fonts, color and external content.

Evidence is the signature. Each version gets one or more byte sequences, the offset where they appear and sometimes a sequence at the end of the file as well. JPEG files, for example, end with the marker FF D9. These signatures are what make file format identification repeatable: two archives running the same tool against the same registry release reach the same answer.

Anatomy of a format record A format record with five parts: identity, signature, specification, relationships and preservation notes, each with an example value for PNG. PNG 1.2: anatomy of a format record IDENTITYPortable Network Graphics.png · image/png SIGNATURE89 50 4E 47 0D 0A 1A 0Aoffset 0, beginning of file SPECIFICATIONISO/IEC 15948, W3C PNGopenly published RELATIONSHIPShas earlier version: PNG 1.1, PNG 1.0compression: DEFLATE, lossless PRESERVATION NOTESwide support, no patents, losslesssuitable for born-digital masters
The five parts most registry entries share, filled in for PNG 1.2. Signature and media type are taken from the PNG specification.

Documentation means the specification itself and who controls it. A format published as an international standard (PNG as ISO/IEC 15948, PDF as ISO 32000) is in a very different position from one whose only description lives inside a vendor's software. Registries record whether the specification is fully disclosed, partially disclosed or not available at all, because that single fact predicts how hard a future migration will be.

Context covers everything around the format: which versions came before and after, which formats it is based on, what compression it uses, which programs create and render it, and which risks are known. A file format reference that keeps these relationships lets a preservation system ask useful questions, such as “which formats in the collection depend on a codec that is no longer maintained?”

Where registries come from

The idea of a shared digital format registry grew out of practical archival work in the early 2000s. PRONOM, launched by The National Archives of the United Kingdom, became the reference most identification tools rely on; its persistent identifiers (PUIDs such as fmt/11, fmt/12 and fmt/13 for successive PNG versions) appear in the technical metadata of archives worldwide. The Library of Congress publishes detailed format descriptions focused on sustainability, and Wikidata now links many formats to their PRONOM identifiers, media types and specifications. The history of format registry projects shows how these efforts tried, with mixed success, to merge into one shared resource.

Why the metadata vocabulary matters

Identifying a format is only useful if the result is written down in a way other systems understand. That is where Dublin Core and the DCMI Metadata Terms come in: dc:format carries the media type, dcterms:FileFormat types the value, and encoding schemes such as IMT state where the value comes from. Language and place need the same discipline, which is why the ISO 639-2 language codes and ISO 3166-1 country codes sit next to the format records here.

How archives use a digital format registry day to day

Most of the work happens at intake. A deposit arrives (a hard drive from a retiring professor, a network share from a closed department, a batch of email exports) and nobody can say with confidence what is on it. The first pass runs an identification tool over every file and writes the result next to the file’s checksum. Out of that pass comes a format profile of the deposit: so many TIFF files, so many PDF versions, a long tail of things nobody expected. The digital format registry is what turns raw byte matches into names, versions and identifiers that mean the same thing in every archive using the same release.

The second job is characterization. Once a file is known to be, say, a TIFF, the archive wants to know more: its dimensions, bit depth, compression, color space, whether it carries embedded metadata. That information goes into technical metadata so that later decisions do not require opening the file again.

Then there is risk watch, which is quieter and easier to forget. A collection that was safe ten years ago may now hold a format whose last rendering application is no longer sold. Because every file carries a registry identifier, a repository can run a simple query (show all objects in format X) whenever the registry or the institution’s own policy flags a format as a concern.

Migration planning closes the loop. When a format is judged at risk, the registry entry tells the planner which formats it relates to, whether a documented successor exists, and whether the conversion is lossless. A planner looking at a few thousand files in an old word processor format needs to know where they should go and what will be lost on the way, and the registry entry is where both questions start.

A small practical point: the registry release matters. Signature files are updated several times a year, and a file that came back unidentified in one release may be identified in the next. Recording the tool version and the signature version with every result is what makes a re-run worth doing.

Identification, validation and characterization

These three words get used loosely, and the difference is worth getting right, because each answers a different question and each is done by different software.

Identification: what is this file?

File format identification assigns a format, ideally a specific version, to a file based on internal evidence. DROID, developed by The National Archives of the United Kingdom, and Siegfried, an independent open-source tool, both match files against the PRONOM signature set and report a PUID. They work fast enough to run across millions of files, and they are designed to say “unknown” when the evidence is weak. That last behavior is a feature. An identification that admits uncertainty is far more useful than one that guesses from the extension.

Validation: does the file follow its specification?

Validation checks whether a file conforms to the rules of its declared format. JHOVE, now maintained under the Open Preservation Foundation, has modules for formats such as PDF, TIFF, JPEG and several audio formats, and reports whether a file is well-formed and valid. A PDF can be identified correctly as PDF 1.4 and still fail validation because its cross-reference table is broken. Identification would not catch that, and an archive that only identifies will find out years later when the file refuses to render.

Validation is also where the most argument happens. Real-world files bend rules constantly, and a strict validator can flag thousands of files that open perfectly well in common software. Archives usually treat a validation failure as a reason to look, not as an automatic rejection.

Characterization: what are this file’s properties?

Characterization extracts technical properties. MediaInfo reports codec, container, duration, frame rate and audio channels for audio and video files. ExifTool reads and writes embedded metadata in images and many other file types, from camera settings to XMP blocks. JHOVE also characterizes as part of validation. None of these tools is a registry; they produce the measurements that a registry’s vocabulary then gives names to, such as a compression lossiness type or a FourCC inside a video container.

The order is usually identification first, then validation and characterization chosen by format. Running a PDF validator on a WAV file wastes time, so the identification result decides which tool runs next. The file format identification guide walks through that sequence with real signatures and offsets.

Format risk and obsolescence

A format does not become unreadable on a particular day. It drifts: the vendor stops updating the application, the operating system that ran it is retired, the people who understood the internals move on. By the time an archive notices, the files are still there and the bytes are intact, but nothing on a current machine will open them.

The Library of Congress sustainability factors give a practical checklist for judging that drift. They ask about disclosure (is the specification public?), adoption (how widely is the format used?), transparency (can the content be read with basic tools?), self-documentation (does the file carry its own metadata?), external dependencies (does it need particular hardware, fonts or software?), the impact of patents, and technical protection mechanisms such as encryption. A format that scores poorly on several of these deserves attention even if it opens fine today.

Disclosure is the factor a digital format registry can record most precisely. A specification may be fully public, available only to members of a consortium, or reverse-engineered by third parties. A record marked as partial disclosure tells a planner that a converter exists but may not cover every feature.

Formats that became hard to read

WordStar is the example most often cited. It dominated word processing on CP/M and early DOS machines, and its files mix text with control codes and high-bit characters that later software did not interpret consistently. The text can usually be recovered; the formatting is another matter.

Proprietary multimedia authoring formats are a harder case. Interactive CD-ROMs from the 1990s were often built with Macromedia Director, and their content depended on a specific player version running on a specific operating system. Adobe ended support for Flash Player at the end of 2020, which left large amounts of web animation and games depending on emulation projects to stay playable. HyperCard stacks on the classic Macintosh fall into the same pattern.

Early computer-aided design files are frequently mentioned in preservation literature as well, since many CAD packages of the 1980s and 1990s used undocumented binary formats tied to one vendor’s release. Hard numbers on how much engineering data was lost this way are scarce, and the cases tend to surface one project at a time, as anecdotes from engineers and archivists, not as a measured trend.

What these examples share is dependency on a single implementation. The formats that aged best (plain text, TIFF, PNG, WAV) are boring in comparison: published, simple, and implemented independently many times. The preservation file formats guide covers those choices format by format.

Linked data and format records

A format record written as a spreadsheet row works inside one institution. As soon as two systems need to agree on what “PDF/A-1b” means, each record needs a stable identity of its own. That is the reason a modern digital format registry tends to publish its data as RDF.

In RDF, every format, every version and every property gets a URI. A statement such as “this format has the media type application/pdf” becomes a triple: the format’s URI, a property URI, a value. Because the property is also a URI, a second system can look it up and know exactly what was meant. The Dublin Core vocabulary pages on this site are written in that spirit: dcterms:FileFormat is a class with a URI, so a record that uses it says something unambiguous.

How identifiers connect

A single format often carries several identifiers at once: a PRONOM PUID, a media type, a Library of Congress format description identifier, a Wikidata item number. None of them replaces the others. The PUID is what identification tools report; the media type is what web servers and Dublin Core records use; the Wikidata item is where the other identifiers are gathered together.

Wikidata is the practical hub. Many file format items carry a dedicated property for the PRONOM identifier and another for the media type, plus links to specifications and to the software that reads the format. A system that holds only a PUID can follow Wikidata to find the media type, and the reverse. That cross-walk is fragile in places (some items mix several versions, others duplicate one another), so a careful repository stores the identifiers it trusts locally and uses linked data to check them.

Labels need the same care. A format may have a preferred name and several others it travels under, which is why a registry records an alias name separately and why SKOS offers skos:altLabel for alternative labels. Search depends on those alternatives; a user looking for “JFIF” should still find the JPEG record.

Where code lists come in

Format records also point outward to language and place. A text format record may state the languages a specification is written in, and a registry entry for a national standard may name the country that issued it. Those values only interoperate when they come from shared lists, which is why the language code type and country code type pages explain which codes are expected and in what form.

How to use this file format reference

A digital format registry is only as useful as the route into it, and different questions lead to different parts of the site. If you have a file in hand and want to know what it is, the identification tool near the top of this page reads its first bytes, and the identification guide linked above explains what to do when the answer is a container or an unknown.

If the question is which format to keep for the long term, start with the preservation guide mentioned in the section on risk. It compares the usual candidates for images, documents, audio, video and data.

If the task is writing a metadata record, the guide to Dublin Core records covers which elements matter, and the term pages give one definition, one example and the usual mistakes for each property. The dc:format page is the natural starting point for anything to do with file formats in Dublin Core.

For questions about the code lists themselves (whether a country code is still assigned, what a special language code means), the code list pages hold the full tables with every form of each code.

And if the interest is in how this file format reference is written and maintained, the project page describes the editorial approach. One detail is worth knowing before using any term page as a citation: each one gives the official URI of the term, and that URI, not the page address, is what belongs in a metadata record.

04 · format explorer

Byte signatures, family by family

.png
PNG image
@0 89 50 4E 47 0D 0A 1A 0A
.jpg
JPEG image
@0 FF D8 FF
.gif
GIF image
@0 47 49 46 38
.tif
TIFF image, little-endian
@0 49 49 2A 00
.tif
TIFF image, big-endian
@0 4D 4D 00 2A
.webp
WebP image
@8 57 45 42 50
05 · DCMI metadata terms

The fifteen elements of Dublin Core

http://purl.org/dc/elements/1.1/creator

dc:creator

An entity primarily responsible for making the resource.

Read the dc:creator guide
06 · code lists

Codes that metadata depends on

ISO 639-2 special codes

CodeMeaning
zxxNo linguistic content; Not applicable
mulMultiple languages
undUndetermined
misUncoded languages
qaa-qtzReserved for local use
All 487 language entries

ISO 3166-1 examples

Alpha-2CountryAlpha-3
BQBonaire, Sint Eustatius and SabaBES · 535
JMJamaicaJAM · 388
LKSri LankaLKA · 144
MUMauritiusMUS · 480
TGTogoTGO · 768
All 249 country codes
08 · questions

Questions about the digital format registry

What is a digital format registry?

A digital format registry is a structured reference that records what each file format is: its names and versions, its byte signature, its specification, the software that reads it and how it relates to other formats. Archives and libraries use it to identify files and to judge how well they will last.

How does a format registry identify a file?

It compares the first bytes of the file with known signatures (magic numbers) at fixed offsets, for example 89 50 4E 47 at byte 0 for PNG. Container formats need a second look inside the package, which is what tools such as DROID and Siegfried do against PRONOM.

Is the file extension enough to know the file format?

No. An extension is a naming convention that anyone can change or drop. Byte signatures are far more reliable, and many formats share an extension or use several, so a digital format registry records both and treats the extension as a hint.

Which digital format registries exist today?

The best known are PRONOM, maintained by The National Archives of the United Kingdom, the Library of Congress Sustainability of Digital Formats site, and the file format items in Wikidata. Many institutions also keep local registries that add their own policy decisions.

What is a PRONOM PUID?

A PUID is the persistent identifier PRONOM assigns to a format version, written like fmt/11 or x-fmt/111. Identification tools report the PUID, which lets a preservation system store an exact format version in its metadata.

What does dc:format record in Dublin Core?

dc:format records the file format, physical medium or dimensions of a resource. For digital files the recommended value is a media type such as application/pdf, ideally drawn from the IANA registry, which DCMI names as the IMT encoding scheme.

What is the difference between Dublin Core and DCMI Metadata Terms?

Dublin Core usually means the fifteen original elements in the dc: namespace. DCMI Metadata Terms is the larger dcterms: vocabulary that repeats those fifteen with formal ranges and adds refinements, classes and encoding schemes such as dcterms:created, dcterms:MediaType and dcterms:ISO3166.

What does the language code zxx mean?

zxx is the ISO 639-2 code for “No linguistic content; Not applicable”. It is the right value for instrumental music, photographs, numeric datasets and other resources where no language applies, and it is different from und, which means the language has not been determined.

How many ISO 3166-1 country codes are there?

ISO 3166-1 currently assigns 249 officially assigned codes, each available as a two-letter alpha-2 code, a three-letter alpha-3 code and a three-digit numeric code. The full list with all three forms is in the country code reference on this site.

Which file formats are best for long-term preservation?

Formats that are openly documented, widely supported and free of encryption or external dependencies hold up best: uncompressed or losslessly compressed TIFF and PNG for images, PDF/A for documents, WAV or FLAC for audio, FFV1 in Matroska for video and CSV or SQLite for data.

Does the identification tool upload files?

No. The tool on this page reads only the first 32 bytes of the chosen file inside the browser and compares them with the signature list. Nothing is uploaded or stored, so it works offline once the page has loaded.

Why does a ZIP signature also match DOCX and EPUB files?

DOCX, XLSX, PPTX, EPUB and ODF documents are ZIP packages with a defined set of files inside. Their outer signature is the ZIP one; telling them apart requires reading the package contents, such as the mimetype entry of an EPUB or the content-types file of an Office document.

What is a magic number in a file?

A magic number is a fixed sequence of bytes, usually at the very start of a file, that a format uses to announce itself. PDF files start with %PDF, GIF files with GIF8, and ZIP files with PK followed by two control bytes.

How do vocabularies like FOAF and SKOS relate to format metadata?

They supply properties that format records borrow: FOAF for people, agents and the pages about them, SKOS for preferred and alternative labels and for editorial or change notes. A registry built on linked data mixes them with DCMI terms instead of reinventing each property.

Who runs Formatory?

Formatory is an independent editorial publication written by an anonymous editorial team. It explains formats, metadata terms and code lists for archivists, librarians, developers and anyone who has to keep digital files usable.