We frequently receive questions about ensuring file accessibility and readability in the future. In digital preservation and data archiving, best practices recommend storing files in recognized, well-documented, open-source formats.
The process of converting files into these well-known formats is called normalization, and we refer to it as future proofing of files.
Normalization is known as the process of;
“Converting copies of original files to one of a small number of non-proprietary, widely used, and preservation-friendly formats during ingest. Normalization standardizes ingested material into a subset of formats stored by an archive and allows the archive to avoid managing many formats into the future.”
[1] https://learn.scholarsportal.info/all-guides/handling-digital-archives/concepts/
Normalization is crucial for ensuring data is stored in file formats with the highest likelihood of long-term survival. This methodology is followed by major national archives worldwide, including the Library of Congress (LoC), British National Archive, Open Preservation Foundation, Digital Preservation Coalition (DPC), and the Norwegian National Archive. Their key responsibility is to assess and establish file format policies for optimal practical outcomes, which may include modifying file sizes and properties.
Piql's approach to file format normalization (future proofing) is straightforward: it is up to you. As the data owner, you decide whether and how to future-proof your files—and we're here to help. You might have valid reasons to keep data in its original format, or your data may already be well-prepared for piqlFilm storage. We're always available to provide guidance and answer your questions.
With piqlConnect we now offer an automatized method of normalizing your files according to the best global standards and thus ensuring perpetual life to your memories. So how does it work?
First you create a package – this is the area where you can upload files and folders like on your own computer. When creating the package, you can choose whether you would like to future proof your data or not.
If you choose to future proof your data, all the files and folders in your package will be scanned upon upload and acted upon. We classify files into one out of three categories of file formats:
The Preferred file formats are those ideally suited for long-term storage and commonly chosen as primary archival formats by leading memory institutions worldwide. You will find a list of preferred formats below.
The Acceptable file formats are widely known and well-documented formats that memory institutions typically accept. If your files are already in these formats, they are suitable for archiving and don't require normalization.
The Unsupported category includes file formats that aren't recognized for long-term storage by the archival community. You may have reasons to keep files in these formats. If so, you must disable future-proofing when creating your package—a decision that cannot be reversed later.
If a file format is marked as 'Unsupported' in piqlConnect, we will attempt to convert it to the appropriate Piql future-proof format if we support that file type. For unsupported formats that we cannot convert, your file will be labeled as 'Unsupported' in the package. In this case, you have three options:
! When future-proofing your files in the package, you can choose to either keep both files (the original and the converted version) or keep only the converted file. If you choose to keep both files, your total data volume will be larger. The software displays the original and future-proofed files together for easy reference.
Below is a list of supported file formats and their classifications as Preferred, Acceptable, or Unsupported. Any file format not listed here will be automatically labeled as 'Unsupported' by our system.