- Version: 0.5.0
- Date: 2023-Jan-06
This specification document is (C) 2022-2023 Even Rouault and licensed under the CC-BY-4.0 terms.
Note: the scope of the copyrighted material does, of course, not extend onto any source or binary code derived from the specification.
A Seek-Optimized ZIP file (SOZip) is a ZIP file that contains one or several Deflate-compressed files that are organized and annotated such that a SOZip-aware reader can perform very fast random access (seek) within a compressed file.
SOZip makes it possible to access large compressed files directly from a .zip file without prior decompression. It is not a new file format, but a profile of the existing ZIP format, done in a fully backward compatible way. ZIP readers that are non-SOZip aware can read a SOZip-enabled file normally and ignore the extended features that support efficient seek capability.
This specification is intended to be general purpose / not domain specific.
SOZip was first developed to serve geospatial use cases, which commonly have large compressed files inside of ZIP archives. In particular, it makes it possible for users to read large Geographic Information Systems (GIS) files using the Shapefile, GeoPackage or FlatGeobuf formats (which have no native provision for compression) compressed in .zip files without prior decompression.
Efficient random access and selective decompression are a requirement to provide acceptable performance in many usage scenarios: spatial index filtering, access to a feature by its identifier, etc.
The SOZip optimization relies on two independent and combined mechanisms:
The first mechanism is the generation of a Deflate compressed stream that is structured in such a way that it contains data chunks that can be compressed and uncompressed independently from preceding and proceeding chunks in the compression stream. Conceptually, such a compressed file could be split into multiple independently compressed files, but from the point of view of a non-SOZip-aware ZIP reader, it will be a fully single legit compressed stream for the whole file. The chunking relies on the block flush mechanisms of the ZLib library, an example of which is the pigz utility with its --independent option. Block flushes are done at a regular interval of the input uncompressed stream, with the consequence of slight degradation of the compression rate. In the rest of this document, this interval is called chunk size. A typical value for it is 32 kilobytes.
The second mechanism is the creation of a hidden index file containing an array that maps file offsets of the uncompressed file, at every chunk size interval, to the corresponding offset in the Deflate compressed stream. This index is the structure that allows SOZip-aware readers to skip about throughout the file.
The below diagram shows the organization in high level records of a SOZip-enabled ZIP consisting of a SOZIP-optimized file my.gpkg:
my.gpkgfile preceded by its local header.my.gpkg.sozip.idxindex file preceded by its local headerCentral directory with an entry corresponding to
my.gpkg(and none for the index file)End of Central directory
If we zoom on the content of those 2 files, we can see that:
the Deflate stream of
my.gpkgconsists in many concatenated independent chunks.
the
.my.gpkg.sozip.idxindex file contains the offsets to the beginning of each chunk.
A ZIP file is said to be SOZip-enabled if it contains one or several Deflate compressed files meeting the following requirements, in additions to the requirements of the .ZIP File Format Specification.
A SOZip-enabled file may contain a mix of SOZip-compressed and regular compressed or uncompressed files.
A file may be SOZIP-compressed only if its uncompressed size is strictly greater than the chunk size (otherwise there is no point in doing the SOZip optimization).
A SOZip-compressed file MUST be created with
compression_method = 8(Deflate).A SOZip-compressed file MUST have a corresponding local file header and a central directory file header.
Those headers MAY use extended fields. Typically for the ZIP64 extension if the compressed and/or uncompressed size of a file exceeds 4 GB. And/or the "Info-ZIP Unicode Path Extra Field" file name extension.
A SOZip writer MUST issue a call to
deflate()ZLib method with theZ_SYNC_FLUSHmode, followed by a call with theZ_FULL_FLUSHflag, at a fixed interval (called chunk size) of the data read from the input uncompressed stream.Z_SYNC_FLUSHandZ_FULL_FLUSHare not required (but may be used) for the final chunk, whose size may be smaller or equal to the chunk size. However the last call todeflate()to encode the last chunk MUST be done with theZ_FINISHflag, to finalize a valid Deflate stream.Note: an explanation of the
Z_SYNC_FLUSHandZ_FULL_FLUSHmode can be found at https://www.bolet.org/~pornin/deflate-flush-fr.htmlThe writer MUST collect the offset of each chunk, except for the initial chunk size whose offset is zero. Offsets MUST be relative to the start of the compressed data.
Note: a pseudo-code (among many possible variations) written in C++, using zlib, can be found in Annex E
Hidden index file
The index file MUST be stored as a uncompressed file.
The index file name MUST be :
${path_to_filename}/.${filename}.sozip.idxwhere${path_to_filename}is the name of the directory if${filename}contains directory paths. For examplemy_dir/.rivers.gpkg.sozip.idxif the filename stored in the archive ismy_dir/rivers.gpkg.sozip.idx- or
.${filename}.sozip.idxif there is no directory path in the filename. For example.rivers.gpkg.sozip.idx
Note the leading dot ('.') character preceding the index filename, to indicate its hidden status.
The index file MUST be preceded by a ZIP local file header (cf paragraph 4.3.7 of the .ZIP File Format Specification)
That local file header MAY use extended fields. Typically for the "Info-ZIP Unicode Path Extra Field" file name extension. If the local file header of the indexed file uses the "Info-ZIP Unicode Path Extra Field" extension, the local file header of the index file MUST also use the "Info-ZIP Unicode Path Extra Field" extension.
That local file header MUST be immediately placed after the compressed file.
That file MUST NOT be listed as a central directory file header, to remain invisible.
The hidden index file is made of a 32-byte header followed by a section of varying size (called offset section) which contains the values of the offsets collected during the generation of the Deflate-compressed data.
In the following, uint32 is a 32-bit unsigned integer, encoded in little-endian order (least significant byte first). uint64 is a 64-bit unsigned integer, encoded in little-endian order.
| Offset | Type | Name | Comment |
|---|---|---|---|
| 0 | uint32 | version | Version number. |
| 4 | uint32 | skip_bytes | Number of bytes to skip after header. |
| 8 | uint32 | chunk_size | Chunk size in bytes. |
| 12 | uint32 | offset_size | Size in bytes of an entry in offset section |
| 16 | uint64 | uncompress_size | Size in bytes of the uncompressed file. |
| 24 | uint64 | compress_size | Size in bytes of the compressed file. |
Specification of fields:
version: MUST be set to 1 for this specificationskip_bytes: number of bytes between the end of the header and the beginning of the offset section. Generally set to 0. This could be set to a non-zero value to store extra content, unspecified currently. SOZip readers SHOULD skip over such extra content.chunk_size: Interval, in uncompressed stream, at which Z_SYNC_FLUSH + Z_FULL_FLUSH are performed. It MUST not be zero (a value of 4096 or bigger is strongly recommended). A value lower than 100 MB is strongly recommended for performance and compatibility with SOZip readers. 32 KB is a generally safe default value.offset_size: Number of bytes to encode each entry of the offset section. This MUST be 8 (uint64 values).uncompress_size: Size in bytes of the uncompressed file (not the index, but the file subject to SOZip compression). This field is redundant with other information found in the local and central directory file headers of the compressed file, and is provides a reader a consistency check of the SOZip index with the compressed file.uncompress_sizemust be strictly greater thanchunk_sizecompress_size: Size in bytes of the compressed file (not the index, but the file subject to SOZip compression). This field is redundant with other information found in the local and central directory file headers of the compressed file, and is here so that a reader can check the consistency of the SOZip index with the compressed file.
The offset section MUST contain exactly
(uncompress_size - 1) / chunk_size(floor rounding) entries, each of sizeoffset_sizebytes.Each entry MUST be a uint64 expressing the offset at which a compressed chunk starts. That offset is a relative offset, with respect to the start of the compressed stream (consequently, the indexed file and its index could be potentially relocated within the .zip file without requiring to regenerate the index). The offset of the first compressed chunk MUST be omitted, as always 0.
The first offset value is thus the offset in the compressed stream where uncompressed bytes in the range
[chunk_size, min(2 * chunk_size, uncompressed_size)[are encoded.The second offset value is the offset in the compressed stream where uncompressed bytes in the range
[2 * chunk_size, min(3 * chunk_size, uncompressed_size)[are encoded. And so on.As a consequence of the generation of the index, values in the offset section MUST be in strictly ascending order, and MUST be strictly lower than
compress_size.
The sozip development branch of 
contains:
a
CPLAddFileInZip()C function that can compress a file and add it to an new or existing ZIP file, and enable the SOZip optimization when relevant.an implementation of the VSIGetFileMetadata() method that can be used on a filename of the form "/vsizip//path/to/my.zip/filename/inside" and with domain = "ZIP" to get information if a SOZip index is available for that file.
a modified /vsizip/ virtual file system that can use the SOZip optimization to perform fast random access to a compressed file within a ZIP.
a new command line utility, sozip, that can be used to create a seek-optimized ZIP file, to append files to an existing ZIP file, list the contents of a ZIP file and display the SOZip optimization status or validate a SOZip file.
Updated Shapefile and GeoPackage drivers that can directly generate SOZip-enabled .shz/.shp.zip or .gpkg.zip files.
This development branch is available in the rouault/sozip Docker image.
Examples:
Create a new ZIP file with an input file called in.gpkg:
docker run --rm -it -v $PWD:$PWD rouault/sozip sozip -j $PWD/out.zip $PWD/in.gpkg
Create a SOZip-optimized zip file called out.zip from an existing ZIP file called in.zip.
docker run --rm -it -v $PWD:$PWD rouault/sozip sozip --convert-from=$PWD/in.zip $PWD/out.zip
List the content of a ZIP file and check if files in it are SOZip-optimized:
docker run --rm -it -v $PWD:$PWD rouault/sozip sozip -l $PWD/in.zip
Validate a SOZip file:
docker run --rm -it -v $PWD:$PWD rouault/sozip sozip --validate $PWD/my.zip
Extracting the polygons extracting a georeferenced window of interest from a remote SOZip-optimized GeoPackage file:
docker run --rm -it -v $PWD:$PWD rouault/sozip \ ogr2ogr $PWD/out.geojson /vsizip//vsicurl/http://even.rouault.free.fr/sozip/nz-building-outlines.gpkg.zip \ -spat 1749900 5946100 1741000 5946200
sozipfile is a fork of Python zipfile module, which implements the SOZip implementation in the ZIP reader and writer, and can be used to check if a file within a ZIP is SOZip-optimized.
The sozip development branch of MapServer, when built against a SOZip-capable GDAL, can generate SOZip-enabled output files if the mapfile has a ZIP output format, such as:
OUTPUTFORMAT NAME "OGRGPKGZIP" DRIVER "OGR/GPKG" MIMETYPE "application/zip; driver=ogr/gpkg" FORMATOPTION "STORAGE=memory" FORMATOPTION "FORM=zip" FORMATOPTION "FILENAME=result.gpkg.zip" END QGIS can read efficiently SOZip files when built against a SOZip-capable GDAL, through the use of GDAL /vsizip/ virtual file system.
SOZip allows multithreaded compression of independent chunks. This is for example used in the GDAL implementation.
SOZip allows multithreaded decompression of independent chunks.
For decompression, faster alternatives to zlib can be used, such as libdeflate. This is for example used in the GDAL implementation.
SOZip has excellent backward compatibility. A data producer may deliver a SOZip enabled file with good confidence that nearly all existing ZIP readers can decompress it (at time of writing, we are not aware of ZIP readers that reject a SOZip enabled file.)
Compression efficiency is reduced by the flushes done to isolate chunks. The larger the chunk size, the more efficient the compression, but random seeking will be less efficient due to more data being decompressed.
SOZip inherits all the limitations of the base ZIP format: in particular update in place of a SOZip optimized file requires rewriting the entire ZIP, or appending the updated version of the modified file at the end of the ZIP (with rewriting of the central header records and end of central directory record).
Why use the Deflate compression and not an alternative compression method ?
Deflate has been chosen as it is supported by all existing ZIP implementations. Other compression methods (LZMA, BZIP2, etc.) are supported more sparsely. Furthermore, given that the SOZip optimization results in non-optimal compression rate, it is likely that compression schemes that offer higher compression than Deflate would perform in a suboptimal way, due to the chunking mechanism resetting the dictionary at each chunk boundary.
Why encoding the hidden index as hidden and not visible ?
This design decision has pros and cons.
Pros:
- End-users will not see those indexes which are not directly useful for them.
- Non SOZip-aware software will not be disturbed by a file they don't expect (some readers could for example expect a precise list of files to be in a .zip)
- If the index file was visible, and the SOZip archive was regenerated by a non SOZip-aware writer that does an edit operation to an existing archive by recreating a new file, it would preserve the index file, but could potentially recompress the compressed file without using the chunked Deflate techniques, which could confuse SOZip readers (although a SOZip reader must use information from the header of the SOZip index to check its consistence with the compressed file).
Cons:
- Appending new files to a SOZip-enabled file may cause the SOZip index to be lost when using some non SOZip-aware ZIP writer. However, ZIP implementations that have an append-in-place strategy will generally preserve the hidden index. Refer to Annex D for a list of known implementations that can append-in-place.
Why encoding the hidden index as a file preceded by a local header ?
We have found at least one reader, Java's java.util.zip.ZipInputStream, which operates in a streaming way, and stops its enumeration of the content of a ZIP file at the first encountered content that is not a local header.
Why placing the hidden index after the compressed file and not before ?
Both can make sense. It has been observed though that if a hidden local header (that is not listed in the central directory entries) is located immediately at the beginning of a zip file, the
7ziputility will expose the hidden file. And, combined with the question at the previous paragraph, if the content is not preceded by a local header,7zipwill emit a warning ("The archive is open with offset"). It is also slightly easier to generate the index after the compressed file, given that the content of the index depends on information collected during creation of the chunked Deflate compressed stream. This also makes it potentially possible for a streaming writer to write a SOZip optimized file.Why not compressing the index file ?
Having it uncompressed makes it easier for implementations. And for very large files, having it uncompressed makes it possible to seek at a random location in a truly constant time. A 64 GB compressed file, with a chunk size of 32 KB, requires a 15.6 MB index. For scenarios where a SOZip file is read in a on-demand piece-wise way from network storage, it would be costly in bandwith to have to download and decompress those 15 MB to read the last chunk of the 64 GB compressed file.
SOZip-enabled files have been tested with the following ZIP capable utilities to check the backward compatibility (non exhaustive list!):
Compatible readers:
Info-ZIP unzip command line utility.
libzip: C library for reading, creating, and modifying zip archives
7zipcommand line utility or graphical interface.WinZipgraphical interface.WinRARgraphical interface.Windows Explorer default ZIP reader.
MacOSX default ZIP extractor.
MacOSX
zipinfocommand line utilityark KDE (Linux/Unix typically) graphical interface
file-roller GNOME (Linux/Unix typically) graphical interface
Java's java.util.zip.ZipFile class.
Java's java.util.zip.ZipInputStream class, with the caveat that it sees the hidden index files (being a streaming reader, it only takes into account local file records)
Python zipfile module
GDAL /vsizip/ virtual file system, in all existing GDAL versions, can read SOZip-enabled files. Versions >= 3.7 will be able to take advantage of the SOZip index (earlier verions will ignore it.)
Partially compatible readers:
- zipdetails requires the use of its main branch, or a version greater than 2.108. Currently released versions (2.108 or earlier) will error out on SOZip files, rejecting them because the Local header record of a .sozip.idx file has no matching Central header record.
Compatible writers:
Info-ZIP zip command line utility, used to create / edit zip files, can append new files to a SOZip-enabled file, if using the -g/--grow option, while preserving their existing SOZip-optimization
Python zipfile module can append new files to a SOZip-enabled file, while preserving their existing SOZip-optimization.
GDAL sozip command line utility can create SOZip-enabled files, and append new files to a SOZip-enabled file, while preserving their existing SOZip-optimization.
GDAL /vsizip/ virtual file system, in all existing GDAL versions, can append new files to a SOZip-enabled file, while preserving their existing SOZip-optimization.
However a number of writers, while attempting to "append" to a SOZip-enabled file, actually create a new file from scratch, and will loose the hidden index.
Licensed under CC0 or MIT at the choice of the user (that is feel free to reuse and adapt without any constaint).
// Setup zlib z_stream zStream; memset(&zStream, 0, sizeof(zStream)); int ret = deflateInit2(&zStream, Z_DEFAULT_COMPRESSION, Z_DEFLATED, -MAX_WBITS, 8, Z_DEFAULT_STRATEGY); // TODO: add error checking std::vector<uint64_t> offsets; constuint64_t start_offset = tell_position(compressed_file_handle); bool first_block = true; uint32_t crc32Val = 0; uint64_t uncompress_size = 0; while (true) { // Acquire input data, up to chunk_size bytes std::vector<uint8_t> uncompressed_data = read(uncompressed_stream, chunk_size); uncompress_size += uncompressed_data.size(); if (!first_block && !uncompressed_data.empty()) { // Track the offset in the compressed stream of the beginning of// the new chunk (except for first chunk, and also if the uncompressed// file size was exactly a multiple of chunk_size) offsets.push_back(uint64_little_endian( tell_position(compressed_file_handle) - start_offset)); } first_block = false; // Prepare output buffer (with security margin for very small chunks,// or chunks that compresses poorly) std::vector<uint8_t> compressed_data(uncompressed_data.size() + uncompressed_data.size() / 10 + 16); // Compress data zStream.avail_in = static_cast<uInt>(uncompressed_data.size()); zStream.next_in = &uncompressed_data[0]; zStream.avail_out = static_cast<uInt>(compressed_data.size()); zStream.next_out = &compressed_data[0]; if (uncompressed_data.size() < chunk_size) { ret = deflate(&zStream, Z_FINISH); // TODO: add error checking } else { ret = deflate(&zStream, Z_SYNC_FLUSH); // TODO: add error checking ret = deflate(&zStream, Z_FULL_FLUSH); // TODO: add error checking } // Write output buffer compressed_data.resize(compressed_data.size() - zStream.avail_out); write(compressed_file_handle, compressed_data); // Update crc32 of uncompressed data crc32Val = crc32(crc32Val, &uncompressed_data[0], static_cast<uInt>(uncompressed_data.size())); // Exit loop if no more input dataif (uncompressed_data.size() < chunk_size) { break; } } if (uncompress_size > 0 ) assert (offsets.size() == (uncompress_size - 1) / chunk_size); // TODO: store crc32Val in local and central header records// CleanupdeflateEnd(&zStream);The chunking process, involving Z_FULL_FLUSH, used during SOZip generation makes it possible to decompress chunks in an independent way.
Given uncompressed_offset being an offset of the uncompressed file, multiple of chunk_size, the reader should retrieve in the offset section of the .sozip.idx (stored in an array compressed_offset_table[]), the start and end offsets of the chunk, like below:
assert (uncompressed_offset % chunk_size) == 0); size_t entry_idx = uncompressed_offset / chunk_size; uint64_t relative_start_offset; if (entry_idx == 0) relative_start_offset = 0; else relative_start_offset = compressed_offset_table[entry_idx - 1]; uint64_t relative_end_offset; if (entry_idx < compressed_offset_table.size() ) relative_end_offset = compressed_offset_table[entry]; else relative_end_offset = compress_size; size_t compressed_chunk_size = relative_end_offset - relative_start_offset; seek(compressed_stream, absolute_offset_of_start_of_deflate_stream + relative_start_offset); std::vector<uint8_t> compressed_chunk_data = read(compressed_stream, compressed_chunk_size);Given that each chunk, except the last one, is not a standalone Deflate stream, reading code must be careful to present it in a way that will not confuse the Deflate decoding library. The only precaution to take is to dynamically patch the last flush Deflate block at the end of the compressed chunk to be advertized as the final Deflate block of the chunk.
Given the use of Z_FULL_FLUSH, the last 5 bytes of a chunk (that is not the last one) should be \x00, \x00, \x00, \xFF, \xFF. The patching operation consists in changing the first byte of this 5 byte sequence from \x00 to \x01 as below:
if (compressed_chunk_size >= 5 && memcmp(compressed_chunk_data.data() + compressed_chunk_size - 5, "\x00\x00\x00\xFF\xFF", 5) == 0) { // Tag this flush Deflate block as the last one of the chunk. compressed_chunk_data[compressed_chunk_size - 5] = 0x01; }With the above, if using zlib, a chunk can then be decompressed with:
z_stream zStream; memset(&zStream, 0, sizeof(zStream)); int err = inflateInit2(&zStream, -MAX_WBITS); // TODO: add error checkingsize_t decompressed_size = chunk_size_for_all_chunks_except_last_one; std::vector<uint8_t> decompressed_data(decompressed_size); zStream.avail_in = compressed_chunk_size; zStream.next_in = compressed_chunk_data; zStream.avail_out = decompressed_size; zStream.next_out = decompressed_data; err = inflate(&zStream, Z_FINISH); if (err == Z_STREAM_END && zStream.avail_in == 0 && zStream.avail_out == 0 ) { // success } inflateEnd(&zStream);Or if using libdeflate:
structlibdeflate_decompressor *decompressor = libdeflate_alloc_decompressor(); // TODO: add error checkingsize_t decompressed_size = chunk_size_for_all_chunks_except_last_one; std::vector<uint8_t> decompressed_data(decompressed_size); size_t actual_decompressed_size = 0; if (libdeflate_deflate_decompress( decompressor, compressed_chunk_data, compressed_chunk_size, decompressed_data.data(), decompressed_size, &actual_decompressed_size) == LIBDEFLATE_SUCCESS && actual_decompressed_size == decompressed_size) { // success } libdeflate_free_decompressor(decompressor);All pseudo-code in this annex is licensed under CC0 or MIT at the choice of the user (that is feel free to reuse and adapt without any constaint).
Examples of SOZip-enabled files can be found in the sozip-examples repository.
The following invokation of GDAL's sozip utility generates a dummy SOZip enabled file that contains a tiny file "foo" with "foo" as content, and using a chunk size of 2 bytes.
printf"foo"> foo sozip --overwrite --enable-sozip=yes --sozip-chunk-size=2 foo.zip foo- Dump of the local file record of the SOZip compressed "foo" file:
| Offset | Type | Values | Comment |
|---|---|---|---|
| 0 | uint32 | 0x04034B50 | Local file header signature (for the compressed file) |
| 4 | uint16 | 20 | version needed to extract |
| 6 | uint16 | 0 | general purpose bit flag |
| 8 | uint16 | 8 | compression method (8=deflate) |
| 10 | uint16 | 32168 | last mod file time |
| 12 | uint16 | 22053 | last mod file date |
| 14 | uint32 | 0x8C736521 | CRC32 |
| 18 | uint32 | 16 | compressed size |
| 22 | uint32 | 3 | uncompressed size |
| 26 | uint16 | 3 | filename length |
| 28 | uint16 | 0 | extra field length |
| 30 | byte[] | 'f', 'o', 'o' | filename |
| 33 | byte[] | 0x4A 0xCB 0x07 | deflate block for first chunk (corresponding to 'f', 'o') |
| 36 | byte[] | 0x00 0x00 0x00 0xFF 0xFF | uncompressed block of size 0 encoding a Z_SYNC_FLUSH |
| 41 | byte[] | 0x00 0x00 0x00 0xFF 0xFF | uncompressed block of size 0 encoding a Z_FULL_FLUSH |
| 46 | byte[] | 0xCB 0x07 0x00 | deflate block for second chunk (corresponding to 'o') |
- Dump of the local file record of the uncompressed ".foo.sozip.idx" index file:
| Offset | Type | Values | Comment |
|---|---|---|---|
| 49 | uint32 | 0x04034B50 | Local file header signature (for the index file) |
| 53 | uint16 | 20 | version needed to extract |
| 55 | uint16 | 0 | general purpose bit flag |
| 57 | uint16 | 0 | compression method (0=uncompressed) |
| 59 | uint16 | 32168 | last mod file time |
| 61 | uint16 | 22053 | last mod file date |
| 63 | uint32 | 0x56FEC86C | CRC32 |
| 67 | uint32 | 36 | compressed size |
| 71 | uint32 | 36 | uncompressed size |
| 75 | uint16 | 14 | filename length |
| 77 | uint16 | 0 | extra field length |
| 79 | byte[] | '.', 'f', 'o', 'o', '.', 's', 'o', 'z', 'i', 'p', '.', 'i', 'd', 'x' | filename |
| 93 | uint32 | 1 | SOZip version |
| 97 | uint32 | 0 | Bytes to skip |
| 101 | uint32 | 2 | chunk_size |
| 105 | uint32 | 8 | offset_size |
| 109 | uint64 | 3 | uncompress_size |
| 117 | uint64 | 16 | compress_size |
| 125 | uint64 | 13 | (relative) offset to the start of the 2nd compressed chunk (absolute offset is 33 + 13 = 46) |
- Dump of the central header record describing the "foo" file:
| Offset | Type | Values | Comment |
|---|---|---|---|
| 133 | uint32 | 0x02014B50 | Central file header signature (compressed file) |
| 137 | uint16 | 0 | version made by |
| 139 | uint16 | 20 | version needed to extract |
| 141 | uint16 | 0 | general purpose bit flag |
| 143 | uint16 | 8 | compression method (8=deflate) |
| 145 | uint16 | 32168 | last mod file time |
| 147 | uint16 | 22053 | last mod file date |
| 151 | uint32 | 0x8C736521 | CRC32 |
| 155 | uint32 | 16 | compressed size |
| 159 | uint32 | 3 | uncompressed size |
| 161 | uint16 | 3 | file name length |
| 163 | uint16 | 0 | extra field length |
| 165 | uint16 | 0 | file comment length |
| 167 | uint16 | 0 | disk number start |
| 169 | uint16 | 0 | internal file attributes |
| 171 | uint32 | 0 | external file attributes |
| 175 | uint32 | 0 | relative offset of local header (0 here because this is the first file, and there's no leading content) |
| 179 | byte[] | 'f', 'o', 'o' | filename |
- Dump of the end of central directory record:
| Offset | Type | Values | Comment |
|---|---|---|---|
| 182 | uint32 | 0x06054B50 | end of central dir signature |
| 186 | uint16 | 0 | number of this disk |
| 188 | uint16 | 0 | number of the disk with the start of the central directory |
| 190 | uint16 | 1 | total number of entries in the central directory on this disk |
| 192 | uint16 | 1 | total number of entries in the central directory |
| 194 | uint32 | 49 | size of the central directory (= 182 - 133) |
| 198 | uint32 | 133 | offset of start of central directory with respect to the starting disk number |
| 202 | uint16 | 0 | .ZIP file comment length |
Output of "zipdetails foo.zip" (using main branch of https://github.com/pmqs/zipdetails):
0000 LOCAL HEADER #1 04034B50 0004 Extract Zip Spec 14 '2.0' 0005 Extract OS 00 'MS-DOS' 0006 General Purpose Flag 0000 [Bits 1-2] 0 'Normal Compression' 0008 Compression Method 0008 'Deflated' 000A Last Mod Time 56257DA8 'Thu Jan 5 16:45:16 2023' 000E CRC 8C736521 0012 Compressed Length 00000010 0016 Uncompressed Length 00000003 001A Filename Length 0003 001C Extra Length 0000 001E Filename 'foo' 0021 PAYLOAD J............... 0031 LOCAL HEADER #2 04034B50 Orphan Entry: No matching central directory 0035 Extract Zip Spec 14 '2.0' 0036 Extract OS 00 'MS-DOS' 0037 General Purpose Flag 0000 0039 Compression Method 0000 'Stored' 003B Last Mod Time 56257DA8 'Thu Jan 5 16:45:16 2023' 003F CRC 56FEC86C 0043 Compressed Length 00000028 0047 Uncompressed Length 00000028 004B Filename Length 000E 004D Extra Length 0000 004F Filename '.foo.sozip.idx' WARNING! Expected Zip header not found at offset 0x5D Skipping 0x24 bytes to Central Directory... 0085 CENTRAL HEADER #1 02014B50 0089 Created Zip Spec 00 '0.0' 008A Created OS 00 'MS-DOS' 008B Extract Zip Spec 14 '2.0' 008C Extract OS 00 'MS-DOS' 008D General Purpose Flag 0000 [Bits 1-2] 0 'Normal Compression' 008F Compression Method 0008 'Deflated' 0091 Last Mod Time 56257DA8 'Thu Jan 5 16:45:16 2023' 0095 CRC 8C736521 0099 Compressed Length 00000010 009D Uncompressed Length 00000003 00A1 Filename Length 0003 00A3 Extra Length 0000 00A5 Comment Length 0000 00A7 Disk Start 0000 00A9 Int File Attributes 0000 [Bit 0] 0 'Binary Data' 00AB Ext File Attributes 00000000 00AF Local Header Offset 00000000 00B3 Filename 'foo' 00B6 END CENTRAL HEADER 06054B50 00BA Number of this disk 0000 00BC Central Dir Disk no 0000 00BE Entries in this disk 0001 00C0 Total Entries 0001 00C2 Size of Central Dir 00000031 00C6 Offset to Central Dir 00000085 00CA Comment Length 0000 WARNINGS * Expected Zip header not found at offset 0x61, skipped 0x24 bytes Done 