Questions tagged [compression]
The compression tag has no summary.
68 questions
5votes
5answers
677views
How does data store compression speed up data warehouses?
I often see the claim that various data warehouse/analytical database systems derive significant performance benefits from compressing their data stores. On the face of it, though, this seems to be ...
- 83.2k
4votes
2answers
620views
How should I handle different hashes of identical files in .zip archive with different 'last changed' date?
We store zipped files in the storage of a cloud provider which contain certain fields (metadata). These files are derived from other, larger files. Every time we (re)generate these files, their 'last ...
- 269
0votes
5answers
235views
Load and process (compressed) data from filesystem in the blink of an eye
We have a huge amount of queries hitting our API that request a minor or major extract of some huge files lying around on our mounted hard drives. The data needs to be extracted from the files and ...
- 493
30votes
7answers
17kviews
How can lossless compression ever exist?
If all data is essentially just a bit string, then all data can be represented as a number. Because a compression algorithm, c(x), must reduce or keep the same length of the input, then the compressed ...
- 475
1vote
1answer
2kviews
How to remove unused code from a jar file? [closed]
I have a jar file, for example foo.jar. My code contains a lot of libraries (almost 75 jar dependencies). I am not using anything like maven or gradle, I'm just using pure java with pure jar files as ...
- 23
2votes
1answer
1kviews
Are Flate compression in PDF and Deflate different algorithms?
I'm trying to make a program that produces pdf files. I've been studying the pdf format specification and specific pdf files whose format I'm trying to mimic. I found the line /FlateDecode in these ...
0votes
1answer
350views
Short and compact barcode
I am writing a c# program where I need to print a lot of small barcodes in a 100x100 grid on a piece of paper. I then scan/photograph the paper and read the barcodes again. Each barcode only need to ...
- 127
4votes
3answers
261views
Is it possible to store N bits of unique combinations, in N-1 bits? If not; why does MD5 get reprimanded for collissions?
Regarding cryptography and the issue of collisions, I posed a question as to whether it was ever possible to store every single possible combination of a bit array of a particular size, in a bit array ...
- 3,633
7votes
2answers
632views
some misunderstanding in concept of Huffman algorithm
What is difference between Average length of codes and Average length of codewords in Huffman Algorithm? is both the same meaning? I get stuck in some facts: I see a fact that marked as False: for a ...
- 183
0votes
1answer
80views
Design Question: Compression with Fast Lookup
I have a multiple files (one per CountryCode) which all get ~5000 entries added to it per day. Each entry in the file looks like (256chars max): {countryCode_customerId:{"ownerId": "...
7votes
2answers
829views
How does conditional compilation impact product quality, security and code complexity? [closed]
Software libraries targetting resource constrained environments like embedded systems use conditional compilation to allow consumers to shave space by removing unused features from the final binaries ...
- 435
-2votes
2answers
551views
Alternative to RLE for short, infrequent runs
I have 3 number arrays that I need to encode into a URL through query parameters. Example: http://localhost:3000/?r=133223333302302040&y=10000000000000000000&b=13333332002100122331 This is a ...
- 127
2votes
2answers
552views
Compressing EBCDIC file vs UTF8
Today I went across a weird case for which I have no explanation, so here I am. I have two files with identical content, but one is encoded in UTF-8 and the other one is in IBM EBCDIC. Both of them ...
- 137
11votes
4answers
2kviews
How to review sql changes more effectively? [duplicate]
From my experience, sql code changes almost always tend to be NOT incremental: someone creates a new stored procedure, or modifies an entire embedded sql query for optimization purposes, or creates a ...
- 235
0votes
2answers
123views
Is it possible to transfer data with a really unique seed of a psudo random number generator
I have thinking about this idea for over 5 years and i don't have the complete technical knowledge to fully grasp the idea I'm having. The premise of the idea is to have an extremely high base number ...
- 105
