Click here to Skip to main content
15,885,914 members
Please Sign up or sign in to vote.
1.00/5 (2 votes)
See more:
we are working on huge data , need to compress morethan 5000 characters of data into 100 Bytes while storing in the DB and while retriving need to uncompress the same .

What I have tried:

not able to think any possible solution
Posted
Updated 7-Feb-20 0:37am

 
Share this answer
 
Quote:
need to compress more than 5000 characters of data into 100 Bytes

Unless your 5000 chars are full of repeated sequences, a ratio of 1/50 is impossible to reach.
You need to understand that compression is not infinite and depend on the data to compress. In some cases, the data can be impossible to compress.
You need to study and understand how compression works.
Data compression - Wikipedia[^]
 
Share this answer
 
It's very unlikely that any generic compression algorithm will get that kind of compression ratio: 5000:100 is compressing data to 1/50th of the original size and unless the data is extremely high in redundancy to start with, that's just not a realistic target.
If I create a 5000 character text string (and text is the easiest thing to compress) and use WinZip on highest compression to reduce it, I don't get anywhere close: 1,566 bytes which is a ration of 5000:1566 or about 1/3rd.

And compression cannot guarantee any "final size" from "any input of 5K" - it doesn't take much to massively change the end result of a compression; many forms of data are already compressed (.JPG, .MP4, .ZIP, and .RAR for example all compress very badly) and further compression may even increase the size of the resulting file as it adds "overhead" information which explains how to decompress it when the time comes.

What I'd suggest is storing the "raw" data as a file, and storing the file location within your 100 characters: because you have control over the filename and path, you can guarantee it will always fit.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900