Data compression in Modern Application
3 likes1,983 views
The document discusses the importance of compression, particularly focusing on lossless codecs and their implementation in systems like Hadoop for optimizing data transfer. It emphasizes the necessity of understanding data flows and carefully selecting compression codecs to enhance web delivery and overall system performance. The key takeaway is that effective compression usage is essential in modern data handling.
Data compression in Modern Application
- 2. Compression as we used to know
- 4. Compression classification We will focus on lossless codecs
- 5. There is always trade off
- 6. There is always trade-off.. ● … ● Is there? It is all about the need and implementation
- 8. Hadoop M/R Data transfer stage
- 9. Terasoft results hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop- examples.jar terasort - Dmapred.compress.map.output=true - Dmapred.reduce.slowstart.completed.maps=0.95 - Dmapred.reduce.tasks=100 - Dmapred.map.output.compression.codec=org.apach e.hadoop.io.compress.GzipCodec teragen-input terasort-output
- 10. Compression for web Demo: using compression to optimize web delivery
- 13. Wrap up ● Compression usage is must in modern systems ● Get to know your data and flows ● Choose codecs carefully. You may no penalty at all to use them!