fig (aka:[phil]) @bad-example.com · Apr 11 1 / 2
dictionary training is sometimes significant, and very tricky rocks has an amount-of-training-data config, you *need* to train enough if using that setting, or you'll get zero benefit (or sometimes negative) leaving it unset (only setting dict size) works consistently but less well
1 likes 1 replies
?
Replies
seems probable that larger dictionaries could have offered better gains for bigger block sizes than what these charts show, but the main takeaway is that dictionaries benefit small data most. crossover seems mostly around 32/64k dictionaries also benefit higher zstd compression levels a bit more