Exercise 2: A Less Skewed Country Distribution — Possible Solution ==================================================================== THE CHANGE ------------------------------ Changed the population weights from this chapter's own 80/7/7/6 (heavily dominated by one country) to a more even 30/25/25/20 across the same four countries (UK, France, Germany, Spain), keeping the identical shard_by_country() hashing function unchanged. RESULT ------------------------------ less-skewed (30/25/25/20) distribution per shard: {3: 468, 1: 532} new spread: 64 | this chapter's own 80/7/7/6 spread was: 726 DOES THE HOTSPOT SHRINK? YES, DRAMATICALLY - BUT NOT TO ZERO ------------------------------ The spread dropped from 726 down to 64 - a genuine, large improvement, confirming that reducing the population skew across country values does directly reduce the resulting shard imbalance. BUT AN HONEST, DEEPER PROBLEM REMAINS ------------------------------ Records still landed on only 2 of the 4 available shards (shard 1 and shard 3) - shards 0 and 2 received ZERO records in BOTH this exercise and this chapter's own original test, regardless of how the population was weighted across the four countries. This is because shard_by_country() only ever sees 4 DISTINCT input values (the four country strings) - hashing four fixed strings mod 4 always produces the exact same four bucket assignments, no matter how many records carry each string. Changing the POPULATION weights changes how many records land in each of those fixed buckets, but it can never change WHICH buckets get used at all. WHY THIS IS A GENUINELY DIFFERENT, MORE FUNDAMENTAL FINDING ------------------------------ This chapter's own text framed the country-key problem as being about skewed real-world data. This exercise reveals a second, independent problem: a shard key with LOW CARDINALITY (few distinct possible values) can never use more shards than it has distinct values, no matter how evenly those values happen to be distributed across users. A shard key needs both enough distinct values AND a reasonably even population across them - user_id (this chapter's own GOOD key) has both; country, even at its best, only ever has the first. WHY THIS WORKS AS AN ANSWER ------------------------------ The distribution is changed while keeping the hashing function and the number of shards identical to this chapter's own setup, the improved spread is reported and compared directly against this chapter's own 726 figure, and the deeper, still-unsolved cause (only 4 distinct input values) is identified rather than treating the reduced spread as full resolution of the problem.