Feb 21, 2022
Ep11. Designing Data-Intensive Applications - Partitioning
这一期我们讨论Designing Data-Intensive Applications书中partitioning这一章的学习笔记。
🔴 这一期偏重技术话题,我们会用很多英文表述技术性专有名词。之前有朋友反馈过中英夹杂对大家收听不方便,希望在意的朋友见谅。如果有不准确或者过时的地方欢迎指正。
# Show Notes
* 📕 Designing Data-Intensive Applications
* What is partitioning?
* A partition is a division of a logical database or its constituent elements into distinct independent parts.
* Main reason: scalability - the query load can be distributed across many processors.
* Youtube / Vitess scaling story
* Single MySQL → Add read replica → Write can’t catchup up → Partition
* How to partition?
* Partitioning by Key Range (e.g., Bigtable)
* Assign a continuous range of keys to each partition
* Pro: range scan is easier, data locality
* Cons: certain access patterns can lead to hot spots (timestamp)
* Cons: finding split points and managing rebalancing is hard
* Partitioning by Hash
* Good hash function: uniformly distribute keys
* Con: no easy range queries
* Cassandra does KKV (partitioning key, sort key, value)
* Hot spots: 3% of Twitter's Servers Dedicated to Justin Bieber
* Secondary indexes: Local index
* Efficient write, expensive read
* ElasticSearch
* Secondary indexes: Global index
* Efficient read, expensive write
* Using Global Secondary Indexes in DynamoDB (这里说错了,DynamoDB 支持 20 global secondary indexes per table)
* Rebalancing partitions
* Move loads to other nodes
* Fixed number of partitions
* New node steals partitions from every existing node
* Notion: 480 partitions
* Dynamic partitioning
* 📈: split partition into 2
* 📉: merge 2 partitions into 1
* Fixed number of partitions per node
* https://www.datastax.com/blog/new-token-allocation-algorithm-cassandra-30
* Operations: full automatic (dangerous) / semi-automatic / full manual (tedious)
* Request Routing
* 3 approaches: nodes talk to each other, separate routing tier, smart client
* Separate coordination service such as ZooKeeper
* Notes by xg
# 联系方式
* 官网: eng.cafe
* 微信公众号: Eng Cafe
* Twitter: @engcafefm
* Youtube: Eng Cafe
* 小宇宙播客
* 泛用型播客客户端: eng.cafe/subscribe
* Email: hi@eng.cafe