Efficient and Scalable Synchronization via Generalized Cache Coherence
Extend cache coherence to synchronization
Background
rack-scale compute-memory disaggregation
-> Needs cache-coherent shared-memory abstractions
- Page-based cache-coherent shared memory: local DRAM on compute blades serves as cache for disaggregated memory, and these caches are kept coherent over high-speed Ethernet
Lock is the problem.
rack-scale disaggregation: high latency of inter-compute & compute-memory links -> lock algorithm became inefficient -> lead to poor application performance scaling -> degrading performance for real-world synchronization-heavy workloads by as much as 1000×
Root cause
due to redundant inter-cache coherence messages over higher-latency Ethernet links. Minimizing the number of coherence transactions in synchronization primitives is essential for scaling the performance of parallel applications on disaggregated memory, since each transaction incurs multiple Ethernet round trips.
Priorworks
- Lock layered atop cache coherence : infer redundant cocherence transactions -> bad scalability.
- Lock bypassing cache coherence : preclude key optimization, high overheads
Goal
synchronization for disaggregated shared memory. but efficient & Scalable
Key Idea
synchronization is a generalization of cache coherence in time and space. cache coherence and lockbased synchronization strive for the same goal: single-writer, multi-reader (SWMR) invariant over some shared state, -> We can adapt the same cache coherence principle to synchronization. -> Lock via generalized cache coherence.
Wait queues : temporal generalization of cache coherence
- Cache coherence protects an “individual” memory access.
- A lock protects a whole “group of” memory accesses/instructions.
Variable-size cache lines : spatial generalization
- Cache-coherence protocols ensure the SWMR invariant at a cache line granularity (i.e., a 4KB page),
- Reader-writer locks require this property for shared states of arbitrary sizes.
Extending existing coherence protocols to support these generalizations
- Eliminate the redundant communication seen in a layered design.
- Can utilize existing optimization for cache coherence protocols.
- Improve lock scalability
Solution
1.a novel Generalized cache Coherence Protocol (GCP)
2. Soul, an end-to-end system implementation of GCP atop a disaggregated shared-memory platform.
Evaluation
against state-of-the-art locks,
- improves the performance of unmodified real-world applications at scale by 1–2 orders of magnitude
- incurring < 8% storage overhead.
Thoughts & Questions
If 2 compartments have similar characteristic but seperated in different layer, we can consider unifying