mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2024-01-19 02:48:24 +00:00
580940bf82
* balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * tracking topic=>broker * merge * comment
44 lines
1.2 KiB
Go
44 lines
1.2 KiB
Go
package pub_balancer
|
|
|
|
/*
|
|
Sequence of operations to ensure ordering
|
|
|
|
Assuming Publisher P10 is publishing to Topic Partition TP10, and Subscriber S10 is subscribing to Topic TP10.
|
|
After splitting Topic TP10 into Topic Partition TP11 and Topic Partition TP21,
|
|
Publisher P11 is publishing to Topic Partition TP11, and Publisher P21 is publishing to Topic Partition TP21.
|
|
Subscriber S12 is subscribing to Topic Partition TP11, and Subscriber S21 is subscribing to Topic Partition TP21.
|
|
|
|
(The last digit is ephoch generation number, which is increasing when the topic partitioning is changed.)
|
|
|
|
The diagram is as follows:
|
|
P10 -> TP10 -> S10
|
|
||
|
|
\/
|
|
P11 -> TP11 -> S11
|
|
P21 -> TP21 -> S21
|
|
|
|
The following is the sequence of events:
|
|
1. Create Topic Partition TP11 and TP21
|
|
2. Close Publisher(s) P10
|
|
3. Close Subscriber(s) S10
|
|
4. Close Topic Partition TP10
|
|
5. Start Publisher P11, P21
|
|
6. Start Subscriber S11, S21
|
|
|
|
The dependency is as follows:
|
|
2 => 3 => 4
|
|
| |
|
|
v v
|
|
1 => (5 | 6)
|
|
|
|
And also:
|
|
2 => 5
|
|
3 => 6
|
|
|
|
For brokers:
|
|
1. Close all publishers for a topic partition
|
|
2. Close all subscribers for a topic partition
|
|
3. Close the topic partition
|
|
|
|
*/
|