gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
10115 commits 28 branches 278 tags 135 MiB
Commit graph

118 commits

Author SHA1 Message Date
chrislu c8090b1f99 send assignments to newly connected subscribers 2024-01-17 22:44:39 -08:00
chrislu c616081ab9 refactor 2024-01-17 22:29:59 -08:00
chrislu bc8d2a01cc create local topic partition from config on filer 2024-01-17 00:15:52 -08:00
chrislu fdf0ea8e11 passing partition info 2024-01-16 09:30:57 -08:00
chrislu be0c426dc7 simplify to LookupTopicPartitions(topic) 2024-01-16 09:30:46 -08:00
chrislu db3670a3a5 simplify api 2024-01-16 08:55:47 -08:00
chrislu 34a78ffad0 remove isForPublish from LookupTopicBrokers 
also adds a return parameter: whether the topic exists or not
 2024-01-16 08:52:42 -08:00
chrislu 3795d8dca8 release local topic partition if no publisher and subscribers 2024-01-16 08:43:07 -08:00
chrislu f782165638 rename 2024-01-16 08:40:45 -08:00
chrislu dd1ec70e62 rename 2024-01-16 08:40:23 -08:00
chrislu ba73199174 reuse local partition 2024-01-15 21:22:41 -08:00
chrislu 7e6497cc1c adjust publisher subscriber 2024-01-15 20:42:46 -08:00
chrislu 026c54a9bb fix publisher 2024-01-15 00:20:21 -08:00
chrislu fa59a5d67e read from disk if not in memory 2024-01-15 00:20:12 -08:00
chrislu 3aa3991f0f adjust client side logs 2024-01-11 23:08:53 -08:00
chrislu 7afaad31a4 wait on local topic partition creation 2024-01-11 23:08:02 -08:00
chrislu 61dbdd0ff6 log ctrl messages 2024-01-11 23:07:32 -08:00
chrislu f750a5e03b passing timestamp 2024-01-11 23:03:55 -08:00
chrislu 45994641e9 lint 2024-01-11 23:03:35 -08:00
chrislu 6a7a679137 passing partition generation timestamp 2024-01-10 22:36:17 -08:00
chrislu 8af61dec91 avoid empty spaces in file name 2024-01-09 09:00:54 -08:00
chrislu aed54eda61 refactor 2024-01-09 08:01:01 -08:00
chrislu d51efddf5c flush to disk 
Need to do: read from disk
 2024-01-08 23:27:02 -08:00
chrislu 49428a303b add batch index for each memory buffer 2024-01-08 00:03:08 -08:00
chrislu d0d24f1e40 rename functions 2024-01-05 17:10:43 -08:00
chrislu aa5b6a8187 adjust logs 2024-01-05 15:35:32 -08:00
chrislu 496fc8fbbf refactor 2024-01-05 15:35:19 -08:00
chrislu e8611ed85d subscribe with partition offset 2024-01-05 15:24:14 -08:00
chrislu 531f854af2 rename functions 2024-01-05 15:16:53 -08:00
chrislu f8787a9761 rename functions 2024-01-05 15:16:41 -08:00
chrislu ddd0fde094 rename functions 2024-01-05 15:16:01 -08:00
chrislu ee41dbb7fc rename functions 2024-01-05 15:14:25 -08:00
chrislu 47a4963d7a subscription start from specified timestamp 2024-01-03 15:57:36 -08:00
chrislu 35869b5c80 subscriber can be notified of the assignment change when topic is just configured 
Next: Subscriber needs to read by the timestamp offset.
 2024-01-03 13:30:30 -08:00
chrislu efb695fd93 lookup existing topic partitions 2024-01-02 17:29:35 -08:00
chrislu 8b4cd50aac log errors 2024-01-01 19:09:17 -08:00
chrislu 32bc8d6a38 adjust wait time 2023-12-31 17:42:44 -08:00
chrislu 458ddbf919 clean up dead code 2023-12-31 13:25:33 -08:00
chrislu 2845230329 passing broker into the assignments 2023-12-28 21:13:49 -08:00
chrislu 093fdc1621 subscriber can get assignments 2023-12-28 20:35:15 -08:00
chrislu c950a40aad subscriber keep connected to the balancer 2023-12-28 11:56:37 -08:00
chrislu e0727071c8 go fmt 2023-12-22 11:33:50 -08:00
chrislu 034db049a0 comment fix 2023-12-22 11:33:00 -08:00
chrislu 61b05fad95 fix test 2023-12-11 20:57:29 -08:00
Chris Lu 580940bf82
Merge accumulated changes related to message queue (#5098) 
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
 2023-12-11 12:05:54 -08:00
chrislu 17710e1ecb adjust logs 2023-11-15 18:11:35 -08:00
chrislu de0b969b36 Revert "rename" 
This reverts commit 35b5264ab7.
 2023-10-12 20:28:11 -07:00
chrislu 35b5264ab7 rename 2023-10-11 21:44:56 -07:00
chrislu bf13f3ced7 add subscriber coordinator 2023-10-02 01:02:27 -07:00
chrislu 734178093e refactor TopicPartition struct 2023-10-02 01:01:45 -07:00