Commit graph

59 commits

Author SHA1 Message Date
Konstantin Lebedev 1169f94310
Fix filer sync set offset (#5197)
* fix: compose 2mount with sync

* fix: DATA RACE
https://github.com/seaweedfs/seaweedfs/issues/5194
https://github.com/seaweedfs/seaweedfs/issues/5195
2024-01-12 10:57:18 -08:00
Chris Lu 580940bf82
Merge accumulated changes related to message queue (#5098)
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
2023-12-11 12:05:54 -08:00
Konstantin Lebedev f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859)
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-25 08:34:12 -07:00
Konstantin Lebedev a40888d343
fix: make dev_tls with volume https (#4866) 2023-09-25 06:47:38 -07:00
Konstantin Lebedev a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856)
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-21 08:20:05 -07:00
Ryan Russell 949607fe90
refactor(compose)!: upgrade to v2 closes #3699 (#3705)
* docs(compose): Update readme for compose v2 upgrade

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade `Makefile` for compose v2

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade yml from version 2 -> 3.9

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-10-16 14:02:33 -07:00
Konstantin Lebedev b9933d5589
master server graceful stop (#3797) 2022-10-06 09:30:30 -07:00
Ryan Russell b51880ed3b
docs(docker): readability fixes (#3684)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-15 01:27:35 -07:00
Patrick Schmidt a73e177ecf
Add an End-to-End workflow for FUSE mount (#3562)
* Add an e2e workflow to test FUSE mount

* Fix deadlocks during concurrent r/w
2022-08-31 09:27:53 -07:00
Konstantin Lebedev bcbdc4cb37 use const multipart uploads folder
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-29 16:21:16 +05:00
Chris Lu e22c2c8012
Merge pull request #3188 from kmlebedev/fix_make_dev
fix ip.bind cmd for docker compose
2022-06-29 01:36:59 -07:00
Konstantin Lebedev 5b388ed6c1 initial advancedtls 2022-06-23 23:32:15 +05:00
Konstantin Lebedev a9f6c4b43d fix ip.bind cmd for docker compose 2022-06-15 16:49:53 +05:00
Konstantin Lebedev b86628f85d fix show empty dir
https://github.com/chrislusf/seaweedfs/issues/3086
2022-06-01 20:10:52 +05:00
Konstantin Lebedev a14728d508 docker compose nextcloud rm mysql 2022-05-23 20:11:32 +05:00
Konstantin Lebedev a34438384e docker compose file for nextcloud testing
https://github.com/chrislusf/seaweedfs/issues/3086
2022-05-23 18:53:35 +05:00
Konstantin Lebedev d12a423aa0 refactor
add some tests

https://github.com/chrislusf/seaweedfs/pull/2996
2022-05-12 12:23:02 +05:00
chrislu 22fbce8fd5 add docker compose file to sync 2 mounts 2022-05-06 03:53:36 -07:00
chrislu b2a43dd044 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-06 01:51:28 -07:00
chrislu 362dec87a3 add tests 2022-05-06 01:50:34 -07:00
Konstantin Lebedev 8342f651f3 fix scanNamed 2022-05-03 15:18:28 +05:00
Konstantin Lebedev d8925b4e83 Merge branch 'new_master' into ydb
# Conflicts:
#	go.mod
#	go.sum
2022-05-03 00:13:57 +05:00
Konstantin Lebedev 319d300d48 fix createDB 2022-05-02 22:23:07 +05:00
Konstantin Lebedev 275c3bb19c ydb initial
https://github.com/chrislusf/seaweedfs/issues/2942
2022-05-01 17:28:58 +05:00
chrislu 192983b464 s3 backend support customizing storage class 2022-04-30 17:36:40 -07:00
Konstantin Lebedev 357aa818fe add raft shell cmds 2022-04-06 15:23:53 +05:00
Konstantin Lebedev b7cdde14ae auto bootstraping and update peers 2022-04-05 13:17:53 +05:00
Konstantin Lebedev 1a1e5778c3 fix cluster status 2022-04-04 18:52:08 +05:00
Konstantin Lebedev 14dd971890 hashicorp raft with state machine 2022-04-04 17:51:51 +05:00
Konstantin Lebedev c514710b7b initial add hashicorp raft 2022-04-04 13:50:56 +05:00
Konstantin Lebedev b6a2b43efe allowDeleteBucketNotEmpty 2022-03-30 22:46:13 +05:00
Konstantin Lebedev 5b90a39954 fix s3 tests:
multipart_upload_incorrect_etag
multipart_resend_first_finishes_last
2022-03-29 19:21:09 +05:00
chrislu 24541fc7ec minor 2022-03-21 00:43:47 -07:00
kmlebedev 4f98553ba9 audit log SignatureVersion 2021-12-10 19:40:32 +05:00
Konstantin Lebedev 10678cde81 audit log config 2021-12-07 18:20:52 +05:00
Konstantin Lebedev 9d6ffa0ea1 GetBucketLifecycleConfigurationHandler 2021-10-14 01:35:33 +05:00
Konstantin Lebedev be4b3ed509 AclHandlers 2021-10-11 15:03:56 +05:00
Konstantin Lebedev b612d5aebd s3 test get w/ If-Match: bogus ETag 2021-05-24 16:59:44 +05:00
Konstantin Lebedev 69c768870b - object write cache control
- object write expires
2021-05-24 15:43:55 +05:00
Chris Lu e3f47ddb76 debug 2021-05-13 00:32:46 -07:00
Chris Lu f51bf61a16 remove wrong parameter 2021-04-30 22:51:30 -07:00
Konstantin Lebedev 995ae91007 add DeleteUserPolicy 2021-04-08 11:16:36 +05:00
Konstantin Lebedev 6b3cc94ac4 minio gateway s3 docker compose file 2021-03-29 14:59:59 +05:00
Chris Lu b1a86cf808 s3: copy object to itself
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Chris Lu 2d567e00aa add mount memory profiling 2021-03-12 02:54:08 -08:00
Konstantin Lebedev d66645ecda add 12 tests
https://github.com/chrislusf/seaweedfs/issues/1810
2021-03-12 12:40:33 +05:00
Konstantin Lebedev 348e21a08c add comments 2021-03-10 14:42:39 +05:00
Konstantin Lebedev 831953c55c allowed wildcard domain 2021-03-10 14:02:13 +05:00
Konstantin Lebedev 0e02f7e258 comma-separated SSL certificate common names 2021-03-10 12:42:44 +05:00
Konstantin Lebedev 190fada1ef TLS allowed commonNames 2021-03-08 21:39:44 +05:00