gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
8094 commits 28 branches 278 tags 135 MiB
Commit graph

5301 commits

Author SHA1 Message Date
Chris Lu 097a12dfb4
Merge pull request #3187 from ningfdx/remote 2022-06-15 06:34:32 -07:00
Chris Lu 55b9f281d1
Merge pull request #3185 from kmlebedev/logging_err_process_range_req 2022-06-15 06:33:33 -07:00
石昌林 78b3728169 add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations 
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply

# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply

# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply

# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply

# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
 2022-06-15 21:07:55 +08:00
ningfd 338705f375 fix(wdclient): GetLocations return 2022-06-15 19:20:13 +08:00
Konstantin Lebedev 31a929f7f2 logging processRangeRequest errors 2022-06-15 15:36:43 +05:00
Chris Lu 4eeeb5f50d
Merge pull request #3180 from blacktear23/issue-3149 2022-06-15 00:55:20 -07:00
yulai.li 358c0c0b37 Fix: provide filer.ui.deleteDir options for server command 2022-06-15 15:53:31 +08:00
Guo Lei 3df84cabf6
Merge branch 'chrislusf:master' into weed_update 2022-06-15 15:28:40 +08:00
guol-fnst 695e8c7555 fix minor bug 2022-06-15 15:26:04 +08:00
zhihao.qu 4d0d1848c6 fix(filer.sync): modify clientName format : from -> to 2022-06-15 13:33:20 +08:00
yulai.li b90d213a87 Change ui.deleteDir default to true 2022-06-15 11:37:37 +08:00
zhihao.qu 42d04c581b feat(filer.sync): add metricsServer in filer.sync. 
Metrics include:
(1) the offset of the filer.sync
(2) the last send timestamp of the filer subscription
 2022-06-15 11:33:18 +08:00
chrislu d4ef06cdcf 3.11 2022-06-14 12:14:52 -07:00
chrislu b347b2fb54 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-14 11:16:51 -07:00
chrislu 36d2756978 volume: avoid writing too much data for range requests in large files 
related to https://github.com/chrislusf/seaweedfs/issues/3178
 2022-06-14 11:16:47 -07:00
yulai.li 28cda854eb Add filer command line parameter to let Filer UI show/hide directory delete button 2022-06-15 00:30:49 +08:00
zhihao.qu 14d82c3dea feat(filer.sync): add offset to path. 2022-06-14 19:46:02 +08:00
guol-fnst 78fc9cdfa0 add "copied from https://github.com/restic/restic/tree/master/internal/selfupdate" 2022-06-14 19:10:54 +08:00
guol-fnst 322dbcf372 add update feature 2022-06-14 19:01:57 +08:00
Chris Lu b22ca85fbb
Merge pull request #3169 from shichanglin5/fix_auth 
fix: When there is no access permission configured before startup, th…
 2022-06-12 23:40:42 -07:00
shichanglin5 f01dd27752 fix: When there is no access permission configured before startup, the authentication does not take effect after configuring the permission after startup 2022-06-13 13:23:26 +08:00
chrislu e437ec5a68 3.10 2022-06-12 19:30:27 -07:00
chrislu d12f431d98 collect volume server status 2022-06-12 11:56:23 -07:00
chrislu 444ac21050 go fmt 2022-06-11 09:51:11 -07:00
chrislu a74e926063 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-11 09:51:02 -07:00
chrislu 61b8c9c361 remote object store gateway: disable tagging for backblaze 2022-06-11 09:50:59 -07:00
Chris Lu 0a8b3948db
Merge pull request #3165 from blacktear23/reload-bug-fix 2022-06-10 07:47:43 -07:00
yulai.li 83f8cdd5cb Bug Fix: force reload filer page when upload/delete/rename/create directory 2022-06-10 22:39:21 +08:00
guol-fnst eabf4d4e64 fix typo 2022-06-10 22:03:11 +08:00
Chris Lu 36757d28a8
Merge pull request #3163 from kmlebedev/metric_replica_placement_mismatch 
stats master_replica_placement_mismatch
 2022-06-10 06:44:16 -07:00
chrislu 9f8b72a54d Revert "Merge pull request #3159 from shichanglin5/_duplicateUUID" 
This reverts commit 37da689319, reversing
changes made to 00d53c34c4.
 2022-06-10 06:38:17 -07:00
Konstantin Lebedev fb57aa431c stats master_replica_placement_mismatch 2022-06-10 15:30:40 +05:00
shichanglin5 a63559ee6c fix: fix command prompt information 2022-06-10 12:46:10 +08:00
Chris Lu 37da689319
Merge pull request #3159 from shichanglin5/_duplicateUUID 
perf: Optimized volume handling duplicateUUID logic to avoid quitting…
 2022-06-09 18:01:02 -07:00
shichanglin5 f5b0c04b14 perf: Optimized volume handling duplicateUUID logic to avoid quitting when volume is actualy normal 
Under normal circumstances, there will be no problems, but when the
master is debugged in the local environment, the volume client cannot
communicate with the master normally, so the sendHeartBeat logic is
restarted, and a new connection is created to report the heartbeat. If
the master has not cleared the uuid of the volume at this time, then The
master will respond to volume duplicateUUIDS, and the volume service
will exit, but in fact the uuid of the volume is not duplicated
 2022-06-09 20:41:16 +08:00
zhihao.qu cd5cca36a4 feat(filer.sync): add fromTsMs. Extract signature from doSubscribeFilerMetaChanges 2022-06-09 10:53:19 +08:00
Chris Lu 4a5135961f
Merge pull request #3154 from kmlebedev/s3_test_invalid_range 2022-06-08 01:45:40 -07:00
Konstantin Lebedev 49f058da85 fix s3 tests: 
ranged_request_invalid_range
ranged_request_empty_object
 2022-06-08 13:25:13 +05:00
chrislu 8a1166c021 auto remove previous leftover socket 2022-06-08 00:32:09 -07:00
chrislu 633a4aed18 filer: fix customizable local socket file name 
fix https://github.com/chrislusf/seaweedfs/issues/3147
 2022-06-08 00:15:28 -07:00
chrislu acbc0286e2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-07 22:33:38 -07:00
chrislu ebd6097e6f fix: sequence read rate is slow when use filer.ChunkReadAt.ReadAt #3074 
related to https://github.com/chrislusf/seaweedfs/pull/3074
 2022-06-07 22:33:35 -07:00
Chris Lu f43ec9f363
Merge pull request #3130 from kmlebedev/fix_rm_parent_dir_via_nextcloud 2022-06-07 08:56:55 -07:00
Konstantin Lebedev c07820178f fix s3 tests 
bucket_list_delimiter_prefix
bucket_list_delimiter_prefix_underscore
bucket_list_delimiter_prefix_ends_with_delimiter
 2022-06-07 14:43:10 +05:00
chrislu 78be3b0f6c fix compilation 2022-06-06 10:09:21 -07:00
chrislu 4fd5f96598 filer: remove replication, collection, disk_type info from entry metadata 
these metadata can change and are not used
 2022-06-06 00:39:35 -07:00
chrislu cbf46de5f4 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-06 00:15:40 -07:00
chrislu 8402af0882 filer: detect ttl based on fs configure setting 2022-06-06 00:15:37 -07:00
ningfd f32142f6f5 add disableXAttr in mount option 2022-06-06 14:09:01 +08:00
chrislu 4a046e4de7 3.09 2022-06-05 19:00:24 -07:00
chrislu 11f1e7996b fix test 2022-06-05 18:50:41 -07:00
chrislu d65bb2c6df mount: file handle locks entry better 
related to https://github.com/chrislusf/seaweedfs/issues/2952
 2022-06-05 18:15:06 -07:00
chrislu 746092a60b add a comment 2022-06-05 16:44:07 -07:00
chrislu 056c480eb0 volume: checksum remove the hashing step in Value() 2022-06-05 15:24:02 -07:00
chrislu ecef844dfc stream read large files 2022-06-05 11:54:04 -07:00
chrislu 48cca4e54f remove unused logic 2022-06-04 19:11:06 -07:00
chrislu 685643d506 needle read into a writer 2022-06-04 19:10:52 -07:00
chrislu 663bc5dc23 add needle reading in chunks 2022-06-04 18:15:39 -07:00
chrislu 18b29f709c refactoring, split file 2022-06-03 23:31:14 -07:00
chrislu 39c3cb04af Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-06-03 08:45:32 -07:00
chrislu f97acdd489 volume.fix.replication fix retry logic 
fix https://github.com/chrislusf/seaweedfs/issues/3136
 2022-06-03 08:45:29 -07:00
creeew 02ae102731 fix filer.sync missing source srv uploaded files to target when target down 2022-06-02 01:28:47 +08:00
Konstantin Lebedev b86628f85d fix show empty dir 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-06-01 20:10:52 +05:00
chrislu 6793bc853c help message when in simulation mode 2022-05-31 14:48:46 -07:00
chrislu 89948a373b fix error reporting for "Need to a hdd disk type" 
related to https://github.com/chrislusf/seaweedfs/issues/3128
 2022-05-31 12:43:55 -07:00
chrislu 8a49240d64 3.08 2022-05-31 11:57:41 -07:00
chrislu 376b005ad3 fix wrong logic 2022-05-31 11:55:58 -07:00
chrislu fbd99d53c1 3.07 2022-05-31 00:36:24 -07:00
chrislu 27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu f4a6da6cb2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 22:47:29 -07:00
chrislu 596c3860ca use final destination to resolve fs configuration 
related to https://github.com/chrislusf/seaweedfs/issues/3075
 2022-05-30 22:47:26 -07:00
Chris Lu ca01ce0524
Merge pull request #3122 from chrislusf/filer-sync-with-peers 
Filer bootstrap from peers
 2022-05-30 21:38:31 -07:00
chrislu 6adc42147f fresh filer store bootstrap from the oldest peer 2022-05-30 21:27:48 -07:00
chrislu 490e0850bf filer.meta.tail add one example of using untilTimeAgo 2022-05-30 20:24:59 -07:00
liubaojiang f0ee3e6f21 reduce the scope of inFlightUploadDataLimitCond lock 2022-05-31 09:49:07 +08:00
chrislu c59068d0f3 refactor 2022-05-30 16:28:36 -07:00
chrislu 682382648e collect cluster node start time 2022-05-30 16:23:52 -07:00
chrislu 968ca95b49 filer.meta.tail: support untilTimeAgo for a range 2022-05-30 16:16:23 -07:00
chrislu f214dfb1f5 stop when in memory log is done 2022-05-30 15:25:21 -07:00
chrislu aece35a64f stop when on disk log is done 2022-05-30 15:20:51 -07:00
chrislu a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu 8902fa6ff6 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 13:28:32 -07:00
Chris Lu 730c9cf601
Merge pull request #3114 from kmlebedev/nextcloud 
Create folders in s3 via nextcloud
 2022-05-30 13:10:10 -07:00
chrislu 5b9347c938 typo 2022-05-30 12:37:41 -07:00
Konstantin Lebedev af562e1a20 rm log info 2022-05-30 23:26:26 +05:00
Konstantin Lebedev 62e5e3822c list self dir 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-05-30 18:09:08 +05:00
Konstantin Lebedev 26f3646961 show fake dirs 
https://github.com/chrislusf/seaweedfs/issues/3086
 2022-05-30 15:03:27 +05:00
chrislu b8f3db0d46 s3: keep auth enabled in case identities are set to empty 
fix https://github.com/chrislusf/seaweedfs/issues/3084
 2022-05-29 19:00:03 -07:00
chrislu 81d6159290 volume: report error if a volume has nil data backend 
fix https://github.com/chrislusf/seaweedfs/issues/3105
 2022-05-29 16:59:30 -07:00
chrislu ca16fbf0ef shell: cluster.ps display filer group 2022-05-29 16:37:14 -07:00
chrislu 9219664505 reset sync offset if peer filer resets 
fix https://github.com/chrislusf/seaweedfs/issues/3089
 2022-05-29 13:11:24 -07:00
Konstantin Lebedev 54356211c2 list empty folder 2022-05-27 13:17:18 +05:00
Kaiwalya Joshi 4598581741
fix: Minor spelling fixes on FilerStorePathTranslator. 2022-05-25 01:11:02 -07:00
chrislu 5b8b022985 remove unused parameter 2022-05-24 18:52:04 -07:00
Robert Coelho 0e6e72d462 mount: ReadDir return EIO on EnsureVisited err 2022-05-24 13:02:59 -04:00
Robert Coelho 1fabbe8a25 mount: cleanup ReadDir rewrite's branches to not assume offsets 2022-05-24 12:48:32 -04:00
Robert Coelho cb422d96f7 mount: rewrite ReadDir to respect input.Offset to fix partial results 2022-05-24 12:16:58 -04:00
Robert Coelho 05602167f4 mount: do not mark meta_cache when there is an err 2022-05-24 12:13:15 -04:00
chrislu 7e25a2d416 reduce busy waiting when reading metadata logs 2022-05-24 00:23:53 -07:00
chrislu 2f846777bb 3.06 2022-05-23 01:21:48 -07:00