chrislu
a85ed3fe8f
minor
2022-07-07 22:27:24 -07:00
chrislu
2ea18cdcc8
remove dead code
2022-07-07 22:26:03 -07:00
chrislu
b7de5c6c43
shell: remove unused filer parameter from shell.toml
2022-07-07 15:01:23 -07:00
chrislu
f2f0482dd3
mount: random read also try to use the local cache first
2022-07-07 11:50:28 -07:00
chrislu
05c53820b9
mount: file handler release memory
2022-07-07 11:49:40 -07:00
chrislu
69ef6459c3
S3 ListObjectsV2 fix pagination under bucket root
...
fix https://github.com/chrislusf/seaweedfs/issues/3166
2022-07-06 02:28:34 -07:00
Konstantin Lebedev
785223e587
rabbitpubsub enable durable
2022-07-06 10:05:29 +05:00
chrislu
57975b93ff
adjust used size reporting
2022-07-04 13:47:42 -07:00
chrislu
a79f5bd0d5
adjust used size reporting
2022-07-04 13:46:32 -07:00
chrislu
3c79c77056
3.14
2022-07-04 13:15:49 -07:00
guol-fnst
5bc895673e
add getting bucket quota
2022-07-04 15:36:44 +08:00
guol-fnst
dd4664e610
fix usage
2022-07-04 11:31:39 +08:00
chrislu
141f662734
edge case checking when volume server does not have capacity to balance
...
fix https://github.com/chrislusf/seaweedfs/issues/3257
2022-07-01 12:16:18 -07:00
chrislu
bee3a7c798
fix tests
2022-07-01 12:09:42 -07:00
Chris Lu
ddb0f6bc74
Merge pull request #3258 from kmlebedev/issues_3255
...
volumeServer.evacuate fix select empty node
2022-07-01 02:36:30 -07:00
Konstantin Lebedev
3d5f1cbc5e
fix pagination
2022-07-01 14:17:49 +05:00
chrislu
06a8b174b5
also remove Sync() for idx file
2022-06-30 13:50:53 -07:00
chrislu
52580743b9
stuck with file.Sync()
...
starting to get weed/storage () - (master) > go test ./...
I0630 12:19:54 65819 needle_map_metric_test.go:26] FileCount expected 10000 actual 10000
I0630 12:19:54 65819 needle_map_metric_test.go:27] DeletedSize expected 1648 actual 1648
I0630 12:19:54 65819 needle_map_metric_test.go:28] ContentSize expected 10000 actual 10000
I0630 12:19:54 65819 needle_map_metric_test.go:29] DeletedCount expected 1648 actual 1959
I0630 12:19:54 65819 needle_map_metric_test.go:30] MaxFileKey expected 10000 actual 10000
I0630 12:19:54 65819 volume_loading.go:136] loading index /var/folders/jv/0rlhdck17jzgt7b3hcnq63mc0000gp/T/TestCompaction909350722/001/1.idx to memory
I0630 12:20:36 65819 volume_vacuum.go:98] Committing volume 1 vacuuming...
panic: test timed out after 10m0s
goroutine 61 [running]:
testing.(*M).startAlarm.func1()
/usr/local/go/src/testing/testing.go:2029 +0x8e
created by time.goFunc
/usr/local/go/src/time/sleep.go:176 +0x32
goroutine 1 [chan receive, 10 minutes]:
testing.(*T).Run(0xc0000cd520, {0x1c09bde?, 0x5ab798ea5c102?}, 0x1c67bd8)
/usr/local/go/src/testing/testing.go:1487 +0x37a
testing.runTests.func1(0xc0000cd520?)
/usr/local/go/src/testing/testing.go:1839 +0x6e
testing.tRunner(0xc0000cd520, 0xc00039fcd8)
/usr/local/go/src/testing/testing.go:1439 +0x102
testing.runTests(0xc0000dadc0?, {0x25b4460, 0x6, 0x6}, {0x2855108?, 0x40?, 0x25c4b80?})
/usr/local/go/src/testing/testing.go:1837 +0x457
testing.(*M).Run(0xc0000dadc0)
/usr/local/go/src/testing/testing.go:1719 +0x5d9
main.main()
_testmain.go:57 +0x1aa
goroutine 21 [chan receive]:
github.com/chrislusf/seaweedfs/weed/glog.(*loggingT).flushDaemon(0x0?)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:883 +0x6a
created by github.com/chrislusf/seaweedfs/weed/glog.init.0
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:410 +0x1bf
goroutine 55 [syscall]:
syscall.syscall(0x44ac0f96?, 0xc00007ad80?, 0xc00098b480?, 0x10588df?)
/usr/local/go/src/runtime/sys_darwin.go:22 +0x4e
syscall.fcntl(0x100000001?, 0xc00098b4b0?, 0x1069a7c?)
/usr/local/go/src/syscall/zsyscall_darwin_amd64.go:319 +0x30
internal/poll.(*FD).Fsync.func1(...)
/usr/local/go/src/internal/poll/fd_fsync_darwin.go:18
internal/poll.ignoringEINTR(...)
/usr/local/go/src/internal/poll/fd_posix.go:74
internal/poll.(*FD).Fsync(0xc00038f1e0?)
/usr/local/go/src/internal/poll/fd_fsync_darwin.go:17 +0xfc
os.(*File).Sync(0xc00012a030)
/usr/local/go/src/os/file_posix.go:168 +0x4e
github.com/chrislusf/seaweedfs/weed/storage/backend.(*DiskFile).Sync(...)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/backend/disk_file.go:84
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).makeupDiff(0xc0000bb440, {0xc000633a40, 0x52}, {0xc000633aa0, 0x52}, {0xc000633b00, 0x52}, {0xc000633b60, 0x52})
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:295 +0x12fa
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).CommitCompact(0xc0000bb440)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum.go:119 +0x3e8
github.com/chrislusf/seaweedfs/weed/storage.TestCompaction(0xc00025a000)
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_vacuum_test.go:89 +0x305
testing.tRunner(0xc00025a000, 0x1c67bd8)
/usr/local/go/src/testing/testing.go:1439 +0x102
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x35f
goroutine 56 [chan receive, 10 minutes]:
github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker.func1()
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:244 +0x41a
created by github.com/chrislusf/seaweedfs/weed/storage.(*Volume).startWorker
/Users/chrislu/go/src/github.com/chrislusf/seaweedfs/weed/storage/volume_write.go:234 +0x56
FAIL github.com/chrislusf/seaweedfs/weed/storage 600.194s
? github.com/chrislusf/seaweedfs/weed/storage/backend [no test files]
ok github.com/chrislusf/seaweedfs/weed/storage/backend/memory_map (cached)
? github.com/chrislusf/seaweedfs/weed/storage/backend/s3_backend [no test files]
ok github.com/chrislusf/seaweedfs/weed/storage/erasure_coding (cached)
? github.com/chrislusf/seaweedfs/weed/storage/idx [no test files]
ok github.com/chrislusf/seaweedfs/weed/storage/needle (cached)
ok github.com/chrislusf/seaweedfs/weed/storage/needle_map (cached)
ok github.com/chrislusf/seaweedfs/weed/storage/super_block (cached)
? github.com/chrislusf/seaweedfs/weed/storage/types [no test files]
? github.com/chrislusf/seaweedfs/weed/storage/volume_info [no test files]
FAIL
2022-06-30 12:32:55 -07:00
Konstantin Lebedev
5ed8165161
fix logic
...
add option targetServer
https://github.com/chrislusf/seaweedfs/issues/3255
2022-06-30 17:58:24 +05:00
石昌林
200a862701
fixed volume xx not found caused by missing VolumeLocation events
...
When the requested master node is the leader, `VolumeLocation` or `ClusterNodeUpdate` may be returned here. If it is `VolumeLocation`, the update will be performed while resetting the vidMap, otherwise the event will be lost
2022-06-30 13:41:56 +08:00
Chris Lu
4ae9884a6e
Merge pull request #3254 from shichanglin5/fix_subscribe_meta
...
[Critical] Avoid synchronizing from scratch due to err
2022-06-29 12:36:28 -07:00
Chris Lu
b98556cff5
Merge pull request #3253 from kmlebedev/constMultipartUploadsFolder
...
use const multipart uploads folder
2022-06-29 12:09:03 -07:00
石昌林
2b5b37231b
Make sure that onChangeEvent is called successfully before updating lastTsNs
2022-06-29 23:57:07 +08:00
石昌林
e17e6ed542
Avoid synchronizing from scratch due to err
2022-06-29 23:31:04 +08:00
Konstantin Lebedev
bcbdc4cb37
use const multipart uploads folder
...
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-29 16:21:16 +05:00
chrislu
a8abab2412
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-29 02:44:14 -07:00
chrislu
449ecc8890
remove max connection age
...
related to https://github.com/chrislusf/seaweedfs/pull/3226
2022-06-29 02:44:12 -07:00
Chris Lu
c6e6e303db
Merge pull request #3200 from lapshin-vitaly/bufix/validate-tags-on-copy
...
validate tags on copy object and add regex for validating tags
2022-06-29 01:37:14 -07:00
Lapshin Vitaliy
7c3d9d0535
remove println
2022-06-29 10:43:02 +03:00
Lapshin Vitaliy
61b2e3f9a2
fix return
2022-06-28 18:31:46 +03:00
Lapshin Vitaliy
606252472c
fix return
2022-06-28 18:24:03 +03:00
Lapshin Vitaliy
e969370913
fix parseTagging function and add test for validate tags
2022-06-28 14:48:55 +03:00
Chris Lu
a79bf9d92a
Merge pull request #3252 from guol-fnst/update_tbk
...
remove hdfs, add tikv for full version update
2022-06-27 22:37:41 -07:00
guol-fnst
74c9f39d48
remove hdfs, add tikv for full version update
2022-06-28 12:35:07 +08:00
石昌林
e1b94eb6b9
fix error: concurrent map writes when add or remove cluster node
2022-06-28 10:34:59 +08:00
chrislu
4b1f48a399
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-27 12:40:50 -07:00
chrislu
4a65159250
fix reading time
2022-06-27 12:40:47 -07:00
xdadrm
3a42269da8
Fixes WebDAV 0-bytes files
...
Fixes the issue where files created via WebDAV show as 0-bytes size when read via fuse.
2022-06-27 20:19:21 +02:00
Chris Lu
4be406a095
Merge pull request #3249 from kmlebedev/issues_2589
...
avoid set currentMaster svc.local k8s discovery service domains
2022-06-27 11:05:07 -07:00
Konstantin Lebedev
9bff097b01
currentMaster redirected
2022-06-27 21:55:16 +05:00
Konstantin Lebedev
6c20a3b622
avoid set currentMaster k8s svc.local discoveruy service domains
...
https://github.com/chrislusf/seaweedfs/issues/2589
2022-06-27 21:47:05 +05:00
Lapshin Vitaliy
1a5981d583
fix test
2022-06-27 18:46:21 +03:00
garenchan
8aa19577f4
fix 3238: handle errors for GenerateDirUuid method
2022-06-27 22:04:50 +08:00
Lapshin Vitaliy
d7c3493d15
Merge branch 'master' into bufix/validate-tags-on-copy
2022-06-27 13:53:57 +03:00
chrislu
b9f7b6fb9a
adjust log message
2022-06-26 23:12:16 -07:00
Chris Lu
fdd880b111
Merge pull request #3237 from shichanglin5/remove_query_collection
...
Remove the collection query param of s3api and let the collection be …
2022-06-26 23:02:11 -07:00
石昌林
15bfc60a88
Remove the collection query param of s3api and let the collection be determined by the filer
2022-06-27 13:47:53 +08:00
yulai.li
af23e63e3f
Improve filer command help, add supported filer store list
2022-06-27 12:09:16 +08:00
chrislu
40a9634c3d
3.13
2022-06-26 20:13:52 -07:00
Chris Lu
cc01f71752
Merge pull request #3233 from blacktear23/filerstore-tikv
...
Add TiKV Filerstore back
2022-06-26 20:12:18 -07:00
yulai.li
824cbe32be
Make tikv filer enable/disable by build tags
2022-06-27 10:57:24 +08:00
chrislu
9dfea44aeb
fix unit
2022-06-26 12:21:38 -07:00
chrislu
3142e77eca
mount: stats report physical size instead of logical size
2022-06-26 12:14:34 -07:00
yulai.li
13159ad0d8
Merge branch 'master' into filerstore-tikv
2022-06-26 22:44:01 +08:00
yulai.li
46e0b629e5
Update tikv client version and add one PC support
2022-06-26 22:43:37 +08:00
chrislu
f77eda4b4f
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-25 09:22:52 -07:00
chrislu
3ce46c3566
fix entry full url lookup
2022-06-25 09:22:49 -07:00
Chris Lu
dc59ccd110
Merge pull request #3228 from shichanglin5/fix_volumeNotFound
...
When the connection with the leader is disconnected, the vidMap shoul…
2022-06-24 23:06:13 -07:00
石昌林
f1ea906c13
Code comment optimization
2022-06-25 11:56:09 +08:00
石昌林
0c8e033e68
When the connection with the leader is disconnected, the vidMap should not be reset immediately, but should be reset after connecting to a new leader
2022-06-24 17:41:46 +08:00
chrislu
9c517d2b35
masterclient: fallback to directly querying master in case of missing volume id location
2022-06-24 02:08:57 -07:00
Chris Lu
280e33092c
Merge pull request #3226 from kmlebedev/advancedtls
...
Advancedtls
2022-06-24 01:34:58 -07:00
chrislu
509a9047db
test compact map with snowflake sequencer
2022-06-23 21:45:51 -07:00
chrislu
217d7dc377
verify seq is always unique
2022-06-23 21:22:05 -07:00
Konstantin Lebedev
2e782dfdcb
rm defer comments
2022-06-24 01:50:13 +05:00
Konstantin Lebedev
7efaafb0ee
fix Authenticate
2022-06-24 01:42:04 +05:00
Konstantin Lebedev
b0aa51d7ef
enable require client cert
2022-06-24 00:29:23 +05:00
Konstantin Lebedev
ea7cdb8b0e
seperate option
2022-06-24 00:26:56 +05:00
Konstantin Lebedev
5b388ed6c1
initial advancedtls
2022-06-23 23:32:15 +05:00
chrislu
0c13fc873c
fix compilation
2022-06-23 00:45:11 -07:00
chrislu
96496d5286
master: broadcast new volume locations to clients to avoid possible racing condition
...
fix https://github.com/chrislusf/seaweedfs/issues/3220
2022-06-23 00:41:33 -07:00
chrislu
52c44d646e
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-22 00:34:27 -07:00
chrislu
530da0cc89
remove unused code
2022-06-22 00:34:25 -07:00
famosss
8a6e7a7df8
Merge branch 'chrislusf:master' into master
2022-06-22 09:58:56 +08:00
zzq09494
9df5ad5309
fix: vacuum create a lot of connections quickly
2022-06-22 09:57:22 +08:00
Chris Lu
d652ce1a36
Merge pull request #3216 from kmlebedev/mk_raft_basedir
...
avoid no such raft date directory
2022-06-21 12:21:30 -07:00
chrislu
6b3beb709d
fix format
2022-06-21 12:21:02 -07:00
Konstantin Lebedev
c1c8dad677
avoid no such raft date directory
...
https://github.com/chrislusf/seaweedfs/issues/3214
2022-06-21 13:47:51 +05:00
chrislu
4dc27e1ed5
3.12
2022-06-20 19:07:00 -07:00
chrislu
492da3dbce
master: put metadata under instance specific folder
2022-06-20 19:04:49 -07:00
chrislu
41dfe27102
Revert "remove max connection age"
...
This reverts commit b9b684194f
2022-06-20 14:49:39 -07:00
chrislu
2295766e2c
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-20 14:16:18 -07:00
chrislu
1d0c53ea56
remote storage: stop supporting hdfs as a remote storage
2022-06-20 14:15:59 -07:00
Stephan
1eb7826909
Fix link to osxfuse github page
2022-06-20 22:36:07 +02:00
chrislu
bf5a5abfb1
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
2022-06-20 13:18:11 -07:00
chrislu
69a2e503ab
non-recursive directory deletion
...
fix a problem during git checkout a different version, where updated directories are deleted even though the directory is not empty.
2022-06-20 13:18:08 -07:00
chrislu
76f26c4b99
mount: delete target entry first during move
2022-06-20 13:15:02 -07:00
Chris Lu
460c7cf68e
Merge pull request #3203 from shichanglin5/volume_copy_preallocate
2022-06-20 07:41:45 -07:00
石昌林
81f7f08708
Determine whether to preallocate according to the master configuration before executing copy volume
2022-06-20 21:12:44 +08:00
石昌林
be5c901bea
Fix global counter key is incorrect & Add read lock when reading counter map to avoid memory problems
2022-06-20 19:16:53 +08:00
Lapshin Vitaliy
5f5fd0bc48
validate tags on copy object and add regex for validating tags
2022-06-20 11:32:58 +03:00
Chris Lu
9e2d6e897e
Merge pull request #3198 from guol-fnst/fix_statuscode
...
AbortMultipartUploadHandler should return 204 instead of 200
2022-06-19 22:57:13 -07:00
LHHDZ
8db9f13bc6
Merge branch 'master' into circuit_breaker
2022-06-20 13:42:18 +08:00
guol-fnst
1d77deccd0
AbortMultipartUploadHandler should return 204 instead of 200
...
https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html
2022-06-20 13:26:22 +08:00
石昌林
9e036df356
remove go.uber.org/atomic
2022-06-20 12:35:29 +08:00
geekboood
fdacd94af5
fix: invalid chunk data when failed to read manifests
2022-06-19 16:54:04 +08:00
石昌林
3dd60529c5
some code optimizations
2022-06-17 19:07:39 +08:00
石昌林
37df209195
add some unit tests and some code optimizes
2022-06-17 17:11:18 +08:00
Chris Lu
d28f8bb88f
Merge pull request #3192 from guol-fnst/weed_update
...
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-16 22:58:29 -07:00
guol-fnst
c7c20881f5
remove "weed" from UsageLine, or weed will not show usage of weed update
2022-06-17 12:47:04 +08:00
