Chris Lu
|
1b942dfa3c
|
add error message
|
2021-09-02 12:32:46 -07:00 |
|
Chris Lu
|
7ce97b59d8
|
go fmt
|
2021-09-01 02:45:42 -07:00 |
|
Chris Lu
|
f811fd0903
|
fix build
|
2021-08-30 02:03:08 -07:00 |
|
Chris Lu
|
5adfdd7982
|
change to a new bloom filter library
|
2021-08-29 21:26:38 -07:00 |
|
Chris Lu
|
05a648bb96
|
refactor: separating out remote.proto
|
2021-08-26 15:18:34 -07:00 |
|
Chris Lu
|
c08ac536ed
|
cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
|
2021-08-25 17:34:29 -07:00 |
|
Chris Lu
|
00c4e06caa
|
cloud drive: s3 configurable force path style
|
2021-08-23 03:30:41 -07:00 |
|
Chris Lu
|
f0cc130849
|
do not force path style for better compatibility
|
2021-08-23 03:09:41 -07:00 |
|
Konstantin Lebedev
|
865b06b7fe
|
The IEEE assembler optimizations has been submitted and will be part of the Go 1.6 standard library.
https://github.com/klauspost/crc32
|
2021-08-17 13:06:48 +05:00 |
|
Chris Lu
|
78e8ddf910
|
Only when tailing volume, the zero-ed cookie should skip checking.
This only happens when checkCookie == false and fsync == false.
|
2021-08-13 02:09:35 -07:00 |
|
Chris Lu
|
a8617c1a39
|
tail volume: fix zero cookie problem from batch deletion
|
2021-08-13 01:54:35 -07:00 |
|
Chris Lu
|
734c980040
|
volume: support concurrent download data size limit
|
2021-08-08 23:25:16 -07:00 |
|
Chris Lu
|
9cc84a910f
|
volume: deletion can skip volume size checking
fix https://github.com/chrislusf/seaweedfs/issues/2225
|
2021-08-01 00:32:51 -07:00 |
|
Chris Lu
|
49c66e88a0
|
volume: change all writes to fsync during graceful stopping
fix https://github.com/chrislusf/seaweedfs/issues/2193
|
2021-07-13 01:29:57 -07:00 |
|
Chris Lu
|
b624090398
|
go fmt
|
2021-07-01 01:21:14 -07:00 |
|
Chris Lu
|
24e11d1e90
|
look back when adding to sorted values
look back when adding to sorted values, before adding it to overflow
|
2021-06-28 22:46:49 -07:00 |
|
Chris Lu
|
fc8dd58aea
|
volume: large_volume version has bug when using in memory index
fix https://github.com/chrislusf/seaweedfs/issues/2162
|
2021-06-28 15:48:07 -07:00 |
|
Chris Lu
|
56eb522b13
|
fix stats when a collection is deleted
|
2021-06-15 21:11:31 -07:00 |
|
Chris Lu
|
6c82326575
|
use bytes.Buffer to reduce memory allocation and gc
|
2021-06-06 13:42:36 -07:00 |
|
Konstantin Lebedev
|
99ef280c7c
|
avoid data loss after restarting a container with a volum server
|
2021-06-02 17:07:19 +05:00 |
|
Chris Lu
|
1aa7e99a89
|
skip file not found error when deleting
|
2021-05-15 09:37:39 -07:00 |
|
qieqieplus
|
ac26080bd2
|
fix concurrent vacuum & delete panic
|
2021-05-05 17:54:50 +08:00 |
|
bingoohuang
|
cf552417a7
|
minFreeSpace refactored
|
2021-04-27 10:37:24 +08:00 |
|
bingoohuang
|
31f1cdeac2
|
minFreeSpace argument allows size like 10GiB
|
2021-04-26 18:48:34 +08:00 |
|
Chris Lu
|
ca998328c2
|
do not add new volumes when below minFreeSpacePercent
fix https://github.com/chrislusf/seaweedfs/issues/2017
|
2021-04-21 23:11:11 -07:00 |
|
Chris Lu
|
a8114da02d
|
avoid thundering herd effect
transient errors may cause thundering herd effect to all trying to recover from remove ec shards
|
2021-04-21 10:17:12 -07:00 |
|
Chris Lu
|
ae74d8f02a
|
fix error message
related to https://github.com/chrislusf/seaweedfs/issues/2012
|
2021-04-21 01:40:16 -07:00 |
|
Chris Lu
|
283d703d50
|
adjust text
|
2021-04-15 11:29:58 -07:00 |
|
Chris Lu
|
8e404a1433
|
go fmt
|
2021-04-02 02:22:26 -07:00 |
|
Chris Lu
|
b465095db1
|
shell: add volume.check.disk to fix inconsistency for replicated volumes
fix https://github.com/chrislusf/seaweedfs/issues/1923
|
2021-03-22 00:03:16 -07:00 |
|
Chris Lu
|
102a951377
|
refactor, split into 2 files
|
2021-03-21 13:05:53 -07:00 |
|
Chris Lu
|
f6e2566ee5
|
better logs for cookie mismatch
|
2021-03-20 12:02:57 -07:00 |
|
Chris Lu
|
35d939a1d2
|
go fmt
|
2021-03-14 13:21:02 -07:00 |
|
Chris Lu
|
cb423312a4
|
prevent nil volume message
|
2021-03-13 11:05:29 -08:00 |
|
Chris Lu
|
972327f966
|
prevent nil volume nm
|
2021-03-13 11:04:51 -08:00 |
|
Chris Lu
|
737bde5ab7
|
refactoring, remove duplicated code
|
2021-03-09 23:23:01 -08:00 |
|
Chris Lu
|
828f6e9f4d
|
volume: auto add missing vif files
fix https://github.com/chrislusf/seaweedfs/issues/1878
|
2021-03-09 12:09:32 -08:00 |
|
qieqieplus
|
5b16820924
|
fix: collection stats won't update if all volumes expired at same iteration
|
2021-03-09 18:49:45 +08:00 |
|
Chris Lu
|
726edab054
|
avoid nil when closing an index
fix https://github.com/chrislusf/seaweedfs/issues/1870
|
2021-03-07 11:03:09 -08:00 |
|
Chris Lu
|
bdfed16d42
|
avoid nil exception
fix https://github.com/chrislusf/seaweedfs/issues/1869
|
2021-03-07 01:49:06 -08:00 |
|
Chris Lu
|
e9eddfb8a2
|
remove wrong writes
|
2021-03-06 14:26:27 -08:00 |
|
Chris Lu
|
38fc200e56
|
CRCWriter consistent with CRC
|
2021-03-06 14:26:27 -08:00 |
|
Chris Lu
|
400de380f4
|
volume server: support tcp direct put/get/delete
|
2021-03-05 02:29:38 -08:00 |
|
Chris Lu
|
c46d16b2f3
|
skip network error when remote volumes start
|
2021-03-01 01:20:06 -08:00 |
|
Chris Lu
|
6e43e8ce17
|
add logs for remote file loading
|
2021-03-01 00:48:59 -08:00 |
|
Chris Lu
|
015d16f43f
|
add vif file versions in case loading superblock fails
|
2021-03-01 00:48:30 -08:00 |
|
Chris Lu
|
d680676d45
|
skip already loaded backends
|
2021-03-01 00:47:03 -08:00 |
|
Chris Lu
|
540441fd38
|
go fmt
|
2021-02-28 20:34:14 -08:00 |
|
Chris Lu
|
2cca07b44c
|
cloud tier to non-AWS s3 gateways
|
2021-02-28 18:59:09 -08:00 |
|
Chris Lu
|
30b30b8fe0
|
volume.tier.move: passing non-empty disk type
|
2021-02-22 01:59:03 -08:00 |
|
Chris Lu
|
03c643aa6b
|
fix test
fileSize need to be divided by 8
|
2021-02-20 21:19:21 -08:00 |
|
Chris Lu
|
7635f6b9fa
|
disk file avoid file.Stat()
|
2021-02-20 20:06:06 -08:00 |
|
Chris Lu
|
a2383b3b12
|
Revert "Revert "avoid file.Stat()""
This reverts commit 7ef4c24f18 .
|
2021-02-20 12:42:09 -08:00 |
|
Chris Lu
|
7ef4c24f18
|
Revert "avoid file.Stat()"
This reverts commit 98c93ca465 .
|
2021-02-20 12:39:33 -08:00 |
|
Chris Lu
|
1ad3200094
|
skip seek() when index file writes
|
2021-02-20 12:39:25 -08:00 |
|
Chris Lu
|
98c93ca465
|
avoid file.Stat()
avoid one Syscall, but did not help on performance though
|
2021-02-20 09:44:17 -08:00 |
|
Chris Lu
|
a1210d1e8d
|
Revert "volume: avoid file.stat(), file.seek() if possible during writes"
This reverts commit c78409a598 .
|
2021-02-19 03:56:27 -08:00 |
|
Chris Lu
|
c78409a598
|
volume: avoid file.stat(), file.seek() if possible during writes
|
2021-02-18 23:22:54 -08:00 |
|
Chris Lu
|
73958e357d
|
add descriptive error if no free volumes
|
2021-02-18 19:10:20 -08:00 |
|
bingoohuang
|
352ac2f271
|
Merge remote-tracking branch 'origin/master'
|
2021-02-18 14:05:51 +08:00 |
|
bingoohuang
|
eab6e31d34
|
use backticks instead of double quotes to avoid escaped additionally in regex
|
2021-02-18 14:05:28 +08:00 |
|
Chris Lu
|
3575d41009
|
go fmt
|
2021-02-17 20:57:08 -08:00 |
|
Chris Lu
|
7403cd43c5
|
only use "" for hdd to avoid two values for the same thing
|
2021-02-16 10:50:42 -08:00 |
|
Chris Lu
|
ebf320ec22
|
avoid divided by zero
|
2021-02-16 05:59:24 -08:00 |
|
Chris Lu
|
3fe628f04e
|
use hdd instead of empty string
|
2021-02-16 03:03:00 -08:00 |
|
Chris Lu
|
f8446b42ab
|
this can compile now!!!
|
2021-02-16 02:47:02 -08:00 |
|
Chris Lu
|
7ce647f27e
|
support customizable disk type
|
2021-02-13 15:42:42 -08:00 |
|
Chris Lu
|
4bd8a692d8
|
disk type can be generic tags
|
2021-02-13 13:50:14 -08:00 |
|
Chris Lu
|
885ca34748
|
volume: fail fast if idx files are missing
fix https://github.com/chrislusf/seaweedfs/issues/1796
|
2021-02-11 00:44:40 -08:00 |
|
Chris Lu
|
821c46edf1
|
Merge branch 'master' into support_ssd_volume
|
2021-02-09 11:37:07 -08:00 |
|
bingoohuang
|
9e06ef66b9
|
fix renaming for 5bytes
|
2021-02-09 09:44:28 +08:00 |
|
Chris Lu
|
a833021132
|
fix refactoring left over
|
2021-02-07 23:03:03 -08:00 |
|
bingoohuang
|
7256902fb0
|
fix typo offset.ToAcutalOffset to offset.ToActualOffset
|
2021-02-07 12:11:51 +08:00 |
|
bingoohuang
|
94ea3bd3a5
|
renaming NeedleMapType to NeedleMapKind
|
2021-02-07 09:00:03 +08:00 |
|
Chris Lu
|
cfb9342a15
|
avoid concurrent map updates to viper
|
2021-01-12 02:28:13 -08:00 |
|
Chris Lu
|
039ad101b1
|
manifest file also need to detect compression
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753585046
|
2021-01-03 01:42:21 -08:00 |
|
Chris Lu
|
4f31c1bb94
|
go fmt
|
2020-12-22 02:34:08 -08:00 |
|
Chris Lu
|
9abf016af1
|
Merge branch 'master' into support_ssd_volume
|
2020-12-17 22:30:19 -08:00 |
|
Chris Lu
|
738c4fd203
|
fix logging when error
address https://github.com/chrislusf/seaweedfs/pull/1685#issuecomment-747845601
|
2020-12-17 19:37:46 -08:00 |
|
Chris Lu
|
daa8157fc2
|
Merge branch 'master' into support_ssd_volume
|
2020-12-17 13:05:20 -08:00 |
|
Chris Lu
|
e2076201d7
|
volume: avoid reprocessing the same volume
fix https://github.com/chrislusf/seaweedfs/issues/1682
|
2020-12-17 13:03:39 -08:00 |
|
Chris Lu
|
986cbdf7d9
|
Revert "Merge pull request #1683 from qieqieplus/master"
This reverts commit 8cb67952db , reversing
changes made to 200e56215a .
|
2020-12-17 12:46:42 -08:00 |
|
Chris Lu
|
f56e6d231e
|
Revert "Merge pull request #1683 from qieqieplus/master"
This reverts commit 8cb67952db , reversing
changes made to 200e56215a .
|
2020-12-17 12:46:20 -08:00 |
|
Chris Lu
|
1bf22c0b5b
|
go fmt
|
2020-12-16 09:14:05 -08:00 |
|
Chris Lu
|
1d88865869
|
passing disk type along
|
2020-12-16 09:10:14 -08:00 |
|
Chris Lu
|
23014b6810
|
Merge branch 'master' into support_ssd_volume
|
2020-12-16 08:26:51 -08:00 |
|
chenqieqie
|
45f902a9b7
|
smaller critical section
|
2020-12-16 20:23:51 +08:00 |
|
chenqieqie
|
4e58a4f24e
|
fix race condition when loading volumes concurrently
|
2020-12-16 18:49:10 +08:00 |
|
Chris Lu
|
2e8dba571b
|
adjust volume server UI
|
2020-12-14 00:51:57 -08:00 |
|
Chris Lu
|
933f2f4cfd
|
find location with matching disk type
|
2020-12-13 23:39:00 -08:00 |
|
Chris Lu
|
94525aa0fd
|
allocate volume by disk type
|
2020-12-13 23:08:21 -08:00 |
|
Chris Lu
|
f6a419c26c
|
disk type configurable for each folder
|
2020-12-13 22:49:56 -08:00 |
|
Chris Lu
|
0d2ec832e2
|
rename from volumeType to diskType
|
2020-12-13 11:59:32 -08:00 |
|
Chris Lu
|
d156c74ec0
|
volume server set volume type and heartbeat to the master
|
2020-12-13 03:11:24 -08:00 |
|
Chris Lu
|
e9cd798bd3
|
adding volume type
|
2020-12-13 00:58:58 -08:00 |
|
Chris Lu
|
ae655033ac
|
adjust logging
|
2020-12-11 16:57:53 -08:00 |
|
Chris Lu
|
d986c7196d
|
use append time instead of filer's own modification time
fix https://github.com/chrislusf/seaweedfs/issues/1669
|
2020-12-11 16:55:18 -08:00 |
|
Chris Lu
|
d171d9f988
|
volume: ensure the volume dat and idx files are always closed
fix https://github.com/chrislusf/seaweedfs/issues/1646
|
2020-12-04 21:50:26 -08:00 |
|
Chris Lu
|
cc839f935d
|
better unit detection
|
2020-12-03 19:23:59 -08:00 |
|
Chris Lu
|
b219ccfe68
|
Update volume_ttl_test.go
|
2020-12-03 12:55:44 -08:00 |
|