Commit graph

51 commits

Author SHA1 Message Date
chrislu ff7b6d779e avoid overwriting variables
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-22 11:49:59 -07:00
chrislu 0a22eea55d collect ec shard from multiple locations
fix https://github.com/seaweedfs/seaweedfs/issues/4365
2023-04-17 22:56:21 -07:00
Ryan Russell 277976bd76
refactor(storage): readability improvements (#3703)
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-09-16 02:43:17 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
justin 3551ca2fcf enhancement: replace sort.Slice with slices.SortFunc to reduce reflection 2022-04-18 10:35:43 +08:00
chrislu 9f9ef1340c use streaming mode for long poll grpc calls
streaming mode would create separate grpc connections for each call.
this is to ensure the long poll connections are properly closed.
2021-12-26 00:15:03 -08:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu 734c980040 volume: support concurrent download data size limit 2021-08-08 23:25:16 -07:00
Chris Lu a8114da02d avoid thundering herd effect
transient errors may cause thundering herd effect to all trying to recover from remove ec shards
2021-04-21 10:17:12 -07:00
Chris Lu ae74d8f02a fix error message
related to https://github.com/chrislusf/seaweedfs/issues/2012
2021-04-21 01:40:16 -07:00
Chris Lu f8446b42ab this can compile now!!! 2021-02-16 02:47:02 -08:00
bingoohuang 7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
Chris Lu bd8b9b0c2e some changes to prepare fix after failed reads 2020-10-08 18:33:06 -07:00
Chris Lu ab759f0ec2 erasure coding: fix EC error if multiple disks are configured in one volume server 2020-08-26 09:16:58 -07:00
Chris Lu c026eb0592 refactoring 2020-08-18 17:39:29 -07:00
Chris Lu 32a1f22730 more logs 2020-06-24 22:07:53 -07:00
Chris Lu 892e726eb9 avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu 72a64a5cf8 use the same context object in order to retry 2020-01-26 14:42:11 -08:00
Chris Lu 37b64a50b4 ec: generate and copy .vif file 2019-12-28 12:44:59 -08:00
Chris Lu 09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00
Chris Lu ee2072dff9 volume: avoid inifinite loop reading ec volume info if error happens 2019-11-27 23:07:53 -08:00
Chris Lu 84c503c6a7 adjust ec reading log level 2019-11-08 22:41:02 -08:00
Chris Lu a14bd31f5b go fmt 2019-06-22 20:05:25 -07:00
Chris Lu f88a8bda7b ec deletion works 2019-06-21 01:14:10 -07:00
Chris Lu 4cea8aefd0 add grpc VolumeEcBlobDelete 2019-06-20 00:17:11 -07:00
Chris Lu 856da7aae2 ec volume support deletes 2019-06-19 22:57:14 -07:00
Chris Lu 935639b908 metrics: disk size for <collection, normal/EC> 2019-06-16 21:56:41 -07:00
Chris Lu 0fdb1e705d collect volume disk usage metrics 2019-06-16 02:44:20 -07:00
Chris Lu 57092d69ac go fmt 2019-06-10 21:33:32 -07:00
Chris Lu d344e0a035 fix ec related bugs 2019-06-05 23:20:26 -07:00
Chris Lu 450f4733ce report empty volume or ec shards 2019-06-05 13:32:33 -07:00
Chris Lu ca8a2bb534 go fmt 2019-06-04 22:04:10 -07:00
Chris Lu 2215e81be7 ui add ec shard statuses 2019-06-04 21:52:37 -07:00
Chris Lu 7e80b2b882 fix multiple bugs 2019-06-03 02:26:31 -07:00
Chris Lu 133b772fb5 destroy ec volume if it is empty 2019-06-01 01:51:28 -07:00
Chris Lu de1a35acdd conditionally fresh the shard locations 2019-05-31 02:22:12 -07:00
Chris Lu 47f1901843 ask for the ec volume version 2019-05-31 00:58:51 -07:00
Chris Lu 689930f092 forget shards that are broken 2019-05-31 00:19:13 -07:00
Chris Lu aaab2c25f5 resolve concurrent modification problem
n and err can be modified concurrently
2019-05-30 13:59:07 -07:00
Chris Lu 5dd67f9acf reading by recover from other shards 2019-05-28 23:48:39 -07:00
Chris Lu 3f9ecee40f working with reading remote intervals 2019-05-28 21:29:07 -07:00
Chris Lu 4f76342cbc WIP
no errors, but not returning file content
* the interval needs to use actual file zie
* need to read the actual version instead of the current version
2019-05-28 00:51:01 -07:00
Chris Lu 2858a954b3 read ec volume shards locations from master 2019-05-28 00:13:13 -07:00
Chris Lu 713596e781 caching ec shard locations 2019-05-27 22:54:58 -07:00
Chris Lu 217cde0a3b refactoring 2019-05-27 21:40:51 -07:00
Chris Lu 03b9291e5d volum server passes grpc option and master location to store 2019-05-27 21:22:23 -07:00
Chris Lu b4b407e403 add grpc ec shard read 2019-05-27 11:59:03 -07:00
Chris Lu a4f3d82c57 convert needle id to ec intervals to read from 2019-05-27 01:29:46 -07:00
Chris Lu db94a41f9e mount/unmount ec shards 2019-05-25 23:23:19 -07:00
Chris Lu 6f4b09b6a4 pb shard info uses ShardBits instead one message for one shard 2019-05-24 13:28:44 -07:00