Commit graph

2398 commits

Author SHA1 Message Date
Chris Lu 5c6c757619 clean up test files 2019-05-29 00:34:30 -07:00
Chris Lu 5dd67f9acf reading by recover from other shards 2019-05-28 23:48:39 -07:00
Chris Lu 3f9ecee40f working with reading remote intervals 2019-05-28 21:29:07 -07:00
Chris Lu 302d9fbc6d adjust error message for file not found 2019-05-28 14:27:21 -07:00
Chris Lu 4f76342cbc WIP
no errors, but not returning file content
* the interval needs to use actual file zie
* need to read the actual version instead of the current version
2019-05-28 00:51:01 -07:00
Chris Lu 2858a954b3 read ec volume shards locations from master 2019-05-28 00:13:13 -07:00
Chris Lu 713596e781 caching ec shard locations 2019-05-27 22:54:58 -07:00
Chris Lu 3a8c1055a2 refactoring ecx to ecVolume 2019-05-27 22:00:36 -07:00
Chris Lu 217cde0a3b refactoring 2019-05-27 21:40:51 -07:00
Chris Lu 03b9291e5d volum server passes grpc option and master location to store 2019-05-27 21:22:23 -07:00
Chris Lu ac1ba3b667 remove comments 2019-05-27 21:21:52 -07:00
Chris Lu b4b407e403 add grpc ec shard read 2019-05-27 11:59:03 -07:00
Chris Lu a4f3d82c57 convert needle id to ec intervals to read from 2019-05-27 01:29:46 -07:00
Chris Lu a463759edf adjust error message 2019-05-26 01:14:42 -07:00
Chris Lu 92f9c7b1aa adjust ec shard status on disconnect 2019-05-26 01:05:08 -07:00
Chris Lu ac14cd5ad0 wdclient need to have ec shard info on initial dump 2019-05-26 01:01:04 -07:00
Chris Lu b58e25e588 broadcast vid->location map to master clients for ec chards 2019-05-26 00:49:15 -07:00
Chris Lu af67d99ca4 incrementally update master ec shards state 2019-05-26 00:21:17 -07:00
Chris Lu db94a41f9e mount/unmount ec shards 2019-05-25 23:23:19 -07:00
Chris Lu 41e8ae61f8 generate, balance, delete copied shards, delete old volume 2019-05-25 14:02:06 -07:00
Chris Lu f0e6574d5e allocate ec shards to volume servers 2019-05-25 02:02:44 -07:00
Chris Lu 6f4b09b6a4 pb shard info uses ShardBits instead one message for one shard 2019-05-24 13:28:44 -07:00
Chris Lu 228850d588 shard id starts from zero 2019-05-24 11:52:23 -07:00
Chris Lu 24b8ad9a14 push the ec shard info to master 2019-05-23 23:47:49 -07:00
Chris Lu 8838176d82 register ec shards to topology 2019-05-23 23:34:29 -07:00
Chris Lu 8a96445f40 register ec shards to each data node 2019-05-23 22:51:18 -07:00
Chris Lu 4659d80035 prepare to register ec shard info in master 2019-05-23 00:42:28 -07:00
Chris Lu e913ee380a add comments 2019-05-23 00:04:24 -07:00
Chris Lu 7180520889 ec shard info can be queried via VolumeList() 2019-05-22 22:44:28 -07:00
Chris Lu 17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00
Chris Lu 54b835e1ae filer: redis cluster add option for password
fix https://github.com/chrislusf/seaweedfs/issues/971
2019-05-20 09:00:30 -07:00
Chris Lu fbbc74abb4 adds VolumeEcGenerateSlices, VolumeEcCopy 2019-05-20 00:53:17 -07:00
Chris Lu ae499fd5aa Merge branch 'master' into erasure_coding 2019-05-19 21:38:18 -07:00
Chris Lu 698dea779c refactoring 2019-05-19 21:37:49 -07:00
Chris Lu 693d6c0f1b clean up tests 2019-05-19 21:16:01 -07:00
Chris Lu 0094ca3f8a mark the webdav as unstable 2019-05-19 21:08:26 -07:00
Chris Lu 6386a3174b able to validate by randomly selected ec files 2019-05-19 14:24:33 -07:00
Chris Lu 7c2c60c376 add locating data inside the ec files 2019-05-19 03:01:58 -07:00
Chris Lu 87f63b9c08 generate ec01~ec14, generate ecx file with sorted needle values 2019-05-18 22:46:24 -07:00
Chris Lu 12dc6608f0 Merge branch 'master' into erasure_coding 2019-05-18 11:16:07 -07:00
Chris Lu f2c4c888f6 1.34 2019-05-17 22:53:30 -07:00
Chris Lu 8123a488cd filer: optimize leveldb parameters 2019-05-17 18:14:25 -07:00
Chris Lu 76cf57a0e1 volume: optimzation for leveldb 2019-05-17 17:34:03 -07:00
Chris Lu 0fe286a6ca filer: optimization for leveldb, add leveldb2
large filer db may see high CPU and disk usage due to background compaction
2019-05-17 17:33:49 -07:00
Chris Lu 7e0c3bb732 filer: adjust recommended mysql meta data type to LONGBLOB
fix https://github.com/chrislusf/seaweedfs/issues/966
2019-05-17 09:11:08 -07:00
Chris Lu bf9d490538 fix for tests 2019-05-17 02:28:20 -07:00
Chris Lu 82b0759493 filer: migrating filer store from persisting shorter structured file id instead of a string 2019-05-17 02:03:23 -07:00
Chris Lu 939de1e832
Update README.md 2019-05-15 23:25:59 -07:00
Chris Lu 8156958ee9 move function to make travis happy 2019-05-15 10:02:44 -07:00
Chris Lu 0a36f628c6 testing RS coding 2019-05-15 01:02:00 -07:00