gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9550 commits 28 branches 278 tags 135 MiB
Commit graph

154 commits

Author SHA1 Message Date
Chris Lu 94525aa0fd allocate volume by disk type 2020-12-13 23:08:21 -08:00
Chris Lu f6a419c26c disk type configurable for each folder 2020-12-13 22:49:56 -08:00
Chris Lu 0d2ec832e2 rename from volumeType to diskType 2020-12-13 11:59:32 -08:00
Chris Lu d156c74ec0 volume server set volume type and heartbeat to the master 2020-12-13 03:11:24 -08:00
Chris Lu ae655033ac adjust logging 2020-12-11 16:57:53 -08:00
Chris Lu 96c48bc8a8 fix test 2020-11-28 03:03:43 -08:00
Chris Lu 2c913dde04 volume: detect and drop volumes with disk IO error 
from Jethro in slack:
is it possible to make the assign request a bit smarter? Currently I’m in the state that a disk failed but all assign request are being send to this volume. It would be cool if the master sees this and stopped using this volume.
e=HTTP(http://x:8089/913,045a782b63176edf) not 200 but 500 Internal Server Error
Body={"size":740167,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"ee4381e202212ff3aee647704c036689"}
e=HTTP(http://x:8089/913,045a782c90240077) not 200 but 500 Internal Server Error
Body={"size":792779,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"c43463ccc11eb6eb2fc306f407a6a953"}
e=HTTP(http://x:8089/913,045a782e6b7901ea) not 200 but 500 Internal Server Error
Body={"size":3962392,"error":"failed to write to local disk: write /mnt/v9/913.dat: input/output error","eTag":"04c91198e9b276c81f11dbf189af5d28"}
 2020-11-28 00:09:29 -08:00
Chris Lu 6d30b21b10 volume: add "-dir.idx" option for separate index storage 
fix https://github.com/chrislusf/seaweedfs/issues/1265
 2020-11-27 03:17:10 -08:00
Chris Lu b11449f955 pass in dir.idx parameter to DiskLocation 2020-11-26 14:59:03 -08:00
Chris Lu 1dd3a6ac36 avoid verbose logs 2020-11-09 15:56:11 -08:00
Chris Lu 0dafcf1f5a volume: detect max volume count changes based on disk usage 
fix https://github.com/chrislusf/seaweedfs/issues/1594
 2020-11-03 14:43:17 -08:00
Konstantin Lebedev 46303c36bf When the volume server is stopped, the master server immediately sees the deletion of volumes 2020-10-31 02:48:25 +05:00
Konstantin Lebedev 1ad1b8c4f6 collection Volume ReadOnly Count with detailed status 2020-10-28 17:38:26 +05:00
Konstantin Lebedev 2fb1fce8a0 avoid old values when missing read only flags 2020-10-25 16:21:40 +05:00
Chris Lu 9104cfa744 reduce locks 2020-10-24 19:40:35 -07:00
Konstantin Lebedev dc2e13092d add number of read only volumes metric 2020-10-15 15:48:40 +05:00
Chris Lu a1c01d716b volume: avoid deadlock when deleting volumes 
fix https://github.com/chrislusf/seaweedfs/issues/1501
 2020-10-01 07:10:03 -07:00
James Hartig 8e54e34576 volume: Don't unmount before deleting volume in copy 
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
 2020-09-01 22:00:07 -04:00
James Hartig 3ccfa4c6ad Added VolumeMarkWritable and VolumeStatus grpc methods 
This is necessary for copy to mark as read-only and then restore the
original state afterwards.
 2020-08-19 11:42:56 -04:00
Chris Lu 7e91ae592c pass in option to read deleted entries 
not working yet
 2020-08-18 17:37:26 -07:00
Chris Lu 6a92f0bc7a refactoring to typed Size 
Go is amazing with refactoring!
 2020-08-18 17:04:28 -07:00
Chris Lu f43146b237 resolve directories if containing home directory 2020-07-16 22:50:14 -07:00
Chris Lu 4d1484628a refactoring 2020-07-03 16:41:30 -07:00
Evgenii Kozlov 0e0db70f55 Set volumes ReadOnly if low free disk space 2020-06-05 18:18:15 +03:00
Chris Lu 5568395edd Revert "Revert "Merge pull request #1299 from song-zhang/master"" 
This reverts commit afb6a1dbb4.
 2020-05-06 15:37:17 -07:00
Chris Lu afb6a1dbb4 Revert "Merge pull request #1299 from song-zhang/master" 
This reverts commit 9016fa19ba, reversing
changes made to 47234760f4.
 2020-05-04 20:34:26 -07:00
zhangsong f9e8702bb4 use async write to persistent file to disk - part1 2020-05-04 17:39:44 +08:00
Chris Lu bafa95045b volume: deletion checks all disk locations 
fix https://github.com/chrislusf/seaweedfs/issues/1283
 2020-04-21 14:49:58 -07:00
Chris Lu e4af63a721 volume server: accept fsync=true in write requests 2020-04-11 21:39:16 -07:00
Chris Lu 9fa065f600 typo in logs 2020-04-09 23:43:09 -07:00
Chris Lu e63a79ade8 better handle lock in case of exception 2020-03-24 18:41:25 -07:00
Chris Lu 3137777d83 volume: automatically detect max volume count 2020-03-22 16:21:42 -07:00
Chris Lu b964bbab3d fix compilation 2020-03-17 10:01:24 -07:00
Chris Lu c3cb6fa1d7 volume: compaction can cause readonly volumes 
address https://github.com/chrislusf/seaweedfs/issues/1233
 2020-03-17 09:43:57 -07:00
Chris Lu 560df51def refactoring 2020-03-15 03:11:26 -07:00
Chris Lu ed0acd1722 go fmt 2020-02-26 16:52:57 -08:00
Chris Lu 40ae533fa3 shell: add volume.configure.replication to change replication for a volume 
fix https://github.com/chrislusf/seaweedfs/issues/1192
 2020-02-02 15:37:23 -08:00
Chris Lu 09ca936c78 shell: add ec.decode command 2019-12-23 12:48:20 -08:00
Chris Lu f61de28c69 volume: add deletion capability for previously readonly volumes 2019-12-19 00:42:46 -08:00
Chris Lu 1becbce657 display remote volumes on volume server ui page 2019-12-02 23:23:54 -08:00
Chris Lu 6383b45bd0 add lock variable 2019-12-02 20:49:50 -08:00
Chris Lu c5c1d83d91 Merge branch 'master' into refactoring_dat_backend 2019-11-08 22:54:41 -08:00
Chris Lu c34ffed43f go fmt 2019-11-08 22:47:50 -08:00
zhangsong 1dd101f782 use read lock to avoid io hang during heartbeat 2019-11-08 20:27:55 +08:00
Chris Lu b7156291a8 adjust variable names 2019-10-29 23:16:43 -07:00
Chris Lu faec9076a4 adjust parameter names 2019-10-21 22:57:01 -07:00
j.laycock 6fc6322c90 Change joeslay paths to chrislusf paths 2019-09-12 14:18:21 +01:00
j.laycock 44ae041e80 rename more memorymapped variables to memoryMapMaxSizeMB 2019-09-05 13:42:23 +01:00
Tom Maxwell 4a878c0006 Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve 2019-09-04 15:27:14 +01:00
j.laycock cbd0a98fa1 Add InMemory to Volume Grow structure 2019-09-03 18:19:02 +01:00
j.laycock 1f01eb78e8 Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct 2019-09-03 17:00:59 +01:00
Tom Maxwell d637d86d22 Changes to try and pass the URL parameters through - in memory flag not working still 2019-09-03 15:41:28 +01:00
j.laycock 595a1beff0 Swap imports to use joeslay 2019-09-02 11:28:40 +01:00
Chris Lu d829df4f59 volume: protect against nil needle map 
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
 2019-08-14 01:08:01 -07:00
Chris Lu fd16adbde1 stricter checking before writes 2019-07-21 11:21:30 -07:00
Chris Lu 898d943b25 refactoring 2019-07-17 23:43:48 -07:00
Chris Lu a3d1296ed9 go fmt 2019-06-27 12:18:59 -07:00
Chris Lu 6883f9e322 mark volume readonly before ec encoding 2019-06-26 23:02:22 -07:00
Chris Lu 115558e5f5 adjust counters 2019-06-17 21:02:50 -07:00
Chris Lu 935639b908 metrics: disk size for <collection, normal/EC> 2019-06-16 21:56:41 -07:00
Chris Lu 0fdb1e705d collect volume disk usage metrics 2019-06-16 02:44:20 -07:00
Chris Lu 450f4733ce report empty volume or ec shards 2019-06-05 13:32:33 -07:00
Chris Lu 713596e781 caching ec shard locations 2019-05-27 22:54:58 -07:00
Chris Lu 03b9291e5d volum server passes grpc option and master location to store 2019-05-27 21:22:23 -07:00
Chris Lu a4f3d82c57 convert needle id to ec intervals to read from 2019-05-27 01:29:46 -07:00
Chris Lu a463759edf adjust error message 2019-05-26 01:14:42 -07:00
Chris Lu db94a41f9e mount/unmount ec shards 2019-05-25 23:23:19 -07:00
Chris Lu 17ac1290c0 volume: load ec shards during heartbeats to master 2019-05-21 22:41:20 -07:00
Chris Lu 9beea63960 memory alignment 
fix https://github.com/chrislusf/seaweedfs/issues/939
 2019-04-24 09:23:19 -07:00
Chris Lu 440111a349 volume: support http status 304 for the same file id 2019-04-21 13:33:23 -07:00
Chris Lu 132921ad41 adjust log or error 2019-04-21 10:14:17 -07:00
Chris Lu 316bd27f75 add remove volumes with version info 2019-04-20 23:53:37 -07:00
Chris Lu 6fc1f53018 shell: add command volume.move 2019-04-20 11:35:23 -07:00
Chris Lu 3b3651dea3 volume: atomic copying file, adds version and stopOffset 2019-04-19 12:29:49 -07:00
Chris Lu e5506152c0 refactoring 2019-04-18 21:43:36 -07:00
Chris Lu 3e8a3a8fec fix race detector found problems 2019-04-14 23:00:37 -07:00
Chris Lu 95e0520182 weed volume: add grpc operation to relicate a volume to local 2019-03-23 11:33:34 -07:00
Chris Lu ece9d13312 volume info collect compact revision number 2019-03-18 09:32:21 -07:00
Chris Lu aca653c08b weed shell: list volumes 2019-03-17 20:27:08 -07:00
bingoohuang ab6be025d7 go fmt and fix some typo 2019-01-17 09:17:19 +08:00
Chris Lu 1478d7ea21 reduce file seek when writing 2018-12-31 15:08:32 -08:00
Chris Lu 0c932d1738 add volume delete grpc API 
fix https://github.com/chrislusf/seaweedfs/issues/808
 2018-12-29 00:03:30 -08:00
Chris Lu 66a353dcb5 remove volume server /admin/volume/delete 2018-10-15 01:26:49 -07:00
Chris Lu b423bb9e2d migrate assign volume to grpc API on volume server 2018-10-15 00:40:46 -07:00
Chris Lu 76cbe8bf33 instant notification of new volumes added or deleted 2018-08-24 01:26:56 -07:00
Chris Lu 7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu d4d7ced922 refactoring: add type for needle id, offset 
later the type size can possibly be adjusted
 2018-07-08 02:28:04 -07:00
Chris Lu 43e3f5724c use fixed list of masters in both filer and volume servers 2018-06-01 00:39:39 -07:00
Chris Lu 942c2cbd7b mv pb to master_pb 2018-05-09 23:11:54 -07:00
Chris Lu bd97cbc523 skip busy reporting back to master 
possible fix for https://github.com/chrislusf/seaweedfs/issues/499
 2017-05-23 21:12:03 -07:00
brstgt e074a54a20 Delete volumes online without restarting volume server 2017-01-20 13:02:37 +01:00
brstgt 492f93416d Mount and unmount volumes online without restarting volume server 2017-01-20 12:49:20 +01:00
Chris Lu 5b0f5f456e volume server get notified of leader change 2017-01-18 09:34:27 -08:00
Chris Lu 3065506b38 volume servers always connect to the master leader 2017-01-13 10:20:40 -08:00
Chris Lu e46c3415f7 gRpc for master~volume heartbeat 2017-01-10 01:01:12 -08:00
Chris Lu ed44f12f6d support Fallocate on linux 2017-01-08 11:01:46 -08:00
sparklxb 86a7c56275 support additional header name-value pairs 2017-01-08 09:16:40 +08:00
Chris Lu 36f9633223 add locks for location.volumes 
fix https://github.com/chrislusf/seaweedfs/issues/392
 2016-11-06 20:55:22 -08:00
Chris Lu d981eb282f refactor volume_read_write.go out of volume.go 2016-07-03 00:10:27 -07:00
霍晓栋 1ef81ac518 filer could detect master nodes healthy status and choose a working one when encountering SPOF 2016-06-07 11:38:40 +08:00