gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
2682 commits 28 branches 278 tags 135 MiB
Commit graph

1327 commits

Author SHA1 Message Date
Chris Lu 02710f9624 refactoring: remove unused functions 2019-09-09 10:03:41 -07:00
Chris Lu 0f861d23a2 avoid nil needle map 
fix https://github.com/chrislusf/seaweedfs/issues/1061
 2019-09-09 09:48:08 -07:00
joeslay d757ed2fe3
Merge pull request #4 from chrislusf/master 
merge seaweed master
 2019-09-06 10:49:27 +01:00
Chris Lu 803345e6af remove glide 2019-09-06 00:15:59 -07:00
j.laycock 63acc8c972 rename mem_buffer to mBuffer 2019-09-05 13:46:29 +01:00
j.laycock 44ae041e80 rename more memorymapped variables to memoryMapMaxSizeMB 2019-09-05 13:42:23 +01:00
j.laycock 5885ab67b3 rename volume property MemoryMap to MemoryMapMaxSizeMB 2019-09-05 11:56:48 +01:00
j.laycock 5a6dfb05fc Only use truncate flag when creating non memory map files, rename and use memory map size for reserving max file size 2019-09-05 11:46:12 +01:00
j.laycock f6146d6ad2 use 0 instead of false for max memory flag 2019-09-05 11:43:18 +01:00
j.laycock 4257582db5 Allocate in 16MB chunks, make creation of memory maps always aligned to 16MB chunks 2019-09-05 11:15:01 +01:00
j.laycock 84f2dc3b53 Remove test code 2019-09-05 11:14:02 +01:00
Chris Lu 20d90dea5a filer: avoid hard-coded upload timeout 2019-09-05 01:26:59 -07:00
Tom Maxwell 4a878c0006 Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve 2019-09-04 15:27:14 +01:00
joeslay 6ee65356e3
Merge pull request #3 from chrislusf/master 
merge seaweed master
 2019-09-04 14:58:04 +01:00
j.laycock cc756ddd79 Rename End_of_file variable, implement platform indepenent memory_map.go 2019-09-04 14:24:17 +01:00
j.laycock 9aa0859697 Remove inmemory from vaccum stuff 2019-09-03 18:20:45 +01:00
j.laycock de5a7e60a2 remove temp hack 2019-09-03 18:20:09 +01:00
j.laycock cbd0a98fa1 Add InMemory to Volume Grow structure 2019-09-03 18:19:02 +01:00
Chris Lu ee8bcb0c12 remove println 2019-09-03 09:22:56 -07:00
j.laycock 1f01eb78e8 Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct 2019-09-03 17:00:59 +01:00
Tom Maxwell d637d86d22 Changes to try and pass the URL parameters through - in memory flag not working still 2019-09-03 15:41:28 +01:00
j.laycock 9a459d984b Do not vacuum memory mapped files 2019-09-03 15:05:43 +01:00
Chris Lu 89a0cb2e11 1.43 2019-09-03 01:31:58 -07:00
j.laycock 0123c7a898 Change to max 2GB for now 2019-09-02 17:46:22 +01:00
joeslay e91139348e
Merge pull request #2 from chrislusf/master 
merge seaweed master
 2019-09-02 11:39:40 +01:00
j.laycock 595a1beff0 Swap imports to use joeslay 2019-09-02 11:28:40 +01:00
Chris Lu 60c9215a00 HCFS: avoid verbose file not found exception stack trace 2019-09-01 21:40:26 -07:00
j.laycock 1e62a2b233 Fix memory not being unmapped 2019-08-30 16:55:33 +01:00
j.laycock 840ccdc35d Refactor to pass memory maps by reference instead of value, fix memory maps not being created properly or written to properly 2019-08-30 12:30:00 +01:00
j.laycock dc50701e7d Overload createFile function, expose File in memory_map struct, delete from memory_map map on delete 2019-08-30 12:30:00 +01:00
j.laycock 523f3a12b3 use Os.File/uintptr instead of windows.Handle in memory map struct 2019-08-30 12:30:00 +01:00
j.laycock 0e5d3b1a70 Move memory_map_windows to new folder, intercept most of the read and write calls to volume dat files 2019-08-30 12:30:00 +01:00
Chris Lu 58168a8c52 volume: truncate dat files when creating 2019-08-26 13:01:57 -07:00
joeslay 36bde84000
Merge pull request #1 from chrislusf/master 
merge changes from chrislusf/seaweed master
 2019-08-23 17:03:37 +01:00
j.laycock 1e89e719b3 Add windows memory map functions to storage package 2019-08-23 16:19:00 +01:00
Chris Lu 35d471a818
Revert "use std encoding lib" 2019-08-23 07:59:09 -07:00
Chris Lu fa160e00f3
Merge pull request #1051 from xushuxun/master 
volume: read dat files' last modified time correctly
 2019-08-20 22:25:37 -07:00
xushuxun 9af9f36b89 volume: read dat files' last modified time no matter dat files are catWrite or readonly 2019-08-21 11:26:01 +08:00
divinerapier e798e4e9c5
use std encoding lib 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-08-19 18:07:54 +08:00
xushuxun 5904d78bd4 weed backup: add ttl and replication parameter 2019-08-16 11:05:22 +08:00
Chris Lu f9c1cd7370 avoid dead lock from dataFileAccessLock 
fix https://github.com/chrislusf/seaweedfs/issues/1043
 2019-08-15 09:24:54 -07:00
xushuxun 46ae35cd7e wdclient: clear client cache when failed to connect to master leader 2019-08-15 15:13:16 +08:00
Chris Lu d829df4f59 volume: protect against nil needle map 
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
 2019-08-14 01:08:01 -07:00
Chris Lu e40634e6b4 volume: fail the volume deletion if compaction is in progress 
fix https://github.com/chrislusf/seaweedfs/issues/1035
 2019-08-12 00:53:50 -07:00
Chris Lu 6f75df8660 server: fix starting default master list 2019-08-10 03:10:03 -07:00
Chris Lu 0dc43ecdca skip writing content if not modified 
related to https://github.com/chrislusf/seaweedfs/issues/1036
 2019-08-09 14:38:54 -07:00
Chris Lu b9f6236f8a filer: fix typo 
fix https://github.com/chrislusf/seaweedfs/issues/1031
 2019-08-06 00:42:19 -07:00
Chris Lu c79274f7eb notes on supported meta data store 2019-08-05 16:45:21 -07:00
Feng Ye fc0498798b Add etcd filestore 2019-08-01 10:16:45 +08:00
Chris Lu ca2b81a944 fix test error 2019-07-31 02:09:13 -07:00
Chris Lu 19360ab10a master: redirect clients to the new leader 2019-07-31 01:54:42 -07:00
Chris Lu 8afd8d35b3 master: followers can also lookup and redirect 
improve scalability
 2019-07-28 03:58:13 -07:00
Chris Lu 2c6cf72e73 refactoring 2019-07-28 01:55:05 -07:00
Chris Lu fb90d63589 shell: Added minor changes to fs.meta.save 
* added an option to set the output file name
* added an option to see verbose output, default to non-verbose mode

related to: https://github.com/chrislusf/seaweedfs/issues/1019
 2019-07-26 09:35:22 -07:00
divinerapier 9e3b77c3ad avoid double warpping 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-07-24 16:41:45 +08:00
Chris Lu 344ce90315 remove weedfuse 2019-07-24 00:42:12 -07:00
Chris Lu 5956dfd08d mount: add umask option 
related to https://github.com/chrislusf/seaweedfs/issues/978
 2019-07-24 00:03:05 -07:00
Chris Lu 3f851feb59 1.42 2019-07-21 21:52:48 -07:00
Chris Lu 862c2cb8e6 go fmt 2019-07-21 21:51:38 -07:00
Chris Lu 79762385bd master: ensure only one exclusive vacuum process 
fix https://github.com/chrislusf/seaweedfs/issues/1011
 2019-07-21 21:49:10 -07:00
Chris Lu f3b99cbfe5 fix wrong alignment fix 2019-07-21 13:50:42 -07:00
Chris Lu 644c5c27b4 refactoring 2019-07-21 13:50:24 -07:00
Chris Lu 33b3366043 volume: avoid compacting wait group 2019-07-21 13:49:59 -07:00
Chris Lu 4b15c8f0c4 volume: lock writables changes 2019-07-21 13:49:09 -07:00
Chris Lu fd16adbde1 stricter checking before writes 2019-07-21 11:21:30 -07:00
Chris Lu 5a135fef87 purge old code 2019-07-21 10:45:36 -07:00
Chris Lu c54d9221b9 write requests also checks cookie if overwrites 
protect against edge cases, avoid https://github.com/chrislusf/seaweedfs/issues/1014
 2019-07-17 23:57:34 -07:00
Chris Lu 898d943b25 refactoring 2019-07-17 23:43:48 -07:00
Chris Lu 0264a7f50a set the max file key when ever possible 2019-07-17 23:23:01 -07:00
Chris Lu c33f423955 stop early if compaction fails 
fix https://github.com/chrislusf/seaweedfs/issues/1015
 2019-07-17 23:22:01 -07:00
Chris Lu aff911c00d skip all azuresb 2019-07-17 01:24:20 -07:00
Chris Lu 0ea98dc6a2 disable azure due to api changes 2019-07-17 00:39:36 -07:00
Chris Lu a7b1b23c58 fix wrong volume count 
fix https://github.com/chrislusf/seaweedfs/issues/1013
 2019-07-17 00:03:15 -07:00
Chris Lu 9e47c4749a shell: add fs.mv 
fix https://github.com/chrislusf/seaweedfs/issues/954
 2019-07-11 09:29:04 -07:00
Chris Lu 1d1b355f9e filer: set file name when uploading to a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1005
 2019-07-11 04:20:03 -07:00
Chris Lu 61b7a650f4 filer.copy default to 32MB chunks 2019-07-10 21:48:27 -07:00
Chris Lu 476a7c269c detect a filer path is a file or a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1004
 2019-07-10 12:11:18 -07:00
Chris Lu 1e2127e261 s3: adding Location for multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/1002
 2019-07-08 12:37:45 -07:00
Chris Lu 62843991f2 s3: adjust object key 2019-07-08 12:37:20 -07:00
Chris Lu cf2804eebd 1.41 2019-07-07 20:40:20 -07:00
Chris Lu 7872fc38ea filer leveldb2: correct directory listing 
fix https://github.com/chrislusf/seaweedfs/issues/1001
 2019-07-07 20:37:48 -07:00
Chris Lu fe2882ce7d fix test 2019-07-02 21:42:06 -07:00
Chris Lu 36250154cc remove println 2019-07-02 21:33:23 -07:00
Chris Lu 4d579f7f57 adjust 2019-07-02 21:28:51 -07:00
Chris Lu 23d36dfbaf 1.40 
enable azure replication
 2019-07-02 21:27:05 -07:00
Chris Lu dcc331cf5a parallelize leveldb 2019-07-02 21:25:53 -07:00
Chris Lu 88fed85bba temp skipping azure 
can not release due to https://github.com/Azure/azure-pipeline-go/issues/23
 2019-06-30 23:17:45 -07:00
Chris Lu ce5aab66e9 add versions 2019-06-30 22:43:10 -07:00
Chris Lu 36022f9416 1.39 2019-06-30 00:50:17 -07:00
Chris Lu 15ecf4e3bf filer: default to leveldb2 2019-06-30 00:44:57 -07:00
Chris Lu ee933cd859 filer: add metrics for go memory, gc stats 2019-06-29 03:02:28 -07:00
Chris Lu 92c7f7e069 fix compilation error 2019-06-28 11:19:08 -07:00
Chris Lu 9f3f2f7c79 protect locations slice 
fix https://github.com/chrislusf/seaweedfs/issues/995
 2019-06-28 09:47:29 -07:00
Chris Lu 327336ecf3 filer: avoid concurrent modification to result slice 
fix https://github.com/chrislusf/seaweedfs/issues/972
 2019-06-28 01:12:41 -07:00
Chris Lu a3d1296ed9 go fmt 2019-06-27 12:18:59 -07:00
Chris Lu ec75b2d761 volume: fix bug with 8000GB version if using in memory index 
fix https://github.com/chrislusf/seaweedfs/issues/994
 2019-06-27 12:18:45 -07:00
Chris Lu 6883f9e322 mark volume readonly before ec encoding 2019-06-26 23:02:22 -07:00
Chris Lu c2960c7a86 defaulting file ttl to volume ttl 
fix https://github.com/chrislusf/seaweedfs/issues/992
 2019-06-26 11:18:23 -07:00
Lei Liu 1d9b75b536
weed.go: remove unused parameter 
Signed-off-by: Lei Liu <liul.stone@gmail.com>
 2019-06-26 10:46:32 +08:00
Chris Lu 359563e95a refactoring 2019-06-25 09:49:27 -07:00