gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
2707 commits 28 branches 278 tags 135 MiB
Commit graph

1348 commits

Author SHA1 Message Date
Chris Lu cf47f657af scaffold for volume server query feature 2019-10-06 22:35:08 -07:00
joeslay 9d03aa6a38
Merge pull request #8 from chrislusf/master 
merge seaweed master
 2019-10-01 17:55:31 +01:00
j.laycock eb27c2b037 Make releaseMemory private and return byte array instead, fix other platform compilation issues, reduce in-memory chunk size. 2019-10-01 12:21:44 +01:00
Lei Liu 2cb348a2c2 s3 api: fix listbucket common_prefixes issue 
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
 2019-09-29 21:15:40 +08:00
Lei Liu 3cc084269c master api: return http 404 when volumeId not exist 
Signed-off-by: Lei Liu <lei01.liu@horizon.ai>
 2019-09-29 14:23:39 +08:00
j.laycock d5f5acb734 limit locking physical memory to 80% of max physical memory 2019-09-20 12:44:29 +01:00
joeslay 48b093b3d2
Merge pull request #7 from chrislusf/master 
merge seaweed master
 2019-09-19 12:00:05 +01:00
j.laycock 476140fd6b minor change to setProcessWorkingSetSize function 2019-09-19 11:59:00 +01:00
Chris Lu 972e881d48 add more logs 
add more logs
 2019-09-16 09:48:30 -07:00
Chris Lu 20ae559437 add logs 
add logs for https://github.com/chrislusf/seaweedfs/issues/1064
 2019-09-16 08:19:10 -07:00
Chris Lu ad3efbb197 tweaking data types 2019-09-14 01:21:51 -07:00
Chris Lu b0e4771135
Merge pull request #1060 from divinerapier/master 
fix: non-thread-safe rand will panic
 2019-09-14 01:06:14 -07:00
divinerapier bb31462b52
fix: thread unsafe 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-09-13 20:06:02 +08:00
j.laycock 6fc6322c90 Change joeslay paths to chrislusf paths 2019-09-12 14:18:21 +01:00
joeslay d8c34b032f
Merge pull request #6 from chrislusf/master 
merge seaweed master
 2019-09-12 11:44:09 +01:00
Chris Lu ae53f63680 filer: recursive deletion optionally ignoring any errors 
fix https://github.com/chrislusf/seaweedfs/issues/1062
 2019-09-11 20:26:20 -07:00
Chris Lu 5e9c65469e volume: skip readonly checking when Destroy() 
fix https://github.com/chrislusf/seaweedfs/issues/1063
 2019-09-11 09:45:09 -07:00
j.laycock c1a928887a Add some comments/documentation 2019-09-11 10:42:37 +01:00
joeslay 9ea9a5e97a
Merge pull request #5 from chrislusf/master 
merge seaweed master
 2019-09-11 10:25:27 +01:00
j.laycock 15a4c91c28 Remove Finalizer 2019-09-10 17:05:54 +01:00
j.laycock 9c9dff7386 Virtual lock memory pages and Set Process Working set size to hint to windows harder not to write pages to disk as much, add finalize function, minor renaming 2019-09-10 16:18:51 +01:00
Chris Lu 02710f9624 refactoring: remove unused functions 2019-09-09 10:03:41 -07:00
Chris Lu 0f861d23a2 avoid nil needle map 
fix https://github.com/chrislusf/seaweedfs/issues/1061
 2019-09-09 09:48:08 -07:00
joeslay d757ed2fe3
Merge pull request #4 from chrislusf/master 
merge seaweed master
 2019-09-06 10:49:27 +01:00
Chris Lu 803345e6af remove glide 2019-09-06 00:15:59 -07:00
j.laycock 63acc8c972 rename mem_buffer to mBuffer 2019-09-05 13:46:29 +01:00
j.laycock 44ae041e80 rename more memorymapped variables to memoryMapMaxSizeMB 2019-09-05 13:42:23 +01:00
j.laycock 5885ab67b3 rename volume property MemoryMap to MemoryMapMaxSizeMB 2019-09-05 11:56:48 +01:00
j.laycock 5a6dfb05fc Only use truncate flag when creating non memory map files, rename and use memory map size for reserving max file size 2019-09-05 11:46:12 +01:00
j.laycock f6146d6ad2 use 0 instead of false for max memory flag 2019-09-05 11:43:18 +01:00
j.laycock 4257582db5 Allocate in 16MB chunks, make creation of memory maps always aligned to 16MB chunks 2019-09-05 11:15:01 +01:00
j.laycock 84f2dc3b53 Remove test code 2019-09-05 11:14:02 +01:00
Chris Lu 20d90dea5a filer: avoid hard-coded upload timeout 2019-09-05 01:26:59 -07:00
Tom Maxwell 4a878c0006 Changed the InMemory bool to a uint32 so that it can be used to alter how much space to reserve 2019-09-04 15:27:14 +01:00
joeslay 6ee65356e3
Merge pull request #3 from chrislusf/master 
merge seaweed master
 2019-09-04 14:58:04 +01:00
j.laycock cc756ddd79 Rename End_of_file variable, implement platform indepenent memory_map.go 2019-09-04 14:24:17 +01:00
j.laycock 9aa0859697 Remove inmemory from vaccum stuff 2019-09-03 18:20:45 +01:00
j.laycock de5a7e60a2 remove temp hack 2019-09-03 18:20:09 +01:00
j.laycock cbd0a98fa1 Add InMemory to Volume Grow structure 2019-09-03 18:19:02 +01:00
Chris Lu ee8bcb0c12 remove println 2019-09-03 09:22:56 -07:00
j.laycock 1f01eb78e8 Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct 2019-09-03 17:00:59 +01:00
Tom Maxwell d637d86d22 Changes to try and pass the URL parameters through - in memory flag not working still 2019-09-03 15:41:28 +01:00
j.laycock 9a459d984b Do not vacuum memory mapped files 2019-09-03 15:05:43 +01:00
Chris Lu 89a0cb2e11 1.43 2019-09-03 01:31:58 -07:00
j.laycock 0123c7a898 Change to max 2GB for now 2019-09-02 17:46:22 +01:00
joeslay e91139348e
Merge pull request #2 from chrislusf/master 
merge seaweed master
 2019-09-02 11:39:40 +01:00
j.laycock 595a1beff0 Swap imports to use joeslay 2019-09-02 11:28:40 +01:00
Chris Lu 60c9215a00 HCFS: avoid verbose file not found exception stack trace 2019-09-01 21:40:26 -07:00
j.laycock 1e62a2b233 Fix memory not being unmapped 2019-08-30 16:55:33 +01:00
j.laycock 840ccdc35d Refactor to pass memory maps by reference instead of value, fix memory maps not being created properly or written to properly 2019-08-30 12:30:00 +01:00
j.laycock dc50701e7d Overload createFile function, expose File in memory_map struct, delete from memory_map map on delete 2019-08-30 12:30:00 +01:00
j.laycock 523f3a12b3 use Os.File/uintptr instead of windows.Handle in memory map struct 2019-08-30 12:30:00 +01:00
j.laycock 0e5d3b1a70 Move memory_map_windows to new folder, intercept most of the read and write calls to volume dat files 2019-08-30 12:30:00 +01:00
Chris Lu 58168a8c52 volume: truncate dat files when creating 2019-08-26 13:01:57 -07:00
joeslay 36bde84000
Merge pull request #1 from chrislusf/master 
merge changes from chrislusf/seaweed master
 2019-08-23 17:03:37 +01:00
j.laycock 1e89e719b3 Add windows memory map functions to storage package 2019-08-23 16:19:00 +01:00
Chris Lu 35d471a818
Revert "use std encoding lib" 2019-08-23 07:59:09 -07:00
Chris Lu fa160e00f3
Merge pull request #1051 from xushuxun/master 
volume: read dat files' last modified time correctly
 2019-08-20 22:25:37 -07:00
xushuxun 9af9f36b89 volume: read dat files' last modified time no matter dat files are catWrite or readonly 2019-08-21 11:26:01 +08:00
divinerapier e798e4e9c5
use std encoding lib 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-08-19 18:07:54 +08:00
xushuxun 5904d78bd4 weed backup: add ttl and replication parameter 2019-08-16 11:05:22 +08:00
Chris Lu f9c1cd7370 avoid dead lock from dataFileAccessLock 
fix https://github.com/chrislusf/seaweedfs/issues/1043
 2019-08-15 09:24:54 -07:00
xushuxun 46ae35cd7e wdclient: clear client cache when failed to connect to master leader 2019-08-15 15:13:16 +08:00
Chris Lu d829df4f59 volume: protect against nil needle map 
fix @mastak reported nil problem in https://github.com/chrislusf/seaweedfs/issues/1037
 2019-08-14 01:08:01 -07:00
Chris Lu e40634e6b4 volume: fail the volume deletion if compaction is in progress 
fix https://github.com/chrislusf/seaweedfs/issues/1035
 2019-08-12 00:53:50 -07:00
Chris Lu 6f75df8660 server: fix starting default master list 2019-08-10 03:10:03 -07:00
Chris Lu 0dc43ecdca skip writing content if not modified 
related to https://github.com/chrislusf/seaweedfs/issues/1036
 2019-08-09 14:38:54 -07:00
Chris Lu b9f6236f8a filer: fix typo 
fix https://github.com/chrislusf/seaweedfs/issues/1031
 2019-08-06 00:42:19 -07:00
Chris Lu c79274f7eb notes on supported meta data store 2019-08-05 16:45:21 -07:00
Feng Ye fc0498798b Add etcd filestore 2019-08-01 10:16:45 +08:00
Chris Lu ca2b81a944 fix test error 2019-07-31 02:09:13 -07:00
Chris Lu 19360ab10a master: redirect clients to the new leader 2019-07-31 01:54:42 -07:00
Chris Lu 8afd8d35b3 master: followers can also lookup and redirect 
improve scalability
 2019-07-28 03:58:13 -07:00
Chris Lu 2c6cf72e73 refactoring 2019-07-28 01:55:05 -07:00
Chris Lu fb90d63589 shell: Added minor changes to fs.meta.save 
* added an option to set the output file name
* added an option to see verbose output, default to non-verbose mode

related to: https://github.com/chrislusf/seaweedfs/issues/1019
 2019-07-26 09:35:22 -07:00
divinerapier 9e3b77c3ad avoid double warpping 
Signed-off-by: divinerapier <poriter.coco@gmail.com>
 2019-07-24 16:41:45 +08:00
Chris Lu 344ce90315 remove weedfuse 2019-07-24 00:42:12 -07:00
Chris Lu 5956dfd08d mount: add umask option 
related to https://github.com/chrislusf/seaweedfs/issues/978
 2019-07-24 00:03:05 -07:00
Chris Lu 3f851feb59 1.42 2019-07-21 21:52:48 -07:00
Chris Lu 862c2cb8e6 go fmt 2019-07-21 21:51:38 -07:00
Chris Lu 79762385bd master: ensure only one exclusive vacuum process 
fix https://github.com/chrislusf/seaweedfs/issues/1011
 2019-07-21 21:49:10 -07:00
Chris Lu f3b99cbfe5 fix wrong alignment fix 2019-07-21 13:50:42 -07:00
Chris Lu 644c5c27b4 refactoring 2019-07-21 13:50:24 -07:00
Chris Lu 33b3366043 volume: avoid compacting wait group 2019-07-21 13:49:59 -07:00
Chris Lu 4b15c8f0c4 volume: lock writables changes 2019-07-21 13:49:09 -07:00
Chris Lu fd16adbde1 stricter checking before writes 2019-07-21 11:21:30 -07:00
Chris Lu 5a135fef87 purge old code 2019-07-21 10:45:36 -07:00
Chris Lu c54d9221b9 write requests also checks cookie if overwrites 
protect against edge cases, avoid https://github.com/chrislusf/seaweedfs/issues/1014
 2019-07-17 23:57:34 -07:00
Chris Lu 898d943b25 refactoring 2019-07-17 23:43:48 -07:00
Chris Lu 0264a7f50a set the max file key when ever possible 2019-07-17 23:23:01 -07:00
Chris Lu c33f423955 stop early if compaction fails 
fix https://github.com/chrislusf/seaweedfs/issues/1015
 2019-07-17 23:22:01 -07:00
Chris Lu aff911c00d skip all azuresb 2019-07-17 01:24:20 -07:00
Chris Lu 0ea98dc6a2 disable azure due to api changes 2019-07-17 00:39:36 -07:00
Chris Lu a7b1b23c58 fix wrong volume count 
fix https://github.com/chrislusf/seaweedfs/issues/1013
 2019-07-17 00:03:15 -07:00
Chris Lu 9e47c4749a shell: add fs.mv 
fix https://github.com/chrislusf/seaweedfs/issues/954
 2019-07-11 09:29:04 -07:00
Chris Lu 1d1b355f9e filer: set file name when uploading to a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1005
 2019-07-11 04:20:03 -07:00
Chris Lu 61b7a650f4 filer.copy default to 32MB chunks 2019-07-10 21:48:27 -07:00
Chris Lu 476a7c269c detect a filer path is a file or a directory 
fix https://github.com/chrislusf/seaweedfs/issues/1004
 2019-07-10 12:11:18 -07:00
Chris Lu 1e2127e261 s3: adding Location for multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/1002
 2019-07-08 12:37:45 -07:00
Chris Lu 62843991f2 s3: adjust object key 2019-07-08 12:37:20 -07:00