gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
1577 commits 28 branches 278 tags 135 MiB
Commit graph

426 commits

Author SHA1 Message Date
Chris Lu 3a6f9c9bcf increase test size 2018-06-23 18:30:13 -07:00
Chris Lu 7f7e4e9885 fix error for deleted files during compaction 
deletion during commit may cause trouble when make up the difference during commitCompact()
 2018-06-23 18:24:59 -07:00
Chris Lu 3699996985 add compaction test 2018-06-23 16:48:19 -07:00
Chris Lu 14c6f12b0c adjust interface parameter name and doc 2018-06-17 13:24:57 -07:00
Chris Lu db0c25b3d8 delete old entry only after successful insert 2018-06-17 13:02:22 -07:00
Chris Lu eb2acd11c2 decouple from viper for filer store 2018-06-17 13:01:57 -07:00
Chris Lu 0467195f07 fix file closing on windows 2018-06-13 00:36:51 -07:00
Chris Lu 4433b4c006 treat 0 sized file as normal 
fix https://github.com/chrislusf/seaweedfs/issues/409
 2018-06-12 21:11:26 -07:00
Chris Lu 03f50180f3 simplifying the leader election by raft 
fixing https://github.com/chrislusf/seaweedfs/issues/629
 2018-06-12 01:54:09 -07:00
Chris Lu 69b4f93830 fix compilation for go tip 2018-06-12 00:58:43 -07:00
Chris Lu 17918f0b82 fix compilation error on go tip 2018-06-11 23:51:36 -07:00
Chris Lu eaf000eff1 fix error on go tip 2018-06-11 23:39:42 -07:00
Chris Lu d2a811eef2 fix compilation 2018-06-11 23:29:09 -07:00
Chris Lu 5bd72696ac weed mount add ttl option 2018-06-11 23:13:33 -07:00
Chris Lu 98110c1697 add collection, replication, ttl to each file 2018-06-10 16:57:32 -07:00
Chris Lu f1273073fc switch to fs.NodeRequestLookuper 
in order to set the entry valid duration
 2018-06-07 00:07:37 -07:00
Chris Lu cec1d97035 recursively move files and directories 2018-06-06 23:39:30 -07:00
Chris Lu ae23e46313 better error log 2018-06-06 23:06:19 -07:00
Chris Lu e755540be9 fix caching during directory listing 2018-06-06 22:55:59 -07:00
Chris Lu daabdfe357 remove nodemap, fix directory listing cache 2018-06-06 22:48:51 -07:00
Chris Lu 5c25d29272 support renaming files 2018-06-06 22:11:01 -07:00
Chris Lu b3447f4375 adjust logging 2018-06-06 02:21:36 -07:00
Chris Lu 6816661b0f fixed file handle by file full path 2018-06-06 02:09:57 -07:00
Chris Lu 299312c805 use separate filer grpc port 2018-06-05 23:37:41 -07:00
Chris Lu 95fe745a0c compatible with previous json rendering 2018-06-04 19:28:59 -07:00
Chris Lu ebf58709d3 remove unused register file function 2018-06-01 23:24:34 -07:00
Chris Lu 43e3f5724c use fixed list of masters in both filer and volume servers 2018-06-01 00:39:39 -07:00
Chris Lu a6f7f9b0b8 add missing modify and create times 2018-05-31 22:50:19 -07:00
Chris Lu 7efeb146c5 fix log 2018-05-31 22:49:55 -07:00
Chris Lu 4d1eedfa28 prepare to merge with master 2018-05-30 22:29:18 -07:00
Chris Lu dc13e10637 fix copy error on 0 size files 2018-05-30 22:28:14 -07:00
Chris Lu a218eaf1f0 fix log 2018-05-30 22:09:24 -07:00
Chris Lu 430eb67489 handle large file copy when write request is larger than buffer 2018-05-30 22:02:21 -07:00
Chris Lu 44acf4b756 fix file name 
f.Name() is not the base file name
 2018-05-30 20:48:13 -07:00
Chris Lu 0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu 26e7cd8c75 assign a different volume on large file copying 2018-05-30 01:05:26 -07:00
Chris Lu 2fe0d479f1 filer.copy supports chunking 2018-05-30 00:54:56 -07:00
Chris Lu 2d7dea1a09 a little refactoring 2018-05-29 23:52:27 -07:00
Chris Lu 8a48baa056 add single chunk file copying to new filer 2018-05-29 23:46:45 -07:00
Chris Lu 78aabf66ce add error details 2018-05-29 23:46:17 -07:00
Chris Lu 4e3ea49cff properly working local write buffer 2018-05-29 01:21:21 -07:00
Chris Lu c4b92e17d0 fix isPerfectAppend 2018-05-28 22:45:52 -07:00
Chris Lu 74332e1a61 minor 2018-05-28 14:32:16 -07:00
Chris Lu be0e88a606 fix chunk size limit for default 0 2018-05-28 13:44:27 -07:00
Chris Lu 5c4480ec6c add mountOptions.chunkSizeLimitMB, remove cmdMount.IsDebug 2018-05-28 13:42:25 -07:00
Chris Lu 8ab7dd9d08 weed mount add options for collection and replication 2018-05-28 13:24:48 -07:00
Chris Lu d0b238d2db cache local writes before flushing to volume server 2018-05-28 12:30:17 -07:00
Chris Lu 07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu dd5661b297 adding favicon to filer also 2018-05-28 02:35:58 -07:00
Chris Lu 9b603f5ffa split filer read into 2 files 2018-05-28 02:24:14 -07:00
Chris Lu 8db9319a06 refactoring: go fmt, reorg 2018-05-27 23:59:49 -07:00
Chris Lu 480a073f1f refactoring: split into 4 files 2018-05-27 23:53:10 -07:00
Chris Lu 5b844d7011 pass reference of master server 
SendHeartbeat passes lock by value: weed_server.MasterServer contains
sync.Mutex
 2018-05-27 11:58:00 -07:00
Chris Lu 8b0718ac92 go vet 2018-05-27 11:56:49 -07:00
Chris Lu 458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu 8647191bee removing filer package! 2018-05-27 11:52:15 -07:00
Chris Lu 03d76479a5 add reds 2018-05-27 11:14:29 -07:00
Chris Lu a6a6b87f1b adjust readme 2018-05-27 11:14:12 -07:00
Chris Lu ab86d263c9 add Cassandra 2018-05-27 00:01:15 -07:00
Chris Lu 603de2d5db skip deletion if entry not found 2018-05-27 00:00:56 -07:00
Chris Lu f124ebab1d fix ordering 2018-05-27 00:00:11 -07:00
Chris Lu 1c0a2a7db8 reformat 2018-05-26 23:59:56 -07:00
Chris Lu 869161a261 support both mysql and postgres 2018-05-26 22:02:49 -07:00
Chris Lu 87b3b84471 simplifying dirhash to 64bit integer 2018-05-26 21:24:03 -07:00
Chris Lu 955eae3500 this works great and support long file names 2018-05-26 14:08:55 -07:00
Chris Lu 2da84ed331 working fine now 
but index length can be improved
 2018-05-26 13:35:56 -07:00
Chris Lu 68bcaff14d mysql can compile, not yet tested! 2018-05-26 05:32:15 -07:00
Chris Lu c1e353491a refactoring variable names 2018-05-26 04:50:55 -07:00
Chris Lu 828e4a5ace fix test error 2018-05-26 04:13:50 -07:00
Chris Lu 9e77563c99 add leveldb store 
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
 2018-05-26 03:49:46 -07:00
Chris Lu c34feca59c refactoring 2018-05-25 23:27:06 -07:00
Chris Lu 6de84c64c6 adding create time 2018-05-25 23:26:40 -07:00
Chris Lu 72fc001d0d remove unused code 2018-05-25 01:30:16 -07:00
Chris Lu ac66c133a5 do not read attributes when file is opened 2018-05-25 01:27:21 -07:00
Chris Lu 6d1bcd4b8c use existing attributes instead of fetching from filer 2018-05-25 01:22:31 -07:00
Chris Lu 0a223838bd refactoring 2018-05-25 00:57:25 -07:00
Chris Lu f8776ad5cd add breadcrumb to filer UI 2018-05-24 23:42:13 -07:00
Chris Lu a01557f4cb adjust filer web page 2018-05-24 23:21:12 -07:00
Chris Lu c11d84f314 fix reading from a url 2018-05-24 23:20:56 -07:00
Chris Lu 7b81cf3762 better logs 2018-05-24 23:20:26 -07:00
Chris Lu 9f4f8de9ad skip printout 2018-05-24 23:20:12 -07:00
Chris Lu e18c7e160c avoid empty chunk view 2018-05-24 23:19:56 -07:00
Chris Lu d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu 00d0274fd7 prepare to read from multiple file chunks 2018-05-23 22:28:54 -07:00
Chris Lu 849b6ec28d seems editing already working 
Need to handle multiple chunks read.
Need to cache local file changes.
 2018-05-23 20:55:24 -07:00
Chris Lu d84b80b795 similar changes as master 2018-05-23 03:23:47 -07:00
Chris Lu 536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00
Chris Lu 1675243f29 maybe speed up a little when loading index 2018-05-23 02:38:11 -07:00
Chris Lu e97c60cc10 avoid overwriting variables 2018-05-23 02:36:19 -07:00
Chris Lu 468514f525 reduce logs 2018-05-23 02:36:06 -07:00
Chris Lu 69b9d8c3c2 fix boltdb variable usage 2018-05-22 10:18:09 -07:00
Chris Lu 873868cc10 not working now 
need to add file handler
 2018-05-22 04:31:44 -07:00
Chris Lu 7362de9a18 weed mount can work well 
TODO: somehow filer url is returning empty content
 2018-05-22 03:26:38 -07:00
Chris Lu 9dd228747c filer copy added uid/gid 2018-05-21 01:25:30 -07:00
Chris Lu f07482382b able to update file content 
having some issue when vi reports file changed.
 2018-05-21 00:00:28 -07:00
Chris Lu cd47528a75 remove unused code 2018-05-20 17:08:54 -07:00
Chris Lu cc66e25cd2 merge intervals 2018-05-20 17:06:09 -07:00
Chris Lu 7ca5052942 create files correctly! 2018-05-19 13:51:44 -07:00
Chris Lu 793dd81ca2 skip permission checking when creating dir or files 2018-05-19 12:40:24 -07:00
Chris Lu ab4ddb1e0e fix directory creation, directory listing 2018-05-19 12:07:15 -07:00
Chris Lu e31c514b00 adding modified time to file chunk 2018-05-16 00:54:44 -07:00
Chris Lu 6bf31467c7 adding empty fsync 
less error now. But still:

vi on write: E514: write error (file system full?)
cp: Input/output error
 2018-05-16 00:54:27 -07:00
Chris Lu b303a02461 cp file can work 
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write

updates having issue
 2018-05-16 00:08:44 -07:00
Chris Lu c7a71d35b0 fix on pagination 2018-05-14 20:27:48 -07:00
Chris Lu 58954bf46f pass file attributes from filer to mount 2018-05-14 02:02:17 -07:00
Chris Lu 67401f1908 fix gRpc nil response 2018-05-14 00:17:22 -07:00
Chris Lu c5cf9bd290 properly working filer 2018-05-13 23:56:16 -07:00
Chris Lu f01d5616b3 add better listing directory entries 2018-05-13 14:02:29 -07:00
Chris Lu a4740ca836 fix compilation error for windows 2018-05-13 00:49:59 -07:00
Chris Lu 2dd5b777f7 prepare for 0.77 release 
right before changing to filer2
 2018-05-13 00:26:23 -07:00
Chris Lu 75a2b05aa3 remove Size from Attr 2018-05-13 00:11:26 -07:00
Chris Lu fc7f81f6e7 remove nlink property from directory node 2018-05-12 22:42:28 -07:00
Chris Lu 9ecbf92569 more tests 2018-05-12 13:50:11 -07:00
Chris Lu 81af1bafba file store adds memdb implementation 2018-05-12 13:45:29 -07:00
Chris Lu a808df5019 visit parent folder first 2018-05-11 02:27:57 -07:00
Chris Lu 8a1d640dc4 starting filer2 2018-05-11 02:20:15 -07:00
Chris Lu 9f345da20f mv filer proto to filer_pb 2018-05-09 23:18:02 -07:00
Chris Lu 942c2cbd7b mv pb to master_pb 2018-05-09 23:11:54 -07:00
Chris Lu 57c5405657 add todo 2018-05-09 04:18:31 -07:00
Chris Lu 6db1c8bafb Revert "adjust the needle mapper" 
This reverts commit e2c7be3a2e.
 2018-05-09 03:57:07 -07:00
Chris Lu e2c7be3a2e adjust the needle mapper 
as discussed on
https://github.com/chrislusf/seaweedfs/pull/642#issuecomment-387606960
 2018-05-09 03:50:14 -07:00
Chris Lu 43a69d20bf change filer API to gRPC 2018-05-08 01:59:43 -07:00
Chris Lu 4936d6c342 adjusting for FUSE 2018-05-07 00:37:47 -07:00
Chris Lu cb6b1d8d41 fix broken master health checking 2018-05-07 00:37:33 -07:00
Chris Lu be58993f47 ensure the same Node is returned 2018-05-05 23:50:34 -07:00
Chris Lu b857cf9d9b format 2018-05-05 23:39:29 -07:00
Chris Lu 456738ba64 refactoring fuse 2018-05-05 22:47:16 -07:00
Chris Lu 2cada598c6 fix test 2018-05-05 15:08:23 -07:00
Chris Lu f3e07c231b adding folder icon 2018-05-05 14:41:40 -07:00
Chris Lu 7f49514f6f fix filer UI 2018-05-05 14:10:50 -07:00
Chris Lu fffb14bc87 better support FUSE Lookup() 2018-05-05 02:01:50 -07:00
Chris Lu 050ab19264 add design 2018-05-04 23:38:42 -07:00
Chris Lu 94a35f25f3 simple refactoring 2018-05-03 00:16:02 -07:00
WenLong LI b9068c1304
use volumes variable 
fix leveldb mount bug
 2018-05-03 12:13:53 +08:00
liwenlong05 02a1597642 fix leveldb mount bug 2018-05-03 11:20:38 +08:00
mtolman e4b1e5c3c8
Fix for issue #635 
Fix for the following issue on master: https://github.com/chrislusf/seaweedfs/issues/635
 2018-04-17 13:26:48 -06:00
Chris Lu d49e91cd78 cassandra use local quorum for better performance 2018-04-16 01:33:05 -07:00
Chris Lu 15b8499f8d refactoring 2018-04-16 01:16:06 -07:00
Chris Lu d712c87ec0 update glide 2018-04-16 00:59:10 -07:00
Chris Lu 6448e0c25d update glide 2018-04-16 00:38:49 -07:00
Chris Lu 8c64d06b3a update redis library 2018-04-16 00:27:35 -07:00
Chris Lu 9cd6b33e97 fix https://github.com/chrislusf/seaweedfs/issues/603 2018-02-27 14:18:00 -08:00
Chris Lu 4b9b9547d4 unused option 2018-02-27 13:57:15 -08:00
Chris Lu fc3924ac70 set file name to base file name 
fixing https://github.com/chrislusf/seaweedfs/issues/632
 2018-02-26 08:01:23 -08:00
tengattack 371ad2e397 fix reserve volume on node 2018-01-17 19:53:41 +08:00
Chris Lu 61ff330e99 adjust instructions 2018-01-15 20:33:12 -08:00
Chris Lu 1d09291f95 update upstream 2018-01-15 20:32:57 -08:00
Chris Lu 70f6740309 better fix for single master restart without peers changing 2017-12-06 00:14:14 -08:00
Chris Lu 9026b3e86e always remember the max volume id 2017-11-28 17:08:59 -08:00
Chris Lu 97f757c669 add debug message 
display the error message for debugging
https://github.com/chrislusf/seaweedfs/issues/603
 2017-11-25 01:31:13 -08:00