gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
1613 commits 28 branches 278 tags 135 MiB
Commit graph

458 commits

Author SHA1 Message Date
Chris Lu 1e1c368ae8 adjust filer help message 2018-07-10 00:22:11 -07:00
Chris Lu 007c4a1965 alt request for dc preferred requests 2018-07-10 00:20:50 -07:00
Chris Lu 8a12fd9ec4 add error checking when loading dat file 2018-07-09 23:31:25 -07:00
Chris Lu 6c4fee85e3 add primary request and backup request when requesting volumes for a data center 2018-07-09 23:18:20 -07:00
Chris Lu 50d351944f filer add data center option 2018-07-09 02:22:48 -07:00
Chris Lu d0982cafa7 refactoring 2018-07-09 00:22:50 -07:00
Chris Lu 5b1fd374be fix all tests 2018-07-08 02:39:04 -07:00
Chris Lu d4d7ced922 refactoring: add type for needle id, offset 
later the type size can possibly be adjusted
 2018-07-08 02:28:04 -07:00
Chris Lu 922032b9bb add filer sub dir list size limit to "weed server" 2018-07-08 02:11:36 -07:00
Chris Lu 702ff48cda add dir list limit option 2018-07-07 02:18:47 -07:00
Chris Lu 2ad45ca04f refactor 2018-07-07 02:03:25 -07:00
Chris Lu 5bfb72d058 faster loading boltdb or leveldb needle map metrics by bloomfilter 
avoid btree
 2018-07-07 00:51:17 -07:00
Chris Lu aba1fe01b3 use constant NeedleIndexSize instead of the number 16 2018-07-07 00:50:14 -07:00
Chris Lu defcd4edc5 adjust log level 2018-07-07 00:48:58 -07:00
Chris Lu c8234a5af6 ensure using none nil attributes 
fix https://github.com/chrislusf/seaweedfs/issues/674
 2018-07-05 23:16:34 -07:00
zhangmingfeng 79d18c69b4 增加chunk图片文件支持width和height 2018-07-05 10:34:17 +08:00
Chris Lu 77fc8c5914 keep alive for gRpc calls 2018-07-03 19:07:55 -07:00
Chris Lu 28e5f20c8e add super block extra for erasure coding 2018-06-28 22:20:37 -07:00
Chris Lu 9e6ad558c5 upgrading dependencies to latest versions 2018-06-28 22:03:47 -07:00
Chris Lu 9bcaa65574 avoid divided by 0 
fix https://github.com/chrislusf/seaweedfs/issues/650
 2018-06-25 01:20:15 -07:00
Chris Lu 0d83c1b91e refactoring 2018-06-25 00:01:53 -07:00
Chris Lu 018a9a20be add test for volume registeration at master 2018-06-24 23:55:27 -07:00
Chris Lu cb19a7da99 add a note 2018-06-24 23:55:06 -07:00
Chris Lu ef98d26c9e ensuring removing registered datanode when disconnecting 2018-06-24 23:20:27 -07:00
Chris Lu afc1532b24 file rename 2018-06-24 23:19:56 -07:00
Chris Lu 233a9a9b6c fix log error 2018-06-24 16:12:41 -07:00
Chris Lu 49f4216280 fix compilation error 2018-06-24 15:37:54 -07:00
Chris Lu 7f6500ae07 super block reserves a couple of bits for future extension 2018-06-24 15:23:03 -07:00
Chris Lu f339e9c284 add support for extra super block info 2018-06-24 15:19:57 -07:00
Chris Lu 9ba335a7c4 prepare for flexible super block 2018-06-24 11:37:08 -07:00
Chris Lu c0122eeef3 start 0.91 beta 2018-06-24 09:29:34 -07:00
Chris Lu cf76711789 releasing 0.90 version 2018-06-23 18:36:27 -07:00
Chris Lu 3a6f9c9bcf increase test size 2018-06-23 18:30:13 -07:00
Chris Lu 7f7e4e9885 fix error for deleted files during compaction 
deletion during commit may cause trouble when make up the difference during commitCompact()
 2018-06-23 18:24:59 -07:00
Chris Lu 3699996985 add compaction test 2018-06-23 16:48:19 -07:00
Chris Lu 14c6f12b0c adjust interface parameter name and doc 2018-06-17 13:24:57 -07:00
Chris Lu db0c25b3d8 delete old entry only after successful insert 2018-06-17 13:02:22 -07:00
Chris Lu eb2acd11c2 decouple from viper for filer store 2018-06-17 13:01:57 -07:00
Chris Lu 0467195f07 fix file closing on windows 2018-06-13 00:36:51 -07:00
Chris Lu 4433b4c006 treat 0 sized file as normal 
fix https://github.com/chrislusf/seaweedfs/issues/409
 2018-06-12 21:11:26 -07:00
Chris Lu 03f50180f3 simplifying the leader election by raft 
fixing https://github.com/chrislusf/seaweedfs/issues/629
 2018-06-12 01:54:09 -07:00
Chris Lu 69b4f93830 fix compilation for go tip 2018-06-12 00:58:43 -07:00
Chris Lu 17918f0b82 fix compilation error on go tip 2018-06-11 23:51:36 -07:00
Chris Lu eaf000eff1 fix error on go tip 2018-06-11 23:39:42 -07:00
Chris Lu d2a811eef2 fix compilation 2018-06-11 23:29:09 -07:00
Chris Lu 5bd72696ac weed mount add ttl option 2018-06-11 23:13:33 -07:00
Chris Lu 98110c1697 add collection, replication, ttl to each file 2018-06-10 16:57:32 -07:00
Chris Lu f1273073fc switch to fs.NodeRequestLookuper 
in order to set the entry valid duration
 2018-06-07 00:07:37 -07:00
Chris Lu cec1d97035 recursively move files and directories 2018-06-06 23:39:30 -07:00
Chris Lu ae23e46313 better error log 2018-06-06 23:06:19 -07:00
Chris Lu e755540be9 fix caching during directory listing 2018-06-06 22:55:59 -07:00
Chris Lu daabdfe357 remove nodemap, fix directory listing cache 2018-06-06 22:48:51 -07:00
Chris Lu 5c25d29272 support renaming files 2018-06-06 22:11:01 -07:00
Chris Lu b3447f4375 adjust logging 2018-06-06 02:21:36 -07:00
Chris Lu 6816661b0f fixed file handle by file full path 2018-06-06 02:09:57 -07:00
Chris Lu 299312c805 use separate filer grpc port 2018-06-05 23:37:41 -07:00
Chris Lu 95fe745a0c compatible with previous json rendering 2018-06-04 19:28:59 -07:00
Chris Lu ebf58709d3 remove unused register file function 2018-06-01 23:24:34 -07:00
Chris Lu 43e3f5724c use fixed list of masters in both filer and volume servers 2018-06-01 00:39:39 -07:00
Chris Lu a6f7f9b0b8 add missing modify and create times 2018-05-31 22:50:19 -07:00
Chris Lu 7efeb146c5 fix log 2018-05-31 22:49:55 -07:00
Chris Lu 4d1eedfa28 prepare to merge with master 2018-05-30 22:29:18 -07:00
Chris Lu dc13e10637 fix copy error on 0 size files 2018-05-30 22:28:14 -07:00
Chris Lu a218eaf1f0 fix log 2018-05-30 22:09:24 -07:00
Chris Lu 430eb67489 handle large file copy when write request is larger than buffer 2018-05-30 22:02:21 -07:00
Chris Lu 44acf4b756 fix file name 
f.Name() is not the base file name
 2018-05-30 20:48:13 -07:00
Chris Lu 0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu 26e7cd8c75 assign a different volume on large file copying 2018-05-30 01:05:26 -07:00
Chris Lu 2fe0d479f1 filer.copy supports chunking 2018-05-30 00:54:56 -07:00
Chris Lu 2d7dea1a09 a little refactoring 2018-05-29 23:52:27 -07:00
Chris Lu 8a48baa056 add single chunk file copying to new filer 2018-05-29 23:46:45 -07:00
Chris Lu 78aabf66ce add error details 2018-05-29 23:46:17 -07:00
Chris Lu 4e3ea49cff properly working local write buffer 2018-05-29 01:21:21 -07:00
Chris Lu c4b92e17d0 fix isPerfectAppend 2018-05-28 22:45:52 -07:00
Chris Lu 74332e1a61 minor 2018-05-28 14:32:16 -07:00
Chris Lu be0e88a606 fix chunk size limit for default 0 2018-05-28 13:44:27 -07:00
Chris Lu 5c4480ec6c add mountOptions.chunkSizeLimitMB, remove cmdMount.IsDebug 2018-05-28 13:42:25 -07:00
Chris Lu 8ab7dd9d08 weed mount add options for collection and replication 2018-05-28 13:24:48 -07:00
Chris Lu d0b238d2db cache local writes before flushing to volume server 2018-05-28 12:30:17 -07:00
Chris Lu 07e0d13d2d filer support reading multiple chunks, with range support 2018-05-28 05:39:12 -07:00
Chris Lu dd5661b297 adding favicon to filer also 2018-05-28 02:35:58 -07:00
Chris Lu 9b603f5ffa split filer read into 2 files 2018-05-28 02:24:14 -07:00
Chris Lu 8db9319a06 refactoring: go fmt, reorg 2018-05-27 23:59:49 -07:00
Chris Lu 480a073f1f refactoring: split into 4 files 2018-05-27 23:53:10 -07:00
Chris Lu 5b844d7011 pass reference of master server 
SendHeartbeat passes lock by value: weed_server.MasterServer contains
sync.Mutex
 2018-05-27 11:58:00 -07:00
Chris Lu 8b0718ac92 go vet 2018-05-27 11:56:49 -07:00
Chris Lu 458ada173e go fmt 2018-05-27 11:52:26 -07:00
Chris Lu 8647191bee removing filer package! 2018-05-27 11:52:15 -07:00
Chris Lu 03d76479a5 add reds 2018-05-27 11:14:29 -07:00
Chris Lu a6a6b87f1b adjust readme 2018-05-27 11:14:12 -07:00
Chris Lu ab86d263c9 add Cassandra 2018-05-27 00:01:15 -07:00
Chris Lu 603de2d5db skip deletion if entry not found 2018-05-27 00:00:56 -07:00
Chris Lu f124ebab1d fix ordering 2018-05-27 00:00:11 -07:00
Chris Lu 1c0a2a7db8 reformat 2018-05-26 23:59:56 -07:00
Chris Lu 869161a261 support both mysql and postgres 2018-05-26 22:02:49 -07:00
Chris Lu 87b3b84471 simplifying dirhash to 64bit integer 2018-05-26 21:24:03 -07:00
Chris Lu 955eae3500 this works great and support long file names 2018-05-26 14:08:55 -07:00
Chris Lu 2da84ed331 working fine now 
but index length can be improved
 2018-05-26 13:35:56 -07:00
Chris Lu 68bcaff14d mysql can compile, not yet tested! 2018-05-26 05:32:15 -07:00
Chris Lu c1e353491a refactoring variable names 2018-05-26 04:50:55 -07:00
Chris Lu 828e4a5ace fix test error 2018-05-26 04:13:50 -07:00
Chris Lu 9e77563c99 add leveldb store 
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
 2018-05-26 03:49:46 -07:00
Chris Lu c34feca59c refactoring 2018-05-25 23:27:06 -07:00
Chris Lu 6de84c64c6 adding create time 2018-05-25 23:26:40 -07:00
Chris Lu 72fc001d0d remove unused code 2018-05-25 01:30:16 -07:00
Chris Lu ac66c133a5 do not read attributes when file is opened 2018-05-25 01:27:21 -07:00
Chris Lu 6d1bcd4b8c use existing attributes instead of fetching from filer 2018-05-25 01:22:31 -07:00
Chris Lu 0a223838bd refactoring 2018-05-25 00:57:25 -07:00
Chris Lu f8776ad5cd add breadcrumb to filer UI 2018-05-24 23:42:13 -07:00
Chris Lu a01557f4cb adjust filer web page 2018-05-24 23:21:12 -07:00
Chris Lu c11d84f314 fix reading from a url 2018-05-24 23:20:56 -07:00
Chris Lu 7b81cf3762 better logs 2018-05-24 23:20:26 -07:00
Chris Lu 9f4f8de9ad skip printout 2018-05-24 23:20:12 -07:00
Chris Lu e18c7e160c avoid empty chunk view 2018-05-24 23:19:56 -07:00
Chris Lu d773e11c7a file handler directly read from volume servers 
this mostly works fine now!

next: need to cache files to local disk
 2018-05-24 01:22:37 -07:00
Chris Lu 00d0274fd7 prepare to read from multiple file chunks 2018-05-23 22:28:54 -07:00
Chris Lu 849b6ec28d seems editing already working 
Need to handle multiple chunks read.
Need to cache local file changes.
 2018-05-23 20:55:24 -07:00
Chris Lu d84b80b795 similar changes as master 2018-05-23 03:23:47 -07:00
Chris Lu 536559f62d copy works, edit somehow still fails 2018-05-23 03:08:46 -07:00
Chris Lu 1675243f29 maybe speed up a little when loading index 2018-05-23 02:38:11 -07:00
Chris Lu e97c60cc10 avoid overwriting variables 2018-05-23 02:36:19 -07:00
Chris Lu 468514f525 reduce logs 2018-05-23 02:36:06 -07:00
Chris Lu 69b9d8c3c2 fix boltdb variable usage 2018-05-22 10:18:09 -07:00
Chris Lu 873868cc10 not working now 
need to add file handler
 2018-05-22 04:31:44 -07:00
Chris Lu 7362de9a18 weed mount can work well 
TODO: somehow filer url is returning empty content
 2018-05-22 03:26:38 -07:00
Chris Lu 9dd228747c filer copy added uid/gid 2018-05-21 01:25:30 -07:00
Chris Lu f07482382b able to update file content 
having some issue when vi reports file changed.
 2018-05-21 00:00:28 -07:00
Chris Lu cd47528a75 remove unused code 2018-05-20 17:08:54 -07:00
Chris Lu cc66e25cd2 merge intervals 2018-05-20 17:06:09 -07:00
Chris Lu 7ca5052942 create files correctly! 2018-05-19 13:51:44 -07:00
Chris Lu 793dd81ca2 skip permission checking when creating dir or files 2018-05-19 12:40:24 -07:00
Chris Lu ab4ddb1e0e fix directory creation, directory listing 2018-05-19 12:07:15 -07:00
Chris Lu e31c514b00 adding modified time to file chunk 2018-05-16 00:54:44 -07:00
Chris Lu 6bf31467c7 adding empty fsync 
less error now. But still:

vi on write: E514: write error (file system full?)
cp: Input/output error
 2018-05-16 00:54:27 -07:00
Chris Lu b303a02461 cp file can work 
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write

updates having issue
 2018-05-16 00:08:44 -07:00
Chris Lu c7a71d35b0 fix on pagination 2018-05-14 20:27:48 -07:00
Chris Lu 58954bf46f pass file attributes from filer to mount 2018-05-14 02:02:17 -07:00
Chris Lu 67401f1908 fix gRpc nil response 2018-05-14 00:17:22 -07:00
Chris Lu c5cf9bd290 properly working filer 2018-05-13 23:56:16 -07:00
Chris Lu f01d5616b3 add better listing directory entries 2018-05-13 14:02:29 -07:00
Chris Lu a4740ca836 fix compilation error for windows 2018-05-13 00:49:59 -07:00
Chris Lu 2dd5b777f7 prepare for 0.77 release 
right before changing to filer2
 2018-05-13 00:26:23 -07:00
Chris Lu 75a2b05aa3 remove Size from Attr 2018-05-13 00:11:26 -07:00
Chris Lu fc7f81f6e7 remove nlink property from directory node 2018-05-12 22:42:28 -07:00
Chris Lu 9ecbf92569 more tests 2018-05-12 13:50:11 -07:00
Chris Lu 81af1bafba file store adds memdb implementation 2018-05-12 13:45:29 -07:00
Chris Lu a808df5019 visit parent folder first 2018-05-11 02:27:57 -07:00
Chris Lu 8a1d640dc4 starting filer2 2018-05-11 02:20:15 -07:00
Chris Lu 9f345da20f mv filer proto to filer_pb 2018-05-09 23:18:02 -07:00
Chris Lu 942c2cbd7b mv pb to master_pb 2018-05-09 23:11:54 -07:00