gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
1708 commits 28 branches 278 tags 135 MiB
Commit graph

539 commits

Author SHA1 Message Date
Chris Lu cfbfc7cb67 fix compilation error 2018-07-28 18:34:15 -07:00
Chris Lu 4e0522a80c adjust API 2018-07-28 18:17:31 -07:00
Chris Lu 888eb2abb5 filer read write all via locations from MasterClient 2018-07-28 14:51:36 -07:00
Chris Lu 1d779389cb MasterClient replicates all vid locations 2018-07-28 14:22:46 -07:00
Chris Lu 01bcc89803 refactor into MasterClient 2018-07-28 02:10:32 -07:00
Chris Lu 1ab8232b55 filer only talk to the master leader 2018-07-28 01:30:03 -07:00
Chris Lu e8d4be579d send initial list of volume location 2018-07-28 01:17:35 -07:00
Chris Lu a12c7b86b0 broadcast messages of new and deleted volumes 2018-07-27 23:09:55 -07:00
Chris Lu f82ac793b4 Merge branch 'master' into add_topo_listener 2018-07-27 02:26:43 -07:00
Chris Lu 10bb76fe6b go fmt 2018-07-27 02:10:42 -07:00
Chris Lu 4d322df95e avoid reparsing the multipart form 2018-07-27 02:10:10 -07:00
Chris Lu 0d98949199 tmp commit 2018-07-27 01:54:45 -07:00
Chris Lu e4b7e31902 add back recursively delete a folder 2018-07-24 22:33:26 -07:00
Chris Lu 57ded4fd80 speed up leveldb bolddb loading 2018-07-24 22:17:56 -07:00
Chris Lu ea8ecf35f4 set needle AppendAtNs value during writes 2018-07-24 02:44:33 -07:00
Chris Lu 655b449270 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2018-07-24 02:20:02 -07:00
Chris Lu 0fb4f59fa6
Merge pull request #697 from chrislusf/add_version3 
Add version3
 2018-07-24 02:19:06 -07:00
Chris Lu 69d1b88b9e starting on 0.93 beta 2018-07-24 02:18:39 -07:00
Chris Lu e8a5ee4f48 releasing 0.92 
ensure the nil problem is resolved.
 2018-07-24 01:54:15 -07:00
Chris Lu 77fadab560 go fmt 2018-07-24 01:38:08 -07:00
Chris Lu 4f317c7e3d working version3 2018-07-24 01:36:04 -07:00
Chris Lu 0b99557c29 fix compilation error 2018-07-24 01:09:32 -07:00
Chris Lu 9bf93e698e refactoring: avoid common code 2018-07-24 00:58:52 -07:00
Chris Lu aae77e8ea3 initialize read seeker 2018-07-24 00:54:15 -07:00
Chris Lu 5a237af973 fix nil if not images 2018-07-24 00:49:53 -07:00
Chris Lu a9069520ae start 0.92 beta2 2018-07-23 01:59:23 -07:00
Chris Lu 04356ad141 re-releasing 0.91 2018-07-23 01:58:13 -07:00
Chris Lu 6039b25bc9 s3API working with listing objects 2018-07-23 01:55:26 -07:00
Chris Lu 3c7ffa2a2c go fmt 2018-07-23 01:16:17 -07:00
Chris Lu ebad3a44ab s3api do not proxy directory requests 2018-07-23 01:15:59 -07:00
Chris Lu 7f32eb1e25 s3 add https support 2018-07-22 21:28:54 -07:00
Chris Lu c133e72f62 print out unsupported methods 2018-07-22 19:55:53 -07:00
Chris Lu ff331178aa
Merge pull request #695 from zhangmingfeng/master 
增加chunk图片文件支持width和height
 2018-07-22 19:54:04 -07:00
Chris Lu 7e76a7d21b
Merge pull request #694 from himyxia/inprove_switch_case 
better way to handler switch case
 2018-07-22 10:29:37 -07:00
Jimmy 1c77d323b0 better way to handler switch case 2018-07-23 01:27:10 +08:00
Chris Lu 5e55f0da4d add WIP 0.92 beta 2018-07-22 02:15:24 -07:00
Chris Lu 19c537b687 releasing 0.91 2018-07-22 02:12:02 -07:00
Chris Lu 20853972b2 s3 API add ListObjectsV2Handler 2018-07-22 02:04:07 -07:00
Chris Lu 056871242b remove println 2018-07-22 01:17:49 -07:00
Chris Lu 0655151b8c go fmt 2018-07-22 01:15:11 -07:00
Chris Lu 6319d84f42 s3 API add ListObjectsV1 2018-07-22 01:14:36 -07:00
Chris Lu 922c614bde fix compilation problem 2018-07-21 20:42:14 -07:00
Chris Lu 16c66e097b adjust default buckets folder 2018-07-21 20:29:33 -07:00
Chris Lu 49375d6031 adjust DELETE response to S3 format 2018-07-21 19:12:44 -07:00
Chris Lu 5068b6ae7d fix compilation 2018-07-21 19:06:09 -07:00
Chris Lu 12825938d7 unused method 2018-07-21 18:50:56 -07:00
Chris Lu 8f240276db s3 add HEAD DELETE 2018-07-21 18:49:47 -07:00
Chris Lu bfa87b4f50 adjust log level 2018-07-21 18:47:23 -07:00
Chris Lu 648626115a fix printing 2018-07-21 18:47:07 -07:00
Chris Lu 12e6ccc581 remove unused code 2018-07-21 17:54:14 -07:00
Chris Lu 702d7ac424 delegating old chunk purging to CreateEntry 
1. filer add file user id and group id
2. auto-chunking set the file name correctly
3. delegating old chunk purging to CreateEntry
 2018-07-21 17:47:59 -07:00
Chris Lu 852af28f91 needle priting format 2018-07-21 17:41:21 -07:00
Chris Lu 6f30a78a6c update also delete old chunks if different from latest chunks 2018-07-21 17:40:00 -07:00
Chris Lu 7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu dc8dd8b486 Merge branch 'master' into add_s3 2018-07-21 15:59:18 -07:00
Chris Lu c98df05ed0 support PUT 2018-07-21 15:58:48 -07:00
Chris Lu feb8eeb830 refactor: separate multipart parsing file 2018-07-21 12:01:35 -07:00
Chris Lu 8480008a9a add s3 upload, and removing mono and multi part upload analyzer 
removing mono and multi part upload analyzer, which were used just to determine the file name
 2018-07-21 10:39:02 -07:00
Chris Lu 80d80daf64 set filer2.ErrNotFound for not found entry 2018-07-20 02:14:18 -07:00
Chris Lu dad733086a fix string printing 2018-07-20 00:58:10 -07:00
Chris Lu e73b859b1d fix compilation error 2018-07-20 00:46:02 -07:00
Chris Lu 568132c0c9 delete bucket also removes collection data 2018-07-20 00:10:01 -07:00
Chris Lu 13e5541e17 FUSE can change file or folder attributes 
FUSE can change file or folder attributes
 2018-07-19 02:17:36 -07:00
Chris Lu a09ef6002a adjust name to avoid confusion 2018-07-19 01:46:18 -07:00
Chris Lu feea33d5e0 add HeadBucket 2018-07-19 01:43:27 -07:00
Chris Lu d429b6296b fix test 2018-07-19 01:27:45 -07:00
Chris Lu c7228fefa7 add bucket creation and deletion 
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
 2018-07-19 01:21:44 -07:00
Chris Lu 702fbdf731 refactoring 2018-07-18 02:39:12 -07:00
Chris Lu 834a25f084 add list all my buckets 2018-07-18 02:37:09 -07:00
Chris Lu 7abfab8e77 add feature to mount a specific filer path to local directory 2018-07-14 21:09:21 -07:00
Chris Lu 3edfe1d28f extend export command to show tombstone + change output format to CSV 
merging https://github.com/chrislusf/seaweedfs/pull/610 and add "-limit" option
 2018-07-14 20:51:17 -07:00
Chris Lu 85eae57e4c Store and replicate timestamp of file deletion in tombstone marker 
merging changes from https://github.com/chrislusf/seaweedfs/pull/609/files
 2018-07-14 20:26:45 -07:00
Chris Lu 67763fb375 fix data center setting error 2018-07-14 14:09:45 -07:00
Chris Lu 6d58916cfd fix compilation error 2018-07-14 13:39:57 -07:00
Chris Lu 842dab07b4 weed mount can request to prioritize to write to a data center 2018-07-14 13:36:28 -07:00
Chris Lu a51aa4c586 avoid extra log messages 
fix https://github.com/chrislusf/seaweedfs/issues/689
 2018-07-13 22:07:32 -07:00
Chris Lu 9e974f6a1b
Merge pull request #686 from qszxnp/filer2_dir_cache_bugfix 
bugfix:delete Directory cache when filer2 delete Directory
 2018-07-12 01:17:32 -07:00
聂鹏 1ed41f8b21 bugfix:check Directory attr before create it 2018-07-12 00:53:31 -04:00
聂鹏 c523bc4f07 bugfix:delete Directory cache when filer2 delete Directory 2018-07-11 21:27:25 -04:00
Chris Lu 2fd48df9d9 fix test 2018-07-11 12:52:48 -07:00
Chris Lu f7afa80250 also remove volumeLayout if collection is empty 2018-07-11 02:01:33 -07:00
Chris Lu 1e1c368ae8 adjust filer help message 2018-07-10 00:22:11 -07:00
Chris Lu 007c4a1965 alt request for dc preferred requests 2018-07-10 00:20:50 -07:00
Chris Lu 8a12fd9ec4 add error checking when loading dat file 2018-07-09 23:31:25 -07:00
Chris Lu 6c4fee85e3 add primary request and backup request when requesting volumes for a data center 2018-07-09 23:18:20 -07:00
Chris Lu 50d351944f filer add data center option 2018-07-09 02:22:48 -07:00
Chris Lu d0982cafa7 refactoring 2018-07-09 00:22:50 -07:00
Chris Lu 5b1fd374be fix all tests 2018-07-08 02:39:04 -07:00
Chris Lu d4d7ced922 refactoring: add type for needle id, offset 
later the type size can possibly be adjusted
 2018-07-08 02:28:04 -07:00
Chris Lu 922032b9bb add filer sub dir list size limit to "weed server" 2018-07-08 02:11:36 -07:00
Chris Lu 702ff48cda add dir list limit option 2018-07-07 02:18:47 -07:00
Chris Lu 2ad45ca04f refactor 2018-07-07 02:03:25 -07:00
Chris Lu 5bfb72d058 faster loading boltdb or leveldb needle map metrics by bloomfilter 
avoid btree
 2018-07-07 00:51:17 -07:00
Chris Lu aba1fe01b3 use constant NeedleIndexSize instead of the number 16 2018-07-07 00:50:14 -07:00
Chris Lu defcd4edc5 adjust log level 2018-07-07 00:48:58 -07:00
Chris Lu c8234a5af6 ensure using none nil attributes 
fix https://github.com/chrislusf/seaweedfs/issues/674
 2018-07-05 23:16:34 -07:00
zhangmingfeng 79d18c69b4 增加chunk图片文件支持width和height 2018-07-05 10:34:17 +08:00
Chris Lu 77fc8c5914 keep alive for gRpc calls 2018-07-03 19:07:55 -07:00
Chris Lu 28e5f20c8e add super block extra for erasure coding 2018-06-28 22:20:37 -07:00
Chris Lu 9e6ad558c5 upgrading dependencies to latest versions 2018-06-28 22:03:47 -07:00
Chris Lu 9bcaa65574 avoid divided by 0 
fix https://github.com/chrislusf/seaweedfs/issues/650
 2018-06-25 01:20:15 -07:00
Chris Lu 0d83c1b91e refactoring 2018-06-25 00:01:53 -07:00
Chris Lu 018a9a20be add test for volume registeration at master 2018-06-24 23:55:27 -07:00
Chris Lu cb19a7da99 add a note 2018-06-24 23:55:06 -07:00
Chris Lu ef98d26c9e ensuring removing registered datanode when disconnecting 2018-06-24 23:20:27 -07:00
Chris Lu afc1532b24 file rename 2018-06-24 23:19:56 -07:00
Chris Lu 233a9a9b6c fix log error 2018-06-24 16:12:41 -07:00
Chris Lu 49f4216280 fix compilation error 2018-06-24 15:37:54 -07:00
Chris Lu 7f6500ae07 super block reserves a couple of bits for future extension 2018-06-24 15:23:03 -07:00
Chris Lu f339e9c284 add support for extra super block info 2018-06-24 15:19:57 -07:00
Chris Lu 9ba335a7c4 prepare for flexible super block 2018-06-24 11:37:08 -07:00
Chris Lu c0122eeef3 start 0.91 beta 2018-06-24 09:29:34 -07:00
Chris Lu cf76711789 releasing 0.90 version 2018-06-23 18:36:27 -07:00
Chris Lu 3a6f9c9bcf increase test size 2018-06-23 18:30:13 -07:00
Chris Lu 7f7e4e9885 fix error for deleted files during compaction 
deletion during commit may cause trouble when make up the difference during commitCompact()
 2018-06-23 18:24:59 -07:00
Chris Lu 3699996985 add compaction test 2018-06-23 16:48:19 -07:00
Chris Lu 14c6f12b0c adjust interface parameter name and doc 2018-06-17 13:24:57 -07:00
Chris Lu db0c25b3d8 delete old entry only after successful insert 2018-06-17 13:02:22 -07:00
Chris Lu eb2acd11c2 decouple from viper for filer store 2018-06-17 13:01:57 -07:00
Chris Lu 0467195f07 fix file closing on windows 2018-06-13 00:36:51 -07:00
Chris Lu 4433b4c006 treat 0 sized file as normal 
fix https://github.com/chrislusf/seaweedfs/issues/409
 2018-06-12 21:11:26 -07:00
Chris Lu 03f50180f3 simplifying the leader election by raft 
fixing https://github.com/chrislusf/seaweedfs/issues/629
 2018-06-12 01:54:09 -07:00
Chris Lu 69b4f93830 fix compilation for go tip 2018-06-12 00:58:43 -07:00
Chris Lu 17918f0b82 fix compilation error on go tip 2018-06-11 23:51:36 -07:00
Chris Lu eaf000eff1 fix error on go tip 2018-06-11 23:39:42 -07:00
Chris Lu d2a811eef2 fix compilation 2018-06-11 23:29:09 -07:00
Chris Lu 5bd72696ac weed mount add ttl option 2018-06-11 23:13:33 -07:00
Chris Lu 98110c1697 add collection, replication, ttl to each file 2018-06-10 16:57:32 -07:00
Chris Lu f1273073fc switch to fs.NodeRequestLookuper 
in order to set the entry valid duration
 2018-06-07 00:07:37 -07:00
Chris Lu cec1d97035 recursively move files and directories 2018-06-06 23:39:30 -07:00
Chris Lu ae23e46313 better error log 2018-06-06 23:06:19 -07:00
Chris Lu e755540be9 fix caching during directory listing 2018-06-06 22:55:59 -07:00
Chris Lu daabdfe357 remove nodemap, fix directory listing cache 2018-06-06 22:48:51 -07:00
Chris Lu 5c25d29272 support renaming files 2018-06-06 22:11:01 -07:00
Chris Lu b3447f4375 adjust logging 2018-06-06 02:21:36 -07:00
Chris Lu 6816661b0f fixed file handle by file full path 2018-06-06 02:09:57 -07:00
Chris Lu 299312c805 use separate filer grpc port 2018-06-05 23:37:41 -07:00
Chris Lu 95fe745a0c compatible with previous json rendering 2018-06-04 19:28:59 -07:00
Chris Lu ebf58709d3 remove unused register file function 2018-06-01 23:24:34 -07:00
Chris Lu 43e3f5724c use fixed list of masters in both filer and volume servers 2018-06-01 00:39:39 -07:00
Chris Lu a6f7f9b0b8 add missing modify and create times 2018-05-31 22:50:19 -07:00
Chris Lu 7efeb146c5 fix log 2018-05-31 22:49:55 -07:00
Chris Lu 4d1eedfa28 prepare to merge with master 2018-05-30 22:29:18 -07:00
Chris Lu dc13e10637 fix copy error on 0 size files 2018-05-30 22:28:14 -07:00
Chris Lu a218eaf1f0 fix log 2018-05-30 22:09:24 -07:00
Chris Lu 430eb67489 handle large file copy when write request is larger than buffer 2018-05-30 22:02:21 -07:00
Chris Lu 44acf4b756 fix file name 
f.Name() is not the base file name
 2018-05-30 20:48:13 -07:00
Chris Lu 0301504184 add mime, use simple insert and update filer store API 
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
 2018-05-30 20:24:57 -07:00
Chris Lu 26e7cd8c75 assign a different volume on large file copying 2018-05-30 01:05:26 -07:00
Chris Lu 2fe0d479f1 filer.copy supports chunking 2018-05-30 00:54:56 -07:00