gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
2033 commits 28 branches 278 tags 135 MiB
Commit graph

778 commits

Author SHA1 Message Date
Chris Lu c28e8a2397 refactoring 2018-12-06 00:44:41 -08:00
Chris Lu ffa2827ab1 fail fast if two notification queues or inputs are enabled 2018-12-06 00:37:59 -08:00
Chris Lu 63619f8b8e fail early if two filer stores are defined 2018-12-06 00:32:36 -08:00
Chris Lu 52b24a9902 default "weed server -filer" to same directory as -mdir 2018-12-05 23:24:25 -08:00
Chris Lu fb43a9d7b9 adjust message 2018-12-05 02:03:03 -08:00
chenwanli 31038acccf fix https://github.com/chrislusf/seaweedfs/issues/780 2018-12-05 14:06:16 +08:00
Chris Lu 650d1af978 HCFS support change user name and group names 2018-12-04 00:46:00 -08:00
Chris Lu b0838db5a8 support hdfs user name and group names 2018-12-04 00:13:40 -08:00
Chris Lu 7ace0efd65 revert prev 2018-12-02 23:25:37 -08:00
Chris Lu 738247d47e do not throw error if not found 2018-12-02 23:20:29 -08:00
Chris Lu c6b646e92c skip creation for root 2018-12-02 23:16:17 -08:00
Chris Lu bea162c34c add warning for empty attributes 2018-12-02 22:57:59 -08:00
Chris Lu f0677c5af1 default root to 0755 2018-12-02 19:59:47 -08:00
Chris Lu db22666a30 add default root directory 2018-12-02 19:42:50 -08:00
Chris Lu a3def2bbd9 for 1.10 2018-11-29 09:15:04 -08:00
Chris Lu 7a6f49cd42 refactor a bit more 
fix https://github.com/chrislusf/seaweedfs/issues/777
 2018-11-29 00:07:54 -08:00
Chris Lu ceca078acb avoid overwriting file or directory 
fix https://github.com/chrislusf/seaweedfs/issues/777
 2018-11-29 00:00:56 -08:00
Chris Lu 1cbd53c01c WIP SeaweedFileSystem added mkdirs, getFileStatus, listStatus, delete 2018-11-25 13:43:26 -08:00
Chris Lu 871dee4674 fix test 2018-11-23 12:37:09 -08:00
Chris Lu 86dd933596 go fmt 2018-11-23 00:26:15 -08:00
Chris Lu 444dfded84 add fs.FSStatfser for SeaweedFS weed mount 2018-11-23 00:24:51 -08:00
Chris Lu 2e32b44061 add template for fs.FSStatfser 2018-11-22 23:05:22 -08:00
Chris Lu 6344469a9b set fuse subtype 2018-11-22 23:04:23 -08:00
Chris Lu d64546c791 adjust logs 2018-11-22 14:31:10 -08:00
Chris Lu e12b6c0306 add subimit count parameter 
related to https://github.com/chrislusf/seaweedfs/issues/631
 2018-11-22 13:45:29 -08:00
Chris Lu 0ed816d4e9 formatting error log 2018-11-22 00:42:56 -08:00
Chris Lu 19d546938a forgot to upgrade version number 2018-11-21 23:30:59 -08:00
Chris Lu 60d3681371 default images.fix.orientation to false 2018-11-21 16:50:42 -08:00
Chris Lu 99256100a0 fix tests 2018-11-21 16:35:27 -08:00
Chris Lu e5ad2223a1 add missing intervals 
fix https://github.com/chrislusf/seaweedfs/issues/774
 2018-11-21 16:25:13 -08:00
Chris Lu c9b3ef58de options to control filer.export verbosity 2018-11-21 15:15:40 -08:00
Chris Lu 2f2e417b68 detailed export 2018-11-21 15:12:47 -08:00
Chris Lu b282e34dc2 async file chunk deletion 2018-11-20 20:56:28 -08:00
Chris Lu 5065d4ab2d master add grpc API for fileid assigning 2018-11-20 11:35:45 -08:00
Chris Lu b318167cd3 batch delete file ids 2018-11-20 11:35:08 -08:00
Chris Lu 06474e5990 1.08 2018-11-18 22:43:05 -08:00
Chris Lu ee3fe07acf fix sort bug 2018-11-18 21:59:53 -08:00
Chris Lu 469863f7f0 merge interval now is quite faster with reusing slices 2018-11-18 21:24:58 -08:00
Chris Lu d2924c7748 merge intervals is a bit faster 2018-11-18 20:31:39 -08:00
Chris Lu cb5a10c6a3 adjust the tests 2018-11-18 17:05:36 -08:00
Chris Lu f1db22d48b add benchmark test for merging intervals 2018-11-18 16:49:08 -08:00
Chris Lu 4fcfc9410f cleanup 2018-11-18 11:51:38 -08:00
Chris Lu 9655dc9ca9 simpler logic 2018-11-18 10:07:30 -08:00
Chris Lu f8eb988347 1.07 2018-11-18 08:10:15 -08:00
Chris Lu 60421b23f0 reduce weed mount memory usage 2018-11-18 07:49:14 -08:00
Chris Lu 2929563294 1.06 2018-11-14 23:51:15 -08:00
Chris Lu d3b7965c76 caching grpc clients 2018-11-14 23:50:46 -08:00
Chris Lu d89774cb7f cache grpc client connections to improve performance 2018-11-14 23:31:39 -08:00
Chris Lu 2e6a3c7b16 reduce memory usage 2018-11-14 22:48:54 -08:00
Chris Lu f224b9fe28 1.05 2018-11-13 23:54:17 -08:00
Chris Lu 11b0a9cb76 fix possible error when reusing file handles 2018-11-13 23:53:17 -08:00
Chris Lu a62adf62ca a bit faster 2018-11-12 11:19:13 -08:00
Chris Lu affe708961 a bit faster 2018-11-11 00:43:30 -08:00
Chris Lu a09eaf8756 adjusting, seems there are some perf benefit 2018-11-11 00:08:22 -08:00
Chris Lu 8178fc9973 adjust logging 2018-11-11 00:07:46 -08:00
Chris Lu 2697d6a4fb fix deleting bucket 2018-11-08 22:15:21 -08:00
Chris Lu 0d095f72e4 update to 1.04 2018-11-08 21:42:45 -08:00
Chris Lu a4ceb051a7 use MD5 for ETag to be consistent with Amazon S3 2018-11-08 21:41:02 -08:00
Chris Lu 6e53c38c2f update to 1.03 2018-11-08 07:37:44 -08:00
Chris Lu cbd94b18a5 improve "ls -al" performance for large directory 2018-11-08 07:37:34 -08:00
Chris Lu 6e11923551 upgrade to 1.02 2018-11-07 11:37:32 -08:00
Chris Lu ed8efb5aef much improved "ls -al" performance 2018-11-07 11:35:13 -08:00
Chris Lu 60db731e36 update to 1.01 2018-11-05 08:54:40 -08:00
Chris Lu f9dbaf6292 delete file by name instead of referencing the dataFile pointer 2018-11-05 08:53:38 -08:00
alex8224 8d3f9c0b75 fix invalid point when delete collection 2018-11-05 19:53:55 +08:00
Chris Lu 1a64a817ac update to 1.00 2018-11-04 22:13:16 -08:00
Chris Lu 85d1d99459 default "weed export to current enabled store" 2018-11-04 12:59:29 -08:00
Chris Lu 6c45787a37 selectively export files 2018-11-04 12:07:33 -08:00
Chris Lu ff4189ab25 go fmt 2018-11-04 11:59:08 -08:00
Chris Lu 8dfac6a4cf working b2 sink 2018-11-04 11:58:59 -08:00
Chris Lu f050b22d6c close file in ScanVolumeFile() 
fix https://github.com/chrislusf/seaweedfs/issues/761
 2018-11-04 00:28:24 -07:00
Chris Lu 93e0dddbf3 add function description 2018-11-03 22:32:21 -07:00
Chris Lu a64613172d bootstrap filer replication with weed filer.export -targetStore=notification 2018-11-03 12:43:45 -07:00
Chris Lu 3674ad9f8e go fmt 2018-11-01 01:12:21 -07:00
Chris Lu db584ff7f8 separate into notification.toml, add gcp pub/sub message queue 2018-11-01 01:11:09 -07:00
Chris Lu 6219a9ad1f filer ordered notifications 2018-10-31 21:48:05 -07:00
Chris Lu 4c97ff3717 support AWS SQS as file change notification message queue 2018-10-31 01:11:19 -07:00
Chris Lu 200cbcde62
Merge pull request #756 from zzoe/patch-1 
Update volume_server_handlers_write.go
 2018-10-30 21:52:19 -07:00
zoe a11c28cb4c
Update volume_server_handlers_write.go 
size declared and not used
 2018-10-31 09:36:32 +08:00
Chris Lu fe31f389b0 add configuration instructions 2018-10-30 02:29:11 -07:00
zoe 29eff0ecd7
Update volume_server_handlers_write.go 
返回实际文件大小，而不是needle的大小
 2018-10-29 21:05:51 +08:00
Chris Lu 9e98bba206 go fmt 2018-10-24 00:00:01 -07:00
Chris Lu 6a756136ef go fmt 2018-10-23 23:59:49 -07:00
Chris Lu c58b73ad9b adding BackBlaze, Azure 2018-10-23 23:59:40 -07:00
Chris Lu b861899919 use released version of azblob 
replace https://github.com/chrislusf/seaweedfs/pull/754
 2018-10-23 02:18:46 -07:00
Chris Lu 82a5d39778 add back volume vacuum cleanup 2018-10-18 23:02:58 -07:00
Chris Lu 496a7398d1 adjust logging 2018-10-18 20:47:30 -07:00
Chris Lu 201c24c110 add locking to volume layout vacuum 2018-10-18 20:34:43 -07:00
Chris Lu 46eb77f9bb move DiskStatus and MemStatus to protobuf 2018-10-15 22:25:28 -07:00
Chris Lu eec951cad2 migrate volume sync to gRpc 2018-10-15 21:44:41 -07:00
Chris Lu f8b2d3cacc move volume mount/unmount on volume server to grpc 2018-10-15 01:48:15 -07:00
Chris Lu 66a353dcb5 remove volume server /admin/volume/delete 2018-10-15 01:26:49 -07:00
Chris Lu fda771c83f migrate volume sync status to grpc API on volume server 2018-10-15 01:19:15 -07:00
Chris Lu b423bb9e2d migrate assign volume to grpc API on volume server 2018-10-15 00:40:46 -07:00
Chris Lu 333709657c adjust log level 2018-10-15 00:40:12 -07:00
Chris Lu f9410bcaa7 adjust log level for deletes 2018-10-15 00:39:56 -07:00
Chris Lu 8301519fb0 migrate delete collection to grpc API on volume server 2018-10-15 00:03:55 -07:00
Chris Lu db152ca540 fix error log format 2018-10-14 23:50:54 -07:00
Chris Lu 67d1e572ca rename files 2018-10-14 23:47:29 -07:00
Chris Lu b1daede91b move volume vacuum to gRpc 2018-10-14 23:12:43 -07:00
Chris Lu 91ac2e0dd9 go fmt 2018-10-14 00:30:20 -07:00
Chris Lu 249eeccf32 0.99 release 2018-10-14 00:19:04 -07:00
Chris Lu 9db53cf256 use the configurable chunk size 2018-10-14 00:18:52 -07:00
Chris Lu ff66269b62 use grpc to replace http APIs for batch volume id lookup and batch delete 
1. remove batch volume id lookup http API /vol/lookup
2. remove batch delete http API /delete
 2018-10-14 00:12:28 -07:00
Chris Lu 3ddcd87098 increase weed mount IO perforance 2018-10-14 00:10:17 -07:00
Chris Lu ab85118233 error handling when kafka not ready 2018-10-13 23:30:00 -07:00
Chris Lu 41aba0c02c fix when no files in filer 2018-10-12 00:45:28 -07:00
Chris Lu 4af6b4eb0e fix master static resource display problem 2018-10-12 00:05:15 -07:00
Chris Lu 556382ff5f adding basic grpc to volume server 2018-10-11 01:16:33 -07:00
Chris Lu 64ed47502b rename file 2018-10-11 00:21:32 -07:00
Chris Lu 08266b7256 go fmt 2018-10-11 00:08:13 -07:00
Chris Lu 1d7e01b754 reformat 2018-10-11 00:05:54 -07:00
Chris Lu da6154b29c refactor volume server to startVolumeServer() 2018-10-11 00:04:31 -07:00
Chris Lu 60d2f1557d refactor func name 2018-10-10 23:19:54 -07:00
Chris Lu b7763c1cb6 releasing 0.98 2018-10-10 21:44:10 -07:00
Chris Lu 9f4c2f87d3 adding filer replication sink to Azure 2018-10-09 01:35:48 -07:00
Chris Lu 453d0be4d4 adjust error format 2018-10-09 01:35:30 -07:00
Alex Wang 4db68669b6 [bugfix] Fix interrupt hook overwritten bug 2018-10-09 15:43:32 +08:00
Chris Lu 96a23591e7 remove comments 2018-10-07 22:21:00 -07:00
Chris Lu dbe33236fb embed static resources via statik 2018-10-07 21:42:57 -07:00
Chris Lu fffbc2d490 embed static resources via statik 2018-10-07 10:54:05 -07:00
Chris Lu a5b5887efc add filer replication to aws s3 2018-10-06 17:10:15 -07:00
Chris Lu 84d4a2c8a0 add filer replication to google cloud storage 2018-10-06 13:08:38 -07:00
Chris Lu f8c2704d2b reformat 2018-10-06 13:04:33 -07:00
Chris Lu 14858f6224 add logging 2018-10-06 13:01:38 -07:00
Chris Lu 04da4c8094 add logging 2018-10-06 13:01:29 -07:00
Chris Lu 4a8ef198d7 add logging 2018-10-06 13:00:33 -07:00
Chris Lu 56c5c7b1b6 add google cloud storage 2018-10-04 01:14:44 -07:00
Chris Lu e8ef501f02 add s3 replication sink 2018-10-03 23:36:52 -07:00
Chris Lu 56a5d5af8d ignore duplicated key error if the store has duplicated primary key prevention 
fix https://github.com/chrislusf/seaweedfs/issues/741
 2018-09-30 00:49:52 -07:00
Chris Lu 98384b62fd adjust description 2018-09-28 01:58:34 -07:00
Chris Lu f11d531d45 adjust logging 2018-09-28 01:54:10 -07:00
Chris Lu 06c43286cd skip error when deletion failed 2018-09-28 00:10:49 -07:00
Chris Lu 746017ea4f releasing 0.97 2018-09-27 21:51:41 -07:00
Chris Lu f7ff84f17f Revert "smalller block size for better accuracy" 
This reverts commit 97ecd20fc9.
 2018-09-27 09:42:24 -07:00
Chris Lu 97ecd20fc9 smalller block size for better accuracy 2018-09-27 09:39:17 -07:00
Chris Lu f53632a84b add block size for linux du 
du already working on Mac
 2018-09-27 09:33:36 -07:00
Chris Lu a362261972 avoid possible racing issue if leveldb or boltdb is used. 
remove the to-be-generated files during commit

fix https://github.com/chrislusf/seaweedfs/issues/738
 2018-09-26 18:45:51 -07:00
Chris Lu 31ed352ab6 replication handle cases when entry already exists 2018-09-25 09:27:03 -07:00
Chris Lu 60c1ada4c1 remove filer grpc API GetEntryAttributes 2018-09-25 09:18:06 -07:00
Chris Lu 578f8a6dec fix empty file reading on linux 2018-09-23 10:01:00 -07:00
Chris Lu 200d9b9ac2 add file name to log 2018-09-23 09:48:55 -07:00
Chris Lu a60767c43a add more instructions 2018-09-23 09:26:39 -07:00
Chris Lu 12ba8b9799 generate useable filer.toml 2018-09-23 09:26:25 -07:00
Chris Lu b1b8c4ed32 join via filepath 2018-09-23 01:46:50 -07:00
Chris Lu b16d47030e save and use last processed kafka offsets 2018-09-23 01:34:40 -07:00
Chris Lu 9fe24991d5 refactoring 2018-09-23 00:40:36 -07:00
Chris Lu 7d6b2a4740 add ETag to upload results and chunks 2018-09-22 22:12:21 -07:00
Chris Lu 420f068376 add more instructions 2018-09-22 22:12:06 -07:00
Chris Lu d3e1f20dcc adjust log level 2018-09-22 22:11:49 -07:00