Commit graph

226 commits

Author SHA1 Message Date
Chris Lu 74fb237727 benchmark can work in secure mode 2019-02-15 00:09:19 -08:00
Chris Lu 215cd27b37 add authorizing fileId write access
need to secure upload/update/delete for benchmark/filer/mount
need to add secure grpc
2019-02-14 00:08:20 -08:00
Chris Lu 4ff4a147b2 cleanup security.Secret 2019-02-09 21:56:32 -08:00
Chris Lu 501bd72b1c wip: add security.toml file 2019-02-09 21:07:12 -08:00
Sergey aa5ccff6d2
fixing of typos 2019-02-06 18:59:15 +05:00
Chris Lu a6bffc58f9 go fmt 2019-01-18 14:15:10 -08:00
Chris Lu 67e2ea72be master add separate grpc port
due to https://github.com/soheilhy/cmux/issues/64

fix https://github.com/chrislusf/seaweedfs/issues/820
fix https://github.com/chrislusf/seaweedfs/issues/840
fix https://github.com/chrislusf/seaweedfs/issues/841
2019-01-18 14:14:47 -08:00
bingoohuang ab6be025d7 go fmt and fix some typo 2019-01-17 09:17:19 +08:00
bingoohuang 6ddfaf33cb extract VolumeFileScanner for ScanVolumeFile 2019-01-16 17:48:59 +08:00
Chris Lu 6740427638 weed mount: reduce default chunk size 2019-01-05 15:17:03 -08:00
Chris Lu 55f7d8f801 remove extra log 2018-12-29 18:00:19 -08:00
Chris Lu 170ed53da5 weed mount: mac add AutoXattr 2018-12-29 13:55:38 -08:00
Chris Lu 9f3836072a weed mount: no mac finder broswe mode 2018-12-29 13:16:23 -08:00
Chris Lu d7bf6c5b91 switching_to_seaweedfs_fuse 2018-12-29 10:31:36 -08:00
Chris Lu 03c7447ad6 set mount point to existing permissions
fix https://github.com/chrislusf/seaweedfs/issues/806
2018-12-28 23:36:13 -08:00
Chris Lu 2a19511ba8 avoid gzip benchmark data
related to https://github.com/chrislusf/seaweedfs/issues/804
2018-12-27 13:10:46 -08:00
Chris Lu ba923e2864 error if the last arguement does not end with forward slash 2018-12-23 18:20:11 -08:00
Chris Lu 49f40cd145 adjust help message 2018-12-22 15:05:15 -08:00
Chris Lu 852ee21835 avoid .gz auto decompression 2018-12-22 13:58:16 -08:00
Chris Lu 93d0ad09af error log format 2018-12-16 17:37:10 -08:00
Chris Lu c28e8a2397 refactoring 2018-12-06 00:44:41 -08:00
Chris Lu ffa2827ab1 fail fast if two notification queues or inputs are enabled 2018-12-06 00:37:59 -08:00
Chris Lu 52b24a9902 default "weed server -filer" to same directory as -mdir 2018-12-05 23:24:25 -08:00
Chris Lu fb43a9d7b9 adjust message 2018-12-05 02:03:03 -08:00
Chris Lu 1cbd53c01c WIP SeaweedFileSystem added mkdirs, getFileStatus, listStatus, delete 2018-11-25 13:43:26 -08:00
Chris Lu 86dd933596 go fmt 2018-11-23 00:26:15 -08:00
Chris Lu 6344469a9b set fuse subtype 2018-11-22 23:04:23 -08:00
Chris Lu 60d3681371 default images.fix.orientation to false 2018-11-21 16:50:42 -08:00
Chris Lu c9b3ef58de options to control filer.export verbosity 2018-11-21 15:15:40 -08:00
Chris Lu 2f2e417b68 detailed export 2018-11-21 15:12:47 -08:00
Chris Lu 2e6a3c7b16 reduce memory usage 2018-11-14 22:48:54 -08:00
Chris Lu a62adf62ca a bit faster 2018-11-12 11:19:13 -08:00
Chris Lu affe708961 a bit faster 2018-11-11 00:43:30 -08:00
Chris Lu a09eaf8756 adjusting, seems there are some perf benefit 2018-11-11 00:08:22 -08:00
Chris Lu cbd94b18a5 improve "ls -al" performance for large directory 2018-11-08 07:37:34 -08:00
Chris Lu 85d1d99459 default "weed export to current enabled store" 2018-11-04 12:59:29 -08:00
Chris Lu 6c45787a37 selectively export files 2018-11-04 12:07:33 -08:00
Chris Lu ff4189ab25 go fmt 2018-11-04 11:59:08 -08:00
Chris Lu 8dfac6a4cf working b2 sink 2018-11-04 11:58:59 -08:00
Chris Lu a64613172d bootstrap filer replication with weed filer.export -targetStore=notification 2018-11-03 12:43:45 -07:00
Chris Lu 3674ad9f8e go fmt 2018-11-01 01:12:21 -07:00
Chris Lu db584ff7f8 separate into notification.toml, add gcp pub/sub message queue 2018-11-01 01:11:09 -07:00
Chris Lu 4c97ff3717 support AWS SQS as file change notification message queue 2018-10-31 01:11:19 -07:00
Chris Lu fe31f389b0 add configuration instructions 2018-10-30 02:29:11 -07:00
Chris Lu c58b73ad9b adding BackBlaze, Azure 2018-10-23 23:59:40 -07:00
Chris Lu fda771c83f migrate volume sync status to grpc API on volume server 2018-10-15 01:19:15 -07:00
Chris Lu b1daede91b move volume vacuum to gRpc 2018-10-14 23:12:43 -07:00
Chris Lu 556382ff5f adding basic grpc to volume server 2018-10-11 01:16:33 -07:00
Chris Lu 08266b7256 go fmt 2018-10-11 00:08:13 -07:00
Chris Lu 1d7e01b754 reformat 2018-10-11 00:05:54 -07:00
Chris Lu da6154b29c refactor volume server to startVolumeServer() 2018-10-11 00:04:31 -07:00
Chris Lu 60d2f1557d refactor func name 2018-10-10 23:19:54 -07:00
Chris Lu 9f4c2f87d3 adding filer replication sink to Azure 2018-10-09 01:35:48 -07:00
Chris Lu 96a23591e7 remove comments 2018-10-07 22:21:00 -07:00
Chris Lu a5b5887efc add filer replication to aws s3 2018-10-06 17:10:15 -07:00
Chris Lu 84d4a2c8a0 add filer replication to google cloud storage 2018-10-06 13:08:38 -07:00
Chris Lu 14858f6224 add logging 2018-10-06 13:01:38 -07:00
Chris Lu 56c5c7b1b6 add google cloud storage 2018-10-04 01:14:44 -07:00
Chris Lu e8ef501f02 add s3 replication sink 2018-10-03 23:36:52 -07:00
Chris Lu 98384b62fd adjust description 2018-09-28 01:58:34 -07:00
Chris Lu 12ba8b9799 generate useable filer.toml 2018-09-23 09:26:25 -07:00
Chris Lu b16d47030e save and use last processed kafka offsets 2018-09-23 01:34:40 -07:00
Chris Lu 9fe24991d5 refactoring 2018-09-23 00:40:36 -07:00
Chris Lu 7d6b2a4740 add ETag to upload results and chunks 2018-09-22 22:12:21 -07:00
Chris Lu 420f068376 add more instructions 2018-09-22 22:12:06 -07:00
Chris Lu 01ceace18e adjust sink options 2018-09-22 00:53:52 -07:00
Chris Lu 402ab598b6 go fmt 2018-09-22 00:12:10 -07:00
Chris Lu 7bb62b9f21 delay file creation without chunks from dir.Create to file.Flush 2018-09-22 00:11:46 -07:00
Chris Lu db69ce89f0 go fmt 2018-09-21 01:56:43 -07:00
Chris Lu a6cfaba018 able to sync the changes 2018-09-21 01:54:29 -07:00
Chris Lu a8cd7df44a let filer choose default replication 2018-09-20 09:00:15 -07:00
Chris Lu d5149e592e add logging on kafka reading 2018-09-19 22:31:45 -07:00
Chris Lu 25fb6f9a46 fix compilation 2018-09-17 02:23:21 -07:00
Chris Lu 779641e9d4 adjust replicated entry name 2018-09-17 01:37:24 -07:00
Chris Lu b7a375f5aa set large limit for "weed filer.export" directory listing 2018-09-17 00:42:36 -07:00
Chris Lu 4e1f1a0b92 set large limit for "weed mount" directory listing 2018-09-17 00:41:33 -07:00
Chris Lu 788acdf527 add WIP filer.replicate 2018-09-17 00:27:56 -07:00
Chris Lu 7ad5a90127 go fmt 2018-09-03 16:47:00 -07:00
Chris Lu 6e3f4d1079 add working filer.export command 2018-08-23 00:02:04 -07:00
Chris Lu 98b8f8649d adding filer.export prototype 2018-08-22 23:54:22 -07:00
Chris Lu ae4e654d2a add example configuration for default redis create-cluster setup 2018-08-22 18:23:19 -07:00
Chris Lu 748870b03f add "weed scaffold" command 2018-08-19 15:36:30 -07:00
Chris Lu f827ada811 merge notification config with filer.toml 2018-08-19 15:17:55 -07:00
Chris Lu f036ef8a3c add filer notification 2018-08-13 01:20:49 -07:00
Chris Lu 75d63db60d randomize raft server startup
also some go fmt
2018-08-12 14:27:14 -07:00
Chris Lu eca4b928d2 ensure master server count is odd 2018-08-12 14:25:31 -07:00
Chris Lu 2e427c66ac small refactoring 2018-08-10 23:47:17 -07:00
Chris Lu 44d8e35988 add listener for volumd id location for benchmark tool 2018-07-31 19:12:36 -07:00
Chris Lu d09606c65b refactoring 2018-07-29 02:25:24 -07:00
Chris Lu 4f317c7e3d working version3 2018-07-24 01:36:04 -07:00
Chris Lu 7f32eb1e25 s3 add https support 2018-07-22 21:28:54 -07:00
Chris Lu 056871242b remove println 2018-07-22 01:17:49 -07:00
Chris Lu 0655151b8c go fmt 2018-07-22 01:15:11 -07:00
Chris Lu 6319d84f42 s3 API add ListObjectsV1 2018-07-22 01:14:36 -07:00
Chris Lu 922c614bde fix compilation problem 2018-07-21 20:42:14 -07:00
Chris Lu 16c66e097b adjust default buckets folder 2018-07-21 20:29:33 -07:00
Chris Lu 7e2031b18f go fmt 2018-07-21 17:39:10 -07:00
Chris Lu c7228fefa7 add bucket creation and deletion
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
2018-07-19 01:21:44 -07:00
Chris Lu 834a25f084 add list all my buckets 2018-07-18 02:37:09 -07:00
Chris Lu 7abfab8e77 add feature to mount a specific filer path to local directory 2018-07-14 21:09:21 -07:00