Commit graph

43 commits

Author SHA1 Message Date
chrislu 87b70a6809 clean up 2022-08-27 00:09:04 -07:00
chrislu c839ce1b19 s3 sink use s3 upload manager
fix https://github.com/seaweedfs/seaweedfs/issues/3531
2022-08-26 23:47:12 -07:00
chrislu e3f40d538d cleaner code 2022-08-20 17:51:30 -07:00
chrislu 11f99836c3 filer.backup: backup small files if the file is saved in filer (saveToFilerLimit > 0)
fix https://github.com/seaweedfs/seaweedfs/issues/3468
2022-08-19 23:00:56 -07:00
chrislu 26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
elee 881a0fe806 ensure compatibility 2022-02-27 04:50:59 -06:00
elee 954ad98e0d set canned acl on replication create 2022-02-27 04:49:31 -06:00
Chris Lu 7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu c08ac536ed cloud drive: add support for Wasabi
* disable md5, sha256 checking to avoid reading one chunk twice
* single threaded upload to avoid chunk swapping (to be enhanced later)
2021-08-25 17:34:29 -07:00
Chris Lu 00c4e06caa cloud drive: s3 configurable force path style 2021-08-23 03:30:41 -07:00
Chris Lu f0cc130849 do not force path style for better compatibility 2021-08-23 03:09:41 -07:00
Chris Lu 45bffc92a8 filer.backup: fix cloud sinks when updating entry 2021-05-26 14:53:11 -07:00
Chris Lu c9722dceb2 adjust log level 2021-02-28 20:26:55 -08:00
Chris Lu 984fdd6192 always use non bucket prefixing url 2021-02-28 16:20:13 -08:00
Chris Lu 9abb041763 filer source: support filerProxy mode 2021-02-28 16:19:47 -08:00
Chris Lu 678c54d705 data sink: add incremental mode 2021-02-28 16:19:03 -08:00
Chris Lu d75bc62196 to read files that are quickly changing 2020-10-22 12:59:15 -07:00
Chris Lu a8624c2e4f read from alternative replica
related to https://github.com/chrislusf/seaweedfs/issues/1512
2020-10-07 22:49:04 -07:00
Chris Lu 387ab6796f filer: cross cluster synchronization 2020-09-09 11:21:23 -07:00
Chris Lu eb7929a971 rename filer2 to filer 2020-09-01 00:21:19 -07:00
Chris Lu c647deace1 file size support set file length
use Attr.FileSize and TotalChunkSize to determine file size
2020-08-15 09:32:47 -07:00
Chris Lu 97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
Chris Lu 4c498e73de filer replication: add s3 endpoint 2020-04-07 17:49:00 -07:00
Chris Lu ae2ee379c0 consistent 64bit size 2020-03-22 01:37:46 -07:00
Chris Lu bec6ec7db6 go fmt 2020-03-17 10:01:55 -07:00
HongyanShen 81610ed006 fix: #1226 2020-03-11 14:37:14 +08:00
Chris Lu 2e3f6ad3a9 filer: remember content is gzipped or not 2020-03-08 21:39:33 -07:00
Chris Lu 13e215ee5c filer: option to encrypt data on volume server 2020-03-06 00:49:47 -08:00
Chris Lu 97ab8a1976 remove ctx if possible 2020-02-25 22:23:59 -08:00
Chris Lu 892e726eb9 avoid reusing context object
fix https://github.com/chrislusf/seaweedfs/issues/1182
2020-02-25 21:50:12 -08:00
Chris Lu d335f04de6 support env variables to overwrite toml file 2020-01-29 09:09:55 -08:00
Chris Lu 9711a6ffaa WIP 2019-11-18 19:24:37 -08:00
Chris Lu b3b42bc947 replicate need to include new entry path 2019-04-16 00:44:31 -07:00
Chris Lu 55bab1b456 add context.Context 2019-03-15 17:20:24 -07:00
Chris Lu 1bfb96f34d optimization for reading whole chunk with gzip encoding 2018-12-07 01:57:55 -08:00
Chris Lu 8dfac6a4cf working b2 sink 2018-11-04 11:58:59 -08:00
Chris Lu 3674ad9f8e go fmt 2018-11-01 01:12:21 -07:00
Chris Lu 4c97ff3717 support AWS SQS as file change notification message queue 2018-10-31 01:11:19 -07:00
Chris Lu 08266b7256 go fmt 2018-10-11 00:08:13 -07:00
Chris Lu a5b5887efc add filer replication to aws s3 2018-10-06 17:10:15 -07:00
Chris Lu f8c2704d2b reformat 2018-10-06 13:04:33 -07:00
Chris Lu 56c5c7b1b6 add google cloud storage 2018-10-04 01:14:44 -07:00
Chris Lu e8ef501f02 add s3 replication sink 2018-10-03 23:36:52 -07:00