gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6880 commits 28 branches 278 tags 135 MiB
Commit graph

886 commits

Author SHA1 Message Date
Chris Lu af71ae11aa master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates() 2021-11-03 01:09:48 -07:00
Chris Lu ab97b17e62 better printout 2021-11-02 23:45:47 -07:00
Chris Lu 5160eb08f7 shell: optionally read filer address from master 2021-11-02 23:38:45 -07:00
Chris Lu edbf6d297b filer.meta.tail: add example to send metadata to elastic search 2021-10-31 18:01:33 -07:00
Chris Lu 3be3c17f59 volume vacuum: avoid timeout with streaming progress report 
fix https://github.com/chrislusf/seaweedfs/issues/2396
 2021-10-24 01:55:34 -07:00
Eng Zer Jun a23bcbb7ec
refactor: move from io/ioutil to io and os package 
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2021-10-14 12:27:58 +08:00
Chris Lu 3e2acf677c removing tikv to resolve "go mod tidy" problem 
tikv is causing "go mod tidy" problem. Need to resolve this before adding tikv back.

go mod tidy
go: finding module for package github.com/coreos/etcd/clientv3/balancer/picker
go: finding module for package cloud.google.com/go/kms/apiv1
go: finding module for package github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
go: finding module for package google.golang.org/grpc/naming
go: finding module for package github.com/coreos/etcd/clientv3/credentials
go: finding module for package github.com/coreos/etcd/clientv3/balancer
go: finding module for package github.com/d4l3k/messagediff
go: found github.com/coreos/etcd/clientv3/balancer in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/balancer/picker in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/balancer/resolver/endpoint in github.com/coreos/etcd v3.3.26+incompatible
go: found github.com/coreos/etcd/clientv3/credentials in github.com/coreos/etcd v3.3.26+incompatible
go: found cloud.google.com/go/kms/apiv1 in cloud.google.com/go/kms v1.0.0
go: found github.com/d4l3k/messagediff in github.com/d4l3k/messagediff v1.2.1
go: finding module for package google.golang.org/grpc/naming
github.com/chrislusf/seaweedfs/weed/filer/tikv imports
	github.com/tikv/client-go/v2/tikv imports
	go.etcd.io/etcd/clientv3 tested by
	go.etcd.io/etcd/clientv3.test imports
	github.com/coreos/etcd/integration imports
	github.com/coreos/etcd/proxy/grpcproxy imports
	google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
 2021-10-10 19:27:02 -07:00
Chris Lu bf218cd59d removing etcd sequencer 
causing go mod tidy problem. If anyone wants this, please help to resolve this first.

github.com/chrislusf/seaweedfs/weed/sequence imports
	go.etcd.io/etcd/client tested by
	go.etcd.io/etcd/client.test imports
	github.com/coreos/etcd/integration imports
	github.com/coreos/etcd/proxy/grpcproxy imports
	google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
 2021-10-10 19:10:46 -07:00
Chris Lu 6a030547a2 server: remove peer check if not starting master 
more fix https://github.com/chrislusf/seaweedfs/issues/2352
 2021-10-04 03:27:10 -07:00
Chris Lu 62c2732fd1 add back logic to check master peers 
fix https://github.com/chrislusf/seaweedfs/issues/2352
 2021-10-04 02:51:26 -07:00
Chris Lu d4bb16e20e
Merge pull request #2354 from chrislusf/bptree 
Add Redis3
 2021-10-04 02:43:15 -07:00
Chris Lu 04662126bb add redis3 2021-10-04 01:04:27 -07:00
Chris Lu 366f522a2d add redis3 2021-10-04 01:01:31 -07:00
Chris Lu b297849147 typo 2021-10-01 23:24:54 -07:00
Chris Lu af207bbaf0 retry both assign volume and uploading data 
fix https://github.com/chrislusf/seaweedfs/issues/2351
 2021-10-01 23:23:39 -07:00
Chris Lu 4a1d4d7462 s3: default to allow empty folder for better performance 2021-09-26 22:34:14 -07:00
Chris Lu a814f3f0a8 adjust metadata tail output 2021-09-25 01:04:51 -07:00
Chris Lu 52fe86df45 use default 10000 for grpc port 2021-09-20 14:05:59 -07:00
Chris Lu 84fdda85e7 go fmt 2021-09-19 12:06:15 -07:00
Chris Lu 94a01fcfcb filer.remote.gateway: add options to include or exclude new bucket names to mirror 2021-09-15 23:04:16 -07:00
Chris Lu b5f4910412 adjust help messages 2021-09-15 22:53:10 -07:00
Chris Lu f0907eb83c add command filre.remote.gateway 2021-09-15 22:48:04 -07:00
Chris Lu 63da4bbb54 separate filer.remote.gateway command to avoid confusion 2021-09-15 22:47:17 -07:00
Chris Lu 2789d10342 go fmt 2021-09-14 10:37:06 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu 2c9d4c8f43 custom grpc port: filer 2021-09-12 02:28:37 -07:00
Chris Lu e690a2be16 custom grpc port: volume server 2021-09-12 02:25:15 -07:00
Chris Lu 232ad2fe65 custom grpc port: master follower 2021-09-12 02:25:00 -07:00
Chris Lu bd1f800efe custom grpc port: master 2021-09-12 02:19:10 -07:00
Chris Lu 2b8ab1534a mysql table DDL: minor, not necessary, but a little more strict 
the directory is already hashed into dirhash. So this change is not strictly required.
 2021-09-08 23:55:18 -07:00
Chris Lu 574485ec69 better IP v6 support 2021-09-07 19:29:42 -07:00
Chris Lu 0128239c0f handle ipv6 addresses 2021-09-07 16:43:54 -07:00
Chris Lu 6923af7280 refactoring 2021-09-06 16:20:49 -07:00
Chris Lu 64f6532fbe update help message 2021-09-06 16:16:22 -07:00
Chris Lu 308d4b2dae filer.remote.sync: createBucketWithRandomSuffix defaults to true 2021-09-06 15:14:26 -07:00
Chris Lu c218ef20c7 filer.remote.sync: automatically detect the primary remote storage 2021-09-06 15:10:55 -07:00
Chris Lu 1702ce5395 remove unused code 2021-09-06 12:01:44 -07:00
Chris Lu 60573fd3e2 option to map remote bucket to trimmed bucket name 2021-09-05 11:55:52 -07:00
Chris Lu bdefdee4e6 filer.remote.sync: add option to add randomized suffix to buckets to avoid conflicts 2021-09-04 22:46:28 -07:00
Chris Lu 98c68d7a79 filer.remote.sync fix upload logic 2021-09-04 18:46:28 -07:00
Chris Lu 16a342eb8a re-ordering 2021-09-04 13:46:44 -07:00
Chris Lu 796b2eb929 remember sync time 2021-09-04 13:46:22 -07:00
Chris Lu 99d4594ee8 update remote storage mount mapping 2021-09-04 05:22:34 -07:00
Chris Lu 13cc2ef723 when creating bucket, use remote storage client for default storage 2021-09-04 04:54:55 -07:00
Chris Lu 4b28c5f6c3 filer.remote.sync: split into buckets mode and single directory mode 2021-09-04 04:35:46 -07:00
Chris Lu 03a31587ce go fmt 2021-09-03 20:42:28 -07:00
Chris Lu 7f2a8246a9 minor 2021-09-02 12:32:59 -07:00
Chris Lu 9a73b0e3c9 refactor 2021-09-02 07:07:16 -07:00
Chris Lu e281f0fa82 refactor 2021-09-02 06:53:21 -07:00
Chris Lu 7ce97b59d8 go fmt 2021-09-01 02:45:42 -07:00
Chris Lu 3bd48c4f29 filer.remote.sync: exit when directory is unmounted 
this will not propagate the deletions back to the cloud
 2021-09-01 01:29:22 -07:00
Chris Lu 8e125339d5 line wrap 2021-08-31 23:30:28 -07:00
Chris Lu 68ea99190f add additionalPathPrefixes 2021-08-31 23:26:26 -07:00
Rain Li e5f9ff983d
Merge branch 'master' into filerstore-tikv 2021-08-30 15:52:54 +08:00
Chris Lu 6deee4c0b9 import hdfs 2021-08-29 18:49:43 -07:00
Chris Lu a31f2907f0 cloud drive: filer.remote.sync supports remove folder 2021-08-29 18:46:28 -07:00
Chris Lu 05d2774117 refactoring 2021-08-28 22:49:38 -07:00
Chris Lu a9b41900d8 handle cases when mounted directory does not exist 2021-08-28 01:56:56 -07:00
Chris Lu 05a648bb96 refactor: separating out remote.proto 2021-08-26 15:18:34 -07:00
yulai.li c1dc5ab4ac Add deleterange_concurrency to filer configuration file 2021-08-26 18:25:08 +08:00
yulai.li de8ef28460 Update filer.toml scaffold to support tikv store 2021-08-26 16:26:27 +08:00
yulai.li 546efeba8f Fix build bug 2021-08-26 16:20:35 +08:00
Chris Lu 6bab20d862 cat entry content 2021-08-24 02:38:32 -07:00
Chris Lu e9ebe24f2e cloud drive: add support for Azure 2021-08-24 01:18:30 -07:00
Chris Lu 258063de26 cloud drive: add google cloud storage 2021-08-23 00:29:27 -07:00
Chris Lu 05fc7db755 filer.copy: large files pack file ids into chunks 2021-08-18 23:46:54 -07:00
Chris Lu 72eb6d5b9d ensure no writes to remote storage if content is not changed 2021-08-15 20:23:41 -07:00
Chris Lu 5a7c40510f format output 2021-08-15 20:07:13 -07:00
Chris Lu 5d5a21ba2d adjust log format 2021-08-15 19:46:45 -07:00
Chris Lu c45c12aedc metadata updates 2021-08-15 19:27:30 -07:00
Chris Lu 49b5e47bd1 retry forever with filer.remote.sync, and some refactoring 2021-08-15 12:38:26 -07:00
Chris Lu c34747c79d rename, fix wrong logic. 2021-08-14 21:46:34 -07:00
Chris Lu 0db2517994 go fmt 2021-08-14 02:55:44 -07:00
Chris Lu 7937db52e1 Filer locationPrefix configure does not exec replication #2257 
fix https://github.com/chrislusf/seaweedfs/issues/2257
 2021-08-14 02:54:13 -07:00
Chris Lu 5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu 6238644c35 remove gateway command 2021-08-12 20:52:04 -07:00
Chris Lu d1d1fc772c move some volume lookup operations to grpc 
jwt related lookup will come in next commit
 2021-08-12 20:33:00 -07:00
Chris Lu 5571f4f70a master: add master.follower to handle read file id lookup requests 2021-08-12 18:10:59 -07:00
Chris Lu 5469019852 adjust data type 2021-08-12 17:54:34 -07:00
Chris Lu 713c035a6e shell: remote.cache remote.uncache 2021-08-09 14:35:18 -07:00
Chris Lu 734c980040 volume: support concurrent download data size limit 2021-08-08 23:25:16 -07:00
Chris Lu c5f38c365d go fmt 2021-08-08 22:30:36 -07:00
Chris Lu df85f7a1eb adjust help message 2021-08-08 22:30:12 -07:00
Chris Lu 7412ccdf88 write back remote entry to local entry after uploading to remote 2021-08-08 17:55:03 -07:00
Chris Lu dcf614a8c3 skip if the remote entry update is because of internal managerial operations 2021-08-08 15:58:10 -07:00
Chris Lu 13e45e1605 filer.remote.sync can work now 2021-08-08 01:21:42 -07:00
Chris Lu 6b743dbbf9 refactor client subscribe metadata 2021-08-04 16:25:46 -07:00
Chris Lu d84c311699 refactoring 2021-08-04 12:30:18 -07:00
Chris Lu d2ddf1dbdb typo 2021-08-03 09:32:31 -07:00
Chris Lu 3afbf04007 add TLS grpc support for filer meta clients 2021-08-03 01:30:35 -07:00
Chris Lu fe60c6ef9a minor changes 2021-08-02 11:49:40 -07:00
byunghwa.yun bdc7730fdb Add autocomplete 2021-08-03 02:56:52 +09:00
Chris Lu 3bb640b786 add -force option following #2228 2021-08-01 20:03:05 -07:00
Chris Lu 58bc3ecf47 add default quietFor value 2021-08-01 15:36:06 -07:00
Chris Lu 2ca1839d77 shell: add volume.deleteEmpty command 2021-08-01 15:33:45 -07:00
Chris Lu 1c7e404abe remove buckets folder option 
the related code still works for old deployments
 2021-08-01 12:23:16 -07:00
Chris Lu 6ba65c3382 customizable debug port 2021-07-31 09:18:41 -07:00
Chris Lu 1ff8285d82 debug from any server 2021-07-31 02:13:21 -07:00
Chris Lu 0d6f45cb46 add debug option to generate full goroutine stack dump 2021-07-31 02:00:01 -07:00
Chris Lu ac28611817 snowflake sequencer need an unique id 
fix https://github.com/chrislusf/seaweedfs/issues/2213
 2021-07-23 20:54:03 -07:00