j.laycock
1f01eb78e8
Rename mem_map to mMap, remove some in_memory variables being passed around, added MemoryMapped member to volume struct
2019-09-03 17:00:59 +01:00
Tom Maxwell
d637d86d22
Changes to try and pass the URL parameters through - in memory flag not working still
2019-09-03 15:41:28 +01:00
j.laycock
595a1beff0
Swap imports to use joeslay
2019-09-02 11:28:40 +01:00
xushuxun
5904d78bd4
weed backup: add ttl and replication parameter
2019-08-16 11:05:22 +08:00
Chris Lu
6f75df8660
server: fix starting default master list
2019-08-10 03:10:03 -07:00
Chris Lu
b9f6236f8a
filer: fix typo
...
fix https://github.com/chrislusf/seaweedfs/issues/1031
2019-08-06 00:42:19 -07:00
Chris Lu
c79274f7eb
notes on supported meta data store
2019-08-05 16:45:21 -07:00
Feng Ye
fc0498798b
Add etcd filestore
2019-08-01 10:16:45 +08:00
Chris Lu
8afd8d35b3
master: followers can also lookup and redirect
...
improve scalability
2019-07-28 03:58:13 -07:00
Chris Lu
2c6cf72e73
refactoring
2019-07-28 01:55:05 -07:00
Chris Lu
344ce90315
remove weedfuse
2019-07-24 00:42:12 -07:00
Chris Lu
5956dfd08d
mount: add umask option
...
related to https://github.com/chrislusf/seaweedfs/issues/978
2019-07-24 00:03:05 -07:00
Chris Lu
61b7a650f4
filer.copy default to 32MB chunks
2019-07-10 21:48:27 -07:00
Chris Lu
23d36dfbaf
1.40
...
enable azure replication
2019-07-02 21:27:05 -07:00
Chris Lu
88fed85bba
temp skipping azure
...
can not release due to https://github.com/Azure/azure-pipeline-go/issues/23
2019-06-30 23:17:45 -07:00
Chris Lu
15ecf4e3bf
filer: default to leveldb2
2019-06-30 00:44:57 -07:00
Chris Lu
c777102da9
go fmt
2019-06-23 15:30:16 -07:00
Chris Lu
d5560f2705
simplify metrics settings
2019-06-23 15:29:49 -07:00
Chris Lu
6f8b335007
fix duplicated flags
2019-06-23 03:11:21 -07:00
Chris Lu
f16375621f
big refactoring
2019-06-23 03:08:27 -07:00
Chris Lu
8da5d5b094
filer.copy: use filer settings, avoid unnecessary command line options
...
fix https://github.com/chrislusf/seaweedfs/issues/968
2019-06-23 01:57:35 -07:00
Chris Lu
d8ed73926d
volume servers get metrics address and interval from the master
2019-06-17 14:51:47 -07:00
Chris Lu
8b43679ae3
refactoring
2019-06-15 12:21:44 -07:00
Chris Lu
5f6c9825f8
volume server: adds basic metrics
2019-06-14 00:54:56 -07:00
Chris Lu
a11525fe4e
filer: adds basic metrics pushing to Prometheus gateway
2019-06-13 02:01:54 -07:00
Chris Lu
e66bddd84c
customizable sleep intervals
2019-06-06 00:39:08 -07:00
Chris Lu
50aa769554
jwt for read access control
2019-06-06 00:29:02 -07:00
Chris Lu
784141c5e6
adjust script parameter
2019-06-05 01:48:03 -07:00
Chris Lu
ede876cfdb
periodic scripts exeuction from leader master
2019-06-05 01:30:24 -07:00
Chris Lu
3f9ecee40f
working with reading remote intervals
2019-05-28 21:29:07 -07:00
Chris Lu
17ac1290c0
volume: load ec shards during heartbeats to master
2019-05-21 22:41:20 -07:00
Chris Lu
54b835e1ae
filer: redis cluster add option for password
...
fix https://github.com/chrislusf/seaweedfs/issues/971
2019-05-20 09:00:30 -07:00
Chris Lu
0094ca3f8a
mark the webdav as unstable
2019-05-19 21:08:26 -07:00
Chris Lu
0fe286a6ca
filer: optimization for leveldb, add leveldb2
...
large filer db may see high CPU and disk usage due to background compaction
2019-05-17 17:33:49 -07:00
Chris Lu
7e0c3bb732
filer: adjust recommended mysql meta data type to LONGBLOB
...
fix https://github.com/chrislusf/seaweedfs/issues/966
2019-05-17 09:11:08 -07:00
Chris Lu
ab22407763
filer ip only use bind ip
2019-05-14 23:43:40 -07:00
Chris Lu
4dfcd2169e
weedfuse: adapt to fstab command line pattern
2019-05-11 01:17:55 -07:00
Chris Lu
9a4fb14ea0
weedfuse: daemonize
2019-05-10 17:08:15 -07:00
Chris Lu
1ca1ec906a
add weedfuse
2019-05-10 15:03:31 -07:00
Chris Lu
d01807ba05
shell: volume.balance add option for all_collections, each_collection
2019-05-06 14:12:19 -07:00
Chris Lu
4c392ca762
use ip.bind to listen for grpc calls
...
help to avoid https://github.com/chrislusf/seaweedfs/issues/952
2019-05-06 08:10:18 -07:00
Chris Lu
25941e0500
master: add jwt expires_after_seconds
2019-05-04 08:42:25 -07:00
Chris Lu
3fbe22b203
better error handling for nil raftServer
...
addressing https://github.com/chrislusf/seaweedfs/issues/717
2019-05-03 22:33:08 -07:00
Chris Lu
b335f81a4f
volume: add option to limit compaction speed
2019-05-03 17:22:39 -07:00
Chris Lu
9d3c945b00
Merge pull request #950 from chrislusf/webdav
...
add WebDav
2019-05-03 01:25:10 -07:00
Chris Lu
b30c14b631
webdav: can read now
2019-05-03 00:24:35 -07:00
Chris Lu
47f14775d7
scaffolding, but seems hard to satisfy mac, windows, and different os versions
...
https://www.reddit.com/r/golang/comments/3wrxo3/webdav_server_that_provide_filesystem_using/
2019-05-02 14:22:10 -07:00
Wine93
32f93fb09e
style: go fmt for all
2019-04-30 03:22:19 +00:00
Chris Lu
82e06a780b
mount: allow nonempty fuse mount to work with Samba
...
possibly fix https://github.com/chrislusf/seaweedfs/issues/936
2019-04-29 10:42:09 -07:00
Chris Lu
9774ede3cf
adjust usage description
2019-04-24 00:25:20 -07:00
Chris Lu
b04d7e3ac0
weed server: also optionally start S3 gateway
2019-04-24 00:18:01 -07:00
Chris Lu
dabc9c9a89
filer: listen on specified ip address
2019-04-24 00:01:39 -07:00
Chris Lu
21fe5bde7c
filer: remove customizable grpc port, fixed to filer port + 10000
2019-04-23 23:58:52 -07:00
Chris Lu
3b3651dea3
volume: atomic copying file, adds version and stopOffset
2019-04-19 12:29:49 -07:00
Chris Lu
730a032137
filer mysql: adjust scaffolding instructions
2019-04-19 09:55:46 -07:00
Chris Lu
ac2727853f
fix needle map entry size
2019-04-19 00:39:34 -07:00
Chris Lu
e5506152c0
refactoring
2019-04-18 21:43:36 -07:00
Chris Lu
a2d34d4802
filer.copy: add adjustable compression level
2019-04-18 20:21:28 -07:00
Chris Lu
13ad5c1966
refactoring
2019-04-17 22:04:49 -07:00
Chris Lu
8ea1ee6dfa
weed shell: add fs.meta.notify, removing filer.export
2019-04-16 01:58:28 -07:00
Chris Lu
2e03f3adef
filer: rename -port.public to -port.readonly
2019-04-11 20:42:55 -07:00
Wine93
361912224d
typo: remove blank
2019-04-11 09:18:53 +00:00
Chris Lu
c0d6cc0067
weed mount: mount to the same folder name
2019-04-10 23:25:41 -07:00
Chris Lu
3555628ad5
weed volume: fix bug, remove boltdb, btree
2019-04-09 10:08:59 -07:00
Chris Lu
8a5ce16e96
weed volume: remove boltdb, btree options, add options for leveldb medium, large
2019-04-09 09:42:06 -07:00
Chris Lu
88ba08cb9e
add large disk support
2019-04-08 22:32:42 -07:00
Chris Lu
000ee725fc
refactor Offset into a struct of bytes
2019-04-08 19:40:56 -07:00
Chris Lu
a41ba79119
Merge pull request #914 from jba/gocdk-receive
...
replication: add GoCDK PubSub support
2019-04-08 10:31:02 -07:00
Chris Lu
d14b614407
weed filer.copy: use existing file owner and gropu id
2019-04-07 11:31:50 -07:00
Chris Lu
c731d9be6d
weed filer.copy: add option to profile the execution
2019-04-07 09:13:24 -07:00
Chris Lu
24c020104a
weed filer.copy: parallelize the copying to increase throughput
...
improvement https://github.com/chrislusf/seaweedfs/issues/915
2019-04-05 23:35:30 -07:00
Jonathan Amsterdam
72920efc20
added entry to scaffold
2019-04-05 13:43:38 -04:00
Chris Lu
300b39b5af
weed filer.copy: use cached filer grpc connection
...
another attemp for https://github.com/chrislusf/seaweedfs/issues/915
2019-04-05 01:09:06 -07:00
Chris Lu
921231268b
weed filer.copy: skip files that can not be opened
...
such as socket
2019-04-05 00:04:00 -07:00
Chris Lu
4f714cef22
default weed filer.copy and weed upload chunk size to 32MB
2019-04-04 23:14:32 -07:00
Chris Lu
af37b374cb
weed shell: fs.cd change current directory
2019-04-04 19:27:51 -07:00
Chris Lu
715a38da1e
weed shell: add fs.cd, fs.pwd to change to a directory and print current directory
2019-04-03 00:20:00 -07:00
Chris Lu
2a52e70a41
weed filer: increase default mysql/postgres dir or name length to 65535
2019-04-01 12:37:54 -07:00
Chris Lu
920b4e56aa
refactoring
2019-03-29 22:53:35 -07:00
Chris Lu
f60d383947
weed filer.copy: support multiple masters
2019-03-29 22:50:14 -07:00
Chris Lu
487ca51491
weed filer.export: fix pagination error
2019-03-28 00:46:15 -07:00
Chris Lu
7a14cdc90c
refactoring, go fmt
2019-03-25 23:18:40 -07:00
Chris Lu
df95ce0b6c
weed backup: efficient delta backup
...
fix https://github.com/chrislusf/seaweedfs/issues/399
2019-03-25 23:01:53 -07:00
Chris Lu
70815e9124
WIP
2019-03-25 09:16:12 -07:00
Chris Lu
a3490b600c
weed filer, weed master: add option to disable http
2019-03-21 16:00:46 -07:00
Jonathan Amsterdam
e966033098
Merge branch 'master' of https://github.com/chrislusf/seaweedfs into gocdk
2019-03-20 16:02:38 -04:00
Jonathan Amsterdam
8db82e2b75
notification: add Go CDK pubsub support
...
Add the gocdk_pub_sub package, which supports the Go Cloud Development
Kit pubsub API.
Link in all current providers.
Update the notification scaffold.
2019-03-20 07:57:58 -04:00
chenwanli
45a52b17fd
benchmark: add replication flag
2019-03-20 16:38:11 +08:00
Chris Lu
da871896c3
weed filer: set grpc port to port + 10000
2019-03-19 05:47:41 -07:00
Chris Lu
657dd2e6c9
add shell command to list all collections
2019-03-16 13:43:16 -07:00
Chris Lu
55bab1b456
add context.Context
2019-03-15 17:20:24 -07:00
Chris Lu
cece860bfd
add context to all filer APIs
2019-03-15 15:55:34 -07:00
Chris Lu
d30600e6d6
fix filer starting under "weed server" when ip is not localhost
...
fix https://github.com/chrislusf/seaweedfs/issues/886
2019-03-15 09:57:33 -07:00
Chris Lu
a4e8aa98a0
weed server: mkdir first, and set the master ip address
...
fix https://github.com/chrislusf/seaweedfs/issues/885
2019-03-15 00:32:42 -07:00
Chris Lu
6fe071175d
mount: add option to disable allow others
...
fix https://github.com/chrislusf/seaweedfs/issues/877
2019-03-10 19:45:40 -07:00
Chris Lu
4773497d2c
add back "/cluster/status"
...
fix https://github.com/chrislusf/seaweedfs/issues/870
2019-03-06 12:10:45 -08:00
Chris Lu
ab32556a51
weed server: fix error coming from moving raft to grpc
...
fix https://github.com/chrislusf/seaweedfs/issues/872
2019-03-06 10:21:49 -08:00
Chris Lu
2812c14520
master: add option to disable http operations
2019-03-02 05:10:05 -08:00
Chris Lu
7bbe24dd28
volume server directly support https
2019-02-25 00:43:36 -08:00
Chris Lu
0fa1be8c4b
adjust formatting and comments
2019-02-19 21:10:10 -08:00
Chris Lu
07af52cb6f
raft change from http to grpc
...
master grpc port is fixed to http port + 10000
2019-02-18 22:38:14 -08:00
Chris Lu
77b9af531d
adding grpc mutual tls
2019-02-18 12:11:52 -08:00
Chris Lu
98a03b38e5
avoid util package depends on security package
2019-02-16 12:23:35 -08:00
Chris Lu
74fb237727
benchmark can work in secure mode
2019-02-15 00:09:19 -08:00
Chris Lu
215cd27b37
add authorizing fileId write access
...
need to secure upload/update/delete for benchmark/filer/mount
need to add secure grpc
2019-02-14 00:08:20 -08:00
Chris Lu
4ff4a147b2
cleanup security.Secret
2019-02-09 21:56:32 -08:00
Chris Lu
501bd72b1c
wip: add security.toml file
2019-02-09 21:07:12 -08:00
Sergey
aa5ccff6d2
fixing of typos
2019-02-06 18:59:15 +05:00
Chris Lu
a6bffc58f9
go fmt
2019-01-18 14:15:10 -08:00
Chris Lu
67e2ea72be
master add separate grpc port
...
due to https://github.com/soheilhy/cmux/issues/64
fix https://github.com/chrislusf/seaweedfs/issues/820
fix https://github.com/chrislusf/seaweedfs/issues/840
fix https://github.com/chrislusf/seaweedfs/issues/841
2019-01-18 14:14:47 -08:00
bingoohuang
ab6be025d7
go fmt and fix some typo
2019-01-17 09:17:19 +08:00
bingoohuang
6ddfaf33cb
extract VolumeFileScanner for ScanVolumeFile
2019-01-16 17:48:59 +08:00
Chris Lu
6740427638
weed mount: reduce default chunk size
2019-01-05 15:17:03 -08:00
Chris Lu
55f7d8f801
remove extra log
2018-12-29 18:00:19 -08:00
Chris Lu
170ed53da5
weed mount: mac add AutoXattr
2018-12-29 13:55:38 -08:00
Chris Lu
9f3836072a
weed mount: no mac finder broswe mode
2018-12-29 13:16:23 -08:00
Chris Lu
d7bf6c5b91
switching_to_seaweedfs_fuse
2018-12-29 10:31:36 -08:00
Chris Lu
03c7447ad6
set mount point to existing permissions
...
fix https://github.com/chrislusf/seaweedfs/issues/806
2018-12-28 23:36:13 -08:00
Chris Lu
2a19511ba8
avoid gzip benchmark data
...
related to https://github.com/chrislusf/seaweedfs/issues/804
2018-12-27 13:10:46 -08:00
Chris Lu
ba923e2864
error if the last arguement does not end with forward slash
2018-12-23 18:20:11 -08:00
Chris Lu
49f40cd145
adjust help message
2018-12-22 15:05:15 -08:00
Chris Lu
852ee21835
avoid .gz auto decompression
2018-12-22 13:58:16 -08:00
Chris Lu
93d0ad09af
error log format
2018-12-16 17:37:10 -08:00
Chris Lu
c28e8a2397
refactoring
2018-12-06 00:44:41 -08:00
Chris Lu
ffa2827ab1
fail fast if two notification queues or inputs are enabled
2018-12-06 00:37:59 -08:00
Chris Lu
52b24a9902
default "weed server -filer" to same directory as -mdir
2018-12-05 23:24:25 -08:00
Chris Lu
fb43a9d7b9
adjust message
2018-12-05 02:03:03 -08:00
Chris Lu
1cbd53c01c
WIP SeaweedFileSystem added mkdirs, getFileStatus, listStatus, delete
2018-11-25 13:43:26 -08:00
Chris Lu
86dd933596
go fmt
2018-11-23 00:26:15 -08:00
Chris Lu
6344469a9b
set fuse subtype
2018-11-22 23:04:23 -08:00
Chris Lu
60d3681371
default images.fix.orientation to false
2018-11-21 16:50:42 -08:00
Chris Lu
c9b3ef58de
options to control filer.export verbosity
2018-11-21 15:15:40 -08:00
Chris Lu
2f2e417b68
detailed export
2018-11-21 15:12:47 -08:00
Chris Lu
2e6a3c7b16
reduce memory usage
2018-11-14 22:48:54 -08:00
Chris Lu
a62adf62ca
a bit faster
2018-11-12 11:19:13 -08:00
Chris Lu
affe708961
a bit faster
2018-11-11 00:43:30 -08:00
Chris Lu
a09eaf8756
adjusting, seems there are some perf benefit
2018-11-11 00:08:22 -08:00
Chris Lu
cbd94b18a5
improve "ls -al" performance for large directory
2018-11-08 07:37:34 -08:00
Chris Lu
85d1d99459
default "weed export to current enabled store"
2018-11-04 12:59:29 -08:00
Chris Lu
6c45787a37
selectively export files
2018-11-04 12:07:33 -08:00
Chris Lu
ff4189ab25
go fmt
2018-11-04 11:59:08 -08:00
Chris Lu
8dfac6a4cf
working b2 sink
2018-11-04 11:58:59 -08:00
Chris Lu
a64613172d
bootstrap filer replication with weed filer.export -targetStore=notification
2018-11-03 12:43:45 -07:00
Chris Lu
3674ad9f8e
go fmt
2018-11-01 01:12:21 -07:00
Chris Lu
db584ff7f8
separate into notification.toml, add gcp pub/sub message queue
2018-11-01 01:11:09 -07:00
Chris Lu
4c97ff3717
support AWS SQS as file change notification message queue
2018-10-31 01:11:19 -07:00
Chris Lu
fe31f389b0
add configuration instructions
2018-10-30 02:29:11 -07:00
Chris Lu
c58b73ad9b
adding BackBlaze, Azure
2018-10-23 23:59:40 -07:00
Chris Lu
fda771c83f
migrate volume sync status to grpc API on volume server
2018-10-15 01:19:15 -07:00
Chris Lu
b1daede91b
move volume vacuum to gRpc
2018-10-14 23:12:43 -07:00