Chris Lu
6344469a9b
set fuse subtype
2018-11-22 23:04:23 -08:00
Chris Lu
60d3681371
default images.fix.orientation to false
2018-11-21 16:50:42 -08:00
Chris Lu
c9b3ef58de
options to control filer.export verbosity
2018-11-21 15:15:40 -08:00
Chris Lu
2f2e417b68
detailed export
2018-11-21 15:12:47 -08:00
Chris Lu
2e6a3c7b16
reduce memory usage
2018-11-14 22:48:54 -08:00
Chris Lu
a62adf62ca
a bit faster
2018-11-12 11:19:13 -08:00
Chris Lu
affe708961
a bit faster
2018-11-11 00:43:30 -08:00
Chris Lu
a09eaf8756
adjusting, seems there are some perf benefit
2018-11-11 00:08:22 -08:00
Chris Lu
cbd94b18a5
improve "ls -al" performance for large directory
2018-11-08 07:37:34 -08:00
Chris Lu
85d1d99459
default "weed export to current enabled store"
2018-11-04 12:59:29 -08:00
Chris Lu
6c45787a37
selectively export files
2018-11-04 12:07:33 -08:00
Chris Lu
ff4189ab25
go fmt
2018-11-04 11:59:08 -08:00
Chris Lu
8dfac6a4cf
working b2 sink
2018-11-04 11:58:59 -08:00
Chris Lu
a64613172d
bootstrap filer replication with weed filer.export -targetStore=notification
2018-11-03 12:43:45 -07:00
Chris Lu
3674ad9f8e
go fmt
2018-11-01 01:12:21 -07:00
Chris Lu
db584ff7f8
separate into notification.toml, add gcp pub/sub message queue
2018-11-01 01:11:09 -07:00
Chris Lu
4c97ff3717
support AWS SQS as file change notification message queue
2018-10-31 01:11:19 -07:00
Chris Lu
fe31f389b0
add configuration instructions
2018-10-30 02:29:11 -07:00
Chris Lu
c58b73ad9b
adding BackBlaze, Azure
2018-10-23 23:59:40 -07:00
Chris Lu
fda771c83f
migrate volume sync status to grpc API on volume server
2018-10-15 01:19:15 -07:00
Chris Lu
b1daede91b
move volume vacuum to gRpc
2018-10-14 23:12:43 -07:00
Chris Lu
556382ff5f
adding basic grpc to volume server
2018-10-11 01:16:33 -07:00
Chris Lu
08266b7256
go fmt
2018-10-11 00:08:13 -07:00
Chris Lu
1d7e01b754
reformat
2018-10-11 00:05:54 -07:00
Chris Lu
da6154b29c
refactor volume server to startVolumeServer()
2018-10-11 00:04:31 -07:00
Chris Lu
60d2f1557d
refactor func name
2018-10-10 23:19:54 -07:00
Chris Lu
9f4c2f87d3
adding filer replication sink to Azure
2018-10-09 01:35:48 -07:00
Chris Lu
96a23591e7
remove comments
2018-10-07 22:21:00 -07:00
Chris Lu
a5b5887efc
add filer replication to aws s3
2018-10-06 17:10:15 -07:00
Chris Lu
84d4a2c8a0
add filer replication to google cloud storage
2018-10-06 13:08:38 -07:00
Chris Lu
14858f6224
add logging
2018-10-06 13:01:38 -07:00
Chris Lu
56c5c7b1b6
add google cloud storage
2018-10-04 01:14:44 -07:00
Chris Lu
e8ef501f02
add s3 replication sink
2018-10-03 23:36:52 -07:00
Chris Lu
98384b62fd
adjust description
2018-09-28 01:58:34 -07:00
Chris Lu
12ba8b9799
generate useable filer.toml
2018-09-23 09:26:25 -07:00
Chris Lu
b16d47030e
save and use last processed kafka offsets
2018-09-23 01:34:40 -07:00
Chris Lu
9fe24991d5
refactoring
2018-09-23 00:40:36 -07:00
Chris Lu
7d6b2a4740
add ETag to upload results and chunks
2018-09-22 22:12:21 -07:00
Chris Lu
420f068376
add more instructions
2018-09-22 22:12:06 -07:00
Chris Lu
01ceace18e
adjust sink options
2018-09-22 00:53:52 -07:00
Chris Lu
402ab598b6
go fmt
2018-09-22 00:12:10 -07:00
Chris Lu
7bb62b9f21
delay file creation without chunks from dir.Create to file.Flush
2018-09-22 00:11:46 -07:00
Chris Lu
db69ce89f0
go fmt
2018-09-21 01:56:43 -07:00
Chris Lu
a6cfaba018
able to sync the changes
2018-09-21 01:54:29 -07:00
Chris Lu
a8cd7df44a
let filer choose default replication
2018-09-20 09:00:15 -07:00
Chris Lu
d5149e592e
add logging on kafka reading
2018-09-19 22:31:45 -07:00
Chris Lu
25fb6f9a46
fix compilation
2018-09-17 02:23:21 -07:00
Chris Lu
779641e9d4
adjust replicated entry name
2018-09-17 01:37:24 -07:00
Chris Lu
b7a375f5aa
set large limit for "weed filer.export" directory listing
2018-09-17 00:42:36 -07:00
Chris Lu
4e1f1a0b92
set large limit for "weed mount" directory listing
2018-09-17 00:41:33 -07:00
Chris Lu
788acdf527
add WIP filer.replicate
2018-09-17 00:27:56 -07:00
Chris Lu
7ad5a90127
go fmt
2018-09-03 16:47:00 -07:00
Chris Lu
6e3f4d1079
add working filer.export command
2018-08-23 00:02:04 -07:00
Chris Lu
98b8f8649d
adding filer.export prototype
2018-08-22 23:54:22 -07:00
Chris Lu
ae4e654d2a
add example configuration for default redis create-cluster setup
2018-08-22 18:23:19 -07:00
Chris Lu
748870b03f
add "weed scaffold" command
2018-08-19 15:36:30 -07:00
Chris Lu
f827ada811
merge notification config with filer.toml
2018-08-19 15:17:55 -07:00
Chris Lu
f036ef8a3c
add filer notification
2018-08-13 01:20:49 -07:00
Chris Lu
75d63db60d
randomize raft server startup
...
also some go fmt
2018-08-12 14:27:14 -07:00
Chris Lu
eca4b928d2
ensure master server count is odd
2018-08-12 14:25:31 -07:00
Chris Lu
2e427c66ac
small refactoring
2018-08-10 23:47:17 -07:00
Chris Lu
44d8e35988
add listener for volumd id location for benchmark tool
2018-07-31 19:12:36 -07:00
Chris Lu
d09606c65b
refactoring
2018-07-29 02:25:24 -07:00
Chris Lu
4f317c7e3d
working version3
2018-07-24 01:36:04 -07:00
Chris Lu
7f32eb1e25
s3 add https support
2018-07-22 21:28:54 -07:00
Chris Lu
056871242b
remove println
2018-07-22 01:17:49 -07:00
Chris Lu
0655151b8c
go fmt
2018-07-22 01:15:11 -07:00
Chris Lu
6319d84f42
s3 API add ListObjectsV1
2018-07-22 01:14:36 -07:00
Chris Lu
922c614bde
fix compilation problem
2018-07-21 20:42:14 -07:00
Chris Lu
16c66e097b
adjust default buckets folder
2018-07-21 20:29:33 -07:00
Chris Lu
7e2031b18f
go fmt
2018-07-21 17:39:10 -07:00
Chris Lu
c7228fefa7
add bucket creation and deletion
...
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
2018-07-19 01:21:44 -07:00
Chris Lu
834a25f084
add list all my buckets
2018-07-18 02:37:09 -07:00
Chris Lu
7abfab8e77
add feature to mount a specific filer path to local directory
2018-07-14 21:09:21 -07:00
Chris Lu
3edfe1d28f
extend export command to show tombstone + change output format to CSV
...
merging https://github.com/chrislusf/seaweedfs/pull/610 and add "-limit" option
2018-07-14 20:51:17 -07:00
Chris Lu
6d58916cfd
fix compilation error
2018-07-14 13:39:57 -07:00
Chris Lu
842dab07b4
weed mount can request to prioritize to write to a data center
2018-07-14 13:36:28 -07:00
Chris Lu
1e1c368ae8
adjust filer help message
2018-07-10 00:22:11 -07:00
Chris Lu
50d351944f
filer add data center option
2018-07-09 02:22:48 -07:00
Chris Lu
d4d7ced922
refactoring: add type for needle id, offset
...
later the type size can possibly be adjusted
2018-07-08 02:28:04 -07:00
Chris Lu
922032b9bb
add filer sub dir list size limit to "weed server"
2018-07-08 02:11:36 -07:00
Chris Lu
702ff48cda
add dir list limit option
2018-07-07 02:18:47 -07:00
Chris Lu
2ad45ca04f
refactor
2018-07-07 02:03:25 -07:00
Chris Lu
77fc8c5914
keep alive for gRpc calls
2018-07-03 19:07:55 -07:00
Chris Lu
5bd72696ac
weed mount add ttl option
2018-06-11 23:13:33 -07:00
Chris Lu
98110c1697
add collection, replication, ttl to each file
2018-06-10 16:57:32 -07:00
Chris Lu
299312c805
use separate filer grpc port
2018-06-05 23:37:41 -07:00
Chris Lu
43e3f5724c
use fixed list of masters in both filer and volume servers
2018-06-01 00:39:39 -07:00
Chris Lu
dc13e10637
fix copy error on 0 size files
2018-05-30 22:28:14 -07:00
Chris Lu
44acf4b756
fix file name
...
f.Name() is not the base file name
2018-05-30 20:48:13 -07:00
Chris Lu
0301504184
add mime, use simple insert and update filer store API
...
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
2018-05-30 20:24:57 -07:00
Chris Lu
26e7cd8c75
assign a different volume on large file copying
2018-05-30 01:05:26 -07:00
Chris Lu
2fe0d479f1
filer.copy supports chunking
2018-05-30 00:54:56 -07:00
Chris Lu
2d7dea1a09
a little refactoring
2018-05-29 23:52:27 -07:00
Chris Lu
8a48baa056
add single chunk file copying to new filer
2018-05-29 23:46:45 -07:00
Chris Lu
4e3ea49cff
properly working local write buffer
2018-05-29 01:21:21 -07:00
Chris Lu
5c4480ec6c
add mountOptions.chunkSizeLimitMB, remove cmdMount.IsDebug
2018-05-28 13:42:25 -07:00
Chris Lu
8ab7dd9d08
weed mount add options for collection and replication
2018-05-28 13:24:48 -07:00
Chris Lu
458ada173e
go fmt
2018-05-27 11:52:26 -07:00
Chris Lu
9e77563c99
add leveldb store
...
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
2018-05-26 03:49:46 -07:00