Chris Lu
2442d56671
adjust package name
2019-02-18 18:03:16 -08:00
Chris Lu
77b9af531d
adding grpc mutual tls
2019-02-18 12:11:52 -08:00
bingoohuang
ab6be025d7
go fmt and fix some typo
2019-01-17 09:17:19 +08:00
Chris Lu
2a75a36b27
set cassandra ttl with entry.TTLSec
2019-01-08 20:26:19 -08:00
byungjo.yoon
58fd2e5a19
Set redis key ttl with entry.TTLSec
2019-01-09 11:48:19 +09:00
Chris Lu
be9a7592a1
incrementally calculate visible intervals
2019-01-05 22:23:44 -08:00
Chris Lu
832ac8129a
reduce pointer usage
2018-12-31 15:10:14 -08:00
Chris Lu
0388d421d2
caching visible intervals for read
...
speeds up 4x in single thread mode
speeds up 30% in 32 threads mode
2018-12-30 00:51:44 -08:00
Chris Lu
253f190f48
weed mount add symlink support
2018-12-25 22:45:44 -08:00
Chris Lu
3ac54792e1
paginate when filer deleting and FUSE mount renaming
2018-12-16 23:20:08 -08:00
Chris Lu
1bfb96f34d
optimization for reading whole chunk with gzip encoding
2018-12-07 01:57:55 -08:00
Chris Lu
c28e8a2397
refactoring
2018-12-06 00:44:41 -08:00
Chris Lu
63619f8b8e
fail early if two filer stores are defined
2018-12-06 00:32:36 -08:00
Chris Lu
52b24a9902
default "weed server -filer" to same directory as -mdir
2018-12-05 23:24:25 -08:00
Chris Lu
b0838db5a8
support hdfs user name and group names
2018-12-04 00:13:40 -08:00
Chris Lu
c6b646e92c
skip creation for root
2018-12-02 23:16:17 -08:00
Chris Lu
f0677c5af1
default root to 0755
2018-12-02 19:59:47 -08:00
Chris Lu
db22666a30
add default root directory
2018-12-02 19:42:50 -08:00
Chris Lu
7a6f49cd42
refactor a bit more
...
fix https://github.com/chrislusf/seaweedfs/issues/777
2018-11-29 00:07:54 -08:00
Chris Lu
ceca078acb
avoid overwriting file or directory
...
fix https://github.com/chrislusf/seaweedfs/issues/777
2018-11-29 00:00:56 -08:00
Chris Lu
86dd933596
go fmt
2018-11-23 00:26:15 -08:00
Chris Lu
99256100a0
fix tests
2018-11-21 16:35:27 -08:00
Chris Lu
e5ad2223a1
add missing intervals
...
fix https://github.com/chrislusf/seaweedfs/issues/774
2018-11-21 16:25:13 -08:00
Chris Lu
b282e34dc2
async file chunk deletion
2018-11-20 20:56:28 -08:00
Chris Lu
b318167cd3
batch delete file ids
2018-11-20 11:35:08 -08:00
Chris Lu
ee3fe07acf
fix sort bug
2018-11-18 21:59:53 -08:00
Chris Lu
469863f7f0
merge interval now is quite faster with reusing slices
2018-11-18 21:24:58 -08:00
Chris Lu
d2924c7748
merge intervals is a bit faster
2018-11-18 20:31:39 -08:00
Chris Lu
cb5a10c6a3
adjust the tests
2018-11-18 17:05:36 -08:00
Chris Lu
f1db22d48b
add benchmark test for merging intervals
2018-11-18 16:49:08 -08:00
Chris Lu
4fcfc9410f
cleanup
2018-11-18 11:51:38 -08:00
Chris Lu
9655dc9ca9
simpler logic
2018-11-18 10:07:30 -08:00
Chris Lu
a64613172d
bootstrap filer replication with weed filer.export -targetStore=notification
2018-11-03 12:43:45 -07:00
Chris Lu
4c97ff3717
support AWS SQS as file change notification message queue
2018-10-31 01:11:19 -07:00
Chris Lu
ff66269b62
use grpc to replace http APIs for batch volume id lookup and batch delete
...
1. remove batch volume id lookup http API /vol/lookup
2. remove batch delete http API /delete
2018-10-14 00:12:28 -07:00
Chris Lu
56a5d5af8d
ignore duplicated key error if the store has duplicated primary key prevention
...
fix https://github.com/chrislusf/seaweedfs/issues/741
2018-09-30 00:49:52 -07:00
Chris Lu
7bb62b9f21
delay file creation without chunks from dir.Create to file.Flush
2018-09-22 00:11:46 -07:00
Chris Lu
db69ce89f0
go fmt
2018-09-21 01:56:43 -07:00
Chris Lu
788acdf527
add WIP filer.replicate
2018-09-17 00:27:56 -07:00
Chris Lu
70e67b7c6d
add notification log
2018-09-16 11:20:36 -07:00
Chris Lu
d923ba2206
renaming msgqueue to notification
2018-09-16 01:18:30 -07:00
Chris Lu
296acc4a0a
go fmt
2018-09-09 16:26:11 -07:00
Chris Lu
164091c269
add s3 multipart upload
2018-09-09 16:25:43 -07:00
Chris Lu
b23883b3b6
fix a bug on error reporting
2018-08-22 18:19:26 -07:00
Chris Lu
f22d6b87e1
go fmt
2018-08-19 15:42:50 -07:00
Chris Lu
748870b03f
add "weed scaffold" command
2018-08-19 15:36:30 -07:00
Chris Lu
95b997fb35
go fmt
2018-08-19 15:18:37 -07:00
Chris Lu
f827ada811
merge notification config with filer.toml
2018-08-19 15:17:55 -07:00
Chris Lu
ee920d4f83
kafka set the right topic
2018-08-19 01:27:30 -07:00
Chris Lu
708acee502
add support for redis cluster
...
fix https://github.com/chrislusf/seaweedfs/issues/705
2018-08-15 00:01:38 -07:00
Chris Lu
ae9b81c18f
fix tests
2018-08-13 01:33:21 -07:00
Chris Lu
f036ef8a3c
add filer notification
2018-08-13 01:20:49 -07:00
Chris Lu
c81f1cda47
correctly recursively delete folders
2018-08-01 01:26:41 -07:00
Chris Lu
d3205a0070
go fmt
2018-07-28 21:02:56 -07:00
Chris Lu
888eb2abb5
filer read write all via locations from MasterClient
2018-07-28 14:51:36 -07:00
Chris Lu
1d779389cb
MasterClient replicates all vid locations
2018-07-28 14:22:46 -07:00
Chris Lu
01bcc89803
refactor into MasterClient
2018-07-28 02:10:32 -07:00
Chris Lu
a12c7b86b0
broadcast messages of new and deleted volumes
2018-07-27 23:09:55 -07:00
Chris Lu
6f30a78a6c
update also delete old chunks if different from latest chunks
2018-07-21 17:40:00 -07:00
Chris Lu
7e2031b18f
go fmt
2018-07-21 17:39:10 -07:00
Chris Lu
80d80daf64
set filer2.ErrNotFound for not found entry
2018-07-20 02:14:18 -07:00
Chris Lu
d429b6296b
fix test
2018-07-19 01:27:45 -07:00
Chris Lu
c7228fefa7
add bucket creation and deletion
...
1. option for "weed s3 -filer.dir.buckets" to choose a folder for buckets
2. create a bucket
3. delete a bucket, recursively delete all metadata on filer
2018-07-19 01:21:44 -07:00
Chris Lu
9e974f6a1b
Merge pull request #686 from qszxnp/filer2_dir_cache_bugfix
...
bugfix:delete Directory cache when filer2 delete Directory
2018-07-12 01:17:32 -07:00
聂鹏
1ed41f8b21
bugfix:check Directory attr before create it
2018-07-12 00:53:31 -04:00
聂鹏
c523bc4f07
bugfix:delete Directory cache when filer2 delete Directory
2018-07-11 21:27:25 -04:00
Chris Lu
77fc8c5914
keep alive for gRpc calls
2018-07-03 19:07:55 -07:00
Chris Lu
14c6f12b0c
adjust interface parameter name and doc
2018-06-17 13:24:57 -07:00
Chris Lu
db0c25b3d8
delete old entry only after successful insert
2018-06-17 13:02:22 -07:00
Chris Lu
eb2acd11c2
decouple from viper for filer store
2018-06-17 13:01:57 -07:00
Chris Lu
98110c1697
add collection, replication, ttl to each file
2018-06-10 16:57:32 -07:00
Chris Lu
5c25d29272
support renaming files
2018-06-06 22:11:01 -07:00
Chris Lu
43e3f5724c
use fixed list of masters in both filer and volume servers
2018-06-01 00:39:39 -07:00
Chris Lu
0301504184
add mime, use simple insert and update filer store API
...
1. add mime type to file in filer
2. purge old chunks if overwrite during insert
2018-05-30 20:24:57 -07:00
Chris Lu
74332e1a61
minor
2018-05-28 14:32:16 -07:00
Chris Lu
07e0d13d2d
filer support reading multiple chunks, with range support
2018-05-28 05:39:12 -07:00
Chris Lu
8db9319a06
refactoring: go fmt, reorg
2018-05-27 23:59:49 -07:00
Chris Lu
8b0718ac92
go vet
2018-05-27 11:56:49 -07:00
Chris Lu
458ada173e
go fmt
2018-05-27 11:52:26 -07:00
Chris Lu
03d76479a5
add reds
2018-05-27 11:14:29 -07:00
Chris Lu
a6a6b87f1b
adjust readme
2018-05-27 11:14:12 -07:00
Chris Lu
ab86d263c9
add Cassandra
2018-05-27 00:01:15 -07:00
Chris Lu
603de2d5db
skip deletion if entry not found
2018-05-27 00:00:56 -07:00
Chris Lu
f124ebab1d
fix ordering
2018-05-27 00:00:11 -07:00
Chris Lu
869161a261
support both mysql and postgres
2018-05-26 22:02:49 -07:00
Chris Lu
87b3b84471
simplifying dirhash to 64bit integer
2018-05-26 21:24:03 -07:00
Chris Lu
955eae3500
this works great and support long file names
2018-05-26 14:08:55 -07:00
Chris Lu
2da84ed331
working fine now
...
but index length can be improved
2018-05-26 13:35:56 -07:00
Chris Lu
68bcaff14d
mysql can compile, not yet tested!
2018-05-26 05:32:15 -07:00
Chris Lu
c1e353491a
refactoring variable names
2018-05-26 04:50:55 -07:00
Chris Lu
828e4a5ace
fix test error
2018-05-26 04:13:50 -07:00
Chris Lu
9e77563c99
add leveldb store
...
1. switch to viper for filer store configuration
2. simplify FindEntry() return values, removing “found”
3. add leveldb store
2018-05-26 03:49:46 -07:00
Chris Lu
c34feca59c
refactoring
2018-05-25 23:27:06 -07:00
Chris Lu
72fc001d0d
remove unused code
2018-05-25 01:30:16 -07:00
Chris Lu
0a223838bd
refactoring
2018-05-25 00:57:25 -07:00
Chris Lu
a01557f4cb
adjust filer web page
2018-05-24 23:21:12 -07:00
Chris Lu
e18c7e160c
avoid empty chunk view
2018-05-24 23:19:56 -07:00
Chris Lu
d773e11c7a
file handler directly read from volume servers
...
this mostly works fine now!
next: need to cache files to local disk
2018-05-24 01:22:37 -07:00
Chris Lu
00d0274fd7
prepare to read from multiple file chunks
2018-05-23 22:28:54 -07:00
Chris Lu
e97c60cc10
avoid overwriting variables
2018-05-23 02:36:19 -07:00
Chris Lu
468514f525
reduce logs
2018-05-23 02:36:06 -07:00
Chris Lu
873868cc10
not working now
...
need to add file handler
2018-05-22 04:31:44 -07:00
Chris Lu
7362de9a18
weed mount can work well
...
TODO: somehow filer url is returning empty content
2018-05-22 03:26:38 -07:00
Chris Lu
9dd228747c
filer copy added uid/gid
2018-05-21 01:25:30 -07:00
Chris Lu
f07482382b
able to update file content
...
having some issue when vi reports file changed.
2018-05-21 00:00:28 -07:00
Chris Lu
cd47528a75
remove unused code
2018-05-20 17:08:54 -07:00
Chris Lu
cc66e25cd2
merge intervals
2018-05-20 17:06:09 -07:00
Chris Lu
793dd81ca2
skip permission checking when creating dir or files
2018-05-19 12:40:24 -07:00
Chris Lu
ab4ddb1e0e
fix directory creation, directory listing
2018-05-19 12:07:15 -07:00
Chris Lu
b303a02461
cp file can work
...
1. consolidate to filer_pb.FileChunk
2. dir add file, mkdir
3. file flush, write
updates having issue
2018-05-16 00:08:44 -07:00
Chris Lu
58954bf46f
pass file attributes from filer to mount
2018-05-14 02:02:17 -07:00
Chris Lu
c5cf9bd290
properly working filer
2018-05-13 23:56:16 -07:00
Chris Lu
f01d5616b3
add better listing directory entries
2018-05-13 14:02:29 -07:00
Chris Lu
75a2b05aa3
remove Size from Attr
2018-05-13 00:11:26 -07:00
Chris Lu
fc7f81f6e7
remove nlink property from directory node
2018-05-12 22:42:28 -07:00
Chris Lu
9ecbf92569
more tests
2018-05-12 13:50:11 -07:00
Chris Lu
81af1bafba
file store adds memdb implementation
2018-05-12 13:45:29 -07:00
Chris Lu
a808df5019
visit parent folder first
2018-05-11 02:27:57 -07:00
Chris Lu
8a1d640dc4
starting filer2
2018-05-11 02:20:15 -07:00