chrislu
|
303bd067b5
|
Revert "rename: delete source entry metadata only, skipping hard links"
This reverts commit 03466f955e .
fix https://github.com/seaweedfs/seaweedfs/issues/3386
|
2022-07-31 22:51:41 -07:00 |
|
chrislu
|
26dbc6c905
|
move to https://github.com/seaweedfs/seaweedfs
|
2022-07-29 00:17:28 -07:00 |
|
chrislu
|
74f60f246f
|
dynamically connect to a filer
|
2022-07-28 23:24:38 -07:00 |
|
chrislu
|
68065128b8
|
add dc and rack
|
2022-07-28 23:22:51 -07:00 |
|
zzq09494
|
01b7aa79fa
|
fix: Buckets are not created and deleted correctly on the filer with the same signature when they are created and deleted
|
2022-07-28 12:04:37 +08:00 |
|
Konstantin Lebedev
|
da9d3e8f6c
|
refactor
|
2022-07-26 11:56:45 +05:00 |
|
Konstantin Lebedev
|
046c3d5ad4
|
fix logic else brake
|
2022-07-26 11:47:11 +05:00 |
|
chrislu
|
ec0edb1ac4
|
filer: fix wrong logic during read
|
2022-07-25 22:40:00 -07:00 |
|
chrislu
|
64f3d6fb6e
|
metadata subscription uses client epoch
|
2022-07-23 10:50:28 -07:00 |
|
chrislu
|
fc8241fb5e
|
leveldb3: add instant dropping bucket
|
2022-07-21 18:48:51 -07:00 |
|
chrislu
|
c93f7ffa44
|
explicit bucket aware declaration
|
2022-07-21 18:23:53 -07:00 |
|
Konstantin Lebedev
|
b6471ecd75
|
err msg with duplicated local subscription detected move to log level 1
https://github.com/chrislusf/seaweedfs/issues/3320
|
2022-07-20 18:40:50 +05:00 |
|
Konstantin Lebedev
|
11e393dbe7
|
err msg with duplicated local subscription detected move to log level 1
https://github.com/chrislusf/seaweedfs/issues/3320
|
2022-07-20 00:45:13 +05:00 |
|
Konstantin Lebedev
|
7b1497ee63
|
Use BackoffSchedule for getLookupFileId
|
2022-07-15 16:05:35 +05:00 |
|
Konstantin Lebedev
|
01996bccf8
|
Use fallback if urls are not found
|
2022-07-15 15:29:15 +05:00 |
|
chrislu
|
1db012485f
|
mount: fix bug during busy writes
fix https://github.com/chrislusf/seaweedfs/issues/3315
|
2022-07-15 01:03:17 -07:00 |
|
chrislu
|
aca20cd9f4
|
more logging related to filer metadata subscription
|
2022-07-15 00:42:13 -07:00 |
|
chrislu
|
fa61074513
|
add clientId logging
|
2022-07-14 12:27:34 -07:00 |
|
Chris Lu
|
1a130125de
|
Merge pull request #3308 from guo-sj/check_duplicate_ak
|
2022-07-13 09:23:37 -07:00 |
|
guosj
|
354b7bdff0
|
replace errors.New(fmt.Sprintf(...)) with fmt.Errorf()
|
2022-07-13 22:49:03 +08:00 |
|
guosj
|
ab1b9697e6
|
supplement check duplicate accesskey
|
2022-07-13 17:28:20 +08:00 |
|
chrislu
|
6e90f7bdd0
|
detect sequential mode with a counter
|
2022-07-13 02:20:03 -07:00 |
|
chrislu
|
1451b389a4
|
Avoid fatal error: sync: Unlock of unlocked RWMutex
fix https://github.com/chrislusf/seaweedfs/issues/3306
|
2022-07-13 00:58:15 -07:00 |
|
chrislu
|
860c207c94
|
typo
|
2022-07-12 02:56:19 -07:00 |
|
guosj
|
5469efa336
|
close SkipCheckParentDirectory flag
|
2022-07-11 09:44:48 +08:00 |
|
chrislu
|
185332b91f
|
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
|
2022-07-08 00:29:41 -07:00 |
|
chrislu
|
28add5a534
|
mount: fix racing conditions
prevent wrong reading when the SingleChunkCacher is started, but not finished yet
|
2022-07-08 00:29:39 -07:00 |
|
duanhongyi
|
1ceab96aba
|
filer tikv support tls
|
2022-07-08 14:23:06 +08:00 |
|
chrislu
|
a85ed3fe8f
|
minor
|
2022-07-07 22:27:24 -07:00 |
|
chrislu
|
2ea18cdcc8
|
remove dead code
|
2022-07-07 22:26:03 -07:00 |
|
chrislu
|
f2f0482dd3
|
mount: random read also try to use the local cache first
|
2022-07-07 11:50:28 -07:00 |
|
石昌林
|
2b5b37231b
|
Make sure that onChangeEvent is called successfully before updating lastTsNs
|
2022-06-29 23:57:07 +08:00 |
|
石昌林
|
e17e6ed542
|
Avoid synchronizing from scratch due to err
|
2022-06-29 23:31:04 +08:00 |
|
yulai.li
|
824cbe32be
|
Make tikv filer enable/disable by build tags
|
2022-06-27 10:57:24 +08:00 |
|
yulai.li
|
46e0b629e5
|
Update tikv client version and add one PC support
|
2022-06-26 22:43:37 +08:00 |
|
LHHDZ
|
8db9f13bc6
|
Merge branch 'master' into circuit_breaker
|
2022-06-20 13:42:18 +08:00 |
|
geekboood
|
fdacd94af5
|
fix: invalid chunk data when failed to read manifests
|
2022-06-19 16:54:04 +08:00 |
|
石昌林
|
78b3728169
|
add s3 circuit breaker support for 'simultaneous request count' and 'simultaneous request bytes' limitations
configure s3 circuit breaker by 'command_s3_circuitbreaker.go':
usage eg:
# Configure the number of simultaneous global (current s3api node) requests
s3.circuit.breaker -global -type count -actions Write -values 1000 -apply
# Configure the number of simultaneous requests for bucket x read and write
s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply
# Configure the total bytes of simultaneous requests for bucket write
s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply
# Disable circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -enable false -apply
# Delete circuit breaker config of bucket 'x'
s3.circuit.breaker -buckets x -delete -apply
|
2022-06-15 21:07:55 +08:00 |
|
chrislu
|
ebd6097e6f
|
fix: sequence read rate is slow when use filer.ChunkReadAt.ReadAt #3074
related to https://github.com/chrislusf/seaweedfs/pull/3074
|
2022-06-07 22:33:35 -07:00 |
|
chrislu
|
4fd5f96598
|
filer: remove replication, collection, disk_type info from entry metadata
these metadata can change and are not used
|
2022-06-06 00:39:35 -07:00 |
|
creeew
|
02ae102731
|
fix filer.sync missing source srv uploaded files to target when target down
|
2022-06-02 01:28:47 +08:00 |
|
chrislu
|
6adc42147f
|
fresh filer store bootstrap from the oldest peer
|
2022-05-30 21:27:48 -07:00 |
|
chrislu
|
c59068d0f3
|
refactor
|
2022-05-30 16:28:36 -07:00 |
|
chrislu
|
682382648e
|
collect cluster node start time
|
2022-05-30 16:23:52 -07:00 |
|
chrislu
|
aece35a64f
|
stop when on disk log is done
|
2022-05-30 15:20:51 -07:00 |
|
chrislu
|
a2b101a737
|
subscribe metadata between a range
|
2022-05-30 15:04:19 -07:00 |
|
chrislu
|
9219664505
|
reset sync offset if peer filer resets
fix https://github.com/chrislusf/seaweedfs/issues/3089
|
2022-05-29 13:11:24 -07:00 |
|
Kaiwalya Joshi
|
4598581741
|
fix: Minor spelling fixes on FilerStorePathTranslator.
|
2022-05-25 01:11:02 -07:00 |
|
chrislu
|
a5cdcc7e63
|
fix reading files larger than 20GB due to manifest resolving bug
|
2022-05-23 01:16:10 -07:00 |
|
chrislu
|
59cc3c0bfd
|
fix reading files larger than 20GB due to manifest resolving bug
this is introduced since Mar 08 2022
|
2022-05-23 01:14:56 -07:00 |
|
Konstantin Lebedev
|
d12a423aa0
|
refactor
add some tests
https://github.com/chrislusf/seaweedfs/pull/2996
|
2022-05-12 12:23:02 +05:00 |
|
Konstantin Lebedev
|
f523aed3c9
|
fixes from comments
https://github.com/chrislusf/seaweedfs/pull/2996
|
2022-05-11 21:50:09 +05:00 |
|
chrislu
|
76da688176
|
adjust log level
|
2022-05-11 00:45:19 -07:00 |
|
chrislu
|
a85720f0f6
|
remove deprecated function
|
2022-05-10 00:02:23 -07:00 |
|
chrislu
|
791c29ae93
|
remove deprecated function
WithQueryCachePolicyKeepInCache is deprecated and already default behavior
|
2022-05-10 00:01:43 -07:00 |
|
chrislu
|
10207a48a4
|
how to setup local test env
|
2022-05-06 02:25:56 -07:00 |
|
chrislu
|
db33b03735
|
add comments
|
2022-05-06 02:19:47 -07:00 |
|
chrislu
|
e350ba8896
|
larger test range
|
2022-05-06 02:17:28 -07:00 |
|
chrislu
|
71e0bcc5d5
|
a bit optimization
|
2022-05-06 02:11:58 -07:00 |
|
chrislu
|
b2a43dd044
|
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
|
2022-05-06 01:51:28 -07:00 |
|
chrislu
|
d5fe5d12b7
|
filer: etcd fix pagination
fix https://github.com/chrislusf/seaweedfs/issues/3016
|
2022-05-06 01:51:19 -07:00 |
|
Konstantin Lebedev
|
e6d2cb5911
|
ydb add empty doc file
|
2022-05-05 11:12:14 +05:00 |
|
Konstantin Lebedev
|
f58adaab25
|
build tag ydb since the growth of the binary is 5 mb
|
2022-05-03 23:20:09 +05:00 |
|
Konstantin Lebedev
|
f127b326bf
|
add options to scaffold
|
2022-05-03 22:54:31 +05:00 |
|
Konstantin Lebedev
|
7b544576af
|
refactor
|
2022-05-03 20:16:00 +05:00 |
|
Konstantin Lebedev
|
04d5dff6bb
|
add support native ttl
|
2022-05-03 17:52:23 +05:00 |
|
Konstantin Lebedev
|
cb3c7a3cdb
|
enable query cache policy instead of prepare
|
2022-05-03 16:03:10 +05:00 |
|
Konstantin Lebedev
|
8342f651f3
|
fix scanNamed
|
2022-05-03 15:18:28 +05:00 |
|
Konstantin Lebedev
|
d8925b4e83
|
Merge branch 'new_master' into ydb
# Conflicts:
# go.mod
# go.sum
|
2022-05-03 00:13:57 +05:00 |
|
Konstantin Lebedev
|
7640e650e5
|
fix queries
|
2022-05-03 00:11:37 +05:00 |
|
Konstantin Lebedev
|
319d300d48
|
fix createDB
|
2022-05-02 22:23:07 +05:00 |
|
Konstantin Lebedev
|
0dc44dda63
|
ydb do Tx or DB
|
2022-05-02 15:33:29 +05:00 |
|
Konstantin Lebedev
|
a3e48831f3
|
ydb BucketAware interface
|
2022-05-02 12:42:20 +05:00 |
|
chrislu
|
94635e9b5c
|
filer: add filer group
|
2022-05-01 21:59:16 -07:00 |
|
Konstantin Lebedev
|
50c4f62ed4
|
ydb-go-sdk move to v3
|
2022-05-02 02:07:47 +05:00 |
|
Konstantin Lebedev
|
21033ff4c3
|
refactor use const CountEntryChunksForGzip
|
2022-05-01 22:28:55 +05:00 |
|
Konstantin Lebedev
|
ec0ed41e37
|
ydb kv interface
|
2022-05-01 21:20:37 +05:00 |
|
Konstantin Lebedev
|
6a052f6ff2
|
ydb Sql interface
|
2022-05-01 20:33:03 +05:00 |
|
Konstantin Lebedev
|
275c3bb19c
|
ydb initial
https://github.com/chrislusf/seaweedfs/issues/2942
|
2022-05-01 17:28:58 +05:00 |
|
chrislu
|
4aa39ef33f
|
conditionally compile sqlite
|
2022-04-21 01:22:38 -07:00 |
|
chrislu
|
3885374edf
|
conditionally build elastic, gocdk to reduce binary size
|
2022-04-21 01:10:46 -07:00 |
|
chrislu
|
cb1e7aa27d
|
fix build
|
2022-04-20 16:21:56 -07:00 |
|
a
|
1d6a9e66b6
|
Merge branch 'master' into a
|
2022-04-20 14:01:42 -07:00 |
|
justin
|
3551ca2fcf
|
enhancement: replace sort.Slice with slices.SortFunc to reduce reflection
|
2022-04-18 10:35:43 +08:00 |
|
eddy-gfx
|
ec53eec94f
|
Merge branch 'master' into a
|
2022-04-06 18:45:55 -05:00 |
|
chrislu
|
bb4beebce3
|
prevent nil
|
2022-04-05 10:49:17 -07:00 |
|
a
|
7e92517571
|
change user and pass to username and password
|
2022-04-01 14:09:25 -05:00 |
|
a
|
bc603e534f
|
diff
|
2022-03-28 16:50:28 +00:00 |
|
chrislu
|
86d4c52d6e
|
Merge branch 'master' of https://github.com/chrislusf/seaweedfs
|
2022-03-26 13:33:20 -07:00 |
|
chrislu
|
21e0898631
|
refactor: change masters from a slice to a map
|
2022-03-26 13:33:17 -07:00 |
|
Chris Lu
|
3153c836a7
|
Merge pull request #2744 from kmlebedev/issues_2545
avoid connect to the old filler address
|
2022-03-26 13:02:49 -07:00 |
|
chrislu
|
cae998eda1
|
Revert "Revert "remove duplicated metadata subscription in filer""
This reverts commit 34b743c481 .
|
2022-03-26 13:00:16 -07:00 |
|
chrislu
|
34b743c481
|
Revert "remove duplicated metadata subscription in filer"
This reverts commit 34742be029 .
Related to https://github.com/chrislusf/seaweedfs/issues/2545
|
2022-03-26 12:33:45 -07:00 |
|
chrislu
|
4e2388e1b5
|
mount: set file size if it is only on remote gateway
|
2022-03-21 02:09:54 -07:00 |
|
elee
|
beb406bbbb
|
fix ls, onBucketDelete still not triggering
|
2022-03-18 22:21:57 -05:00 |
|
elee
|
411c0df3fe
|
switch to multi collection, change readme
|
2022-03-18 21:51:16 -05:00 |
|
elee
|
1cea6c73d3
|
update readme
|
2022-03-18 00:34:19 -05:00 |
|
elee
|
25be96832a
|
update readme with index info
|
2022-03-18 00:33:08 -05:00 |
|
elee
|
06f23aa675
|
put in delete folder children query
|
2022-03-18 00:29:48 -05:00 |
|
elee
|
0701feeb17
|
added more to readme
|
2022-03-18 00:26:11 -05:00 |
|
elee
|
171c27ddf1
|
zz
|
2022-03-17 22:35:30 -05:00 |
|
elee
|
2f0cdcdceb
|
update readme
|
2022-03-17 21:26:16 -05:00 |
|
elee
|
423ce57cde
|
prefix search, bucket implemented
|
2022-03-17 21:12:25 -05:00 |
|
elee
|
bf745bdccb
|
revise comment
|
2022-03-17 14:09:34 -05:00 |
|
elee
|
a0df993cef
|
cleanup, add a few comments
|
2022-03-17 14:08:36 -05:00 |
|
elee
|
94883e2fad
|
ok then
|
2022-03-17 05:23:12 -05:00 |
|
elee
|
f3ab6769e9
|
make start from indexing work
|
2022-03-17 05:09:13 -05:00 |
|
elee
|
31571dd96e
|
fix file listing
|
2022-03-17 05:07:40 -05:00 |
|
elee
|
b25d03340b
|
Merge branch 'a' of github.com:gfxlabs/seaweedfs into a
|
2022-03-17 04:51:59 -05:00 |
|
elee
|
921535001a
|
arangodb adapter
|
2022-03-17 04:49:26 -05:00 |
|
chrislu
|
6886167d5f
|
minor optimization
|
2022-03-17 00:03:28 -07:00 |
|
chrislu
|
4042fdf3bb
|
rename to skipCheckParentDir
related to https://github.com/chrislusf/seaweedfs/pull/2761
It's better to default to false.
|
2022-03-16 23:55:31 -07:00 |
|
zzq09494
|
40b0033fa7
|
go fmt
|
2022-03-17 14:19:48 +08:00 |
|
zzq09494
|
ee5417a5b3
|
fix: ut test error
|
2022-03-17 11:16:46 +08:00 |
|
zzq09494
|
81cce4b4c3
|
filer: support uploading file without needEnsureParentDir
|
2022-03-17 10:53:47 +08:00 |
|
zzq09494
|
a6a8892255
|
Revert "filer: support uploading file without needEnsureParentDir"
This reverts commit a93c4947ba .
|
2022-03-17 10:27:17 +08:00 |
|
zzq09494
|
a93c4947ba
|
filer: support uploading file without needEnsureParentDir
|
2022-03-17 10:18:23 +08:00 |
|
chrislu
|
fbc9f0eb64
|
minor
|
2022-03-14 03:19:16 -07:00 |
|
chrislu
|
2f4cd20f90
|
tests skip reader pattern monitoring
|
2022-03-13 18:15:53 -07:00 |
|
chrislu
|
53513475bf
|
mount: add back random read support
avoid too much memory used also
|
2022-03-13 01:38:52 -08:00 |
|
Konstantin Lebedev
|
834210a9dc
|
avoid connect to the old filler address
https://github.com/chrislusf/seaweedfs/issues/2545
|
2022-03-10 15:24:45 +05:00 |
|
zzq09494
|
2ea18fca48
|
fix:support some databases which not support 'IF NOT EXISTS' keyword of 'create table command'
|
2022-03-10 13:37:34 +08:00 |
|
banjiaojuhao
|
f7f2a597dd
|
minor
|
2022-03-08 16:22:55 +08:00 |
|
banjiaojuhao
|
d61bea9038
|
[bugfix] filer: In file modification, old chunks will be mis-deleted when they are merged(Manifestized).
|
2022-03-08 16:22:55 +08:00 |
|
chrislu
|
6f3ec989a7
|
fix manifest batch size
|
2022-03-07 00:07:53 -08:00 |
|
banjiaojuhao
|
a07c93f4dd
|
filer: reset bytesBuffer before use
|
2022-03-06 21:07:36 +08:00 |
|
chrislu
|
da76af187f
|
mount: avoid possible index out of bounds error
|
2022-03-04 22:36:01 -08:00 |
|
chrislu
|
6fbbc78574
|
stream reading a whole chunk
|
2022-03-02 13:50:46 -08:00 |
|
chrislu
|
941ced60a4
|
download 2 chunks if at the beginning of a file
|
2022-02-27 03:57:24 -08:00 |
|
chrislu
|
551d00d51a
|
prefetch other chunks when stream reading
|
2022-02-26 23:20:45 -08:00 |
|
chrislu
|
7b1a713d2a
|
remove dead code
|
2022-02-26 03:23:15 -08:00 |
|
chrislu
|
3a58b7bac6
|
a little safer
|
2022-02-26 03:23:06 -08:00 |
|
chrislu
|
3345a50d9b
|
prefetch 2 chunks
|
2022-02-26 03:06:17 -08:00 |
|
chrislu
|
86ce69f709
|
remove logs
|
2022-02-26 03:00:20 -08:00 |
|
chrislu
|
28b395bef4
|
better control for reader caching
|
2022-02-26 02:16:47 -08:00 |
|
chrislu
|
72c0233938
|
less logs
|
2022-02-25 15:34:24 -08:00 |
|
chrislu
|
03466f955e
|
rename: delete source entry metadata only, skipping hard links
|
2022-02-25 02:57:54 -08:00 |
|
chrislu
|
8080fe4cc1
|
logs
|
2022-02-25 02:56:23 -08:00 |
|
chrislu
|
202a29d014
|
refactoring
|
2022-02-25 01:17:26 -08:00 |
|
chrislu
|
be3fc77391
|
mount2: use consistent inode
|
2022-02-25 00:53:27 -08:00 |
|
chrislu
|
ceaf993a27
|
mount2: add rdev
|
2022-02-24 14:51:25 -08:00 |
|
chrislu
|
b05962b90e
|
rename: handle hard links
|
2022-02-24 02:59:00 -08:00 |
|
chrislu
|
56da3494cb
|
leveldb3: adjust memory allocation for each bucket
|
2022-02-20 13:40:13 -08:00 |
|
chrislu
|
7bc67399e4
|
listing for filer stores without prefixed query: break if no more progress
|
2022-02-19 10:12:51 -08:00 |
|
chrislu
|
248c0c8087
|
minor
|
2022-02-19 09:54:09 -08:00 |
|
chrislu
|
4ee0a6f47b
|
filer store: reduce one possible listing operation
|
2022-02-19 00:43:42 -08:00 |
|
chrislu
|
b3594278c9
|
optimize a bit
|
2022-02-18 23:36:10 -08:00 |
|
chrislu
|
df51e0c042
|
mongodb: remove ErrNoDocuments checking since it is only for creating single record, not for querying
|
2022-02-15 16:18:36 -08:00 |
|
chrislu
|
98cce6a150
|
mongodb: ensure closing cursor
|
2022-02-15 16:15:44 -08:00 |
|
banjiaojuhao
|
b5ec346700
|
FilerStore: add redis_lua
|
2022-02-15 20:54:57 +08:00 |
|