Chris Lu
|
0b82edc0d2
|
filer: avoid stuck uploader
fix https://github.com/chrislusf/seaweedfs/issues/1980
reverting the file upload batch executor
|
2021-04-09 13:05:15 -07:00 |
|
Chris Lu
|
93f4146ffa
|
properly release the view cache
|
2021-04-09 12:36:39 -07:00 |
|
Merlin Gaillard
|
f952f979d1
|
filer: return 304 when If-Modified-Since == Last-Modified
|
2021-04-09 15:04:17 +02:00 |
|
Merlin Gaillard
|
4d4acc715e
|
s3api: handle 304 response code from filer
|
2021-04-09 12:13:19 +02:00 |
|
Chris Lu
|
6deb647a8f
|
mount: fix possible memory leak
if many files are read repeatedly, their metadata are accumulated in memory. This fix cleared the metadata after the file is read.
|
2021-04-08 19:47:31 -07:00 |
|
Konstantin Lebedev
|
ba175f81b5
|
add auth aws signV4
|
2021-04-08 17:40:47 +05:00 |
|
Konstantin Lebedev
|
995ae91007
|
add DeleteUserPolicy
|
2021-04-08 11:16:36 +05:00 |
|
Philippe Pepiot
|
42a761ee20
|
Fix typo in weed filer long help
|
2021-04-07 22:47:23 +02:00 |
|
Chris Lu
|
3be061994f
|
skip connection reset error
fix https://github.com/chrislusf/seaweedfs/issues/1971
this is because the connections are pooled but the volume server has reset the connection
|
2021-04-07 00:54:13 -07:00 |
|
Chris Lu
|
c5b08bac1b
|
remove mac specific mount options
|
2021-04-07 00:54:13 -07:00 |
|
Konstantin Lebedev
|
f5f8eec8e2
|
fix get filerGrpcAddress
|
2021-04-06 13:53:56 +05:00 |
|
Konstantin Lebedev
|
011e6e90ee
|
Merge branch 'upstreamMaster' into iamapipr
|
2021-04-06 13:50:33 +05:00 |
|
Konstantin Lebedev
|
ed79baa30f
|
add tests
|
2021-04-06 13:43:08 +05:00 |
|
Chris Lu
|
2327c0756b
|
fix to avoid loop
|
2021-04-05 23:24:26 -07:00 |
|
Chris Lu
|
a37eca78cd
|
2.38
|
2021-04-05 19:41:54 -07:00 |
|
Chris Lu
|
c131764c34
|
ensure error is sent back in channel
fix https://github.com/chrislusf/seaweedfs/issues/1966
avoid shared readErr variable
|
2021-04-05 19:40:12 -07:00 |
|
Chris Lu
|
0f64f5b9c8
|
mount: add readOnly option
fix https://github.com/chrislusf/seaweedfs/issues/1961
|
2021-04-04 21:40:58 -07:00 |
|
Chris Lu
|
6eee200c13
|
2.37
|
2021-04-04 18:45:48 -07:00 |
|
Chris Lu
|
fbb82a5c9c
|
skip limiting if limit is zero
|
2021-04-04 18:38:33 -07:00 |
|
Chris Lu
|
8251d1140e
|
refactor
|
2021-04-04 18:38:33 -07:00 |
|
Chris Lu
|
5dca98fe71
|
fix typo
|
2021-04-03 11:37:13 -07:00 |
|
Chris Lu
|
5e64f65632
|
ensure tables are created
fix https://github.com/chrislusf/seaweedfs/issues/1957
|
2021-04-02 12:02:26 -07:00 |
|
Chris Lu
|
319743d330
|
go fmt
|
2021-04-02 02:22:59 -07:00 |
|
Chris Lu
|
8e404a1433
|
go fmt
|
2021-04-02 02:22:26 -07:00 |
|
Chris Lu
|
bdf2ddddfd
|
revert to same implementation as before
This reverts commit 7e8edc3c4a .
|
2021-04-02 02:21:38 -07:00 |
|
Chris Lu
|
67e019d54b
|
add missing changes
|
2021-04-02 01:11:44 -07:00 |
|
Chris Lu
|
7e8edc3c4a
|
refactoring
|
2021-04-02 01:10:24 -07:00 |
|
Chris Lu
|
cc0df36a9e
|
smaller file chunks are faster
|
2021-04-01 02:21:59 -07:00 |
|
Chris Lu
|
cefe66f159
|
dedicated upload processor
avoid thundering effect of overloading volume servers
|
2021-04-01 02:21:40 -07:00 |
|
Chris Lu
|
1f984d2645
|
refactor buffer pool
|
2021-04-01 02:20:00 -07:00 |
|
Chris Lu
|
b5880334fc
|
refactor
|
2021-03-30 21:07:34 -07:00 |
|
Chris Lu
|
e79e2ddeed
|
remove unused MoveEvents
|
2021-03-30 20:42:44 -07:00 |
|
Chris Lu
|
e06807a55b
|
adjust logs
|
2021-03-30 20:36:06 -07:00 |
|
Chris Lu
|
ac875976c0
|
filer, volume: add concurrent upload size limit to avoid OOM
add some back pressure when writes are slow
|
2021-03-30 02:10:53 -07:00 |
|
Chris Lu
|
a1e18a1384
|
server add memory profiling
|
2021-03-30 02:10:53 -07:00 |
|
LazyDBA247-Anyvision
|
9385e2224d
|
fix postgres2 upsertQuery syntax also
|
2021-03-30 10:51:52 +03:00 |
|
Chris Lu
|
a5662c9e5e
|
adjust ordering
|
2021-03-30 00:26:57 -07:00 |
|
Chris Lu
|
bec74c3e6a
|
Merge pull request #1948 from LazyDBA247-Anyvision/master
add enableUpsert=true
|
2021-03-30 00:25:01 -07:00 |
|
LazyDBA247-Anyvision
|
9f1cab179c
|
use a compatible pg syntax for upsert
|
2021-03-30 10:21:27 +03:00 |
|
Chris Lu
|
a95929e53c
|
reduce default concurrentWriters to 32
|
2021-03-30 00:17:52 -07:00 |
|
LazyDBA247-Anyvision
|
7f44d953b5
|
fix GetBool
|
2021-03-30 01:36:02 +03:00 |
|
LazyDBA247-Anyvision
|
4c51e6a660
|
add enableUpsert=true
and rename config to upsertQuery
|
2021-03-30 00:32:03 +03:00 |
|
Chris Lu
|
b61e9f6a1a
|
adjust ordering
|
2021-03-29 00:49:50 -07:00 |
|
Chris Lu
|
1ebdbad222
|
Merge pull request #1946 from LazyDBA247-Anyvision/master
Adding Customisation - insertQuery for postgres/mysql
|
2021-03-29 00:42:48 -07:00 |
|
Konstantin Lebedev
|
8a95f9c10c
|
iam GetUser
|
2021-03-29 12:01:44 +05:00 |
|
LazyDBA247-Anyvision
|
96c62bd34d
|
scaffold - insertQuery
Example for upsert query for mysql & postgres variants
|
2021-03-29 10:01:12 +03:00 |
|
LazyDBA247-Anyvision
|
4a02389eb0
|
Adding custom insertQuery support for postgres/2 mysql/2
|
2021-03-29 09:58:13 +03:00 |
|
Chris Lu
|
6b7aa9633f
|
2.36
|
2021-03-28 19:09:06 -07:00 |
|
Chris Lu
|
bd7471d877
|
refactor
|
2021-03-25 12:05:59 -07:00 |
|
李海
|
06be5dc6c3
|
log snowflake sequencer nodeid's hex when start, it'll be in part of new assigned fid
|
2021-03-25 18:59:40 +08:00 |
|
李海
|
69b2dab9c6
|
add a snowflake sequencer as more robust fid generator, but less compressable than small auto-inc id
|
2021-03-25 18:49:26 +08:00 |
|
Konstantin Lebedev
|
40938d6a47
|
SaveInsideFiler S3 Configuration
|
2021-03-25 12:34:11 +05:00 |
|
Konstantin Lebedev
|
9f26f2815c
|
SaveAs S3 Configuration
|
2021-03-25 12:34:11 +05:00 |
|
Konstantin Lebedev
|
82b0463fac
|
handler PutUserPolicy
https://docs.aws.amazon.com/IAM/latest/APIReference/API_PutUserPolicy.html
|
2021-03-25 12:34:11 +05:00 |
|
Konstantin Lebedev
|
d7719d0542
|
base handlers
|
2021-03-25 12:34:11 +05:00 |
|
Konstantin Lebedev
|
03c7953254
|
init Iam Api Server
|
2021-03-25 12:34:11 +05:00 |
|
Chris Lu
|
82dfe06066
|
Merge pull request #1933 from kmlebedev/missingKeysThreshold
add missingKeysThreshold for fsck
|
2021-03-24 10:56:25 -07:00 |
|
Konstantin Lebedev
|
020a5d40c3
|
avoid counter is null
|
2021-03-24 22:51:22 +05:00 |
|
Konstantin Lebedev
|
df6cf0a2fa
|
nonRepairThreshold
|
2021-03-24 22:07:13 +05:00 |
|
Konstantin Lebedev
|
9ce82a9cd2
|
GOMAXPROCS use defaults to the value of runtime.NumCPU
https://github.com/chrislusf/seaweedfs/issues/1942
|
2021-03-24 18:45:58 +05:00 |
|
Konstantin Lebedev
|
60972f1c97
|
rename option repairThreshold
|
2021-03-24 13:24:49 +05:00 |
|
Chris Lu
|
a801332b0d
|
filer: return 409 if file conflicts with a directory
fix https://github.com/chrislusf/seaweedfs/issues/1938
|
2021-03-24 01:03:11 -07:00 |
|
Chris Lu
|
c42b95c596
|
more help message
fix https://github.com/chrislusf/seaweedfs/issues/1937
|
2021-03-23 17:27:57 -07:00 |
|
Konstantin Lebedev
|
40a9e88c07
|
rename var
|
2021-03-23 21:42:35 +05:00 |
|
Konstantin Lebedev
|
d21a5bf5d0
|
add missingKeysThreshold for fsck
|
2021-03-23 15:04:07 +05:00 |
|
Chris Lu
|
85cff10787
|
mysql: ensure name column is case sensitive
fix https://github.com/chrislusf/seaweedfs/issues/1931
|
2021-03-23 00:46:50 -07:00 |
|
Chris Lu
|
4abb511db3
|
make a local copy of the in memory cached data
|
2021-03-22 22:33:07 -07:00 |
|
Chris Lu
|
3cbc40fa48
|
avoid creating multiple reader
|
2021-03-22 22:32:47 -07:00 |
|
Chris Lu
|
2f7c7afdec
|
refactor: remove unused parameter
|
2021-03-22 22:13:19 -07:00 |
|
Chris Lu
|
d48dd0c738
|
use []byte directly instead of bytes.Buffer
|
2021-03-22 22:12:57 -07:00 |
|
Chris Lu
|
8eb14967ef
|
shell: volume.check.disk adds optional slow mode
fix https://github.com/chrislusf/seaweedfs/issues/1930
|
2021-03-22 21:01:43 -07:00 |
|
Chris Lu
|
288369cfc7
|
mount: release resources when Forget() is called
address https://github.com/chrislusf/seaweedfs/issues/1929
|
2021-03-22 16:22:19 -07:00 |
|
Chris Lu
|
5d931eff27
|
avoid possible nil
fix https://github.com/chrislusf/seaweedfs/issues/1928
The nil was because of `dn.Parent().UnlinkChildNode(dn.Id())` in topo.UnRegisterDataNode() function, when the dn leaves the cluster.
|
2021-03-22 13:24:07 -07:00 |
|
Chris Lu
|
f315eb2bb8
|
fix build error
|
2021-03-22 00:12:53 -07:00 |
|
Chris Lu
|
1dd5bc134c
|
2.35
|
2021-03-22 00:05:09 -07:00 |
|
Chris Lu
|
b465095db1
|
shell: add volume.check.disk to fix inconsistency for replicated volumes
fix https://github.com/chrislusf/seaweedfs/issues/1923
|
2021-03-22 00:03:16 -07:00 |
|
Chris Lu
|
df461402cc
|
ensure entry attribute is not nill
fix https://github.com/chrislusf/seaweedfs/issues/1926
|
2021-03-21 22:27:30 -07:00 |
|
Chris Lu
|
102a951377
|
refactor, split into 2 files
|
2021-03-21 13:05:53 -07:00 |
|
Chris Lu
|
f6e2566ee5
|
better logs for cookie mismatch
|
2021-03-20 12:02:57 -07:00 |
|
Chris Lu
|
27e24162f2
|
minor
|
2021-03-20 06:37:03 -07:00 |
|
Chris Lu
|
41cc6e661b
|
fs.mv ensures there are 2 arguments
|
2021-03-20 06:34:13 -07:00 |
|
Chris Lu
|
b2c1c209a5
|
refactor
|
2021-03-20 06:34:13 -07:00 |
|
Chris Lu
|
e9b1853d63
|
go fmt
|
2021-03-19 01:31:56 -07:00 |
|
Chris Lu
|
b1a86cf808
|
s3: copy object to itself
fix https://github.com/chrislusf/seaweedfs/issues/1922
|
2021-03-19 01:31:49 -07:00 |
|
Konstantin Lebedev
|
bf94149920
|
add line
|
2021-03-17 23:41:34 +05:00 |
|
Konstantin Lebedev
|
c5705e7a35
|
dir + slash key does not exist
https://github.com/chrislusf/seaweedfs/issues/1917
|
2021-03-17 23:40:42 +05:00 |
|
Konstantin Lebedev
|
eb54993a4e
|
Merge branch 'upstreamMaster' into check_chunkviews_mr
# Conflicts:
# weed/filer/filechunk_manifest.go
# weed/filer/stream.go
# weed/replication/repl_util/replication_util.go
# weed/util/fasthttp_util.go
|
2021-03-16 15:29:49 +05:00 |
|
Chris Lu
|
9672f9e1b2
|
2.34
|
2021-03-16 03:01:15 -07:00 |
|
Chris Lu
|
69694a17be
|
reverting 7d57664c2d
|
2021-03-16 02:59:26 -07:00 |
|
Chris Lu
|
c00dd5e62e
|
report error if entry not found
|
2021-03-16 02:55:49 -07:00 |
|
Konstantin Lebedev
|
06da02739d
|
CheckAllChunkViews() for HEAD requests only
|
2021-03-16 14:15:17 +05:00 |
|
Chris Lu
|
91a3ac9731
|
2.33
|
2021-03-16 00:36:06 -07:00 |
|
Chris Lu
|
4b1ed227d1
|
revert fasthttp changes
related to https://github.com/chrislusf/seaweedfs/issues/1907
|
2021-03-16 00:33:14 -07:00 |
|
Chris Lu
|
10164d0386
|
Merge pull request #1905 from wuh-fnst/s3ListCounterErr
s3: count correction for the number of files in the dir
|
2021-03-15 20:54:58 -07:00 |
|
Chris Lu
|
22a07a12d2
|
[mount] [regression] filer.path option results in empty mount in 2.32
fix https://github.com/chrislusf/seaweedfs/issues/1906
|
2021-03-15 20:46:25 -07:00 |
|
wuh-fnst
|
050a7f9599
|
s3: count correction for the number of files in the dir
|
2021-03-16 09:22:03 +08:00 |
|
Konstantin Lebedev
|
3a3699867b
|
Status PartialContent for Content-Range response
|
2021-03-15 23:30:22 +05:00 |
|
Konstantin Lebedev
|
90510e3137
|
If WriteHeader is not called explicitly, the first call to Write
will trigger an implicit WriteHeader(http.StatusOK).
WriteHeader: Only one header may be written. Go does not currently!
|
2021-03-15 21:22:59 +05:00 |
|
Konstantin Lebedev
|
46b9f5cff4
|
add debug logging
|
2021-03-15 19:33:33 +05:00 |
|
Konstantin Lebedev
|
7194a5e7bf
|
avoid http error: superfluous response.WriteHeader
https://github.com/chrislusf/seaweedfs/issues/1838
|
2021-03-15 18:52:59 +05:00 |
|
Chris Lu
|
9f00f95bfb
|
2.32
|
2021-03-14 21:32:00 -07:00 |
|
Chris Lu
|
a4cfffc264
|
shell: fix moving volume, volume server evacuate
fix https://github.com/chrislusf/seaweedfs/issues/1534
|
2021-03-14 21:29:55 -07:00 |
|
Chris Lu
|
ca100568f7
|
fix parsing
|
2021-03-14 20:50:14 -07:00 |
|
Chris Lu
|
352ba23f83
|
revert previous change
revert 29e62aba00
|
2021-03-14 20:49:56 -07:00 |
|
Chris Lu
|
29e62aba00
|
possible fix for volume balance
address https://github.com/chrislusf/seaweedfs/issues/1534
|
2021-03-14 20:23:19 -07:00 |
|
Chris Lu
|
35d939a1d2
|
go fmt
|
2021-03-14 13:21:02 -07:00 |
|
Chris Lu
|
2d4c2db81d
|
filer: leveldb, rocksdb auto create store directory
fix https://github.com/chrislusf/seaweedfs/issues/1901
|
2021-03-14 13:20:14 -07:00 |
|
Chris Lu
|
6d3a96eb56
|
filer: mysql2, postgres2 trigger actions on bucket creation and deletion
fix https://github.com/chrislusf/seaweedfs/issues/1877
|
2021-03-13 22:07:39 -08:00 |
|
Chris Lu
|
cb423312a4
|
prevent nil volume message
|
2021-03-13 11:05:29 -08:00 |
|
Chris Lu
|
972327f966
|
prevent nil volume nm
|
2021-03-13 11:04:51 -08:00 |
|
Chris Lu
|
2a68ddb963
|
default to empty host to bind to tcp4 and tcp6
|
2021-03-12 15:32:51 -08:00 |
|
Chris Lu
|
5fbcaaf37a
|
fullpath() test is not needed
|
2021-03-12 09:45:25 -08:00 |
|
Chris Lu
|
7d57664c2d
|
mount: internals switch to filer.Entry instead of protobuf
|
2021-03-12 00:36:38 -08:00 |
|
wuh-fnst
|
4c1d945e46
|
make List correctly judge whether it is the last file
|
2021-03-12 11:22:26 +08:00 |
|
Chris Lu
|
cca66c7fbe
|
print out the url
|
2021-03-11 18:39:45 -08:00 |
|
Chris Lu
|
7f887c78d2
|
fix
|
2021-03-11 18:18:16 -08:00 |
|
Chris Lu
|
fcc52d1e4e
|
shell: add s3.clean.uploads to clean up stale multipart uploads
fix https://github.com/chrislusf/seaweedfs/issues/1855
|
2021-03-11 18:05:24 -08:00 |
|
Chris Lu
|
40dc5ac904
|
mount: add a bit retry when connecting to filer during startup
|
2021-03-11 16:36:43 -08:00 |
|
Chris Lu
|
aba47fd9e8
|
reduce repeated calls to dir.FullPath()
|
2021-03-11 14:08:20 -08:00 |
|
Chris Lu
|
d084334ffd
|
Merge pull request #1883 from kmlebedev/passDelNonEmptyFolder
s3 delete-objects pass OK for fail to delete non-empty folder
|
2021-03-11 11:01:44 -08:00 |
|
Chris Lu
|
0e35836cc1
|
Merge pull request #1890 from kmlebedev/ZeroDataSize
error if read chunk zero data size
|
2021-03-11 10:51:48 -08:00 |
|
Konstantin Lebedev
|
58cdcc6d6e
|
error if urls not found
|
2021-03-11 23:34:36 +05:00 |
|
Chris Lu
|
79280e1513
|
escape special characters
fix https://github.com/chrislusf/seaweedfs/issues/1884
|
2021-03-11 09:49:40 -08:00 |
|
Konstantin Lebedev
|
4db21012c1
|
error if read chunk zero data size
|
2021-03-11 22:38:59 +05:00 |
|
wuheng
|
828fbf3fb7
|
s3: "isLast" returns true when the file does not exist
|
2021-03-11 15:20:50 +08:00 |
|
Chris Lu
|
5d446673a6
|
fix error printing
|
2021-03-10 17:13:04 -08:00 |
|
Chris Lu
|
ca546f47e1
|
s3: escape object key if containing special characters
fix https://github.com/chrislusf/seaweedfs/issues/1884
|
2021-03-10 13:19:28 -08:00 |
|
Chris Lu
|
105fcd7bfc
|
Merge pull request #1872 from kmlebedev/grpc_auth
TLS allowed CommonNames
|
2021-03-10 11:17:49 -08:00 |
|
Konstantin Lebedev
|
bb3af2d70b
|
no directoriesWithDeletion
|
2021-03-10 19:52:41 +05:00 |
|
Konstantin Lebedev
|
15b018da34
|
s3 delete-objects pass OK for fail to delete non-empty folder
|
2021-03-10 19:41:35 +05:00 |
|
Konstantin Lebedev
|
348e21a08c
|
add comments
|
2021-03-10 14:42:39 +05:00 |
|
Konstantin Lebedev
|
831953c55c
|
allowed wildcard domain
|
2021-03-10 14:02:13 +05:00 |
|
Konstantin Lebedev
|
4bf93d6e63
|
comma-separated
|
2021-03-10 12:43:13 +05:00 |
|
Konstantin Lebedev
|
0e02f7e258
|
comma-separated SSL certificate common names
|
2021-03-10 12:42:44 +05:00 |
|
Chris Lu
|
737bde5ab7
|
refactoring, remove duplicated code
|
2021-03-09 23:23:01 -08:00 |
|
Chris Lu
|
5ba4b479f8
|
properly lock file.entry object
fix https://github.com/chrislusf/seaweedfs/issues/1882
|
2021-03-09 23:08:38 -08:00 |
|
Chris Lu
|
5be83dd4a9
|
add comments
|
2021-03-09 14:13:48 -08:00 |
|
Chris Lu
|
9d8ca513d9
|
refactor
|
2021-03-09 13:21:26 -08:00 |
|
Chris Lu
|
f5b5e4da2d
|
2.31
|
2021-03-09 12:52:16 -08:00 |
|
Chris Lu
|
828f6e9f4d
|
volume: auto add missing vif files
fix https://github.com/chrislusf/seaweedfs/issues/1878
|
2021-03-09 12:09:32 -08:00 |
|
Chris Lu
|
387c6f4218
|
Revert "delete the folder object in multi_object_delete"
This reverts commit 2e89c8c9ae .
|
2021-03-09 10:07:27 -08:00 |
|
qieqieplus
|
5b16820924
|
fix: collection stats won't update if all volumes expired at same iteration
|
2021-03-09 18:49:45 +08:00 |
|
Konstantin Lebedev
|
190fada1ef
|
TLS allowed commonNames
|
2021-03-08 21:39:44 +05:00 |
|
Konstantin Lebedev
|
c6d3735605
|
permitCommonNames
https://github.com/chrislusf/seaweedfs/issues/1841
https://jbrandhorst.com/post/grpc-auth/
|
2021-03-08 13:16:17 +05:00 |
|
Chris Lu
|
d888e9a90f
|
2.30
|
2021-03-07 15:01:19 -08:00 |
|
Chris Lu
|
60686a0bb7
|
make tcp optional
|
2021-03-07 14:45:36 -08:00 |
|
Chris Lu
|
3b2a9c98ef
|
fix nil
|
2021-03-07 11:45:35 -08:00 |
|
Chris Lu
|
df72dc206d
|
stats collect chunk upload retry count
|
2021-03-07 11:26:15 -08:00 |
|
Chris Lu
|
726edab054
|
avoid nil when closing an index
fix https://github.com/chrislusf/seaweedfs/issues/1870
|
2021-03-07 11:03:09 -08:00 |
|
Chris Lu
|
bdfed16d42
|
avoid nil exception
fix https://github.com/chrislusf/seaweedfs/issues/1869
|
2021-03-07 01:49:06 -08:00 |
|