Chris Lu
bbc8ae110f
report errors
2020-07-14 11:27:00 -07:00
Chris Lu
17f2d05cde
filer: ensure logs are persisted
2020-07-14 11:25:50 -07:00
Chris Lu
ddec7b2bb9
go fmt
2020-07-14 07:34:16 -07:00
Chris Lu
4f6096c7f0
add reading from persisted logs for local filer store
2020-07-13 22:55:28 -07:00
Chris Lu
87b5031714
change variable visibility
2020-07-13 08:19:48 -07:00
Chris Lu
7597831cac
filer: leveldb2 supports peers also
2020-07-13 00:05:40 -07:00
Chris Lu
5ba894bb72
leveldb2: support subscribing from peers
2020-07-13 00:05:40 -07:00
Chris Lu
5a4f258a8c
1.85
2020-07-12 17:34:09 -07:00
Chris Lu
4eecc6abc6
filer: emit metadata notification correctly for batch deletion
2020-07-12 17:32:19 -07:00
Chris Lu
1dd2c76532
refactoring
2020-07-12 17:31:24 -07:00
Chris Lu
224103e13b
aggregated logs has empty flushFn
2020-07-11 09:12:03 -07:00
Chris Lu
bfabb4159b
refactoring
2020-07-11 09:11:15 -07:00
Chris Lu
f2e8ad6241
refactoring
2020-07-11 06:35:54 -07:00
Chris Lu
2d1df97933
mount: fatal error if can not process meta data
2020-07-11 06:16:48 -07:00
Chris Lu
e3e00d36c5
reset cache for each new release
2020-07-11 06:16:17 -07:00
Chris Lu
7ad0bce2f5
remove old code
2020-07-10 23:03:22 -07:00
Chris Lu
e4ef3814e7
fix compilation
2020-07-10 10:02:59 -07:00
Chris Lu
e7c04af1d0
skip http.StatusPartialContent in case of error and superfluous response.WriteHeader
2020-07-10 08:56:23 -07:00
李远军
8c318470dd
Revert "remove fixJpgOrientation"
...
This reverts commit de5ca9b2
minimise the revert changes
2020-07-10 10:28:28 +08:00
李远军
6608cb5f43
Revert "remove fixJpgOrientation"
...
This reverts commit de5ca9b2
2020-07-10 10:08:36 +08:00
cheng.li01
dad1161c70
fix dn.volumes Iterate when write issue
2020-07-08 19:57:19 +08:00
Chris Lu
54d6b3c30f
filer: support running multiple mount to different filers avoiding duplicated cache
...
fix https://github.com/chrislusf/seaweedfs/issues/1382
2020-07-07 23:18:44 -07:00
Chris Lu
4bed1663e2
adjust log level
2020-07-07 23:17:31 -07:00
Chris Lu
482d13468b
remove println
2020-07-07 23:17:17 -07:00
Chris Lu
a2eb680f34
filer: default filer store directory
...
1. set default filer store directory
2. set peers, avoiding empty string counted as 1.
2020-07-07 23:06:48 -07:00
Chris Lu
3c269da37f
fix compilation
2020-07-06 09:46:24 -07:00
Chris Lu
d198e8c6d6
filer: support cross filer meta data sync if sharing the same store
2020-07-06 09:24:54 -07:00
Chris Lu
0b8cc9b626
aggregate multiple filer metadata chagne events
2020-07-05 23:05:02 -07:00
Chris Lu
49929e0869
rename
2020-07-05 15:52:36 -07:00
Chris Lu
70d8a3a1d3
add SubscribeLocalMetadata without checking persisted meta logs
2020-07-05 15:50:07 -07:00
Chris Lu
55e40b08fc
refactoring
2020-07-05 15:43:06 -07:00
Chris Lu
881e0fde2e
rename
2020-07-05 08:55:09 -07:00
Chris Lu
353bea8ddb
refactoring
2020-07-03 22:25:35 -07:00
Chris Lu
ce3630e7a4
refactoring
2020-07-03 22:16:59 -07:00
Chris Lu
59152334c7
default to 1%
2020-07-03 16:42:53 -07:00
Chris Lu
acf4a9ea99
volume: min free space default to 1%
2020-07-03 16:42:02 -07:00
Chris Lu
4d1484628a
refactoring
2020-07-03 16:41:30 -07:00
Chris Lu
faa5c2e89a
refactoring
2020-07-03 16:34:31 -07:00
Chris Lu
37d5b3ba12
replication: pass isFromOtherCluster also to EventNotification
...
EventNotification is consistent with message queue and metadata logs.
2020-07-01 08:06:20 -07:00
Chris Lu
31e23e9783
filer: support active<=>active filer replication
2020-06-30 22:53:57 -07:00
Chris Lu
df75b5c98d
passing ctx for metadata logging
2020-06-28 14:34:51 -07:00
Chris Lu
12f6b683fc
default to 8 volumes
...
in case all the volumes are allocated for a collection
2020-06-28 14:34:21 -07:00
Chris Lu
42a338d7b0
FUSE mount: clean up file handles during renaming
...
resolve dir rename when file is still open. Need to clean the file handles as soon as possible.
These can happen out of order:
file rename, then file release
file release, then file rename
2020-06-28 13:41:00 -07:00
Chris Lu
95a1860d65
remove empty calls for list cache
2020-06-28 10:25:54 -07:00
Chris Lu
6498d0c86b
remove list cache
2020-06-28 10:21:52 -07:00
Chris Lu
9033a7d369
removed async option
2020-06-28 10:18:32 -07:00
Chris Lu
1bb8cae65d
reverting and working
2020-06-28 10:14:17 -07:00
Chris Lu
b813fac4a3
1.84
2020-06-28 07:28:21 -07:00
Chris Lu
20d9cfae86
volume: following md5 base64 encoding
...
fix https://github.com/chrislusf/seaweedfs/issues/1374
2020-06-28 07:25:50 -07:00
Chris Lu
f0f884edd5
1.83
2020-06-27 15:46:59 -07:00
Chris Lu
c2797cc6de
add logs
2020-06-27 15:46:53 -07:00
Chris Lu
62260e526b
add logging
2020-06-27 12:54:37 -07:00
Chris Lu
bef356ce4c
since we already know the chunk size, no need to iterate
2020-06-27 12:51:04 -07:00
Chris Lu
a808b3b5df
incase the memory data is too small
2020-06-27 11:59:15 -07:00
Chris Lu
3dbd51c3c2
a little bit more efficient
2020-06-26 10:02:37 -07:00
Chris Lu
212b6e7d42
error logging
2020-06-26 10:01:55 -07:00
Chris Lu
48b23f2fdd
FUSE mount: prevent concurrent modification
2020-06-26 10:00:48 -07:00
Chris Lu
3cec4b3c49
FUSE mount: fix file id written twice?
...
fix https://github.com/chrislusf/seaweedfs/issues/1373
2020-06-25 20:11:49 -07:00
Chris Lu
212c64241b
checking ecx file
2020-06-25 16:36:06 -07:00
Chris Lu
a4dc25cd22
volume: generate ec shard first, before generating the ec index file.
...
possible fix https://github.com/chrislusf/seaweedfs/issues/1364
the ec index file is used to determine whether the ec files are ready or not
2020-06-25 11:06:58 -07:00
Chris Lu
9fd7cdadf1
fix
2020-06-25 10:45:34 -07:00
Chris Lu
3b638d3994
add more ec encoding logging
2020-06-25 09:43:38 -07:00
Chris Lu
e8e0d629af
ensure correct file mode
2020-06-24 23:26:54 -07:00
Chris Lu
32a1f22730
more logs
2020-06-24 22:07:53 -07:00
Chris Lu
5be12eea37
zstd fix
2020-06-24 11:39:12 -07:00
Chris Lu
2ff37ccdbd
testing parsing the upload
...
The mime type is always the value passed in.
Compress or not depends on the content detection, file name extension, and compression ratio.
If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu
c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation
...
Optional md5 validation of uploads
2020-06-24 11:33:44 -07:00
Kimbsen
4eeab2a379
remove unused imports
2020-06-24 13:37:00 +02:00
Kimbsen
ffddecebef
do md5 validation AFTER decompression
2020-06-24 13:35:13 +02:00
Chris Lu
fe60db404a
prepare for zstd
2020-06-23 09:12:02 -07:00
Kimbsen
6b1e93ba0b
check request header not part header for content-encoding
2020-06-23 17:04:24 +02:00
Kimbsen
38626cb584
don't forget the last few bytes
2020-06-23 16:53:49 +02:00
Kimbsen
8d1f6cd567
added optional md5 verification of uploaded data
2020-06-23 16:24:11 +02:00
Chris Lu
a4e3cffe0a
S3: DeleteObject API also support deleting a folder recursively
2020-06-22 10:01:00 -07:00
Chris Lu
98493e0152
volume: add default minFreeSpacePercent
...
fix https://github.com/chrislusf/seaweedfs/issues/1366
2020-06-21 08:44:06 -07:00
Chris Lu
e7c36f740a
refactoring
2020-06-21 08:38:00 -07:00
Chris Lu
8b4a32e782
adjust help message
2020-06-21 08:34:24 -07:00
Chris Lu
b4ec04d464
fix the help message
2020-06-21 08:31:20 -07:00
Chris Lu
de5ca9b258
remove fixJpgOrientation
2020-06-20 12:50:40 -07:00
Chris Lu
3080c197e3
rename UnCompressData to DecompressData
2020-06-20 08:16:16 -07:00
Chris Lu
97239ce6f1
rename filechunk is_gzipped to is_compressed
2020-06-20 08:15:49 -07:00
Chris Lu
a735fd767c
adjust uncompress data
2020-06-20 08:01:00 -07:00
Chris Lu
ca3516ac6d
adjust protoc
2020-06-20 08:00:25 -07:00
Chris Lu
e912fd15e3
renaming
2020-06-19 22:45:27 -07:00
Chris Lu
16fe132a20
detect mime type before replicating to other volume servers
2020-06-19 22:11:36 -07:00
Chris Lu
044841c885
master: always clear previous master meta data directory
2020-06-19 20:42:16 -07:00
Chris Lu
2eeb7551f9
less noisy disk size checking
2020-06-19 15:06:33 -07:00
Chris Lu
c48b407053
1.82
2020-06-19 13:47:27 -07:00
Chris Lu
b78a2dc59b
adjust help message
2020-06-19 12:55:09 -07:00
Chris Lu
f7a45d448f
FUSE mount: lazy loading meta cache
2020-06-19 09:45:42 -07:00
Chris Lu
c0283eee1a
more test cases
2020-06-19 09:45:42 -07:00
Chris Lu
55b6efb755
fix checking visited nodes
2020-06-19 09:45:42 -07:00
Chris Lu
0e7c1a300b
Merge pull request #1363 from DXist/fix_utc
...
Return last modified in UTC
2020-06-19 08:49:00 -07:00
Rinat Shigapov
320d70d305
return last modified in UTC
2020-06-19 10:26:40 +03:00
Chris Lu
546709c1de
check has visited by the boundary
2020-06-18 22:28:17 -07:00
Chris Lu
0e3b970c0c
added bounded tree to track exploration boundary
2020-06-18 21:09:58 -07:00
Chris Lu
ae1994cbc1
erasure coding: fix cases where there are no .ecj files
2020-06-18 09:52:35 -07:00
Chris Lu
b74eced7a3
mount: a fix to prevent possible repeated calls
...
related to https://github.com/bazil/fuse/issues/130
2020-06-17 13:55:16 -07:00
Chris Lu
7ccc31853c
hard limit volume file size
2020-06-16 00:27:48 -07:00
Chris Lu
48880edc2c
volume: fix typo in minFreeSpacePercent option
2020-06-14 20:37:05 -07:00
Chris Lu
a0464fc267
fix help
...
fix https://github.com/chrislusf/seaweedfs/issues/1359
2020-06-12 22:06:21 -07:00
Chris Lu
0d1d83a5a5
mount: add option to auto create dir to mount
2020-06-12 11:46:58 -07:00
Chris Lu
00f96d86a1
fix typo
2020-06-11 10:53:25 -07:00
Chris Lu
63a1db64c5
Merge pull request #1358 from DXist/s3fixes
...
Return NotSuchKey error code for bucket S3 DeleteObject method
2020-06-11 10:37:19 -07:00
Rinat Shigapov
283b749ff1
keep 204 response
2020-06-11 19:00:34 +03:00
Rinat Shigapov
fafc41a27f
return xml encoded NotFound status code for s3 delete
2020-06-11 17:53:15 +03:00
Rinat Shigapov
2d2c5dfa39
proxy status code of filer delete response
2020-06-11 15:00:47 +03:00
Chris Lu
628b27ef3b
purge old cache implementation
2020-06-11 01:50:00 -07:00
Chris Lu
b9365de47b
fix volume server pprof
2020-06-10 13:10:10 -07:00
Chris Lu
718052656a
Merge pull request #1355 from Kimbsen/add_http_pprof
...
Add --pprof to volume servers to enable http pprof handlers
2020-06-10 13:05:04 -07:00
Chris Lu
95a9686eaf
1.81
2020-06-10 13:00:42 -07:00
Chris Lu
c66f357715
report error correctly
2020-06-10 10:59:25 -07:00
Kimbsen
03e8f976d9
added --pprof to volume servers
2020-06-10 11:52:24 +02:00
Chris Lu
c38066be89
also add it for webdav
2020-06-09 18:07:03 -07:00
Chris Lu
4598473251
fix possible EOF problem
2020-06-09 18:04:40 -07:00
Chris Lu
bb3e86fbd5
auto create non-existing cache directory
...
fix https://github.com/chrislusf/seaweedfs/issues/1350
2020-06-08 08:49:57 -07:00
Chris Lu
bdbe5ddbe9
volume server: add a fix for dead lock
...
https://github.com/chrislusf/seaweedfs/issues/1353
2020-06-08 08:12:59 -07:00
Chris Lu
b8e346cc86
1.80
2020-06-05 15:29:25 -07:00
Chris Lu
45a9d852d3
go fmt
2020-06-05 15:27:10 -07:00
Chris Lu
03c2f03509
Merge pull request #1351 from ekozlov-search/master
...
Set volumes ReadOnly if low free disk space
2020-06-05 09:56:46 -07:00
Chris Lu
ebd1759741
fix ReadAt
...
fix https://github.com/chrislusf/seaweedfs/issues/1344
2020-06-05 09:54:14 -07:00
Evgenii Kozlov
2410ffbd79
Name fix
2020-06-05 19:00:38 +03:00
Evgenii Kozlov
0e0db70f55
Set volumes ReadOnly if low free disk space
2020-06-05 18:18:15 +03:00
Chris Lu
393ab07c7c
disable pulse seconds
...
there are a few other hard coded timing settings, which will cause disconnected volume servers.
2020-06-04 10:52:01 -07:00
Evgenii Kozlov
48f9ff52cf
Discs statistics on Windows platform.
2020-06-02 11:52:16 +03:00
Chris Lu
bc2ec6774d
inject git version into build
2020-06-02 00:10:38 -07:00
bingoohuang
1f8782a1ed
try showing the first 100 volume ids and an extra ...
2020-05-29 16:15:33 +08:00
bingoohuang
1a642b9876
add Volume Ids column only for max 100 volumes for convenience in the master ui.
2020-05-29 15:37:58 +08:00
bingoohuang
accb4964b7
util.BytesToHumanReadable
2020-05-29 10:00:07 +08:00
bingoohuang
21d0a013d5
rename 'needle' which collides with imported package name
2020-05-28 17:00:00 +08:00
bingoohuang
41cab5d69f
log JSON response if httpStatus >= 400
2020-05-28 08:56:55 +08:00
Chris Lu
ef2b3a0801
leveldb recover error handling
2020-05-26 00:03:44 -07:00
Chris Lu
5d3ec22975
refactoring
2020-05-26 00:03:31 -07:00
Chris Lu
d4235afe4d
support multiple locks
2020-05-25 18:39:53 -07:00
bingoohuang
ecdeef8c66
simplify func(w http.ResponseWriter, r *http.Request) to http.HandlerFunc
2020-05-25 21:00:12 +08:00
Chris Lu
e5a0787653
filer: leveldb store add auto recovery, report correct error
...
fix https://github.com/chrislusf/seaweedfs/issues/1326
2020-05-22 10:54:42 -07:00
Chris Lu
ed9e4dc749
1.79
...
fix compilation problem
2020-05-19 08:21:51 -07:00
Chris Lu
7ff2b0121d
specify constant data type to uint64 instead of default int
...
fix https://github.com/chrislusf/seaweedfs/issues/1324
2020-05-19 07:55:23 -07:00
Chris Lu
11b33c0760
fix compilation problem
2020-05-18 10:53:31 -07:00
Chris Lu
ac48c899e8
add printouts for synchronizing metadata which can be slow for large folders
2020-05-18 10:31:12 -07:00
Chris Lu
0c856e8387
avoid possible wrong arguments
...
fix https://github.com/chrislusf/seaweedfs/issues/1320
2020-05-18 10:07:12 -07:00
Chris Lu
2bfd810912
combine msgBroker into weed server command
2020-05-17 22:44:20 -07:00
Chris Lu
85aeead3d2
1.78
2020-05-17 20:28:04 -07:00
Chris Lu
8005786d9a
logs
2020-05-17 20:27:10 -07:00
Chris Lu
75179544c3
logging
2020-05-17 20:20:21 -07:00
Chris Lu
91b91d6cb7
add error to avoid copying not found volume
...
fix https://github.com/chrislusf/seaweedfs/issues/1317
2020-05-17 20:20:12 -07:00
Chris Lu
d3925d086c
add delete channel
2020-05-17 17:42:42 -07:00
Chris Lu
e0e31e67a8
Merge pull request #1318 from chrislusf/msg_channel
...
Add messaging, add channel
2020-05-17 17:39:16 -07:00
Chris Lu
081ee6fe34
reduc logs
2020-05-17 17:38:31 -07:00
Chris Lu
bff09fb74f
fix looping
2020-05-17 17:38:21 -07:00