Chris Lu
6d30b21b10
volume: add "-dir.idx" option for separate index storage
...
fix https://github.com/chrislusf/seaweedfs/issues/1265
2020-11-27 03:17:10 -08:00
Chris Lu
3c229eb677
refactor
2020-11-26 17:21:55 -08:00
Chris Lu
c0fb4a3766
add comments
2020-11-26 15:22:42 -08:00
Chris Lu
6783ae5f8f
rename
2020-11-26 15:19:43 -08:00
Chris Lu
97f706f35b
just refactoring
2020-11-26 15:08:05 -08:00
Chris Lu
b11449f955
pass in dir.idx parameter to DiskLocation
2020-11-26 14:59:03 -08:00
Chris Lu
0da7ecfd29
go fmt
2020-11-26 11:22:30 -08:00
Chris Lu
92f906b6fc
remove zstd
...
fix https://github.com/chrislusf/seaweedfs/issues/1629
2020-11-21 13:06:45 -08:00
Chris Lu
1dd3a6ac36
avoid verbose logs
2020-11-09 15:56:11 -08:00
Chris Lu
0dafcf1f5a
volume: detect max volume count changes based on disk usage
...
fix https://github.com/chrislusf/seaweedfs/issues/1594
2020-11-03 14:43:17 -08:00
Chris Lu
de86945aeb
go fmt
2020-10-31 16:45:38 -07:00
Chris Lu
9708df47d1
ensure offset in the index file is positive
2020-10-31 15:50:01 -07:00
Konstantin Lebedev
46303c36bf
When the volume server is stopped, the master server immediately sees the deletion of volumes
2020-10-31 02:48:25 +05:00
Chris Lu
6560ac6466
volume loading: trim out unreachable idx file content
...
fix https://github.com/chrislusf/seaweedfs/issues/1583
2020-10-29 22:25:23 -07:00
Konstantin Lebedev
1ad1b8c4f6
collection Volume ReadOnly Count with detailed status
2020-10-28 17:38:26 +05:00
Chris Lu
744c5594da
volume: automatically trim out unreachable entries
2020-10-28 01:14:39 -07:00
Chris Lu
3e925faddd
trim out name extension
2020-10-28 01:14:05 -07:00
shibinbin
a67eb1afa1
fix: restart volumeserver sometime occurred out of memory error
2020-10-28 12:03:06 +08:00
Chris Lu
53c3aad875
volume: add a note file to avoid incomplete volume files
...
fix https://github.com/chrislusf/seaweedfs/issues/1567
2020-10-27 15:56:49 -07:00
Chris Lu
06c15ab35c
volume: add special handling for .dat larger than 32GB
2020-10-27 13:11:56 -07:00
Konstantin Lebedev
2fb1fce8a0
avoid old values when missing read only flags
2020-10-25 16:21:40 +05:00
Chris Lu
9104cfa744
reduce locks
2020-10-24 19:40:35 -07:00
Chris Lu
e9d40b80b2
less verbose logs
2020-10-24 09:42:54 -07:00
Chris Lu
575d7952a1
add available resource stats
...
fix https://github.com/chrislusf/seaweedfs/issues/1555
2020-10-22 09:13:47 -07:00
Chris Lu
4ff2ceee33
UI fix on rendering EC volumes
...
addressing UI problem with https://github.com/chrislusf/seaweedfs/issues/1551
2020-10-21 22:05:58 -07:00
Konstantin Lebedev
dc2e13092d
add number of read only volumes metric
2020-10-15 15:48:40 +05:00
Chris Lu
bd8b9b0c2e
some changes to prepare fix after failed reads
2020-10-08 18:33:06 -07:00
Chris Lu
d522df0ecd
adjust log level
2020-10-04 01:35:32 -07:00
Chris Lu
a1c01d716b
volume: avoid deadlock when deleting volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1501
2020-10-01 07:10:03 -07:00
Chris Lu
103fafe00b
weed export: print out [start,stop) content range
2020-09-14 22:57:23 -07:00
Chris Lu
f2723c1bc8
do not idx file format
...
revert c9ab8d05fa
2020-09-12 12:42:36 -07:00
Chris Lu
446e476a11
go fmt
2020-09-12 04:08:03 -07:00
Chris Lu
ea26a98753
volume: validate volume correctness if last entry is a deletion
2020-09-12 04:07:04 -07:00
Chris Lu
9d4bdfcfdf
fix volume integrity checking
2020-09-11 11:34:10 -07:00
Chris Lu
401ccf1509
fix test
2020-09-10 15:13:23 -07:00
Chris Lu
c9ab8d05fa
fixes for reading deleted fid
2020-09-10 14:42:52 -07:00
James Hartig
8e54e34576
volume: Don't unmount before deleting volume in copy
...
If we unmount first and then delete, the delete fails because the volume
was unmounted. Delete ends up doing the same thing as the unmount anyways.
2020-09-01 22:00:07 -04:00
Chris Lu
38e06d783d
volume: check disk space before compaction
...
fix https://github.com/chrislusf/seaweedfs/issues/1440
2020-08-31 18:10:53 -07:00
Chris Lu
dbb10e0f0b
testing with more than 30days
2020-08-31 17:59:29 -07:00
Chris Lu
99ecf63276
go fmt
2020-08-29 22:28:33 -07:00
Chris Lu
ab759f0ec2
erasure coding: fix EC error if multiple disks are configured in one volume server
2020-08-26 09:16:58 -07:00
Chris Lu
d1cf39f180
fix logging
2020-08-26 08:55:15 -07:00
James Hartig
3ccfa4c6ad
Added VolumeMarkWritable and VolumeStatus grpc methods
...
This is necessary for copy to mark as read-only and then restore the
original state afterwards.
2020-08-19 11:42:56 -04:00
Chris Lu
3b4b1d4a77
fix tests
2020-08-19 01:37:56 -07:00
Chris Lu
ed4b43b419
adjust logs
2020-08-18 23:42:09 -07:00
Chris Lu
fe01191b5b
support read option readDeleted=true
2020-08-18 19:22:16 -07:00
Chris Lu
6ccd7f0a4d
refactoring
2020-08-18 18:01:37 -07:00
Chris Lu
c026eb0592
refactoring
2020-08-18 17:39:29 -07:00
Chris Lu
7e91ae592c
pass in option to read deleted entries
...
not working yet
2020-08-18 17:37:26 -07:00
Chris Lu
ee11d98650
refactoring
2020-08-18 17:35:19 -07:00
Chris Lu
6a92f0bc7a
refactoring to typed Size
...
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu
627b081b67
adjust logs
2020-08-16 16:32:22 -07:00
Chris Lu
c03bb180eb
fix error reporting
2020-08-14 00:44:02 -07:00
Chris Lu
ab6e5c0dc4
adjust error message
2020-08-08 10:18:32 -07:00
Chris Lu
20e2ac1add
filer: store md5 metadata for files uploaded by filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00
Chris Lu
4ecfa9879d
volume: report Content-MD5 in response header
2020-08-06 05:22:53 -07:00
Chris Lu
cbd80253e3
better needle id format
2020-08-04 22:09:07 -07:00
Chris Lu
f89186a3cf
better logging
2020-08-01 11:19:57 -07:00
Chris Lu
943fa4ea0c
fix compilation error
2020-07-22 16:11:17 -07:00
Chris Lu
49446af516
simplify
2020-07-22 13:43:54 -07:00
James Hartig
0be3bb4a87
idx: Switch WalkIndexFile to accept generic io.ReaderAt
...
This is necessary to use it with in-memory idx files.
2020-07-22 15:01:16 -04:00
Chris Lu
97d97f3528
go code can read and write chunk manifest
2020-07-19 17:59:43 -07:00
Chris Lu
f43146b237
resolve directories if containing home directory
2020-07-16 22:50:14 -07:00
Chris Lu
ddec7b2bb9
go fmt
2020-07-14 07:34:16 -07:00
Chris Lu
f2e8ad6241
refactoring
2020-07-11 06:35:54 -07:00
李远军
6608cb5f43
Revert "remove fixJpgOrientation"
...
This reverts commit de5ca9b2
2020-07-10 10:08:36 +08:00
Chris Lu
4bed1663e2
adjust log level
2020-07-07 23:17:31 -07:00
Chris Lu
4d1484628a
refactoring
2020-07-03 16:41:30 -07:00
Chris Lu
faa5c2e89a
refactoring
2020-07-03 16:34:31 -07:00
Chris Lu
20d9cfae86
volume: following md5 base64 encoding
...
fix https://github.com/chrislusf/seaweedfs/issues/1374
2020-06-28 07:25:50 -07:00
Chris Lu
212c64241b
checking ecx file
2020-06-25 16:36:06 -07:00
Chris Lu
9fd7cdadf1
fix
2020-06-25 10:45:34 -07:00
Chris Lu
3b638d3994
add more ec encoding logging
2020-06-25 09:43:38 -07:00
Chris Lu
32a1f22730
more logs
2020-06-24 22:07:53 -07:00
Chris Lu
2ff37ccdbd
testing parsing the upload
...
The mime type is always the value passed in.
Compress or not depends on the content detection, file name extension, and compression ratio.
If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu
c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation
...
Optional md5 validation of uploads
2020-06-24 11:33:44 -07:00
Kimbsen
4eeab2a379
remove unused imports
2020-06-24 13:37:00 +02:00
Kimbsen
ffddecebef
do md5 validation AFTER decompression
2020-06-24 13:35:13 +02:00
Chris Lu
fe60db404a
prepare for zstd
2020-06-23 09:12:02 -07:00
Kimbsen
6b1e93ba0b
check request header not part header for content-encoding
2020-06-23 17:04:24 +02:00
Kimbsen
38626cb584
don't forget the last few bytes
2020-06-23 16:53:49 +02:00
Kimbsen
8d1f6cd567
added optional md5 verification of uploaded data
2020-06-23 16:24:11 +02:00
Chris Lu
de5ca9b258
remove fixJpgOrientation
2020-06-20 12:50:40 -07:00
Chris Lu
3080c197e3
rename UnCompressData to DecompressData
2020-06-20 08:16:16 -07:00
Chris Lu
e912fd15e3
renaming
2020-06-19 22:45:27 -07:00
Chris Lu
16fe132a20
detect mime type before replicating to other volume servers
2020-06-19 22:11:36 -07:00
Chris Lu
ae1994cbc1
erasure coding: fix cases where there are no .ecj files
2020-06-18 09:52:35 -07:00
Chris Lu
7ccc31853c
hard limit volume file size
2020-06-16 00:27:48 -07:00
Chris Lu
bdbe5ddbe9
volume server: add a fix for dead lock
...
https://github.com/chrislusf/seaweedfs/issues/1353
2020-06-08 08:12:59 -07:00
Chris Lu
45a9d852d3
go fmt
2020-06-05 15:27:10 -07:00
Evgenii Kozlov
2410ffbd79
Name fix
2020-06-05 19:00:38 +03:00
Evgenii Kozlov
0e0db70f55
Set volumes ReadOnly if low free disk space
2020-06-05 18:18:15 +03:00
Chris Lu
ef2b3a0801
leveldb recover error handling
2020-05-26 00:03:44 -07:00
Chris Lu
e5a0787653
filer: leveldb store add auto recovery, report correct error
...
fix https://github.com/chrislusf/seaweedfs/issues/1326
2020-05-22 10:54:42 -07:00
Chris Lu
7ff2b0121d
specify constant data type to uint64 instead of default int
...
fix https://github.com/chrislusf/seaweedfs/issues/1324
2020-05-19 07:55:23 -07:00
Chris Lu
11b33c0760
fix compilation problem
2020-05-18 10:53:31 -07:00
zhangsong
b161baa3f0
fix volume vacuum test unit
2020-05-07 19:58:06 +08:00
zhangsong
3703c7e176
write request keep compatible with before
2020-05-06 21:35:32 +08:00
zhangsong
f9e8702bb4
use async write to persistent file to disk - part1
2020-05-04 17:39:44 +08:00
Chris Lu
6190fd665d
printout error
2020-04-27 12:41:31 -07:00