Eric Yang
b324a6536c
ADHOC: add read needle meta grpc ( #3581 )
...
* ADHOC: add read needle meta grpc
* add test
* nit
Co-authored-by: root <root@HQ-10MSTD3EY.roblox.local>
2022-09-06 23:51:27 -07:00
chrislu
26dbc6c905
move to https://github.com/seaweedfs/seaweedfs
2022-07-29 00:17:28 -07:00
chrislu
36d2756978
volume: avoid writing too much data for range requests in large files
...
related to https://github.com/chrislusf/seaweedfs/issues/3178
2022-06-14 11:16:47 -07:00
chrislu
11f1e7996b
fix test
2022-06-05 18:50:41 -07:00
chrislu
056c480eb0
volume: checksum remove the hashing step in Value()
2022-06-05 15:24:02 -07:00
chrislu
ecef844dfc
stream read large files
2022-06-05 11:54:04 -07:00
chrislu
685643d506
needle read into a writer
2022-06-04 19:10:52 -07:00
chrislu
663bc5dc23
add needle reading in chunks
2022-06-04 18:15:39 -07:00
chrislu
18b29f709c
refactoring, split file
2022-06-03 23:31:14 -07:00
chrislu
433fde4b18
move error to a separate file
...
This file contains metric names for all errors
The naming convention is ErrorSomeThing = "error.some.thing"
2022-02-04 22:57:51 -08:00
Konstantin Lebedev
3f4e17aa24
error metrics for filer and store
2022-02-04 14:07:14 +05:00
Eng Zer Jun
a23bcbb7ec
refactor: move from io/ioutil to io and os package
...
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-14 12:27:58 +08:00
Chris Lu
d688e10ed1
do not try to compress if not sure about the file content
2021-10-07 13:29:00 -07:00
Chris Lu
332f5ad3a8
revert temporary changes that disabled compression
...
fix https://github.com/chrislusf/seaweedfs/issues/2362
2021-10-07 13:24:16 -07:00
Chris Lu
5956a8b05a
adjust comment
2021-09-27 01:58:03 -07:00
Chris Lu
1b942dfa3c
add error message
2021-09-02 12:32:46 -07:00
Konstantin Lebedev
865b06b7fe
The IEEE assembler optimizations has been submitted and will be part of the Go 1.6 standard library.
...
https://github.com/klauspost/crc32
2021-08-17 13:06:48 +05:00
Chris Lu
9cc84a910f
volume: deletion can skip volume size checking
...
fix https://github.com/chrislusf/seaweedfs/issues/2225
2021-08-01 00:32:51 -07:00
Chris Lu
b624090398
go fmt
2021-07-01 01:21:14 -07:00
Chris Lu
6c82326575
use bytes.Buffer to reduce memory allocation and gc
2021-06-06 13:42:36 -07:00
Chris Lu
8e404a1433
go fmt
2021-04-02 02:22:26 -07:00
Chris Lu
b465095db1
shell: add volume.check.disk to fix inconsistency for replicated volumes
...
fix https://github.com/chrislusf/seaweedfs/issues/1923
2021-03-22 00:03:16 -07:00
Chris Lu
35d939a1d2
go fmt
2021-03-14 13:21:02 -07:00
Chris Lu
38fc200e56
CRCWriter consistent with CRC
2021-03-06 14:26:27 -08:00
Chris Lu
400de380f4
volume server: support tcp direct put/get/delete
2021-03-05 02:29:38 -08:00
Chris Lu
540441fd38
go fmt
2021-02-28 20:34:14 -08:00
Chris Lu
03c643aa6b
fix test
...
fileSize need to be divided by 8
2021-02-20 21:19:21 -08:00
Chris Lu
1ad3200094
skip seek() when index file writes
2021-02-20 12:39:25 -08:00
Chris Lu
039ad101b1
manifest file also need to detect compression
...
fix https://github.com/chrislusf/seaweedfs/issues/1724#issuecomment-753585046
2021-01-03 01:42:21 -08:00
Chris Lu
cc839f935d
better unit detection
2020-12-03 19:23:59 -08:00
Chris Lu
b219ccfe68
Update volume_ttl_test.go
2020-12-03 12:55:44 -08:00
Chris Lu
a83b8e7b01
ttl fix: 3y was converted into 96m
2020-12-03 10:53:29 -08:00
Chris Lu
0da7ecfd29
go fmt
2020-11-26 11:22:30 -08:00
Chris Lu
92f906b6fc
remove zstd
...
fix https://github.com/chrislusf/seaweedfs/issues/1629
2020-11-21 13:06:45 -08:00
Chris Lu
06c15ab35c
volume: add special handling for .dat larger than 32GB
2020-10-27 13:11:56 -07:00
Chris Lu
103fafe00b
weed export: print out [start,stop) content range
2020-09-14 22:57:23 -07:00
Chris Lu
dbb10e0f0b
testing with more than 30days
2020-08-31 17:59:29 -07:00
Chris Lu
99ecf63276
go fmt
2020-08-29 22:28:33 -07:00
Chris Lu
6a92f0bc7a
refactoring to typed Size
...
Go is amazing with refactoring!
2020-08-18 17:04:28 -07:00
Chris Lu
ab6e5c0dc4
adjust error message
2020-08-08 10:18:32 -07:00
Chris Lu
20e2ac1add
filer: store md5 metadata for files uploaded by filer
...
fix https://github.com/chrislusf/seaweedfs/issues/1412
2020-08-06 10:04:17 -07:00
Chris Lu
4ecfa9879d
volume: report Content-MD5 in response header
2020-08-06 05:22:53 -07:00
Chris Lu
cbd80253e3
better needle id format
2020-08-04 22:09:07 -07:00
Chris Lu
f89186a3cf
better logging
2020-08-01 11:19:57 -07:00
Chris Lu
97d97f3528
go code can read and write chunk manifest
2020-07-19 17:59:43 -07:00
李远军
6608cb5f43
Revert "remove fixJpgOrientation"
...
This reverts commit de5ca9b2
2020-07-10 10:08:36 +08:00
Chris Lu
20d9cfae86
volume: following md5 base64 encoding
...
fix https://github.com/chrislusf/seaweedfs/issues/1374
2020-06-28 07:25:50 -07:00
Chris Lu
2ff37ccdbd
testing parsing the upload
...
The mime type is always the value passed in.
Compress or not depends on the content detection, file name extension, and compression ratio.
If the content is already compressed, need to know the content size.
2020-06-24 11:39:12 -07:00
Chris Lu
c21f4ebfee
Merge pull request #1371 from Kimbsen/content_md5_validation
...
Optional md5 validation of uploads
2020-06-24 11:33:44 -07:00
Kimbsen
4eeab2a379
remove unused imports
2020-06-24 13:37:00 +02:00