gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9720 commits 28 branches 278 tags 135 MiB
Commit graph

6281 commits

Author SHA1 Message Date
chrislu 875f562779 server side send response at least once per second 2023-09-06 23:16:41 -07:00
chrislu 984b6c54cf ack interval 128 2023-09-06 23:15:29 -07:00
chrislu 2492a45499 ack interval 2023-09-06 22:39:46 -07:00
Lars Lehtonen f24c7e803f weed/storage/erasure_coding: Close() after error handling 2023-09-06 07:35:54 -07:00
Lars Lehtonen d762b12e11 weed/storage/erasure_coding: fix dropped test error 2023-09-06 07:35:54 -07:00
Lars Lehtonen a879e1bbb5 weed/storage/erasure_coding: remove unused err from encodeDatFile() signature 2023-09-06 07:35:54 -07:00
Patrick Schmidt 98dcec0ee2 Clean up old signature hash pools 2023-09-05 10:33:27 -07:00
Patrick Schmidt cdd817edf9 Improve S3 request signing performance 
This change is caching HMAC hashers for repeated use in subsequent
requests and chunks, so they don't have to be initialized from
scratch every time.
On my local computer this gives me ~5-6 times faster signature
calculation and ~5-6.5% more throughput in S3 requests. The smaller
the payload the better the throughput gets.
 2023-09-05 10:33:27 -07:00
chrislu ba67e6ca29 api for sub 2023-09-04 21:43:50 -07:00
chrislu 9e4f985698 publish, benchmark 2023-09-04 21:43:30 -07:00
Lars Lehtonen e6385e5e8b
weed/util: fix dropped errors (#4803) 2023-09-01 13:10:45 -07:00
chrislu cb470d44df can pub and sub 2023-09-01 00:36:51 -07:00
chrislu 1eb2da46d5 connect and publish 2023-08-28 09:02:12 -07:00
chrislu 504ae8383a protoc version 2023-08-28 09:01:25 -07:00
chrislu dbcba75271 rename to lookup 2023-08-27 18:59:04 -07:00
chrislu c9caf33119 move functions 2023-08-27 18:33:46 -07:00
chrislu 4d6c18d86f pub sub initial tests 2023-08-27 17:50:59 -07:00
chrislu 4eb8e8624d rename 2023-08-27 13:14:39 -07:00
chrislu 1990456670 sub 2023-08-27 13:13:14 -07:00
chrislu 905911853d adjust proto 2023-08-26 13:39:21 -07:00
Nico D'Cotta 796b7508f3
Implement SRV lookups for filer (#4767) 2023-08-24 07:08:56 -07:00
chrislu 733db2bc88 turn on streaming assign file id 2023-08-23 00:34:21 -07:00
chrislu 99f037b958 streaming assign file ids 2023-08-23 00:31:33 -07:00
chrislu 94b7e2a37c add stream assign server side implementation 2023-08-22 09:59:04 -07:00
chrislu ccedad5196 refactor files 2023-08-22 09:54:06 -07:00
chrislu 010c5e91e3 add stream assign proto 2023-08-22 09:53:54 -07:00
chrislu a1149f060b upgrade jwt 2023-08-22 08:26:31 -07:00
chrislu c49caff5a1 change from deprecated jwt.StandardClaims to new jwt.RegisteredClaims 2023-08-21 23:42:39 -07:00
chrislu c45665eb97 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-08-21 09:47:36 -07:00
SmsS4 f61490966f
Add time to first byte metric for s3 (#4768) 
* Add time to first byte metric for s3

* Change second to millisecond
 2023-08-21 00:42:39 -07:00
chrislu 01d70c21f3 Squashed commit of the following: 
commit 32f4b1a13057d56b6de487cdb80ff7c205af01a6
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 22:52:19 2023 -0700

    fix compilation

commit e77ad33b7ca0423138fbae26a4433b60923a9588
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 22:46:44 2023 -0700

    pub

commit f431f30cc7ca277ca299e3cd118c05537fb9f5c3
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 13:27:39 2023 -0700

    fix generic type

commit 4e9dcb18293fd1e3e306e2dceb995dfd67a35e1d
Merge: 30f942580 16e3f2d52
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 12:47:14 2023 -0700

    Merge branch 'master' into pubsub

commit 30f942580ad1bb32ae94aade2e3a21ec3ab63e21
Author: chrislu <chris.lu@gmail.com>
Date:   Sun Aug 20 11:10:58 2023 -0700

    wip

commit f8b00980bc2f3879bb43decffd9a08d842f196f2
Author: chrislu <chris.lu@gmail.com>
Date:   Tue Jul 25 09:14:35 2023 -0700

    add design document

commit 08d2bebe42a26ebc39f1542f54d99e73620727dd
Author: chrislu <chris.lu@gmail.com>
Date:   Tue Jul 25 09:14:06 2023 -0700

    minor

commit bcfa7982b262a40fcdce6fc6613fad2ce07c13da
Author: chrislu <chris.lu@gmail.com>
Date:   Tue Jul 25 09:13:49 2023 -0700

    rename
 2023-08-20 22:53:05 -07:00
chrislu 16e3f2d528 fix log buffer test 2023-08-20 12:46:15 -07:00
chrislu 3e9c32a3f0 refactoring to use atomic bool 
fix compilation
 2023-08-20 12:13:06 -07:00
SmsS4 1e72579ac7
Fix s3.clean.uploads (#4766) 2023-08-19 06:30:18 -07:00
chrislu 6792db21dc ErrInvalidPart when completed parts has extra parts 
fix https://github.com/seaweedfs/seaweedfs/discussions/4746
 2023-08-17 00:51:42 -07:00
chrislu f5c4b642bb re-ordering 2023-08-16 23:54:23 -07:00
chrislu 3852307e94 renaming 2023-08-16 23:47:43 -07:00
chrislu 6c7fa567d4 add separate cache directory for write buffers 2023-08-16 23:39:21 -07:00
SmsS4 ea1b259c12
Update weed fix description (#4745) 2023-08-13 03:50:53 -07:00
Nikita Mochalov 4b1ba7f5b2
Save disk space metrics immediately (#4740) 2023-08-10 06:52:32 -07:00
Nikita Mochalov e6a49dc533
Fix resource leaks (#4737) 
* Fix division by zero

* Fix file handle leak

* Fix file handle leak

* Fix file handle leak

* Fix goroutine leak
 2023-08-09 15:30:36 -07:00
chrislu 3365468d0d added an error message 2023-08-08 20:35:21 -07:00
wusong 3ec977cd6c
minor (#4723) 2023-08-01 02:34:59 -07:00
chrislu c09100f928 fix building mongo index 2023-07-24 11:09:51 -07:00
chrislu 7eafa3420b 3.55 2023-07-23 23:11:37 -07:00
SmoothDenis 51bcc219ea
s3api should return 500 code from filer (#4699) 2023-07-22 07:22:38 -07:00
wusong 33b9192536
Fix large file chunks deleted by mistake (#4678) 2023-07-17 07:52:14 -07:00
chrislu 358b3a4894 3.54 2023-07-16 22:47:57 -07:00
chrislu 6f588b5b18 fix refactoring mistake 
fix https://github.com/seaweedfs/seaweedfs/issues/4639
 2023-07-16 11:22:48 -07:00
mervynzhang df400e6c71
Concurrency works better (#4663) 
Co-authored-by: mervyn.zhang <mervyn.zhang@sap.com>
 2023-07-12 23:04:54 -07:00
wusong 0cb9ddd8ec
Fix data loss: add lock for metacache (#4664) 
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-07-11 22:23:32 -07:00
Henco Appel 89542db2d0
Fix S3 API multipart upload on Windows (#4666) 
The file paths were being having '/' changed to '\' by filepath.Dir()
resulting in a file being created with '\' separators, but when trying
to read the files, the same wasn't happening.

Co-authored-by: Hendrik Appel <happel@europe.altair.com>
 2023-07-11 09:05:14 -07:00
Henco Appel 680c2e76f1
Fix the segfault from not setting s3 https port (#4667) 2023-07-11 08:03:20 -07:00
Konstantin Lebedev 2c076ebd96
avoid logging nil error (#4659) 
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-07-10 23:38:52 -07:00
LHHDZ 3730772e05
refresh max volume count after adjust (#4662) 
Signed-off-by: changlin.shi <changlin.shi@ly.com>
 2023-07-10 23:37:33 -07:00
chrislu c278bac263 avoid nil needle map 
fix https://github.com/seaweedfs/seaweedfs/issues/4640
 2023-07-07 22:16:58 -07:00
chrislu 0606b59a3e mount: create mount root on filer 
fix https://github.com/seaweedfs/seaweedfs-csi-driver/issues/127
 2023-07-07 09:30:32 -07:00
chrislu 14546f0ad8 minor 2023-07-06 09:12:28 -07:00
chrislu ab30f489c3 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-07-06 00:33:02 -07:00
chrislu 31b2751aff clone volume locations in case they are changed 
fix https://github.com/seaweedfs/seaweedfs/issues/4642
 2023-07-06 00:32:58 -07:00
wusong 61553beba2
Fix DataBackend nil pointer (#4641) 2023-07-04 23:22:10 -07:00
chrislu a315490f7d proxy to master uses http address 
fix https://github.com/seaweedfs/seaweedfs/issues/4607
 2023-07-04 11:45:21 -07:00
chrislu 2aa59ab37c fix copying level db files during commi 
fix https://github.com/seaweedfs/seaweedfs/issues/4635
 2023-07-04 11:28:12 -07:00
wusong c44a6552d2
Fix buff use (#4628) 2023-07-03 08:33:34 -07:00
Amir H. Yeganemehr f0d3c874b6
Unix Socket listener for S3 server (#4621) 
Add localSocket option to s3 server
 2023-06-26 16:22:45 -07:00
chrislu 2c4c2f0994 3.53 2023-06-25 22:51:14 -07:00
chrislu f193942adc lock on append metadata operation 2023-06-25 22:07:23 -07:00
chrislu 61c42f9991 adjust lock APIs 2023-06-25 20:30:20 -07:00
chrislu aafb874ad2 lock until cancelled 2023-06-25 19:31:25 -07:00
chrislu c4f44d56f7 add distributed lock client 2023-06-25 17:38:34 -07:00
chrislu f8aa5ea844 adjust filer.proto 2023-06-25 17:38:19 -07:00
chrislu 13a8593ab9 adjust errors 2023-06-25 17:37:54 -07:00
chrislu 06471dac9d init lock ring 2023-06-25 15:28:16 -07:00
chrislu 5149b3d07b filer can proxy to peer filer holding the lock 2023-06-25 15:28:01 -07:00
chrislu 868f7875d7 refactor 2023-06-25 14:30:58 -07:00
chrislu 26eff062c8 refactor 2023-06-25 14:17:58 -07:00
chrislu 464a71a373 add distributed lock manager 2023-06-25 14:14:40 -07:00
chrislu ee4f7cd636 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-06-25 00:58:24 -07:00
chrislu 3fd659df2a add distributed lock manager 2023-06-25 00:58:21 -07:00
Damiano Albani 0d9ba8c612
Support templating name of files stored via Rclone backend (#4606) 2023-06-24 07:08:24 -07:00
chrislu a82c44972b refactor 2023-06-20 00:12:59 -07:00
chrislu 14a19170e2 refactor 2023-06-19 23:51:04 -07:00
chrislu 84ced2abf5 add LockRing 2023-06-19 23:48:33 -07:00
chrislu fe1716ccdf remove deprecated tests 2023-06-19 18:43:51 -07:00
chrislu 1ce725728d minor 2023-06-19 18:43:41 -07:00
chrislu 11549706ef refactor 2023-06-19 18:21:07 -07:00
chrislu 8ec1bc2c99 remove unused cluster node leader 2023-06-19 18:19:13 -07:00
Bai Jie 3b88ab42aa
remove duplicate fileCount query (#4588) 2023-06-18 00:14:14 -07:00
Bai Jie 44b9d72ef0
doIsEmpty() return error if v.DataBackend is nil (#4587) 2023-06-18 00:13:40 -07:00
chrislu 94fbf02ea2 generate jwt token when looking up needle id 
fix https://github.com/seaweedfs/seaweedfs/issues/4577
 2023-06-17 00:11:23 -07:00
柏杰 0b0fb9b9e4
avoid data race read volume.IsEmpty (#4574) 
* avoid data race read volume.IsEmpty

-   avoid phantom read isEmpty for onlyEmpty
-   use `v.DataBackend.GetStat()` in v.dataFileAccessLock scope

* add Destroy(onlyEmpty: true) test

* add Destroy(onlyEmpty: false) test

* remove unused `IsEmpty()`

* change literal `8` to `SuperBlockSize`
 2023-06-14 14:39:58 -07:00
Konstantin Lebedev 1e22d5caf2
fix get file stats for IsEmpty (#4576) 2023-06-14 01:43:30 -07:00
Konstantin Lebedev 4dd890d4a2
optional https port for s3 (#4482) 
Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-13 23:58:49 -07:00
chrislu 3fbf4f6189 Revert "Initial patch for redis version 7 support (#4572)" 
This reverts commit a25bca0692.
 2023-06-12 22:27:38 -07:00
Konstantin Lebedev 4527ead295
fix from comment delete volume is empty (#4573) 
* fix from coments
https://github.com/seaweedfs/seaweedfs/pull/4561

* fix tests

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-12 22:22:46 -07:00
Aaron Gipson a25bca0692
Initial patch for redis version 7 support (#4572) 2023-06-12 13:30:03 -07:00
Konstantin Lebedev 25535e9c36
Delete volume is empty (#4561) 
* use onlyEmpty for deleteVolume
https://github.com/seaweedfs/seaweedfs/issues/4559

* fix IsEmpty

* fix test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
 2023-06-12 10:42:44 -07:00
柏杰 68feffa4e6
avoid overwriting non-nil uploadReaderToChunks.uploadErr with nil value (#4560) 2023-06-10 23:15:27 -07:00
Konstantin Lebedev 5ee04d20fa
Healthz check for deadlocks (#4558) 2023-06-09 09:42:48 -07:00
wusong d6af3a8eb6
fix 404 not found (#4536) 
Signed-off-by: wang wusong <wangwusong@virtaitech.com>
Co-authored-by: wang wusong <wangwusong@virtaitech.com>
 2023-06-06 09:14:49 -07:00