gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
9182 commits 28 branches 278 tags 135 MiB
Commit graph

6020 commits

Author SHA1 Message Date
chrislu 0932437a1c Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-02-01 02:38:09 -08:00
chrislu c47f75eb5b fix null content type caused exception in hdfs client 
fix https://github.com/seaweedfs/seaweedfs/issues/4170
 2023-02-01 02:38:05 -08:00
wusong fe573f26a8
memchunk save content twice (#4174) 2023-01-31 19:36:59 -08:00
Neo b9b613a78e
filter system log dir does not make subscribe event (#4172) 2023-01-31 18:56:11 -08:00
wusong 665917fdf1
[mount]: delete redundant code (#4173) 2023-01-31 18:54:46 -08:00
a 0d085ec175
make directory lister trigger if mime type is httpd/unix-directory (#4169) 2023-01-31 06:54:40 -08:00
chrislu 4d71af87f3 3.41 2023-01-29 22:56:01 -08:00
chrislu ef70415c70 skipping manifest chunk volume id change 2023-01-29 22:55:06 -08:00
chrislu 6e8dbf8868 add fs.meta.changeVolumeId 2023-01-29 22:51:58 -08:00
wusong de081c0d64
[mount] fix metacache update (#4161) 2023-01-29 07:55:27 -08:00
Guo Lei b63d1e9c6b
fix index (#4158) 2023-01-28 06:55:27 -08:00
chrislu 81fdf3651b grpc connection to filer add sw-client-id header 2023-01-20 01:48:12 -08:00
chrislu bfe5d910c6 use one readerCache for the whole file 2023-01-16 22:43:02 -08:00
chrislu 4614e85efa adjust help message 2023-01-15 21:28:36 -08:00
chrislu c24678153c use memory chunks only for sequential writes 2023-01-15 21:10:08 -08:00
chrislu 2885ba0e50 3.40 2023-01-15 20:12:05 -08:00
chrislu 12f8251d32 avoid index out of slice range 2023-01-14 13:31:22 -08:00
chrislu 313fd17ae4 allow random writes for mem chunk 2023-01-13 20:32:18 -08:00
Chris Lu 1cd2e64aac
merge chunks during upload (#4130) 
* merge chunks during upload

* fix test
 2023-01-12 23:07:36 -08:00
chrislu 2452f3b2f7 sort chunks by offset 2023-01-12 20:29:07 -08:00
chrislu 361d97941b mysql: Missed escaping column name 
fix https://github.com/seaweedfs/seaweedfs/issues/4129
 2023-01-11 08:46:56 -08:00
chrislu efcffe4f77 no need to set chunks during flush 2023-01-10 10:01:52 -08:00
chrislu 85f1b70207 reduce group size from 256MB to 64MB 2023-01-10 01:35:44 -08:00
chrislu 86657ed3a8 avoid repeatedly adding manifest chunks 2023-01-10 01:35:10 -08:00
chrislu 28fe578944 avoid possible nil entry 2023-01-10 01:07:34 -08:00
chrislu 47fa5ef979 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-01-10 00:52:50 -08:00
chrislu 9e0d526163 fix compilation 2023-01-10 00:52:05 -08:00
chrislu 7f49c59c14 cluster.ps add filer meta sync progress 2023-01-10 00:51:25 -08:00
chrislu 340e7c3a2e chunk group remove manifestChunks and reset sections in SetChunks() 2023-01-10 00:49:31 -08:00
chrislu e650c8397d ensure section.chunks is always garbage free 2023-01-10 00:48:01 -08:00
chrislu 75bdd4a0d1 refactor 2023-01-10 00:46:46 -08:00
Guo Lei 83e29faa55
use time.NewTicker instead of time.Tick. (#4119) 2023-01-09 06:47:08 -08:00
chrislu 51d4a4b28d 3.39 2023-01-08 18:20:22 -08:00
chrislu 0b52f08226 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2023-01-08 18:03:25 -08:00
chrislu 469c959e6a add missing options for webdav 2023-01-08 18:03:22 -08:00
monchickey 3e2c9ea73d
Add image cropping. (#4117) 2023-01-06 09:28:07 -08:00
chrislu 296fdc296c mount: faster add chunks 2023-01-06 01:03:29 -08:00
chrislu 2abf817580 fix for stream reader 
fix https://github.com/seaweedfs/seaweedfs/issues/4112
 2023-01-05 11:19:21 -08:00
chrislu e367444586 add notes for full version 2023-01-04 09:52:25 -08:00
chrislu 5423790b2c add back previous chunk upload selection algo 2023-01-03 22:50:39 -08:00
chrislu 7bdae5172e batch delete EC needles 
fix https://github.com/seaweedfs/seaweedfs/issues/4107
 2023-01-03 22:05:26 -08:00
zemul 6b4c033431
add mount log (#4101) 
* filer.backup use replication.source.filer

* add mount log

* Revert "filer.backup use replication.source.filer"

This reverts commit 07bf6f956c.

* fix

Co-authored-by: zemul <zhouzemiao@ihuman.com>
 2023-01-03 00:00:45 -08:00
chrislu 48f2edc065 3.38 2023-01-02 23:23:02 -08:00
Chris Lu d4566d4aaa
more solid weed mount (#4089) 
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
 2023-01-02 23:20:45 -08:00
CommanderRoot c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 (#4094) 2023-01-01 05:07:53 -08:00
CommanderRoot c2280e94cf
filer.store.mysql: Replace deprecated upsert syntax (#4096) 2023-01-01 05:06:57 -08:00
CommanderRoot 265a56630b
filer.store.mysql: Escape table columns in SQL query (#4095) 2023-01-01 05:06:41 -08:00
Guo Lei ed5f3f073b
add more help message, in case of misunderstanding (#4092) 2022-12-29 07:05:05 -08:00
Guo Lei d8cfa1552b
support enable/disable vacuum (#4087) 
* stop vacuum

* suspend/resume vacuum

* remove unused code

* rename

* rename param
 2022-12-28 01:36:44 -08:00
Guo Lei 5f993a5f20
Fix ide warning (#4082) 
fix warning message from IDE
"redundant type from array, slice, or map composite literal"
 2022-12-26 00:03:47 -08:00