gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
5293 commits 28 branches 278 tags 135 MiB
Commit graph

3365 commits

Author SHA1 Message Date
Chris Lu 98c93ca465 avoid file.Stat() 
avoid one Syscall, but did not help on performance though
 2021-02-20 09:44:17 -08:00
bingoohuang ee082ae402 revert comment temporarily 2021-02-20 16:59:38 +08:00
bingoohuang ee7cdf3668 revert ErrXyz to ErrorXyz temporarily. 2021-02-20 16:58:08 +08:00
bingoohuang 50df484d86 support command line arguments in a configuration file so we can weed master -config=master.conf, weed volume -config=volume.conf and etc. 2021-02-20 16:52:57 +08:00
bingoohuang f69356f589
Merge pull request #3 from chrislusf/master 2021-02-20 16:45:02 +08:00
Chris Lu 565f7a6e72 Update data_node.go 2021-02-19 14:22:36 -08:00
Chris Lu a37473ae60 add back volume ids 
address https://github.com/chrislusf/seaweedfs/issues/1792#issuecomment-782339576
 2021-02-19 14:22:12 -08:00
Chris Lu a1210d1e8d Revert "volume: avoid file.stat(), file.seek() if possible during writes" 
This reverts commit c78409a598.
 2021-02-19 03:56:27 -08:00
Chris Lu b961cd6208 add WIP message 2021-02-19 03:39:19 -08:00
Chris Lu f465d63b5d grpc should fail when heart beating to master 
fix https://github.com/chrislusf/seaweedfs/issues/1820
 2021-02-19 03:37:29 -08:00
Chris Lu c576ad04ac fix volume server display for volumes 2021-02-19 01:38:56 -08:00
bingoohuang 7ffe736d20 renaming to ErrXyz 2021-02-19 15:59:55 +08:00
Chris Lu c78409a598 volume: avoid file.stat(), file.seek() if possible during writes 2021-02-18 23:22:54 -08:00
Chris Lu 29575dadc5 remove extra files 2021-02-18 19:14:58 -08:00
Chris Lu 73958e357d add descriptive error if no free volumes 2021-02-18 19:10:20 -08:00
Chris Lu 776f497469 filer: fs.configure should try to read from entry.content also 
related to https://github.com/chrislusf/seaweedfs/issues/1792
 2021-02-18 17:07:02 -08:00
Chris Lu e1992c83de webdav adds disktype 2021-02-18 15:45:44 -08:00
Chris Lu c2ad6f1047 webdav add replication setting 
fix https://github.com/chrislusf/seaweedfs/issues/1817
 2021-02-18 12:15:09 -08:00
Chris Lu 816ef0ed30 add more help message 
fix https://github.com/chrislusf/seaweedfs/issues/1816
 2021-02-18 10:34:49 -08:00
bingoohuang 352ac2f271 Merge remote-tracking branch 'origin/master' 2021-02-18 14:05:51 +08:00
bingoohuang eab6e31d34 use backticks instead of double quotes to avoid escaped additionally in regex 2021-02-18 14:05:28 +08:00
Chris Lu 3575d41009 go fmt 2021-02-17 20:57:08 -08:00
Chris Lu 6daa932f5c refactoring to get master function, instead of passing master values directly 
this will enable retrying later
 2021-02-17 20:55:55 -08:00
Chris Lu dd9f3a0104 add sleep between upload retries 2021-02-17 20:55:11 -08:00
Chris Lu aa17311063 s3: set filename when uploading file chunks 2021-02-17 20:54:53 -08:00
Chris Lu 3f8b0da677 filer: do not print password on error 
fix https://github.com/chrislusf/seaweedfs/issues/1809
 2021-02-17 02:13:52 -08:00
Chris Lu cd866664a8 skip JWT if fileId is empty 
related to https://github.com/chrislusf/seaweedfs/issues/1808
 2021-02-16 15:39:12 -08:00
Chris Lu 632c94d438 Merge branch 'extend_to_disk_type' 2021-02-16 11:06:13 -08:00
Chris Lu 0611233f16 adjust printing 2021-02-16 10:55:30 -08:00
Chris Lu 68775d29e3 fix tests 2021-02-16 10:51:03 -08:00
Chris Lu 7403cd43c5 only use "" for hdd to avoid two values for the same thing 2021-02-16 10:50:42 -08:00
Chris Lu b314d78e97 fix print 2021-02-16 10:48:28 -08:00
Chris Lu 53ca7e66ef avoid dead lock 2021-02-16 10:48:16 -08:00
Chris Lu 38efc6f572 simplify 2021-02-16 10:47:32 -08:00
Chris Lu ad70107eb4 ec balance among nodes with HDD 2021-02-16 06:22:50 -08:00
Chris Lu 3097b9a9b7 fix existence checking 2021-02-16 05:59:43 -08:00
Chris Lu ebf320ec22 avoid divided by zero 2021-02-16 05:59:24 -08:00
Chris Lu a0c6db361c avoid nil 2021-02-16 05:33:38 -08:00
Chris Lu 36f95e50a9 avoid possible nil disk info 2021-02-16 05:13:48 -08:00
Chris Lu 43101ccea0 move to the empty nodes first 2021-02-16 04:27:16 -08:00
Chris Lu 38bbef7ec1 avoid nil 2021-02-16 04:16:46 -08:00
Chris Lu cb9cc29518 volume.list display; fix updating maxVolumeCount for disk 2021-02-16 03:55:24 -08:00
Chris Lu 3fe628f04e use hdd instead of empty string 2021-02-16 03:03:00 -08:00
Chris Lu f8446b42ab this can compile now!!! 2021-02-16 02:47:02 -08:00
Chris Lu b9b5b932c5 filer: postgres2 mysql2 avoid repeatedly creating tables 2021-02-15 16:19:24 -08:00
Chris Lu 71f0c19515 2.26 2021-02-15 13:38:38 -08:00
Chris Lu 8b169aa47c s3: Critical - S3 Delete Multiple Objects, delete the bucket 
fix https://github.com/chrislusf/seaweedfs/issues/1806
 2021-02-15 13:38:29 -08:00
Chris Lu 50202d1391
Merge pull request #1805 from LazyDBA247-Anyvision/master 
better postgresql connection pool management
 2021-02-14 22:26:29 -08:00
Chris Lu 8726ef34c1 2.25 2021-02-14 22:23:43 -08:00
LazyDBA247-Anyvision 7f458d5e78 better postgres connection pool management 
adding SetConnMaxLifetime configuration (https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime)
to enable refresh of stale connections.
 2021-02-15 07:45:09 +02:00
Chris Lu 0bc3a1f9e8 disk type only supports hdd and ssd, not ready for random tags yet 2021-02-14 11:38:43 -08:00
LazyDBA247-Anyvision 51b4963e2e postgres2 & memsql2 
add escape (quote identifiers) for the dynamic sql
so tables (collections) with special characters will work.
 2021-02-14 13:14:36 +02:00
Chris Lu 876dbe3d26 collect disk types 2021-02-13 23:25:16 -08:00
Chris Lu 861f8fab9b refactoring 2021-02-13 22:34:12 -08:00
Chris Lu 881f05f5a0 adjust help message 2021-02-13 16:13:52 -08:00
Chris Lu ef76365ec2 adjust help message 2021-02-13 15:47:08 -08:00
Chris Lu 7ce647f27e support customizable disk type 2021-02-13 15:42:42 -08:00
Chris Lu 4bd8a692d8 disk type can be generic tags 2021-02-13 13:50:14 -08:00
Chris Lu 712b3e9e53 mount: release need to avoid nil reader 
fix https://github.com/chrislusf/seaweedfs/issues/1803
 2021-02-13 13:39:37 -08:00
Chris Lu 4ce56bac08 fix format error 2021-02-12 06:44:26 -08:00
Chris Lu 0604afcda7 mount: remove unnecessary flush on release 2021-02-12 06:01:04 -08:00
Chris Lu 7d9dc3c6a2 use fasthttp lib to read 2021-02-12 05:59:52 -08:00
Chris Lu 487e435679 adjust http max idle connections per host 
related to https://github.com/chrislusf/seaweedfs/issues/1802
 2021-02-12 03:47:15 -08:00
Chris Lu 0f426ce34d
Merge pull request #1801 from kmlebedev/recoveringRabbitMQ 
Do reconnect to RabbitMQ
 2021-02-11 01:05:03 -08:00
Konstantin Lebedev 94eac4f00e Do reconnect to RabbitMQ 
https://github.com/google/go-cloud/issues/2958
https://github.com/chrislusf/seaweedfs/issues/1773
 2021-02-11 13:59:36 +05:00
Chris Lu 885ca34748 volume: fail fast if idx files are missing 
fix https://github.com/chrislusf/seaweedfs/issues/1796
 2021-02-11 00:44:40 -08:00
Chris Lu a0e84c4fbc go fmt 2021-02-10 23:41:05 -08:00
Chris Lu c959aaa19b mount: load directory based on correct filer path 
fix https://github.com/chrislusf/seaweedfs/issues/1799
 2021-02-10 23:14:18 -08:00
Konstantin Lebedev 0cfed8c3cb restart replication if RabbitMQ connection closed 
https://github.com/google/go-cloud/issues/2958
 2021-02-10 15:29:05 +05:00
Chris Lu 770393a48c volume: add capability to change disk type when moving a volume 2021-02-09 23:58:08 -08:00
Chris Lu 821c46edf1 Merge branch 'master' into support_ssd_volume 2021-02-09 11:37:07 -08:00
Chris Lu a833021132 fix refactoring left over 2021-02-07 23:03:03 -08:00
Chris Lu 5e4b5109dd 2.24 2021-02-07 22:29:43 -08:00
Chris Lu 15c60cbb26 close the grpc connection after 10 hours 
related to https://github.com/chrislusf/seaweedfs/issues/1782
 2021-02-07 03:50:01 -08:00
bingoohuang 7256902fb0 fix typo offset.ToAcutalOffset to offset.ToActualOffset 2021-02-07 12:11:51 +08:00
bingoohuang 94ea3bd3a5 renaming NeedleMapType to NeedleMapKind 2021-02-07 09:00:03 +08:00
Chris Lu c3af72d950 fier store: fix elastic search regression 
fix https://github.com/chrislusf/seaweedfs/issues/1774
 2021-02-04 01:30:14 -08:00
Konstantin Lebedev 2fb4c60540 fix log warn 2021-02-03 21:34:32 +05:00
Konstantin Lebedev 24ec17219b RabbitMQ delay retry with Dead Letter Exchange 
https://github.com/chrislusf/seaweedfs/issues/1773

https://github.com/google/go-cloud/issues/2952
 2021-02-03 21:32:30 +05:00
Chris Lu a331bbb3ae filer: should return 204 on DELETE to nonexistent file 
related to

https://github.com/chrislusf/seaweedfs/issues/1776
https://github.com/chrislusf/seaweedfs/issues/1160
 2021-02-03 00:40:31 -08:00
Chris Lu 5353e38469 S3: should return 204 on DELETE to nonexistent file 
fix https://github.com/chrislusf/seaweedfs/issues/1776
 2021-02-03 00:35:44 -08:00
Chris Lu 2396ac234c filer: fix elastic search pagination 
possible fix for https://github.com/chrislusf/seaweedfs/issues/1774
 2021-02-02 11:34:12 -08:00
Chris Lu 609daaf387 s3: DeleteMultipleObjectsHandler clean up leftover empty folders 
fix https://github.com/chrislusf/seaweedfs/issues/1772
 2021-02-01 10:49:17 -08:00
Chris Lu d67ccb66c0 webdav: can start together with "weed server" or "weed filer" 2021-01-31 22:16:52 -08:00
Chris Lu 76430790b9 webdav: cache to version specific folder 2021-01-31 22:02:03 -08:00
Chris Lu 318a3d2efc 2.23 2021-01-31 20:18:06 -08:00
Chris Lu 4be51c0701 filer: leveldb and hbase may miss files when listing large directories more than 1024 
fix https://github.com/chrislusf/seaweedfs/issues/1768
 2021-01-31 20:11:44 -08:00
Chris Lu 1102ae32c4 fix concurrent map reads 2021-01-31 18:26:26 -08:00
Chris Lu 314dc1c957 filer: etcd store fix listing 
fix https://github.com/chrislusf/seaweedfs/issues/1767
 2021-01-30 13:38:44 -08:00
Chris Lu d475c89fcc go fmt 2021-01-28 15:23:46 -08:00
Chris Lu 19295600f9 mount: change option name to volumeServerAccess, with publicUrl and filerProxy modes 2021-01-28 15:23:16 -08:00
Chris Lu 990fa69bfe add back AdjustedUrl() related code 2021-01-28 14:36:29 -08:00
Chris Lu 9292796ec2 fmt 2021-01-28 14:29:14 -08:00
Chris Lu c2bf1a88ac delete from the deepest directory first when checking empty folders 2021-01-28 14:28:40 -08:00
Chris Lu e9d8201925 s3: batch purge empty folders 2021-01-28 13:20:06 -08:00
Chris Lu cf252fc0cd mount: report error when Flush() 
The error in Release() is not working.

See https://github.com/jaderhs/libfuse/blob/master/FAQ

related to https://github.com/chrislusf/seaweedfs/issues/1765
 2021-01-28 04:46:37 -08:00
Chris Lu 9a06c35da4 replicate: incremental sink only contains new and updated files 
address da08402ba2
 2021-01-28 02:39:22 -08:00
Chris Lu da08402ba2 replicate: use creation time for local incremental file sink 
related to https://github.com/chrislusf/seaweedfs/pull/1762
 2021-01-28 02:17:41 -08:00
Chris Lu 822f1ade9d
Merge pull request #1762 from kmlebedev/backupsink 
replication to create time date directory
 2021-01-28 02:03:33 -08:00
Konstantin Lebedev be1062b7fc rename in scaffold 2021-01-28 14:59:20 +05:00