gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
6087 commits 28 branches 278 tags 135 MiB
Commit graph

3974 commits

Author SHA1 Message Date
Chris Lu 3bb640b786 add -force option following #2228 2021-08-01 20:03:05 -07:00
byunghwa.yun 9e839cb5cc Add force option in volume.deleteEmpty command 2021-08-02 11:37:20 +09:00
Chris Lu 56ee1d5ef1 2.61 2021-08-01 15:50:19 -07:00
Chris Lu 81b255df8b Merge branch 'master' into add_remote_storage 2021-08-01 15:44:21 -07:00
Chris Lu 58bc3ecf47 add default quietFor value 2021-08-01 15:36:06 -07:00
Chris Lu 2ca1839d77 shell: add volume.deleteEmpty command 2021-08-01 15:33:45 -07:00
Chris Lu 89933c46d2 s3: skip hidden directories in /buckets folder 2021-08-01 12:28:08 -07:00
Chris Lu 1c7e404abe remove buckets folder option 
the related code still works for old deployments
 2021-08-01 12:23:16 -07:00
Chris Lu cb1dbd3135 refactor 2021-08-01 11:53:46 -07:00
Chris Lu 1161c70dab Merge branch 'master' into add_remote_storage 2021-08-01 01:10:13 -07:00
Chris Lu 9cc84a910f volume: deletion can skip volume size checking 
fix https://github.com/chrislusf/seaweedfs/issues/2225
 2021-08-01 00:32:51 -07:00
Chris Lu 767edd3c08 rename 2021-07-31 23:52:09 -07:00
Chris Lu 9df7d16791 read <- remote_storage 2021-07-31 22:39:38 -07:00
Chris Lu f5a69a0e44 Merge branch 'master' into add_remote_storage 2021-07-31 09:39:48 -07:00
Chris Lu 6ba65c3382 customizable debug port 2021-07-31 09:18:41 -07:00
byunghwa.yun 5d4438a72f Fix typo 2021-07-31 22:23:52 +09:00
Chris Lu 1ff8285d82 debug from any server 2021-07-31 02:13:21 -07:00
Chris Lu 0d6f45cb46 add debug option to generate full goroutine stack dump 2021-07-31 02:00:01 -07:00
Chris Lu a3290faf17 shell command to calculate size / number of volumes in a collection 
fix https://github.com/chrislusf/seaweedfs/issues/2224
 2021-07-30 15:49:46 -07:00
Chris Lu c6f992b2a3 remove dead code 2021-07-30 15:18:01 -07:00
Chris Lu 899963ac20 remote storage location changed to struct 2021-07-29 02:08:55 -07:00
Chris Lu c090d6bb25 add ReadRemote(), add read remote setup when filer starts 2021-07-28 22:43:12 -07:00
Chris Lu 4deac06da0 log fix 2021-07-28 22:25:04 -07:00
divanikus 5c6270a93a shell: ability to use wildcards for collections, all collections if ommited 2021-07-27 23:53:01 +03:00
Chris Lu 035b0bae29 refactor 2021-07-27 03:32:24 -07:00
Chris Lu 1752eeb538 remote.mount saves the mapping 2021-07-27 03:26:35 -07:00
Chris Lu 4b94b03d90 directory to remote storage mapping 2021-07-27 01:16:28 -07:00
Chris Lu 99b599aa8a remote.mount 2021-07-26 22:53:44 -07:00
Chris Lu 5dede5d38d 2.60 2021-07-25 22:09:09 -07:00
Chris Lu 35f70c51b0 refactor 2021-07-25 03:37:37 -07:00
Chris Lu 72eec84167 shell: add fs.mkdir 2021-07-25 02:37:39 -07:00
byunghwa.yun 2b28a818f1 Fix mysql sql for batch delete 2021-07-25 11:06:14 +09:00
Chris Lu 10464f47d0
Merge pull request #2217 from combineads/add_webp_format 2021-07-24 09:42:41 -07:00
byunghwa.yun 7374b5b473 Add webp extension for resizing 2021-07-24 14:32:31 +09:00
byunghwa.yun 2595f269d1 Add the webp image type 2021-07-24 14:26:40 +09:00
Chris Lu ac28611817 snowflake sequencer need an unique id 
fix https://github.com/chrislusf/seaweedfs/issues/2213
 2021-07-23 20:54:03 -07:00
Chris Lu 5c14da0f1e filer.copy: fail early if assign request failed 
fix https://github.com/chrislusf/seaweedfs/issues/2216
 2021-07-23 20:01:43 -07:00
Chris Lu 4cc2165061 weed -h subcommand list is not sorted, makes discovery difficult 
fix https://github.com/chrislusf/seaweedfs/issues/2214
 2021-07-23 18:44:53 -07:00
Chris Lu 10fc478557 scaffold -config= should output to stdout 
fix https://github.com/chrislusf/seaweedfs/issues/2212
 2021-07-23 18:41:25 -07:00
Chris Lu 092932af8f shell: faster bucket delete 
avoid waiting for large object metadata
 2021-07-23 11:25:46 -07:00
Chris Lu bdb632fa62
Merge pull request #2211 from qieqieplus/filer-bloom-filter 
add bloom filter for filer leveldb/rocksdb
 2021-07-22 23:35:54 -07:00
qieqieplus 2b46df13f8 add bloom filter for filer leveldb/rocksdb 2021-07-23 14:05:59 -06:00
Chris Lu 60f5c0a2f5 fix security alert on github.com/dgrijalva/jwt-go 
resolve https://github.com/chrislusf/seaweedfs/security/dependabot/go.mod/github.com%2Fdgrijalva%2Fjwt-go/open
 2021-07-22 10:22:09 -07:00
Chris Lu 182288f860 filer: fix mysql, postgres batch delete error 2021-07-22 08:23:20 -07:00
Chris Lu 84d91f143f mount: hide /etc folder also from mount 2021-07-22 00:40:16 -07:00
Chris Lu 7359193e97 go fmt 2021-07-21 14:38:12 -07:00
Chris Lu cc32436d63
Merge pull request #2209 from combineads/fuse_option 
Fix error handling
 2021-07-21 12:37:20 -07:00
Chris Lu 67537ff05b
Merge pull request #2208 from combineads/add_rm_options 2021-07-21 11:08:43 -07:00
byunghwa.yun e0c7708b03 Fix error handling 2021-07-21 22:48:04 +09:00
byunghwa.yun 41d559f54e Add fs rm options 2021-07-21 21:10:36 +09:00
Chris Lu 70effac0d3 configure and store remote configurations 2021-07-21 02:24:34 -07:00
Chris Lu 3138805b33
Merge pull request #2200 from combineads/add_fs_rm 
weed shell: fs.rm remove file or directory
 2021-07-21 00:35:54 -07:00
Chris Lu a125c8fbe2
Merge pull request #2204 from nivekuil/tokenpolicy 
cassandra: use LocalQuorum for all queries
 2021-07-20 15:37:31 -07:00
nivekuil b9a67d46c5 cassandra: use LocalQuorum for all queries 
This changes this filer store from eventual to strong consistency at the cost
of read performance.
 2021-07-20 14:47:39 -07:00
Chris Lu 54c8bc8673 Merge branch 'remote_overlay' 2021-07-19 23:18:35 -07:00
Chris Lu 7ab389e7ec optimization: improve random range query for large files 2021-07-19 23:07:22 -07:00
Chris Lu 93c37cfded
Update filechunk_manifest.go 2021-07-19 18:41:41 -07:00
byunghwa.yun b23b307e08 Add fs rm 2021-07-20 08:49:28 +09:00
Chris Lu b938df97a2 remove unused parameter 2021-07-19 02:59:12 -07:00
Chris Lu 450222dd64 add remote to filer.Entry and filer_pb entry, add RemoteConf 2021-07-19 02:47:27 -07:00
Chris Lu 8dc5def435 proto: add remote 2021-07-19 01:12:31 -07:00
Chris Lu 99155e4880 add -retry option for volume.fix.replication 
fix https://github.com/chrislusf/seaweedfs/issues/2191
 2021-07-16 12:13:46 -07:00
Chris Lu 2f209675ab Added -retry option for volumeServer.evacuate 
related to https://github.com/chrislusf/seaweedfs/issues/2191
 2021-07-16 12:08:21 -07:00
Chris Lu fb7a1be1c4 refactor 2021-07-16 11:47:37 -07:00
nivekuil 2faf96f002 cassandra: Use LocalOne instead of One consistency 2021-07-16 04:29:46 -07:00
Chris Lu 4be5b4ff5c [volume.check.disk] skip errors and check next volumes 
fix https://github.com/chrislusf/seaweedfs/issues/2129
 2021-07-16 02:03:32 -07:00
Chris Lu e95166d739 S 2021-07-15 17:43:38 -07:00
Chris Lu a45bbc0b75 2.59 2021-07-15 15:52:22 -07:00
Chris Lu f0042f62dd readable logs 2021-07-15 12:17:48 -07:00
Chris Lu 18c40686d9 s3: multipart upload miss data if file is chunked in 4MB 
fix https://github.com/chrislusf/seaweedfs/issues/2195
 2021-07-15 11:56:28 -07:00
Chris Lu 5a838dbe53 fix compilation 2021-07-13 11:40:21 -07:00
Chris Lu 6103649ffb shell: volume.check.disk adds retries in case the volumes are just moved 
related to https://github.com/chrislusf/seaweedfs/issues/2194
 2021-07-13 11:19:56 -07:00
Chris Lu 49c66e88a0 volume: change all writes to fsync during graceful stopping 
fix https://github.com/chrislusf/seaweedfs/issues/2193
 2021-07-13 01:29:57 -07:00
Chris Lu 01adc567aa shell: volume.fsck deletes a volume is the volume has only orphaned data. 
fix https://github.com/chrislusf/seaweedfs/issues/2190
 2021-07-12 11:22:00 -07:00
Chris Lu 297b41266b 2.58 2021-07-12 01:33:47 -07:00
Chris Lu b194f91f47 add version to filer configuration response 2021-07-12 01:23:20 -07:00
bingoohuang 9e48bff8ee fix typo in fs.configure help message text 2021-07-12 13:51:21 +08:00
Chris Lu d013d6d968 shell: volume.fsck "reallyDeleteFromVolume" should send padded file ids to delete 
fix https://github.com/chrislusf/seaweedfs/issues/2188
 2021-07-10 23:16:06 -07:00
Chris Lu 3d624d1e16 rename 2021-07-09 03:19:21 -07:00
Chris Lu ecce300964 s3 config read via grpc 2021-07-09 02:48:03 -07:00
Chris Lu da7bd62822 /etc files are stored inside metadata store 2021-07-09 02:33:14 -07:00
Chris Lu a6d73e0a66
Merge pull request #2185 from bingoohuang/master 
show RemoteVolumes/EcVolumes only if it is not empty
 2021-07-06 00:28:07 -07:00
bingoohuang ed57a55eae show RemoteVolumes/EcVolumes only if it is not empty 2021-07-06 15:20:18 +08:00
Chris Lu 44b50b2fdf another fix related to 64 bit alignment 
fix https://github.com/chrislusf/seaweedfs/issues/2177
 2021-07-05 16:30:43 -07:00
Chris Lu 141388367e
Merge pull request #2183 from bingoohuang/master 
extract embed html of master/volume/filer ui to separate files
 2021-07-05 03:22:26 -07:00
bingoohuang 44a2538f67 extract embed html of master/volume/filer ui to separate files 2021-07-05 18:09:44 +08:00
Chris Lu e2da647fa3
Merge pull request #2182 from qieqieplus/fix-event-notification 
Subscribe meta data: sync empty notification with timestamp
 2021-07-05 02:00:53 -07:00
qieqieplus 233103f6b2 sync empty notification with timestamp 2021-07-05 16:01:16 +08:00
bingoohuang 5dbbe19c8b extract embed toml example to separate files 2021-07-05 11:16:49 +08:00
Chris Lu 5bcc77b46c volume: default readMode to proxy 2021-07-03 15:55:56 -07:00
Chris Lu f5fa0b08fd 2.57 2021-07-03 15:10:57 -07:00
Chris Lu a024254ad7 logging 2021-07-03 14:51:01 -07:00
Chris Lu d39b2689a5 S3 authorization: StreamingSigned enforces access control 
fix https://github.com/chrislusf/seaweedfs/issues/2180
 2021-07-03 14:50:53 -07:00
Chris Lu fa0dab6029 mount: rename also recursively move file handles 
related to https://github.com/chrislusf/seaweedfs/issues/2169
 2021-07-03 02:59:35 -07:00
Chris Lu 8fe75692ee volume: address "unaligned 64-bit atomic operation" 
fix https://github.com/chrislusf/seaweedfs/issues/2177
 2021-07-02 13:57:43 -07:00
Chris Lu 2cb8b31ea7
Merge pull request #2179 from nivekuil/tokenpolicy 
Cassandra: Use TokenAwareHostPolicy with fallback to localDC by default
 2021-07-02 13:51:53 -07:00
nivekuil 8425705643 Cassandra: Use TokenAwareHostPolicy by default with fallback 
See https://pkg.go.dev/github.com/gocql/gocql#hdr-Data_center_awareness_and_query_routing
 2021-07-02 13:50:01 -07:00
Chris Lu d8bda0b229 locate the weed binary 2021-07-02 13:33:17 -07:00
danielflira 586e066897 fix weed fuse parameters parsing 2021-07-02 16:52:52 -03:00
lyg 1a4db87e19 fix s3 metadata error with multipart upload 2021-07-02 11:00:42 +08:00
Chris Lu c6d4c16079 S3: add metadata with multipart upload 
fix https://github.com/chrislusf/seaweedfs/issues/2173
 2021-07-01 19:12:11 -07:00
Chris Lu 2420c60fc4 log reading adds delay between retries 2021-07-01 14:01:25 -07:00
Chris Lu 067eb15e70 remove debug messages 2021-07-01 01:24:07 -07:00
Chris Lu b624090398 go fmt 2021-07-01 01:21:14 -07:00
Chris Lu 215b169562 mount: recursively rename locally 2021-07-01 01:19:31 -07:00
Chris Lu c795183887 return node itself as directory handler 2021-07-01 01:19:31 -07:00
Chris Lu 30dbe98dff Update Makefile 2021-07-01 01:19:31 -07:00
Chris Lu 07f20155fd small optimization 2021-07-01 01:19:31 -07:00
danielflira a66112c9d2 configurable fusermount path 2021-07-01 00:07:54 -03:00
Kevin Liu 253b4ed898
remote -> proxy in server to match volume flags 
noticed this was missed in https://github.com/chrislusf/seaweedfs/pull/2168
 2021-06-30 13:05:45 -07:00
zhangsong 20d33ae025 add proxy mode to read non-local volumes 2021-06-30 18:33:18 +08:00
zhangsong 7566782c2e add proxy mode to read non-local volumes 2021-06-30 17:28:37 +08:00
陈杨文 6f683e6572
also failed on mips64 
alpine support mips64, will try to build on next version
 2021-06-30 04:36:00 +08:00
Chris Lu 41db292332 skip s390 ppc64le 
due to https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/22703
 2021-06-29 10:06:38 -07:00
Chris Lu 3668d10664 range query for mp4 video play 
fix https://github.com/chrislusf/seaweedfs/issues/2156
 2021-06-29 02:13:29 -07:00
Chris Lu 24e11d1e90 look back when adding to sorted values 
look back when adding to sorted values, before adding it to overflow
 2021-06-28 22:46:49 -07:00
Chris Lu fc8dd58aea volume: large_volume version has bug when using in memory index 
fix https://github.com/chrislusf/seaweedfs/issues/2162
 2021-06-28 15:48:07 -07:00
Chris Lu a2979aa051 2.56 2021-06-27 23:33:45 -07:00
Chris Lu 08377fecb8 fixes 2021-06-27 23:32:57 -07:00
Chris Lu 17477b37d5 sleep before re-reading the messages 
If there are no more metadata changes and the client disconnects, it would go into a busy loop without this fix.
 2021-06-27 06:31:04 -07:00
Chris Lu c764596f96 filer: slow metadata topic read may lose some change events 
fix https://github.com/chrislusf/seaweedfs/issues/2117
 2021-06-27 05:54:16 -07:00
Chris Lu cc7714fdbe logging changes to debug 2021-06-27 05:54:16 -07:00
thephoenixofthevoid 8d70ba2eaa Fix: Loosing environment variables at StartProcess 2021-06-26 15:20:27 +03:00
Chris Lu 1cac2f2278 shell: volume.fsck checks missing chunks in filer 
fix https://github.com/chrislusf/seaweedfs/issues/2154
 2021-06-24 23:56:24 -07:00
Chris Lu 9dd09bbb33 refactor 2021-06-24 17:22:53 -07:00
Chris Lu c2e0a75c1f adjust logs 2021-06-24 12:46:00 -07:00
Chris Lu 78b1fb921c adjust log level 2021-06-23 20:59:54 -07:00
Chris Lu 05af54ad10 2.55 
trigger migrated travis build
 2021-06-23 00:41:04 -07:00
Chris Lu d4b0dcf442
Merge pull request #2147 from jonaz/process_metrics 
Add process metrics of weed itself
 2021-06-22 04:13:17 -07:00
Jonas Falck 829b195084 Add process metrics of weed itself 2021-06-22 13:09:42 +02:00
Chris Lu d474ce6fe3 master: avoid repeated leader redirection 
fix https://github.com/chrislusf/seaweedfs/issues/2146
 2021-06-21 22:56:07 -07:00
danielflira f1d207a0fe start weed with mount in background 2021-06-20 02:48:46 -03:00
Chris Lu b3eb4fecc7 2.54 2021-06-19 03:48:15 -07:00
Chris Lu f24bb9e688 mount: fix for deletion stopped working since 2.53 
fix https://github.com/chrislusf/seaweedfs/issues/2138

due to 4d0cbd2700
 2021-06-19 03:46:39 -07:00
Chris Lu 88d52adfdd remove unused fields 2021-06-18 15:35:22 -07:00
Chris Lu 56eb522b13 fix stats when a collection is deleted 2021-06-15 21:11:31 -07:00
Chris Lu 055374a50b FUSE: skip flushing if file is deleted 
related to https://github.com/chrislusf/seaweedfs/issues/2110
 2021-06-15 12:45:23 -07:00
Konstantin Lebedev 6aa1a56ec8 avoid crashes Galera Cluster 
https://github.com/chrislusf/seaweedfs/issues/2125
 2021-06-15 18:12:39 +05:00
Chris Lu ebe971da2e
Merge pull request #2127 from nagy/check-history-error 
Check for history file errors as well
 2021-06-15 03:29:21 -07:00
Chris Lu 7a81caa31e 2.53 2021-06-13 17:12:34 -07:00
Chris Lu 4d0cbd2700 skip cookie checking if from grpc api 2021-06-13 16:16:11 -07:00
Chris Lu 7225cb4ac5 add block and mutex profiling 2021-06-13 16:15:54 -07:00
Chris Lu 28a4a1f8d6 fix for mysql2 postgres2 on fast dropping buckets 2021-06-13 07:31:56 -07:00
Chris Lu 1e76fc994a filer.copy: zero fileSize for directories 2021-06-13 07:09:06 -07:00
Chris Lu ed6aa13520 minor 2021-06-12 18:36:25 -07:00
Chris Lu ee6c67682c minor 2021-06-12 02:52:41 -07:00
Chris Lu 9357911a95 remove all bucket metadata 
fix https://github.com/chrislusf/seaweedfs/issues/2118
 2021-06-10 23:37:54 -07:00
Chris Lu b71c3cfba4 avoid possible empty object 2021-06-10 22:17:53 -07:00
Chris Lu 310e31424e adjust the error output 
fix https://github.com/chrislusf/seaweedfs/issues/2123
 2021-06-10 21:55:13 -07:00
Chris Lu 8b382a8209 refactor 2021-06-10 21:50:21 -07:00