gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
4127 commits 28 branches 278 tags 135 MiB
Commit graph

2527 commits

Author SHA1 Message Date
Chris Lu edb9d65e05
Merge pull request #1431 from kmlebedev/wip-prefix-search 
Wip prefix search
 2020-08-31 10:22:14 -07:00
Konstantin Lebedev 9a195bebfd accurate limit 2020-08-31 22:13:13 +05:00
Konstantin Lebedev 18b98f9747 not convert err to string 2020-08-31 21:55:18 +05:00
Konstantin Lebedev 22f32e75c4 rename 2020-08-31 21:52:05 +05:00
Konstantin Lebedev a8de332beb fix 2020-08-31 21:45:44 +05:00
Konstantin Lebedev 7f013d77d6 add var UnsupportedListDirectoryPrefixedErr 2020-08-31 21:43:32 +05:00
Konstantin Lebedev 60a86cfe04 add const 2020-08-31 21:35:16 +05:00
Konstantin Lebedev 82ea121d09 rm func 2020-08-31 18:50:06 +05:00
Konstantin Lebedev 2ea638f865 filer store wrapper can implement the logic to filter by prefi 2020-08-31 17:13:56 +05:00
Chris Lu 408e339c53 also delete the manifest chunk itself 2020-08-31 00:16:03 -07:00
Chris Lu dd52037c27 1.92 2020-08-30 22:56:35 -07:00
Chris Lu 33d8c6c617 change log to warning 2020-08-30 21:01:44 -07:00
Chris Lu e62d7f221f a bit more logs 2020-08-30 21:00:24 -07:00
Chris Lu 22fe4ae573 mount: fix for UrBackup 2020-08-30 21:00:09 -07:00
Chris Lu ae84a9ee2e add logs 2020-08-30 20:59:04 -07:00
Chris Lu a41588279a change log level 5 to 4 2020-08-30 20:12:04 -07:00
Chris Lu 56244fb9a1 fix hard coded host address 2020-08-30 10:23:35 -07:00
Chris Lu f2a8574448 filer and mount deletion resolves manifest chunks also 2020-08-30 02:07:14 -07:00
Chris Lu 99ecf63276 go fmt 2020-08-29 22:28:33 -07:00
Chris Lu ef75ce8a34 use UTC for all time related folders 2020-08-29 21:01:14 -07:00
Chris Lu b69cb74c03 read meta logs by timestamp 
pass in event ts when moving logs
meta aggregator reads in memory logs only
 2020-08-29 17:37:19 -07:00
Chris Lu 063c9ddac5 adjust logs 2020-08-29 11:56:22 -07:00
Chris Lu ca658a97c5 add signatures to messages to avoid double processing 2020-08-28 23:48:48 -07:00
Chris Lu 63ad1abcce watch: follow changes for a pattern 2020-08-28 19:43:04 -07:00
Chris Lu b7a654a318 file mime avoid saving application/octet-stream 2020-08-28 19:42:40 -07:00
Chris Lu 707192f966 fix help message 2020-08-27 00:13:38 -07:00
Chris Lu 0be6863c87 rename 2020-08-26 22:40:15 -07:00
Chris Lu 50f7e2e89f add log messages 2020-08-26 09:51:20 -07:00
Chris Lu 6bc618051c log message 2020-08-26 09:32:56 -07:00
Chris Lu 9dae4e9d3e fix logs 2020-08-26 09:31:11 -07:00
Chris Lu 4c1a3187bf fix log error 2020-08-26 09:25:54 -07:00
Chris Lu ab759f0ec2 erasure coding: fix EC error if multiple disks are configured in one volume server 2020-08-26 09:16:58 -07:00
Chris Lu d1cf39f180 fix logging 2020-08-26 08:55:15 -07:00
Chris Lu 83012f543a s: avoid possible completeMultipartUpload re-tries that can overwrite the uploaded file 2020-08-26 08:50:24 -07:00
Konstantin Lebedev 464d4c82ec stop send heartbeat before stop volume server 2020-08-26 16:16:11 +05:00
Chris Lu 1901f15cd2 volume server: remove whitelist for status checking 
Volume liveness/readiness not work if jwt enabled, I try use param
-whiteList
but jwt breaks
 2020-08-25 02:27:47 -07:00
Chris Lu 81e5124faf s3: list bucket permission change from admin to read 
fix https://github.com/chrislusf/seaweedfs/issues/1430
 2020-08-24 11:22:45 -07:00
Chris Lu 5f55a87101 close http response 2020-08-24 00:32:44 -07:00
Chris Lu 0b301b0b47 consume all response body in order to release requests 2020-08-23 21:32:29 -07:00
Chris Lu 2cdad77d82 fix checkin error 2020-08-23 18:30:11 -07:00
Chris Lu 81a8aa6581 1.91 2020-08-23 17:28:40 -07:00
Chris Lu 51346a5930 always keep the manifest list of chunks 2020-08-23 17:15:12 -07:00
Chris Lu c1d1677a28 keep manifest chunks forever 2020-08-23 16:59:01 -07:00
Chris Lu 98175548c2 adjust deletion ordering 2020-08-23 16:27:12 -07:00
Chris Lu f0e50e7b26 fix test 2020-08-23 16:12:34 -07:00
Chris Lu aee27ccbe1 multiple fixes 
* adjust isOpen count
* move ContinuousDirtyPages lock to filehandle
* fix problem with MergeIntoVisibles, avoid reusing slices
* let filer delete the garbage
 2020-08-23 15:48:02 -07:00
Chris Lu c19245886c simpler test 2020-08-23 14:20:27 -07:00
Chris Lu df816a58fe add tests 2020-08-23 14:09:25 -07:00
Chris Lu 77393d3d30 add file handle locking when changing file entry 2020-08-23 00:35:50 -07:00
Chris Lu d60bcbf08a sorting chunks 2020-08-23 00:00:36 -07:00
Chris Lu 5e6b714836 add random test 2020-08-22 16:33:00 -07:00
Chris Lu 42ddbcc3a2 SQL insert falls back to update 
fix https://github.com/chrislusf/seaweedfs/issues/1429
 2020-08-20 07:52:46 -07:00
Chris Lu f48567c5c6 remove unused function 2020-08-19 22:53:49 -07:00
Chris Lu c45ba5d7d4 fix listObjectsV2 response format 
fix https://github.com/chrislusf/seaweedfs/issues/1426 issue 1
 2020-08-19 12:08:04 -07:00
James Hartig 3ccfa4c6ad Added VolumeMarkWritable and VolumeStatus grpc methods 
This is necessary for copy to mark as read-only and then restore the
original state afterwards.
 2020-08-19 11:42:56 -04:00
Chris Lu 3b4b1d4a77 fix tests 2020-08-19 01:37:56 -07:00
Chris Lu 839634097f also do flush on release 2020-08-19 01:27:10 -07:00
Chris Lu c27e18aa6a read possible old deleted chunks 2020-08-19 00:43:07 -07:00
Chris Lu a78772d5ea avoid shutdown in the middle of running 2020-08-19 00:42:02 -07:00
Chris Lu ae9bc4a508 logs 2020-08-18 23:42:26 -07:00
Chris Lu ed4b43b419 adjust logs 2020-08-18 23:42:09 -07:00
Chris Lu 9f1e0aeef5 delete chunks in the last step 2020-08-18 23:40:53 -07:00
Chris Lu 99d05f758c adjust logs 2020-08-18 23:39:18 -07:00
Chris Lu fe01191b5b support read option readDeleted=true 2020-08-18 19:22:16 -07:00
Chris Lu 6ccd7f0a4d refactoring 2020-08-18 18:01:37 -07:00
Chris Lu c026eb0592 refactoring 2020-08-18 17:39:29 -07:00
Chris Lu 7e91ae592c pass in option to read deleted entries 
not working yet
 2020-08-18 17:37:26 -07:00
Chris Lu 51ecb49db3 for debugging 2020-08-18 17:35:39 -07:00
Chris Lu ee11d98650 refactoring 2020-08-18 17:35:19 -07:00
Chris Lu 6a92f0bc7a refactoring to typed Size 
Go is amazing with refactoring!
 2020-08-18 17:04:28 -07:00
Chris Lu 618b2f6829 release resources only when needed to 2020-08-18 12:53:08 -07:00
Chris Lu 208849702d logs 2020-08-18 12:52:54 -07:00
Chris Lu 1fcd083db3 printout data size 2020-08-18 09:09:29 -07:00
Chris Lu 3e5339337a minor 2020-08-18 08:50:14 -07:00
Chris Lu 85001cbec7 properly report io.EOF 2020-08-18 08:18:54 -07:00
Chris Lu 0ca45a5cbc adjust logs 2020-08-18 00:34:15 -07:00
Chris Lu cd43738245 fix reading when filling zeros 2020-08-18 00:32:01 -07:00
Chris Lu ecb3ce46be adjust error logs 2020-08-18 00:31:42 -07:00
Chris Lu 30fe424469 add one more test case 2020-08-17 22:47:27 -07:00
Chris Lu 1b68ba953b fix for out of range reads 2020-08-17 22:46:32 -07:00
Chris Lu 56fbd2c211 fix reading 2020-08-17 21:17:32 -07:00
Chris Lu 09e126bae5 refactoring: use interface 2020-08-17 20:20:08 -07:00
Chris Lu be4d42b8e2 rename 2020-08-17 20:15:53 -07:00
Chris Lu 0625e63648 count 0 as part of the reads 2020-08-17 20:14:40 -07:00
Chris Lu 0be4b6e7f0 logs 2020-08-17 16:05:40 -07:00
Chris Lu 97e54a80d4 rename variables 2020-08-17 16:05:13 -07:00
Chris Lu abdaf9958d possibly read more 2020-08-17 16:04:56 -07:00
Chris Lu 9d46c7bc78 rename 2020-08-17 11:12:10 -07:00
Chris Lu 4ccfdaeb4d prevent nil 2020-08-17 10:07:34 -07:00
Chris Lu 24c8e6bcb4 minor optimization 2020-08-17 10:03:34 -07:00
Chris Lu f5837b7000 report error first 2020-08-16 23:49:10 -07:00
Chris Lu 2ac27616bc fix possible out of range bytes 
avoid buff out of range resp.Data = buff[:totalRead]
 2020-08-16 23:47:34 -07:00
Chris Lu ff200398bb 1.90 2020-08-16 21:18:50 -07:00
Chris Lu 4ceeba9e70 streaming reads 2020-08-16 21:07:46 -07:00
Chris Lu e72953dff7 logs 2020-08-16 21:06:03 -07:00
Chris Lu 627b081b67 adjust logs 2020-08-16 16:32:22 -07:00
Chris Lu 22e5132b3a adjust log level 2020-08-16 16:25:11 -07:00
Chris Lu ee0f92a6be reduce memory allocation 2020-08-16 16:24:40 -07:00
Chris Lu 2ba817afac read randomly written data 2020-08-16 15:16:46 -07:00
Chris Lu 6111b265e7 fix compilation 2020-08-16 01:38:16 -07:00
Chris Lu 4a77f0820a clean up logs 2020-08-16 01:37:50 -07:00
Chris Lu 8c9e6eaacd fix tests 2020-08-16 01:35:52 -07:00
Chris Lu b71df82292 1.89 2020-08-16 00:57:28 -07:00
Chris Lu 1d9ea30b72 fix ViewFromVisibleIntervals 2020-08-16 00:49:26 -07:00
Chris Lu aec7f32b02 fix reader_at 2020-08-16 00:49:08 -07:00
Chris Lu 5d80fc2ec7 adjust logs 2020-08-15 21:09:31 -07:00
Chris Lu e0bfd3161a update metadata only if changed 2020-08-15 19:56:24 -07:00
Chris Lu 003d48da21 adjust logs 2020-08-15 19:55:28 -07:00
Chris Lu 6ee8d952d2 adjust log level 2020-08-15 18:24:35 -07:00
Chris Lu 3e1395b767 adjust log message 2020-08-15 17:06:16 -07:00
Chris Lu a22ee30596 fix nil 2020-08-15 17:01:42 -07:00
Chris Lu 0d60e67816 ensure meta data changes are updated 2020-08-15 14:15:07 -07:00
Chris Lu 5b43bddf20 proper deletion ordering 
delete central file store first, then delete local cache
 2020-08-15 09:33:41 -07:00
Chris Lu c647deace1 file size support set file length 
use Attr.FileSize and TotalChunkSize to determine file size
 2020-08-15 09:32:47 -07:00
Chris Lu c03bb180eb fix error reporting 2020-08-14 00:44:02 -07:00
Chris Lu eb493283dd deletion error report 2020-08-14 00:23:01 -07:00
Chris Lu a7f669044e rename also applies to open file handle 2020-08-14 00:22:49 -07:00
Chris Lu edfa73782f adjust log level 2020-08-14 00:22:21 -07:00
Chris Lu 0983060a90 increase default volume file size limit to 1024 
avoid possible large chunk size set on mount or filer
 2020-08-13 09:07:22 -07:00
Chris Lu 090612492c fix compilation 2020-08-13 00:07:56 -07:00
Chris Lu d43129d27f add back handles lock 2020-08-12 23:52:13 -07:00
Chris Lu f735d579d3 adjust comment 2020-08-12 13:22:19 -07:00
Chris Lu 8824a9755c remove directory cache 2020-08-12 13:11:04 -07:00
Chris Lu 83cad3da79 add retry file upload 2020-08-11 20:30:11 -07:00
Chris Lu 152a6cbc2b minor adjustments 2020-08-10 20:42:27 -07:00
Chris Lu f1b40567e3
Merge pull request #1419 from robin1900/master 
fix bug:  two same volumeId in different collections
 2020-08-10 20:38:55 -07:00
cheng.li01 25fbff5d52 fix bug: two same volumeId in different collections 
1, there will be two leader when master server startup in a few seconds
2, raft server will get a leader even there is only one master, so there is no need to do hard code to set the server to be leader
 2020-08-10 16:37:47 +08:00
Chris Lu e74dc4e4bc add back fs node cache for renaming 2020-08-09 21:56:09 -07:00
Chris Lu f86c7d911a remove println 2020-08-09 14:42:25 -07:00
Chris Lu 9ecc1170a3 istObjects和listObjectsV2不能查询子目录 #1418 
fix https://github.com/chrislusf/seaweedfs/issues/1418
 2020-08-09 14:35:53 -07:00
Chris Lu 3f4aff5dde s3: fix delimiter in list response 2020-08-09 09:09:35 -07:00
Chris Lu b056707770 1.88 2020-08-08 22:55:12 -07:00
Chris Lu 9832653e1d FUSE mount: proper error with deleting non empty folder 2020-08-08 21:37:36 -07:00
Chris Lu 3b1a95ac26 filer refactoring: same auto chunking logic for POST and PUT, no size limit 2020-08-08 12:02:06 -07:00
Chris Lu 67348e7b15 less noisy heartbeat logs 2020-08-08 10:53:35 -07:00
Chris Lu bee0d7e5eb lower log priority for noisy heartbeat 2020-08-08 10:52:13 -07:00
Chris Lu ae00cce4bd support POST and PUT auto chunking 2020-08-08 10:45:37 -07:00
Chris Lu bd8bfdae07 refactoring 2020-08-08 10:18:43 -07:00
Chris Lu ab6e5c0dc4 adjust error message 2020-08-08 10:18:32 -07:00
Chris Lu 828a5ae429 check signature only when auth is enabled 2020-08-08 09:11:40 -07:00
Chris Lu 20e2ac1add filer: store md5 metadata for files uploaded by filer 
fix https://github.com/chrislusf/seaweedfs/issues/1412
 2020-08-06 10:04:17 -07:00
Chris Lu 93ea0801ea volume: the variable for the master node may be stale? 
related to https://github.com/chrislusf/seaweedfs/issues/1414
 2020-08-06 09:48:54 -07:00
Chris Lu 4ecfa9879d volume: report Content-MD5 in response header 2020-08-06 05:22:53 -07:00
Chris Lu 41007ced77 remove logging 2020-08-06 04:26:29 -07:00
Chris Lu 2b74abf766 S3: configurable access for anonymous user 
fix https://github.com/chrislusf/seaweedfs/issues/1413
 2020-08-06 03:41:34 -07:00
Устюжанин Антон Александрович 9ca011e3ca ListDirectoryPrefixedEntries 2020-08-06 00:38:59 +05:00
Устюжанин Антон Александрович b231f7bdab ListDirectoryPrefixedEntries 2020-08-06 00:37:42 +05:00
Устюжанин Антон Александрович a457c308ad test ListDirectoryPrefixedEntries 2020-08-06 00:24:31 +05:00
Устюжанин Антон Александрович 2d3b355fb6 test ListDirectoryPrefixedEntries 2020-08-05 23:56:06 +05:00
Устюжанин Антон Александрович 67f134ebd7 test ListDirectoryPrefixedEntries 2020-08-05 23:44:48 +05:00