gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity
3890 commits 28 branches 278 tags 135 MiB
Commit graph

2320 commits

Author SHA1 Message Date
Chris Lu f89186a3cf better logging 2020-08-01 11:19:57 -07:00
Chris Lu 49d6dcfbc6 dynamically change the log level for reading the disk status 2020-08-01 11:19:57 -07:00
limd b41b7ea4d0 fix spark read s3 bug (sc.binaryFiles) 2020-08-01 01:08:30 +08:00
Chris Lu a6b59d50f7 1.87 2020-07-30 13:52:04 -07:00
Chris Lu eed525b717 FUSE mount: remove DirListCacheLimit 
outdated parameter
 2020-07-28 09:24:39 -07:00
Chris Lu a566bfc6e1 s3: use bucket in the domain 
fix https://github.com/chrislusf/seaweedfs/issues/1405
 2020-07-28 08:47:27 -07:00
limd 92c32f9d46 PutObject: fix create folder bug 2020-07-28 00:58:42 +08:00
Chris Lu 54cd582653 refactor 2020-07-26 12:58:58 -07:00
Chris Lu c50ec1599c auth for presigned put object part 2020-07-26 12:09:50 -07:00
Chris Lu ea0180de4e auth for presigned put operation 2020-07-26 12:06:45 -07:00
Chris Lu 3faa0aa448 rename 2020-07-25 20:09:19 -07:00
Chris Lu 023a1efdf2 set filename in Content-Disposition header 2020-07-25 20:06:40 -07:00
popstk a09fcfcf20 fix s3api auth bug 2020-07-25 17:14:49 +08:00
popstk feb52400be fix s3api copy object handler SerializationError 2020-07-25 15:52:31 +08:00
Chris Lu d40de39e75 FUSE: do not change crtime, uid, gid on save 2020-07-24 10:13:45 -07:00
Chris Lu 8dfeba8023 FUSE: skip changing to empty uid and gid during flush 
Skip uid and gid changes if request uid and gid are zero

mongodb lock file, and many interim files, has empty uid and gid

I0724 10:05:51 93643 filehandle.go:173] /db/diagnostic.data/metrics.interim fh 1333342842031408359 flush Flush [ID=0x3 Node=0x50 Uid=0 Gid=0 Pid=178] 0x10 fl=0x0 lk=0x0
 2020-07-24 10:06:43 -07:00
Chris Lu dd29b8c81d fix compilation 2020-07-24 09:36:05 -07:00
Chris Lu 5788f1f6bf remove fsnode cache 
fix https://github.com/chrislusf/seaweedfs/issues/1400
 2020-07-24 09:25:54 -07:00
Chris Lu 6962ca48d3 fix compilation 2020-07-23 21:31:23 -07:00
Chris Lu df7eee523b FUSE: implement dir fsync 
fix https://github.com/chrislusf/seaweedfs/issues/1398
 2020-07-23 21:09:40 -07:00
Chris Lu 3bbac0820b FUSE: faster rename implementation 2020-07-23 21:08:42 -07:00
Chris Lu 943fa4ea0c fix compilation error 2020-07-22 16:11:17 -07:00
Chris Lu 49446af516 simplify 2020-07-22 13:43:54 -07:00
James Hartig 229f11c660 Added VolumeNeedleStatus volume server grpc method 
This is needed for the diffing tool to get the cookie for a needle
 2020-07-22 15:02:21 -04:00
James Hartig 0be3bb4a87 idx: Switch WalkIndexFile to accept generic io.ReaderAt 
This is necessary to use it with in-memory idx files.
 2020-07-22 15:01:16 -04:00
Chris Lu 7a7e70b6cc 1.86 2020-07-20 23:39:58 -07:00
Chris Lu 885c624bce volume.fsck: follow manifest chunks 2020-07-20 22:02:05 -07:00
Chris Lu 44057a4de1 clean up chunks in manifest 2020-07-20 22:01:39 -07:00
Chris Lu 0b2e06268b use merge factor 1000 2020-07-20 18:30:09 -07:00
Chris Lu d02c0fe0c0 refactoring 2020-07-20 03:34:06 -07:00
Chris Lu 60d14a9800 mount: fix difference with storage format in local cache 2020-07-20 00:17:50 -07:00
Chris Lu 97d97f3528 go code can read and write chunk manifest 2020-07-19 17:59:43 -07:00
李远军 4162766c46 Fix the bug of "http://xx/xx,xxx.PNG?width=100" 
If you want to do a resize a UPPER ext pic file, there is a bug caused by commit 308688c8d0 (also see https://github.com/chrislusf/seaweedfs/blob/master/weed/images/resizing.go#L43)
 2020-07-17 19:57:35 +08:00
Chris Lu f43146b237 resolve directories if containing home directory 2020-07-16 22:50:14 -07:00
Chris Lu befb396892 mount: resolve home directory 
fix https://github.com/chrislusf/seaweedfs/issues/1391
 2020-07-16 22:38:55 -07:00
Chris Lu 2b2ca7d1dc stop if the directory can not be found 
related to https://github.com/chrislusf/seaweedfs/issues/1391
 2020-07-16 13:24:53 -07:00
Chris Lu 2286d27730 rename 2020-07-15 14:19:15 -07:00
Chris Lu 0db4204c81 add is_file_chunks to filer.proto 2020-07-14 22:42:52 -07:00
Chris Lu bbc8ae110f report errors 2020-07-14 11:27:00 -07:00
Chris Lu 17f2d05cde filer: ensure logs are persisted 2020-07-14 11:25:50 -07:00
Chris Lu ddec7b2bb9 go fmt 2020-07-14 07:34:16 -07:00
Chris Lu 4f6096c7f0 add reading from persisted logs for local filer store 2020-07-13 22:55:28 -07:00
Chris Lu 87b5031714 change variable visibility 2020-07-13 08:19:48 -07:00
Chris Lu 7597831cac filer: leveldb2 supports peers also 2020-07-13 00:05:40 -07:00
Chris Lu 5ba894bb72 leveldb2: support subscribing from peers 2020-07-13 00:05:40 -07:00
Chris Lu 5a4f258a8c 1.85 2020-07-12 17:34:09 -07:00
Chris Lu 4eecc6abc6 filer: emit metadata notification correctly for batch deletion 2020-07-12 17:32:19 -07:00
Chris Lu 1dd2c76532 refactoring 2020-07-12 17:31:24 -07:00
Chris Lu 224103e13b aggregated logs has empty flushFn 2020-07-11 09:12:03 -07:00
Chris Lu bfabb4159b refactoring 2020-07-11 09:11:15 -07:00
Chris Lu f2e8ad6241 refactoring 2020-07-11 06:35:54 -07:00
Chris Lu 2d1df97933 mount: fatal error if can not process meta data 2020-07-11 06:16:48 -07:00
Chris Lu e3e00d36c5 reset cache for each new release 2020-07-11 06:16:17 -07:00
Chris Lu 7ad0bce2f5 remove old code 2020-07-10 23:03:22 -07:00
Chris Lu e4ef3814e7 fix compilation 2020-07-10 10:02:59 -07:00
Chris Lu e7c04af1d0 skip http.StatusPartialContent in case of error and superfluous response.WriteHeader 2020-07-10 08:56:23 -07:00
李远军 8c318470dd Revert "remove fixJpgOrientation" 
This reverts commit de5ca9b2

minimise the revert changes
 2020-07-10 10:28:28 +08:00
李远军 6608cb5f43 Revert "remove fixJpgOrientation" 
This reverts commit de5ca9b2
 2020-07-10 10:08:36 +08:00
cheng.li01 dad1161c70 fix dn.volumes Iterate when write issue 2020-07-08 19:57:19 +08:00
Chris Lu 54d6b3c30f filer: support running multiple mount to different filers avoiding duplicated cache 
fix https://github.com/chrislusf/seaweedfs/issues/1382
 2020-07-07 23:18:44 -07:00
Chris Lu 4bed1663e2 adjust log level 2020-07-07 23:17:31 -07:00
Chris Lu 482d13468b remove println 2020-07-07 23:17:17 -07:00
Chris Lu a2eb680f34 filer: default filer store directory 
1. set default filer store directory
2. set peers, avoiding empty string counted as 1.
 2020-07-07 23:06:48 -07:00
Chris Lu 3c269da37f fix compilation 2020-07-06 09:46:24 -07:00
Chris Lu d198e8c6d6 filer: support cross filer meta data sync if sharing the same store 2020-07-06 09:24:54 -07:00
Chris Lu 0b8cc9b626 aggregate multiple filer metadata chagne events 2020-07-05 23:05:02 -07:00
Chris Lu 49929e0869 rename 2020-07-05 15:52:36 -07:00
Chris Lu 70d8a3a1d3 add SubscribeLocalMetadata without checking persisted meta logs 2020-07-05 15:50:07 -07:00
Chris Lu 55e40b08fc refactoring 2020-07-05 15:43:06 -07:00
Chris Lu 881e0fde2e rename 2020-07-05 08:55:09 -07:00
Chris Lu 353bea8ddb refactoring 2020-07-03 22:25:35 -07:00
Chris Lu ce3630e7a4 refactoring 2020-07-03 22:16:59 -07:00
Chris Lu 59152334c7 default to 1% 2020-07-03 16:42:53 -07:00
Chris Lu acf4a9ea99 volume: min free space default to 1% 2020-07-03 16:42:02 -07:00
Chris Lu 4d1484628a refactoring 2020-07-03 16:41:30 -07:00
Chris Lu faa5c2e89a refactoring 2020-07-03 16:34:31 -07:00
Chris Lu 37d5b3ba12 replication: pass isFromOtherCluster also to EventNotification 
EventNotification is consistent with message queue and metadata logs.
 2020-07-01 08:06:20 -07:00
Chris Lu 31e23e9783 filer: support active<=>active filer replication 2020-06-30 22:53:57 -07:00
Chris Lu df75b5c98d passing ctx for metadata logging 2020-06-28 14:34:51 -07:00
Chris Lu 12f6b683fc default to 8 volumes 
in case all the volumes are allocated for a collection
 2020-06-28 14:34:21 -07:00
Chris Lu 42a338d7b0 FUSE mount: clean up file handles during renaming 
resolve dir rename when file is still open. Need to clean the file handles as soon as possible.

These can happen out of order:
file rename, then file release
file release, then file rename
 2020-06-28 13:41:00 -07:00
Chris Lu 95a1860d65 remove empty calls for list cache 2020-06-28 10:25:54 -07:00
Chris Lu 6498d0c86b remove list cache 2020-06-28 10:21:52 -07:00
Chris Lu 9033a7d369 removed async option 2020-06-28 10:18:32 -07:00
Chris Lu 1bb8cae65d reverting and working 2020-06-28 10:14:17 -07:00
Chris Lu b813fac4a3 1.84 2020-06-28 07:28:21 -07:00
Chris Lu 20d9cfae86 volume: following md5 base64 encoding 
fix https://github.com/chrislusf/seaweedfs/issues/1374
 2020-06-28 07:25:50 -07:00
Chris Lu f0f884edd5 1.83 2020-06-27 15:46:59 -07:00
Chris Lu c2797cc6de add logs 2020-06-27 15:46:53 -07:00
Chris Lu 62260e526b add logging 2020-06-27 12:54:37 -07:00
Chris Lu bef356ce4c since we already know the chunk size, no need to iterate 2020-06-27 12:51:04 -07:00
Chris Lu a808b3b5df incase the memory data is too small 2020-06-27 11:59:15 -07:00
Chris Lu 3dbd51c3c2 a little bit more efficient 2020-06-26 10:02:37 -07:00
Chris Lu 212b6e7d42 error logging 2020-06-26 10:01:55 -07:00
Chris Lu 48b23f2fdd FUSE mount: prevent concurrent modification 2020-06-26 10:00:48 -07:00
Chris Lu 3cec4b3c49 FUSE mount: fix file id written twice? 
fix https://github.com/chrislusf/seaweedfs/issues/1373
 2020-06-25 20:11:49 -07:00
Chris Lu 212c64241b checking ecx file 2020-06-25 16:36:06 -07:00
Chris Lu a4dc25cd22 volume: generate ec shard first, before generating the ec index file. 
possible fix https://github.com/chrislusf/seaweedfs/issues/1364

the ec index file is used to determine whether the ec files are ready or not
 2020-06-25 11:06:58 -07:00
Chris Lu 9fd7cdadf1 fix 2020-06-25 10:45:34 -07:00
Chris Lu 3b638d3994 add more ec encoding logging 2020-06-25 09:43:38 -07:00