Commit graph

5230 commits

Author SHA1 Message Date
chrislu fbd99d53c1 3.07 2022-05-31 00:36:24 -07:00
chrislu 27732ecfa4 move s3 related constants from package http to s3_constants 2022-05-30 22:57:41 -07:00
chrislu f4a6da6cb2 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 22:47:29 -07:00
chrislu 596c3860ca use final destination to resolve fs configuration
related to https://github.com/chrislusf/seaweedfs/issues/3075
2022-05-30 22:47:26 -07:00
Chris Lu ca01ce0524
Merge pull request #3122 from chrislusf/filer-sync-with-peers
Filer bootstrap from peers
2022-05-30 21:38:31 -07:00
chrislu 6adc42147f fresh filer store bootstrap from the oldest peer 2022-05-30 21:27:48 -07:00
chrislu 490e0850bf filer.meta.tail add one example of using untilTimeAgo 2022-05-30 20:24:59 -07:00
chrislu c59068d0f3 refactor 2022-05-30 16:28:36 -07:00
chrislu 682382648e collect cluster node start time 2022-05-30 16:23:52 -07:00
chrislu 968ca95b49 filer.meta.tail: support untilTimeAgo for a range 2022-05-30 16:16:23 -07:00
chrislu f214dfb1f5 stop when in memory log is done 2022-05-30 15:25:21 -07:00
chrislu aece35a64f stop when on disk log is done 2022-05-30 15:20:51 -07:00
chrislu a2b101a737 subscribe metadata between a range 2022-05-30 15:04:19 -07:00
chrislu 8902fa6ff6 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-30 13:28:32 -07:00
Chris Lu 730c9cf601
Merge pull request #3114 from kmlebedev/nextcloud
Create folders in s3 via nextcloud
2022-05-30 13:10:10 -07:00
chrislu 5b9347c938 typo 2022-05-30 12:37:41 -07:00
Konstantin Lebedev af562e1a20 rm log info 2022-05-30 23:26:26 +05:00
Konstantin Lebedev 62e5e3822c list self dir
https://github.com/chrislusf/seaweedfs/issues/3086
2022-05-30 18:09:08 +05:00
Konstantin Lebedev 26f3646961 show fake dirs
https://github.com/chrislusf/seaweedfs/issues/3086
2022-05-30 15:03:27 +05:00
chrislu b8f3db0d46 s3: keep auth enabled in case identities are set to empty
fix https://github.com/chrislusf/seaweedfs/issues/3084
2022-05-29 19:00:03 -07:00
chrislu 81d6159290 volume: report error if a volume has nil data backend
fix https://github.com/chrislusf/seaweedfs/issues/3105
2022-05-29 16:59:30 -07:00
chrislu ca16fbf0ef shell: cluster.ps display filer group 2022-05-29 16:37:14 -07:00
chrislu 9219664505 reset sync offset if peer filer resets
fix https://github.com/chrislusf/seaweedfs/issues/3089
2022-05-29 13:11:24 -07:00
Konstantin Lebedev 54356211c2 list empty folder 2022-05-27 13:17:18 +05:00
Kaiwalya Joshi 4598581741
fix: Minor spelling fixes on FilerStorePathTranslator. 2022-05-25 01:11:02 -07:00
chrislu 5b8b022985 remove unused parameter 2022-05-24 18:52:04 -07:00
Robert Coelho 0e6e72d462 mount: ReadDir return EIO on EnsureVisited err 2022-05-24 13:02:59 -04:00
Robert Coelho 1fabbe8a25 mount: cleanup ReadDir rewrite's branches to not assume offsets 2022-05-24 12:48:32 -04:00
Robert Coelho cb422d96f7 mount: rewrite ReadDir to respect input.Offset to fix partial results 2022-05-24 12:16:58 -04:00
Robert Coelho 05602167f4 mount: do not mark meta_cache when there is an err 2022-05-24 12:13:15 -04:00
chrislu 7e25a2d416 reduce busy waiting when reading metadata logs 2022-05-24 00:23:53 -07:00
chrislu 2f846777bb 3.06 2022-05-23 01:21:48 -07:00
chrislu a5cdcc7e63 fix reading files larger than 20GB due to manifest resolving bug 2022-05-23 01:16:10 -07:00
chrislu 59cc3c0bfd fix reading files larger than 20GB due to manifest resolving bug
this is introduced since Mar 08 2022
2022-05-23 01:14:56 -07:00
chrislu acc318e12b 3.05 2022-05-22 21:43:59 -07:00
Chris Lu cb64983c6b
Merge pull request #3079 from ztinpn/hotfix_fsync 2022-05-19 10:33:42 -07:00
tianzhang 66747ee9c9 hotfix_fsync fix fsync 2022-05-19 23:17:17 +08:00
Chris Lu 8efe55f981
Merge pull request #3059 from guol-fnst/avoid_dup_vol
avoid loading duplicated volume directory
2022-05-17 23:48:57 -07:00
guol-fnst 8e2426a879 revert changes of pb file
orignal protoc-gen-go-grpc verion is 1.1.0
2022-05-18 11:52:14 +08:00
guol-fnst b12944f9c6 fix naming convention
notify volume server of duplicate directoris
improve searching efficiency
2022-05-17 15:41:49 +08:00
guol-fnst 076595fbdd just exit in case of duplicated volume directories were loaded 2022-05-17 15:41:49 +08:00
guol-fnst 8fab39e775 rename UUID file
fix typo
move locationUUID  into DiskLocation
2022-05-17 15:41:47 +08:00
Chris Lu 72e7dcde51
Merge pull request #3060 from natmaka/master 2022-05-16 07:42:58 -07:00
Nat Makarevitch b72f0634d7
minor (typos...), done while reading around 2022-05-16 22:11:33 +08:00
guol-fnst de6aa9cce8 avoid duplicated volume directory 2022-05-16 19:33:51 +08:00
Konstantin Lebedev 44f53ceda6 fix collectionIsMismatch charset 2022-05-16 13:23:23 +05:00
Konstantin Lebedev 10d435f2c2 fix skip loop 2022-05-16 13:16:27 +05:00
Konstantin Lebedev 279053572c avoid delete volume replica if collection mismatch 2022-05-16 13:07:05 +05:00
chrislu 9ff0d99002 3.04 2022-05-15 21:32:21 -07:00
chrislu 866fed1eab avoid grpc name conflict
fix https://github.com/chrislusf/seaweedfs/issues/3055
2022-05-15 21:31:29 -07:00
chrislu cd2d10118c 3.03 2022-05-15 18:17:07 -07:00
chrislu 271b5aed96 shell: volume.fsck add a note for -reallyDeleteFromVolume option 2022-05-15 11:07:04 -07:00
chrislu 4076d74813 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-15 00:43:39 -07:00
chrislu 29198720f2 s3: add grpc server to accept configuration changes 2022-05-15 00:43:37 -07:00
Chris Lu ea4918394c
Merge pull request #3047 from shichanglin5/master
fix the problem of metadata and tagging loss when files are copied
2022-05-13 22:52:48 -07:00
shichanglin5 688d55488c test(s3api_object_copy_handlers_test.go): some unit tests have been added to the processMetadata & processMetadataBytes methods of s3api_object_copy_handlers.go 2022-05-14 10:40:29 +08:00
chrislu 735038b2c1 backup do not need to use preallocation
fix https://github.com/chrislusf/seaweedfs/issues/3044
2022-05-13 13:46:52 -07:00
shichanglin5 1166dead00 fix the problem of metadata and tagging loss when files are copied
by adding processing of metadata and tagging in s3 api CopyObject (judging whether to copy or overwrite according to the directive header)
2022-05-13 19:46:20 +08:00
Konstantin Lebedev d12a423aa0 refactor
add some tests

https://github.com/chrislusf/seaweedfs/pull/2996
2022-05-12 12:23:02 +05:00
Konstantin Lebedev f523aed3c9 fixes from comments
https://github.com/chrislusf/seaweedfs/pull/2996
2022-05-11 21:50:09 +05:00
chrislu bff1ccc1de fix compilation 2022-05-11 00:52:15 -07:00
chrislu 76da688176 adjust log level 2022-05-11 00:45:19 -07:00
chrislu 3e20336bde fix atomic add int64
fix https://github.com/chrislusf/seaweedfs/issues/3038#issuecomment-1123269831
2022-05-11 00:44:35 -07:00
chrislu a85720f0f6 remove deprecated function 2022-05-10 00:02:23 -07:00
chrislu 25559eeb92 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-10 00:01:46 -07:00
chrislu 791c29ae93 remove deprecated function
WithQueryCachePolicyKeepInCache is deprecated and already default behavior
2022-05-10 00:01:43 -07:00
qiang.zeng 06cb337b82 fix typo 2022-05-09 14:21:16 +08:00
chrislu a4ca3ed1f0 3.02 2022-05-08 22:28:32 -07:00
chrislu 8bc16f1149 rename 2022-05-06 23:38:22 -07:00
chrislu 139e039c44 filer.sync: pass attributes for mount
fix https://github.com/chrislusf/seaweedfs/issues/3012
2022-05-06 03:54:12 -07:00
chrislu 10207a48a4 how to setup local test env 2022-05-06 02:25:56 -07:00
chrislu db33b03735 add comments 2022-05-06 02:19:47 -07:00
chrislu e350ba8896 larger test range 2022-05-06 02:17:28 -07:00
chrislu 71e0bcc5d5 a bit optimization 2022-05-06 02:11:58 -07:00
chrislu b2a43dd044 Merge branch 'master' of https://github.com/chrislusf/seaweedfs 2022-05-06 01:51:28 -07:00
chrislu d5fe5d12b7 filer: etcd fix pagination
fix https://github.com/chrislusf/seaweedfs/issues/3016
2022-05-06 01:51:19 -07:00
Chris Lu a69bf53405
Merge pull request #2996 from kmlebedev/ydb
ydb filer store
2022-05-05 15:34:37 -07:00
Konstantin Lebedev e6d2cb5911 ydb add empty doc file 2022-05-05 11:12:14 +05:00
chrislu ef6c6c450e avoid fatal error if port is already in use 2022-05-04 14:55:14 -07:00
chrislu f7366a9668 skip unix socket mode for windows
fix https://github.com/chrislusf/seaweedfs/issues/3013
2022-05-04 10:14:34 -07:00
Konstantin Lebedev f58adaab25 build tag ydb since the growth of the binary is 5 mb 2022-05-03 23:20:09 +05:00
Konstantin Lebedev f127b326bf add options to scaffold 2022-05-03 22:54:31 +05:00
Konstantin Lebedev 7b544576af refactor 2022-05-03 20:16:00 +05:00
chrislu f17cd0d5cd return false if not found
fix https://github.com/chrislusf/seaweedfs/issues/3011
2022-05-03 07:18:34 -07:00
chrislu b201edb9df fix wrong assignment 2022-05-03 07:12:33 -07:00
Konstantin Lebedev 04d5dff6bb add support native ttl 2022-05-03 17:52:23 +05:00
Konstantin Lebedev cb3c7a3cdb enable query cache policy instead of prepare 2022-05-03 16:03:10 +05:00
Konstantin Lebedev 8342f651f3 fix scanNamed 2022-05-03 15:18:28 +05:00
chrislu 9271866d1e fix segmentation violation
fix https://github.com/chrislusf/seaweedfs/issues/3000
2022-05-02 23:56:11 -07:00
Konstantin Lebedev 2d27afb515 Merge branch 'new_master' into ydb 2022-05-03 10:47:39 +05:00
chrislu 76148ce0f7 use storage_class for backend configuration
fix https://github.com/chrislusf/seaweedfs/issues/3008
2022-05-02 12:20:01 -07:00
Konstantin Lebedev d8925b4e83 Merge branch 'new_master' into ydb
# Conflicts:
#	go.mod
#	go.sum
2022-05-03 00:13:57 +05:00
Konstantin Lebedev 7640e650e5 fix queries 2022-05-03 00:11:37 +05:00
Konstantin Lebedev 319d300d48 fix createDB 2022-05-02 22:23:07 +05:00
Konstantin Lebedev 0dc44dda63 ydb do Tx or DB 2022-05-02 15:33:29 +05:00
Konstantin Lebedev a3e48831f3 ydb BucketAware interface 2022-05-02 12:42:20 +05:00
chrislu 73961e24d8 3.01 2022-05-01 23:33:45 -07:00
chrislu 00c1dfec4f go fmt 2022-05-01 23:16:29 -07:00
chrislu ce1e8e58ae fix test 2022-05-01 23:15:58 -07:00
chrislu 2c0fdf3693 fix tests 2022-05-01 23:14:04 -07:00