seaweedfs/weed/shell
Chris Lu d4566d4aaa
more solid weed mount (#4089)
* compare chunks by timestamp

* fix slab clearing error

* fix test compilation

* move oldest chunk to sealed, instead of by fullness

* lock on fh.entryViewCache

* remove verbose logs

* revert slat clearing

* less logs

* less logs

* track write and read by timestamp

* remove useless logic

* add entry lock on file handle release

* use mem chunk only, swap file chunk has problems

* comment out code that maybe used later

* add debug mode to compare data read and write

* more efficient readResolvedChunks with linked list

* small optimization

* fix test compilation

* minor fix on writer

* add SeparateGarbageChunks

* group chunks into sections

* turn off debug mode

* fix tests

* fix tests

* tmp enable swap file chunk

* Revert "tmp enable swap file chunk"

This reverts commit 985137ec47.

* simple refactoring

* simple refactoring

* do not re-use swap file chunk. Sealed chunks should not be re-used.

* comment out debugging facilities

* either mem chunk or swap file chunk is fine now

* remove orderedMutex  as *semaphore.Weighted

not found impactful

* optimize size calculation for changing large files

* optimize performance to avoid going through the long list of chunks

* still problems with swap file chunk

* rename

* tiny optimization

* swap file chunk save only successfully read data

* fix

* enable both mem and swap file chunk

* resolve chunks with range

* rename

* fix chunk interval list

* also change file handle chunk group when adding chunks

* pick in-active chunk with time-decayed counter

* fix compilation

* avoid nil with empty fh.entry

* refactoring

* rename

* rename

* refactor visible intervals to *list.List

* refactor chunkViews to *list.List

* add IntervalList for generic interval list

* change visible interval to use IntervalList in generics

* cahnge chunkViews to *IntervalList[*ChunkView]

* use NewFileChunkSection to create

* rename variables

* refactor

* fix renaming leftover

* renaming

* renaming

* add insert interval

* interval list adds lock

* incrementally add chunks to readers

Fixes:
1. set start and stop offset for the value object
2. clone the value object
3. use pointer instead of copy-by-value when passing to interval.Value
4. use insert interval since adding chunk could be out of order

* fix tests compilation

* fix tests compilation
2023-01-02 23:20:45 -08:00
..
command_cluster_check.go remove old raft servers if they don't answer to pings for too long (#3398) 2022-08-23 23:18:21 -07:00
command_cluster_ps.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_cluster_raft_add.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_cluster_raft_ps.go show raft leader via shell (#3796) 2022-10-06 07:10:41 -07:00
command_cluster_raft_remove.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_collection_delete.go remove old raft servers if they don't answer to pings for too long (#3398) 2022-08-23 23:18:21 -07:00
command_collection_list.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_ec_balance.go refactor(shell): Decending -> Descending (#3675) 2022-09-14 12:06:48 -07:00
command_ec_common.go refactor to change capacity data type 2022-10-09 18:58:10 -07:00
command_ec_decode.go refactor(command_ec_decode): exisitngEcIndexBits -> `existingEcInde… (#3674) 2022-09-14 12:02:33 -07:00
command_ec_encode.go shell: stop long running jobs if lock is lost 2022-08-22 14:12:23 -07:00
command_ec_encode_test.go refactor(shell): Decending -> Descending (#3675) 2022-09-14 12:06:48 -07:00
command_ec_rebuild.go refactor(command_ec_rebuild): rebultErr -> rebuildErr (#3679) 2022-09-14 13:10:27 -07:00
command_ec_test.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_fs_cat.go refactor filer_pb.Entry and filer.Entry to use GetChunks() 2022-11-15 06:33:36 -08:00
command_fs_cd.go simplify parsing filer host and port 2020-03-23 21:26:15 -07:00
command_fs_configure.go refactor(shell): readability improvements (#3704) 2022-09-16 02:43:49 -07:00
command_fs_du.go refactor filer_pb.Entry and filer.Entry to use GetChunks() 2022-11-15 06:33:36 -08:00
command_fs_ls.go refactor filer_pb.Entry and filer.Entry to use GetChunks() 2022-11-15 06:33:36 -08:00
command_fs_meta_cat.go refactor filer_pb.Entry and filer.Entry to use GetChunks() 2022-11-15 06:33:36 -08:00
command_fs_meta_load.go fs.meta.load load any dirs with prefix "important" (#3747) 2022-09-27 09:58:46 -07:00
command_fs_meta_notify.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_fs_meta_save.go ADHOC: fix fs.meta.save typo (#3993) 2022-11-20 21:05:00 -08:00
command_fs_mkdir.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_fs_mv.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_fs_pwd.go fix pwd printout 2020-03-24 02:40:29 -07:00
command_fs_rm.go refactor(command_fs_rm): entiries -> entries (#3670) 2022-09-14 11:24:27 -07:00
command_fs_tree.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_fs_verify.go more solid weed mount (#4089) 2023-01-02 23:20:45 -08:00
command_lock_unlock.go refactor(shell): readability improvements (#3704) 2022-09-16 02:43:49 -07:00
command_mount_configure.go refactor(socket mount): Update socket mount pattern to `/tmp/seaweedf… (#3662) 2022-09-14 09:14:44 -07:00
command_mq_topic_list.go add an empty placeholder for mq.topic.list 2022-07-28 23:24:38 -07:00
command_remote_cache.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_remote_configure.go also migrate jsonpb 2022-08-17 12:42:03 -07:00
command_remote_meta_sync.go refactor(shell): readability improvements (#3704) 2022-09-16 02:43:49 -07:00
command_remote_mount.go go fmt 2022-09-14 23:06:44 -07:00
command_remote_mount_buckets.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_remote_uncache.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_remote_unmount.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_bucket_create.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_bucket_delete.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_bucket_list.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_bucket_quota.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_bucket_quota_check.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_circuitbreaker.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_s3_circuitbreaker_test.go refactor(shell): readability improvements (#3704) 2022-09-16 02:43:49 -07:00
command_s3_clean_uploads.go refactor(shell): readability improvements (#3704) 2022-09-16 02:43:49 -07:00
command_s3_configure.go refactor: Directory readability (#3665) 2022-09-14 10:11:31 -07:00
command_volume_balance.go adjust help message 2022-10-24 09:25:08 -07:00
command_volume_balance_test.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_check_disk.go [volume.chek.disk] sync of deletions the fix (#3923) 2022-10-30 20:32:46 -07:00
command_volume_configure_replication.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_copy.go feat(weed.move): add a speed limit parameter of moving files (#3478) 2022-08-21 23:08:31 -07:00
command_volume_delete.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_delete_empty.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_fix_replication.go go fmt 2022-09-14 23:06:44 -07:00
command_volume_fix_replication_test.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_fsck.go more solid weed mount (#4089) 2023-01-02 23:20:45 -08:00
command_volume_list.go volume.list show volumes only from the specified dc/rack/dn (#4024) 2022-11-30 06:30:55 -08:00
command_volume_list_test.go tests add parsing ec shard info 2022-10-09 22:47:55 -07:00
command_volume_mark.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_mount.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_move.go feat(weed.move): add a speed limit parameter of moving files (#3478) 2022-08-21 23:08:31 -07:00
command_volume_server_evacuate.go move volume: find target volume server by exiting/max ratio 2022-10-09 22:30:59 -07:00
command_volume_server_evacuate_test.go fix TestVolumeServerEvacuate 2022-07-12 14:56:34 +05:00
command_volume_server_leave.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_tier_download.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_tier_move.go go fmt 2022-09-14 23:06:44 -07:00
command_volume_tier_upload.go shell: volume.tier.upload clean up replicated copies 2022-10-16 17:02:56 -07:00
command_volume_unmount.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_vacuum.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
command_volume_vacuum_disable.go add more help message, in case of misunderstanding (#4092) 2022-12-29 07:05:05 -08:00
command_volume_vacuum_enable.go add more help message, in case of misunderstanding (#4092) 2022-12-29 07:05:05 -08:00
commands.go [volume.fsck] check needles status from volume server (#3926) 2022-10-31 11:33:04 -07:00
shell_liner.go refactor and fix strings.Split 2022-08-07 01:34:32 -07:00
volume.list.txt Revert "a new test case" 2022-09-10 15:35:42 -07:00
volume.list2.txt tests add parsing ec shard info 2022-10-09 22:47:55 -07:00