nrd.li/akkoma
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
14505 commits 2 branches 0 tags 272 MiB
Commit graph

8785 commits

Author SHA1 Message Date
FloatingGhost a036a01a1e mix format 2022-07-04 17:38:16 +01:00
floatingghost 364b6969eb Use finch everywhere (#33) 
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/33
 2022-07-04 16:30:38 +00:00
sfr 058bf96798 implement Move activities (#45) 
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/45
Co-authored-by: sfr <sol@solfisher.com>
Co-committed-by: sfr <sol@solfisher.com>
 2022-07-04 16:29:39 +00:00
FloatingGhost 95ef3a8b1e Use Akkoma modification for collections 2022-07-03 19:36:30 +01:00
FloatingGhost 05081cd81b Add collection fetching module 2022-07-03 19:20:59 +01:00
FloatingGhost 0a3a552696 Add support for a first reference in pinned objects 2022-07-03 17:25:20 +01:00
floatingghost 3f7c64f30b Fix broken attachments from owncast (#31) 
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/31
 2022-07-01 11:14:55 +00:00
floatingghost 2937495712 fix ES import from live (#29) 
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/29
 2022-06-30 18:44:31 +00:00
FloatingGhost bc9e76cce7 Add documentation for ES search 2022-06-30 17:36:57 +01:00
FloatingGhost 635a3c223a Add elasticsearch tests 2022-06-30 16:53:21 +01:00
FloatingGhost 1ecdb19de5 Refactor ES on top of search behaviour 2022-06-30 16:28:31 +01:00
Ekaterina Vaartis 563b964690 Change updateId to uid because apparently that's the new name 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis b7462040cc Change the meilisearch key auth to conform to 0.25.0 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 7f53aa400b Don't try removing deleted users and such from index as posts 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 7aebff799b Fix meilisearch tests and jobs for oban 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 5def4a7d49 Use oban for search indexing 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 54b2a86f47 Add a search backend behaviour 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis a4914add8c Don't support meilisearch < 0.24.0, since it breaks things 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis cc3319ac1d Make chunk size configurable 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis bac70a2bc1 Implement suggestions from the Meilisearch MR 
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis c128798418 Rename Activity.Search to Search.DatabaseSearch 2022-06-29 20:49:44 +01:00
Ekaterina Vaartis ea582fbf93 Rename search.ex to database_search.ex and add search/2 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis 0769f06bd1 Style fixes 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis 5dd908552c Move the search.ex file so credo doesn't complain 2022-06-29 20:48:59 +01:00
Ekaterina Vaartis 86971fceaa Support reindexing meilisearch >=0.24.0 
It has has a different error code key
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 9e7d7ebd48 Add a reindex option 
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis d99a2be351 Only add local posts to index in activity_pub 
Remote ones are already added in another place
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 51faa28568 Set content-type to application/json 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis afcdd2f437 Modify some meilisearch variables 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 37913c8ddf Use proper deleted object for removing from index 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis a937a98df5 Don't try removing from index again in common_api 
It's already removed in the side effects of the pipeline
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 0802c0666b Move add_to_index / remove_from_index to Pleroma.Actitivy.Search 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 5ed1759091 Reorder ranking rules for (maybe) better results 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 41f3572152 Fix activity being passed to objec_to_search_data 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis d1079f1aa3 Add the meilisearch.stats command 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis d5cc272a91 Add a message with a count of posts to index 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis dbf556cdcf Implement meilisearch auth 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 5360cc1097 Make indexing logs rewrite themselves 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 0cf3654907 Rework task indexing to share code with the main module 
The code in the main module now scrubs new posts too
 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 117f525fd6 Adjust content indexing to skip more unneeded stuff 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 14ef6ce80f Mark only content as searchable for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis b3401ba7bd Also index incoming federated posts 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 52a872432d Make the chunk size smaller 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis a586ce0ddd Use content instead of source and scrub it 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis c3a04166a0 Tweak search ordering to hopefully return newer results 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 7b3701e6b9 Make meilisearch sort on publish date converted to unix time 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis abf82a63ec Make the indexing batch differently and more, show number indexed 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis e5ac2ffa07 Ensure only indexing public posts and implement clearing and delete 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 41db5c8653 Add logging to milisiearch index and make it use desc(id) 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis e961cf2689 Initial meilisearch implementation, doesn't delete posts yet 2022-06-29 20:48:28 +01:00