Commit graph

6811 commits

Author SHA1 Message Date
rinpatch 6bf8eee5f9 ReverseProxy tesla client: remove handling of old_conn
This is no longer relevant because we use a custom
FollowRedirects middleware now
2020-07-26 20:52:00 +03:00
rinpatch d4fbec62a3 ReverseProxy: Fix a gun connection leak when there is an error with no
body

- Modify `close/1` function to do the same thing it does for hackney,
which is - close the client rather than the whole connection
- Release the connection when there is no body to chunk
2020-07-26 19:30:17 +03:00
Alibek Omarov 6107440ea0 OpenAPI: remove accidentally pasted buffer data 2020-07-26 13:59:46 +00:00
Alibek Omarov b31844d6e0 OpenAPI: Replace actor_id by account_id to follow ChatMessage schema 2020-07-26 13:54:56 +00:00
Mark Felder 65a1b048a8 Ensure Oban is available during mix tasks.
Fixes: mix pleroma.user rm username
2020-07-24 12:06:56 -05:00
Mark Felder 9be6668236 Fix mix tasks that make HTTP calls by starting the Gun connection pool 2020-07-24 12:05:42 -05:00
Lain Soykaf 3d13fb05f8 Side Effects: On undoing, put information about the undone object. 2020-07-24 14:40:22 +02:00
Lain Soykaf 91f3cf9bc6 Pipeline: Add embedded object federation. 2020-07-24 14:06:41 +02:00
Maksim Pechnikov 61ef1fca4b remove duplicate module 2020-07-24 08:35:06 +03:00
feld 51627a10e5 Merge branch 'unblock-domain-via-query' into 'develop'
Allow unblocking a domain via query params

Closes #1971

See merge request pleroma/pleroma!2783
2020-07-23 20:06:11 +00:00
Mark Felder 4bfad0b483 Support blocking via query parameters as well and document the change. 2020-07-23 12:59:40 -05:00
Haelwenn 6100b90209 Merge branch '1976-status-view-fixes' into 'develop'
StatusView: Handle badly formatted emoji reactions.

Closes #1976

See merge request pleroma/pleroma!2788
2020-07-23 16:58:36 +00:00
Maksim Pechnikov 67ab9a7928 Merge branch 'develop' into issue/1934-welcome-email 2020-07-23 16:36:27 +03:00
lain 76aa49a0c5 Merge branch '1973-chats-fix-with-restrict-unauthenticated' into 'develop'
[#1973] Fixed accounts rendering in GET /api/v1/pleroma/chats with truish :restrict_unauthenticated setting

Closes #1973

See merge request pleroma/pleroma!2791
2020-07-23 12:52:02 +00:00
Ivan Tashkinov 9ea51a6de5 [#2791] AccountView: renamed :force option to :skip_visibility_check. 2020-07-23 15:08:30 +03:00
lain b2ba70eec4 Merge branch 'bugfix/mrf-reject-logging' into 'develop'
Fix #783: Make MRF reject log as info level instead of error

Closes #783

See merge request pleroma/pleroma!2758
2020-07-23 11:08:29 +00:00
lain 250e0369c7 Merge branch 'issue/1878' into 'develop'
[#1878] fix reset confirmation email in admin section

See merge request pleroma/pleroma!2751
2020-07-23 10:56:33 +00:00
lain a81581472f Merge branch 'linkify' into 'develop'
Fix Linkify

See merge request pleroma/pleroma!2792
2020-07-23 10:28:52 +00:00
Maksim Pechnikov 7991ddad58 added warning to use old keys 2020-07-23 09:01:37 +03:00
Ivan Tashkinov 6f5f7af607 [#1973] Fixed accounts rendering in GET /api/v1/pleroma/chats with truish :restrict_unauthenticated.
Made `Pleroma.Web.MastodonAPI.AccountView.render("show.json", _)` demand :for or :force option in order to prevent incorrect rendering of empty map instead of expected user representation with truish :restrict_unauthenticated setting.
2020-07-22 19:06:00 +03:00
lain 1c9752cff4 Merge branch 'fix/pleroma-api-emoji-packs' into 'develop'
Fix /api/pleroma/emoji/packs index endpoint.

See merge request pleroma/pleroma!2777
2020-07-22 15:09:32 +00:00
Maksim Pechnikov db0224d174 added check user email for welcome email 2020-07-22 16:00:49 +03:00
Lain Soykaf 0cb9e1da74 StatusView: Handle badly formatted emoji reactions. 2020-07-22 14:44:06 +02:00
Maksim Pechnikov 5879d36854 fix sender for welcome email 2020-07-22 15:34:47 +03:00
Maksim Pechnikov 8d6f7aec21 Merge branch 'develop' into issue/1023 2020-07-22 09:24:16 +03:00
Mark Felder 341a8f3500 Skip the correct plug 2020-07-21 17:26:59 -05:00
lain 5b1eeb06d8 Revert "Merge branch 'revert-2b5d9eb1' into 'develop'"
This reverts merge request !2784
2020-07-21 22:18:17 +00:00
lain 696c13ce54 Revert "Merge branch 'linkify' into 'develop'"
This reverts merge request !2677
2020-07-21 22:17:34 +00:00
Mark Felder 97c60b6a43 Merge branch 'develop' into fix/pleroma-api-emoji-packs 2020-07-21 16:25:45 -05:00
lain 2b5d9eb105 Merge branch 'linkify' into 'develop'
AutoLinker --> Linkify, update to latest version

See merge request pleroma/pleroma!2677
2020-07-21 19:35:43 +00:00
lain fa2421dd7e Merge branch 'gopher-fix' into 'develop'
Gopher: using atom keys in search params

See merge request pleroma/pleroma!2775
2020-07-21 19:31:03 +00:00
Egor Kislitsyn bdb3375933
Allow unblocking a domain via query params 2020-07-21 16:06:46 +04:00
Maksim Pechnikov 3edaecae96 added welcome email 2020-07-21 09:25:53 +03:00
Alex Gleason 204dddcfaa
Pleroma.Formatter can have partial updates 2020-07-20 13:45:05 -05:00
href 4bac25e6f5 Don't enable Pleroma.HTTP.Middleware.FollowRedirects unless Gun is used 2020-07-18 13:17:38 +02:00
KokaKiwi 7ce722ce3e
Fix /api/pleroma/emoji/packs index endpoint. 2020-07-18 12:55:04 +02:00
Haelwenn 10d46235cf Merge branch 'fix-1958' into 'develop'
OpenAPI: Add :id to follower/following endpoints, fixes #1958

Closes #1958

See merge request pleroma/pleroma!2772
2020-07-17 18:54:30 +00:00
Alex Gleason 48f8b26c92
OpenAPI: Add :id to follower/following endpoints, fixes #1958 2020-07-17 12:35:28 -05:00
Alex Gleason 5756843736
Fully delete users with status :approval_pending 2020-07-17 12:19:41 -05:00
Mark Felder 20a496d2cb Expose the post formats in /api/v1/instance 2020-07-17 10:45:41 -05:00
Alexander Strizhakov af376cbffb
using atom keys in search params 2020-07-17 18:06:05 +03:00
Mark Felder 62438530e2 TOTP length is configurable, so we can't hardcode this here. 2020-07-17 08:19:49 -05:00
Alex Gleason 5e74556703
Sanitize reason param in POST /api/v1/accounts 2020-07-16 20:25:53 -05:00
Alex Gleason 38425ebdbf
Merge remote-tracking branch 'upstream/develop' into linkify 2020-07-16 14:51:36 -05:00
Mark Felder 3be64556db Improve TOTP token and recovery input fields in OAuth login 2020-07-16 13:11:03 -05:00
Maksim 16da9f5cfd Apply 1 suggestion(s) to 1 file(s) 2020-07-16 13:30:28 +00:00
Maksim c72676d22f Apply 1 suggestion(s) to 1 file(s) 2020-07-16 13:30:17 +00:00
Maksim Pechnikov 6c59fe259d Merge branch 'develop' into issue/1023 2020-07-16 08:53:21 +03:00
Maksim Pechnikov 4e0e19a706 update task messages 2020-07-16 08:52:14 +03:00
Alex Gleason 9ce95fa68f
Use approval_required in /api/v1/instance 2020-07-15 17:04:30 -05:00
Mark Felder b2d398b1d0 Merge branch 'develop' into refactor/gun-pool-registry 2020-07-15 13:34:27 -05:00
Mark Felder 9d30bacace Merge branch 'develop' into refactor/notification_settings 2020-07-15 09:30:23 -05:00
rinpatch 0fe36b311c Merge branch 'features/mrf-reasons' into 'develop'
Add rejection reason to our MRFs

See merge request pleroma/pleroma!2759
2020-07-15 14:00:22 +00:00
Haelwenn (lanodan) Monnier d29b8997f4
MastoAPI: fix & test giving MRF reject reasons 2020-07-15 15:25:33 +02:00
rinpatch 7115c5f82e ConnectionPool.Worker: do not stop with an error when there is a timeout
This produced error log messages about GenServer termination
every time the connection was not open due to a timeout.

Instead we stop with `{:shutdown, <gun_error>}` since shutting down
when the connection can't be established is normal behavior.
2020-07-15 15:58:08 +03:00
href 6d583bcc3b Set a default timeout for Gun adapter timeout 2020-07-15 15:26:35 +03:00
href afd378f84c host is now useless 2020-07-15 15:26:35 +03:00
href ce1a42bd04 Simplify TLS opts
- `verify_fun` is not useful now
- use `customize_check_hostname` (OTP 20+ so OK)
- `partial_chain` is useless as of OTP 21.1 (wasn't there, but hackney/..
uses it)
2020-07-15 15:26:35 +03:00
href 53ba6815b1 parentheses... 2020-07-15 15:26:35 +03:00
href 23d714ed30 Fix race in enforcer/reclaimer start 2020-07-15 15:26:35 +03:00
href 6a0f2bdf8c Ensure connections error get known by the caller 2020-07-15 15:26:35 +03:00
href 46dd276d68 ConnectionPool.Worker: Open gun conn in continue instead of init 2020-07-15 15:26:35 +03:00
rinpatch a705637dcf Connection Pool: fix LRFU implementation to not actually be LRU
The numbers of the native time unit were so small the CRF was always 1,
making it an LRU. This commit switches the time to miliseconds and changes
the time delta multiplier to the one yielding mostly highest hit rates according
to the paper
2020-07-15 15:26:35 +03:00
rinpatch 9b73c35ca8 Request limiter setup: consider {:error, :existing} a success
When the application restarts (which happens after certain config
changes), the limiters are not destroyed, so `ConcurrentLimiter.new`
will produce {:error, :existing}
2020-07-15 15:26:35 +03:00
rinpatch 12fa5541f0 FollowRedirects: Unconditionally release the connection if there is an error
There is no need for streaming the body if there is no body
2020-07-15 15:26:35 +03:00
rinpatch 37f1e781cb Gun adapter helper: fix wildcard cert issues on OTP 23
See https://bugs.erlang.org/browse/ERL-1260 for more info.

The ssl match function is basically copied from mint, except
that `:string.lowercase/1` was replaced by `:string.casefold`.
It was a TODO in mint's code, so might as well do it since we don't need
to support OTP <20.

Closes #1834
2020-07-15 15:26:35 +03:00
rinpatch 007843b75e Add documentation for new connection pool settings and remove some
`:retry_timeout` and `:retry` got removed because reconnecting on failure is
something the new pool intentionally doesn't do.

`:max_overflow` had to go in favor of `:max_waiting`, I didn't reuse the key because
the settings are very different in their behaviour.

`:checkin_timeout` got removed in favor of `:connection_acquisition_wait`,
I didn't reuse the key because the settings are somewhat different.

I didn't do any migrations/deprecation warnings/changelog entries because
these settings were never in stable.
2020-07-15 15:26:35 +03:00
rinpatch 7882f28569 Use erlang monotonic time for CRF calculation 2020-07-15 15:26:35 +03:00
rinpatch 00926a63fb Adapter Helper: Use built-in ip address type 2020-07-15 15:26:35 +03:00
rinpatch 4128e3a84a HTTP: Implement max request limits 2020-07-15 15:26:35 +03:00
rinpatch 94c8f3cfaf Use a custom pool-aware FollowRedirects middleware 2020-07-15 15:26:35 +03:00
rinpatch 281ddd5e37 Connection pool: fix connections being supervised by gun_sup 2020-07-15 15:26:35 +03:00
rinpatch 1b15cb066c Connection pool: Add client death tracking
While running this in production I noticed a number of ghost
processes with all their clients dead before they released the connection,
so let's track them to log it and remove them from clients
2020-07-15 15:26:35 +03:00
rinpatch e94ba05e52 Connection pool: Fix a possible infinite recursion if the pool is exhausted 2020-07-15 15:26:35 +03:00
rinpatch 7738fbbaf5 Connection pool: implement logging and telemetry events 2020-07-15 15:26:25 +03:00
rinpatch 0ffde499b8 Connection Pool: register workers using :via 2020-07-15 15:24:47 +03:00
rinpatch ec9d0d146b Connection pool: Fix race conditions in limit enforcement
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
2020-07-15 15:17:27 +03:00
rinpatch d08b157699 Connection pool: check that there actually is a result
Sometimes connections died before being released to the pool, resulting
in MatchErrors
2020-07-15 15:17:27 +03:00
rinpatch fffbcffb8c Connection Pool: don't enforce pool limits if no new connection needs to be opened 2020-07-15 15:17:27 +03:00
rinpatch 58a4f350a8 Refactor gun pooling and simplify adapter option insertion
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.

Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
2020-07-15 15:17:27 +03:00
Alex Gleason 0d004a9d04
Email admins when a new unapproved account is up for review 2020-07-14 20:31:20 -05:00
Alex Gleason 48983e9421
Merge remote-tracking branch 'upstream/develop' into by-approval 2020-07-14 18:56:40 -05:00
Alex Gleason df3d1bf5e5
Add :approval_pending to User @type account_status 2020-07-14 18:56:36 -05:00
Alex Gleason 20d24741af
AdminAPI: Add PATCH /api/pleroma/admin/users/approve endpoint 2020-07-14 18:02:44 -05:00
Alex Gleason 33f1b29b2c
AdminAPI: Filter users by need_approval 2020-07-14 14:29:03 -05:00
Alex Gleason b750129da1
AdminAPI: Return approval_pending with users 2020-07-14 13:47:05 -05:00
Maksim Pechnikov 1dd767b8c7 Include port in host for signatures 2020-07-14 21:44:08 +03:00
Mark Felder 777a7edc6b Lint and fix test to match new log message 2020-07-14 13:15:37 -05:00
Mark Felder 37297a8482 Improve error messages 2020-07-14 13:12:16 -05:00
Alex Gleason a1570ba6ad
AdminAPI: Return registration_reason with users 2020-07-14 13:08:39 -05:00
feld 3f65f2ea79 Merge branch 'feature/1922-media-proxy-whitelist' into 'develop'
Support for hosts with scheme in MediaProxy whitelist setting

Closes #1922

See merge request pleroma/pleroma!2754
2020-07-14 18:07:44 +00:00
feld 63798e89de Merge branch 'fix/scrubber-recompile-warnings-as-errors' into 'develop'
Fix in-db configuration in dev environment

See merge request pleroma/pleroma!2764
2020-07-14 18:05:36 +00:00
feld 2909dc873b Merge branch '1940-admin-token-oauthless-auth' into 'develop'
[#1940] Reinstated OAuth-less `admin_token` authentication

Closes #1940

See merge request pleroma/pleroma!2760
2020-07-14 16:48:26 +00:00
Ivan Tashkinov 124b4709dc [#1940] Added admin_token param (as admin_api_params/0) to existing Admin API OpenAPI operations. 2020-07-14 19:31:05 +03:00
feld 246f49d685 Merge branch '1916-transmogrifier-skipping-oddities-in-address-fields' into 'develop'
Transmogrifier: filtering weirdness in address fields

Closes #1916

See merge request pleroma/pleroma!2762
2020-07-14 15:17:25 +00:00
rinpatch e6ccc25565 Fix in-db configuration in dev environment
Previously, in-db configuration only worked when `warnings_as_errors`
was disabled because re-compiling scrubbers on application restart
created a warning about module conflicts. This patch fixes that
by enabling `ignore_module_conflict` option of the compiler at runtime,
and enables `warnings_as_errors` in prod since there is no reason
to keep it disabled anymore.
2020-07-14 13:16:54 +03:00
Haelwenn (lanodan) Monnier 858d9fc7e8
MRF Policies: Return a {:reject, reason} instead of {:reject, nil} 2020-07-14 11:04:16 +02:00
Ivan Tashkinov 9b225db7d8 [#1940] Applied rate limit for requests with bad admin_token. Added doc warnings on admin_token setting. 2020-07-14 11:58:41 +03:00
Ivan Tashkinov cf3f8cb72a [#1940] Reinstated OAuth-less admin_token authentication. Refactored UserIsAdminPlug (freed from checking admin scopes presence). 2020-07-19 21:35:57 +03:00
Maksim Pechnikov 3062f86613 added generated pleroma.env 2020-07-14 09:05:34 +03:00