Merge branch 'develop' into issue/2099

2020-09-18 07:08:37 +03:00 · 2020-09-18 07:08:37 +03:00 · 2ec0dcf001
parent 5a8ea0a5b0 49584a9928
commit 2ec0dcf001
62 changed files with 1421 additions and 159 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -9,8 +9,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

 - Renamed `:await_up_timeout` in `:connections_pool` namespace to `:connect_timeout`, old name is deprecated.
 - Renamed `:timeout` in `pools` namespace to `:recv_timeout`, old name is deprecated.
+- The `discoverable` field in the `User` struct will now add a NOINDEX metatag to profile pages when false.
+- Users with the `discoverable` field set to false will not show up in searches.
 - Minimum lifetime for ephmeral activities changed to 10 minutes and made configurable (`:min_lifetime` option).
-
 ### Removed

 - **Breaking:** `Pleroma.Workers.Cron.StatsWorker` setting from Oban `:crontab` (moved to a simpler implementation).
@ -19,12 +20,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Removed `:managed_config` option. In practice, it was accidentally removed with 2.0.0 release when frontends were
 switched to a new configuration mechanism, however it was not officially removed until now.

-## unreleased-patch - ???
+## [2.1.2] - 2020-09-17
+
+### Security
+
+- Fix most MRF rules either crashing or not being applied to objects passed into the Common Pipeline (ChatMessage, Question, Answer, Audio, Event).

 ### Fixed

- Welcome Chat messages preventing user registration with MRF Simple Policy applied to the local instance
- Mastodon API: the public timeline returning an error when the `reply_visibility` parameter is set to `self` for an unauthenticated user
+- Welcome Chat messages preventing user registration with MRF Simple Policy applied to the local instance.
+- Mastodon API: the public timeline returning an error when the `reply_visibility` parameter is set to `self` for an unauthenticated user.
+- Mastodon Streaming API: Handler crashes on authentication failures, resulting in error logs.
+- Mastodon Streaming API: Error logs on client pings.
+- Rich media: Log spam on failures. Now the error is only logged once per attempt.
+
+### Changed
+
+- Rich Media: A HEAD request is now done to the url, to ensure it has the appropriate content type and size before proceeding with a GET.
+
+### Upgrade notes
+
+1. Restart Pleroma

 ## [2.1.1] - 2020-09-08

@ -41,6 +57,12 @@ switched to a new configuration mechanism, however it was not officially removed
 ### Added
 - Rich media failure tracking (along with `:failure_backoff` option).

+<details>
+  <summary>Admin API Changes</summary>
+
+- Add `PATCH /api/pleroma/admin/instance_document/:document_name` to modify the Terms of Service and Instance Panel HTML pages via Admin API
+</details>
+
 ### Fixed
 - Default HTTP adapter not respecting pool setting, leading to possible OOM.
 - Fixed uploading webp images when the Exiftool Upload Filter is enabled by skipping them
--- a/docs/API/admin_api.md
+++ b/docs/API/admin_api.md
@ -1334,3 +1334,166 @@ Loads json generated from `config/descriptions.exs`.
 { }

 ```
+
+## GET /api/pleroma/admin/users/:nickname/chats
+
+### List a user's chats
+
+- Params: None
+
+- Response:
+
+```json
+[
+   {
+      "sender": {
+        "id": "someflakeid",
+        "username": "somenick",
+        ...
+      },
+      "receiver": {
+        "id": "someflakeid",
+        "username": "somenick",
+        ...
+      },
+      "id" : "1",
+      "unread" : 2,
+      "last_message" : {...}, // The last message in that chat
+      "updated_at": "2020-04-21T15:11:46.000Z"
+   }
+]
+```
+
+## GET /api/pleroma/admin/chats/:chat_id
+
+### View a single chat
+
+- Params: None
+
+- Response:
+
+```json
+{
+  "sender": {
+    "id": "someflakeid",
+    "username": "somenick",
+    ...
+  },
+  "receiver": {
+    "id": "someflakeid",
+    "username": "somenick",
+    ...
+  },
+  "id" : "1",
+  "unread" : 2,
+  "last_message" : {...}, // The last message in that chat
+  "updated_at": "2020-04-21T15:11:46.000Z"
+}
+```
+
+## GET /api/pleroma/admin/chats/:chat_id/messages
+
+### List the messages in a chat
+
+- Params: `max_id`, `min_id`
+
+- Response:
+
+```json
+[
+  {
+    "account_id": "someflakeid",
+    "chat_id": "1",
+    "content": "Check this out :firefox:",
+    "created_at": "2020-04-21T15:11:46.000Z",
+    "emojis": [
+      {
+        "shortcode": "firefox",
+        "static_url": "https://dontbulling.me/emoji/Firefox.gif",
+        "url": "https://dontbulling.me/emoji/Firefox.gif",
+        "visible_in_picker": false
+      }
+    ],
+    "id": "13",
+    "unread": true
+  },
+  {
+    "account_id": "someflakeid",
+    "chat_id": "1",
+    "content": "Whats' up?",
+    "created_at": "2020-04-21T15:06:45.000Z",
+    "emojis": [],
+    "id": "12",
+    "unread": false
+  }
+]
+```
+
+## DELETE /api/pleroma/admin/chats/:chat_id/messages/:message_id
+
+### Delete a single message
+
+- Params: None
+
+- Response:
+
+```json
+{
+  "account_id": "someflakeid",
+  "chat_id": "1",
+  "content": "Check this out :firefox:",
+  "created_at": "2020-04-21T15:11:46.000Z",
+  "emojis": [
+    {
+      "shortcode": "firefox",
+      "static_url": "https://dontbulling.me/emoji/Firefox.gif",
+      "url": "https://dontbulling.me/emoji/Firefox.gif",
+      "visible_in_picker": false
+    }
+  ],
+  "id": "13",
+  "unread": false
+}
+```
+
+## `GET /api/pleroma/admin/instance_document/:document_name`
+
+### Get an instance document
+
+- Authentication: required
+
+- Response:
+
+Returns the content of the document
+
+```html
+<h1>Instance panel</h1>
+```
+
+## `PATCH /api/pleroma/admin/instance_document/:document_name`
+- Params:
+  - `file` (the file to be uploaded, using multipart form data.)
+
+### Update an instance document
+
+- Authentication: required
+
+- Response:
+
+``` json
+{
+  "url": "https://example.com/instance/panel.html"
+}
+```
+
+## `DELETE /api/pleroma/admin/instance_document/:document_name`
+
+### Delete an instance document
+
+- Response:
+
+``` json
+{
+  "url": "https://example.com/instance/panel.html"
+}
+```
--- a/lib/mix/tasks/pleroma/config.ex
+++ b/lib/mix/tasks/pleroma/config.ex
@ -32,7 +32,8 @@ def run(["migrate_from_db" | options]) do

  @spec migrate_to_db(Path.t() | nil) :: any()
  def migrate_to_db(file_path \\ nil) do
-    if Pleroma.Config.get([:configurable_from_database]) do
+    with true <- Pleroma.Config.get([:configurable_from_database]),
+         :ok <- Pleroma.Config.DeprecationWarnings.warn() do
      config_file =
        if file_path do
          file_path
@ -46,7 +47,8 @@ def migrate_to_db(file_path \\ nil) do

      do_migrate_to_db(config_file)
    else
-      migration_error()
+      :error -> deprecation_error()
+      _ -> migration_error()
    end
  end

@ -120,6 +122,10 @@ defp migration_error do
    )
  end

+  defp deprecation_error do
+    shell_error("Migration is not allowed until all deprecation warnings have been resolved.")
+  end
+
  if Code.ensure_loaded?(Config.Reader) do
    defp config_header, do: "import Config\r\n\r\n"
    defp read_file(config_file), do: Config.Reader.read_imports!(config_file)
--- a/lib/pleroma/chat.ex
+++ b/lib/pleroma/chat.ex
@ -6,7 +6,9 @@ defmodule Pleroma.Chat do
  use Ecto.Schema

  import Ecto.Changeset
+  import Ecto.Query

+  alias Pleroma.Chat
  alias Pleroma.Repo
  alias Pleroma.User

@ -69,4 +71,12 @@ def bump_or_create(user_id, recipient) do
      conflict_target: [:user_id, :recipient]
    )
  end
+
+  @spec for_user_query(FlakeId.Ecto.CompatType.t()) :: Ecto.Query.t()
+  def for_user_query(user_id) do
+    from(c in Chat,
+      where: c.user_id == ^user_id,
+      order_by: [desc: c.updated_at]
+    )
+  end
 end
--- a/lib/pleroma/config/deprecation_warnings.ex
+++ b/lib/pleroma/config/deprecation_warnings.ex
@ -26,6 +26,10 @@ def check_hellthread_threshold do
      !!!DEPRECATION WARNING!!!
      You are using the old configuration mechanism for the hellthread filter. Please check config.md.
      """)
+
+      :error
+    else
+      :ok
    end
  end

@ -47,17 +51,26 @@ def mrf_user_allowlist do

      config :pleroma, :mrf_user_allowlist, #{inspect(rewritten, pretty: true)}
      """)
+
+      :error
+    else
+      :ok
    end
  end

  def warn do
-    check_hellthread_threshold()
-    mrf_user_allowlist()
-    check_old_mrf_config()
-    check_media_proxy_whitelist_config()
-    check_welcome_message_config()
-    check_gun_pool_options()
-    check_activity_expiration_config()
+    with :ok <- check_hellthread_threshold(),
+         :ok <- mrf_user_allowlist(),
+         :ok <- check_old_mrf_config(),
+         :ok <- check_media_proxy_whitelist_config(),
+         :ok <- check_welcome_message_config(),
+         :ok <- check_gun_pool_options(),
+         :ok <- check_activity_expiration_config() do
+      :ok
+    else
+      _ ->
+        :error
+    end
  end

  def check_welcome_message_config do
@ -74,6 +87,10 @@ def check_welcome_message_config do
      \n* `config :pleroma, :instance, welcome_user_nickname` is now `config :pleroma, :welcome, :direct_message, :sender_nickname`
      \n* `config :pleroma, :instance, welcome_message` is now `config :pleroma, :welcome, :direct_message, :message`
      """)
+
+      :error
+    else
+      :ok
    end
  end

@ -101,8 +118,11 @@ def move_namespace_and_warn(config_map, warning_preface) do
          end
      end)

-    if warning != "" do
+    if warning == "" do
+      :ok
+    else
      Logger.warn(warning_preface <> warning)
+      :error
    end
  end

@ -115,6 +135,10 @@ def check_media_proxy_whitelist_config do
      !!!DEPRECATION WARNING!!!
      Your config is using old format (only domain) for MediaProxy whitelist option. Setting should work for now, but you are advised to change format to scheme with port to prevent possible issues later.
      """)
+
+      :error
+    else
+      :ok
    end
  end

@ -157,6 +181,9 @@ def check_gun_pool_options do
      Logger.warn(Enum.join([warning_preface | pool_warnings]))

      Config.put(:pools, updated_config)
+      :error
+    else
+      :ok
    end
  end

--- a/lib/pleroma/moderation_log.ex
+++ b/lib/pleroma/moderation_log.ex
@ -320,6 +320,19 @@ def insert_log(%{
    |> insert_log_entry_with_message()
  end

+  @spec insert_log(%{actor: User, action: String.t(), subject_id: String.t()}) ::
+          {:ok, ModerationLog} | {:error, any}
+  def insert_log(%{actor: %User{} = actor, action: "chat_message_delete", subject_id: subject_id}) do
+    %ModerationLog{
+      data: %{
+        "actor" => %{"nickname" => actor.nickname},
+        "action" => "chat_message_delete",
+        "subject_id" => subject_id
+      }
+    }
+    |> insert_log_entry_with_message()
+  end
+
  @spec insert_log_entry_with_message(ModerationLog) :: {:ok, ModerationLog} | {:error, any}
  defp insert_log_entry_with_message(entry) do
    entry.data["message"]
@ -627,6 +640,17 @@ def get_log_entry_message(%ModerationLog{
    "@#{actor_nickname} updated users: #{users_to_nicknames_string(subjects)}"
  end

+  @spec get_log_entry_message(ModerationLog) :: String.t()
+  def get_log_entry_message(%ModerationLog{
+        data: %{
+          "actor" => %{"nickname" => actor_nickname},
+          "action" => "chat_message_delete",
+          "subject_id" => subject_id
+        }
+      }) do
+    "@#{actor_nickname} deleted chat message ##{subject_id}"
+  end
+
  defp nicknames_to_string(nicknames) do
    nicknames
    |> Enum.map(&"@#{&1}")
--- a/lib/pleroma/user/search.ex
+++ b/lib/pleroma/user/search.ex
@ -52,6 +52,7 @@ defp search_query(query_string, for_user, following) do
    |> base_query(following)
    |> filter_blocked_user(for_user)
    |> filter_invisible_users()
+    |> filter_discoverable_users()
    |> filter_internal_users()
    |> filter_blocked_domains(for_user)
    |> fts_search(query_string)
@ -122,6 +123,10 @@ defp filter_invisible_users(query) do
    from(q in query, where: q.invisible == false)
  end

+  defp filter_discoverable_users(query) do
+    from(q in query, where: q.discoverable == true)
+  end
+
  defp filter_internal_users(query) do
    from(q in query, where: q.actor_type != "Application")
  end
--- a/lib/pleroma/web/activity_pub/mrf.ex
+++ b/lib/pleroma/web/activity_pub/mrf.ex
@ -5,16 +5,34 @@
 defmodule Pleroma.Web.ActivityPub.MRF do
  @callback filter(Map.t()) :: {:ok | :reject, Map.t()}

-  def filter(policies, %{} = object) do
+  def filter(policies, %{} = message) do
    policies
-    |> Enum.reduce({:ok, object}, fn
-      policy, {:ok, object} -> policy.filter(object)
+    |> Enum.reduce({:ok, message}, fn
+      policy, {:ok, message} -> policy.filter(message)
      _, error -> error
    end)
  end

  def filter(%{} = object), do: get_policies() |> filter(object)

+  def pipeline_filter(%{} = message, meta) do
+    object = meta[:object_data]
+    ap_id = message["object"]
+
+    if object && ap_id do
+      with {:ok, message} <- filter(Map.put(message, "object", object)) do
+        meta = Keyword.put(meta, :object_data, message["object"])
+        {:ok, Map.put(message, "object", ap_id), meta}
+      else
+        {err, message} -> {err, message, meta}
+      end
+    else
+      {err, message} = filter(message)
+
+      {err, message, meta}
+    end
+  end
+
  def get_policies do
    Pleroma.Config.get([:mrf, :policies], []) |> get_policies()
  end
--- a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex
@ -20,9 +20,17 @@ defp string_matches?(string, pattern) do
    String.match?(string, pattern)
  end

-  defp check_reject(%{"object" => %{"content" => content, "summary" => summary}} = message) do
+  defp object_payload(%{} = object) do
+    [object["content"], object["summary"], object["name"]]
+    |> Enum.filter(& &1)
+    |> Enum.join("\n")
+  end
+
+  defp check_reject(%{"object" => %{} = object} = message) do
+    payload = object_payload(object)
+
    if Enum.any?(Pleroma.Config.get([:mrf_keyword, :reject]), fn pattern ->
-         string_matches?(content, pattern) or string_matches?(summary, pattern)
+         string_matches?(payload, pattern)
       end) do
      {:reject, "[KeywordPolicy] Matches with rejected keyword"}
    else
@ -30,12 +38,12 @@ defp check_reject(%{"object" => %{"content" => content, "summary" => summary}} =
    end
  end

-  defp check_ftl_removal(
-         %{"to" => to, "object" => %{"content" => content, "summary" => summary}} = message
-       ) do
+  defp check_ftl_removal(%{"to" => to, "object" => %{} = object} = message) do
+    payload = object_payload(object)
+
    if Pleroma.Constants.as_public() in to and
         Enum.any?(Pleroma.Config.get([:mrf_keyword, :federated_timeline_removal]), fn pattern ->
-           string_matches?(content, pattern) or string_matches?(summary, pattern)
+           string_matches?(payload, pattern)
         end) do
      to = List.delete(to, Pleroma.Constants.as_public())
      cc = [Pleroma.Constants.as_public() | message["cc"] || []]
@ -51,35 +59,24 @@ defp check_ftl_removal(
    end
  end

-  defp check_replace(%{"object" => %{"content" => content, "summary" => summary}} = message) do
-    content =
-      if is_binary(content) do
-        content
-      else
-        ""
-      end
-
-    summary =
-      if is_binary(summary) do
-        summary
-      else
-        ""
-      end
-
-    {content, summary} =
+  defp check_replace(%{"object" => %{} = object} = message) do
+    object =
+      ["content", "name", "summary"]
+      |> Enum.filter(fn field -> Map.has_key?(object, field) && object[field] end)
+      |> Enum.reduce(object, fn field, object ->
+        data =
          Enum.reduce(
            Pleroma.Config.get([:mrf_keyword, :replace]),
-        {content, summary},
-        fn {pattern, replacement}, {content_acc, summary_acc} ->
-          {String.replace(content_acc, pattern, replacement),
-           String.replace(summary_acc, pattern, replacement)}
-        end
+            object[field],
+            fn {pat, repl}, acc -> String.replace(acc, pat, repl) end
          )

-    {:ok,
-     message
-     |> put_in(["object", "content"], content)
-     |> put_in(["object", "summary"], summary)}
+        Map.put(object, field, data)
+      end)
+
+    message = Map.put(message, "object", object)
+
+    {:ok, message}
  end

  @impl true
--- a/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex
@ -28,8 +28,7 @@ def filter(%{"actor" => actor} = message) do
        }"
      )

-      subchain
-      |> MRF.filter(message)
+      MRF.filter(subchain, message)
    else
      _e -> {:ok, message}
    end
--- a/lib/pleroma/web/activity_pub/pipeline.ex
+++ b/lib/pleroma/web/activity_pub/pipeline.ex
@ -26,13 +26,17 @@ def common_pipeline(object, meta) do

      {:error, e} ->
        {:error, e}
+
+      {:reject, e} ->
+        {:reject, e}
    end
  end

  def do_common_pipeline(object, meta) do
    with {_, {:ok, validated_object, meta}} <-
           {:validate_object, ObjectValidator.validate(object, meta)},
-         {_, {:ok, mrfd_object}} <- {:mrf_object, MRF.filter(validated_object)},
+         {_, {:ok, mrfd_object, meta}} <-
+           {:mrf_object, MRF.pipeline_filter(validated_object, meta)},
         {_, {:ok, activity, meta}} <-
           {:persist_object, ActivityPub.persist(mrfd_object, meta)},
         {_, {:ok, activity, meta}} <-
@ -40,7 +44,7 @@ def do_common_pipeline(object, meta) do
         {_, {:ok, _}} <- {:federation, maybe_federate(activity, meta)} do
      {:ok, activity, meta}
    else
-      {:mrf_object, {:reject, _}} -> {:ok, nil, meta}
+      {:mrf_object, {:reject, message, _}} -> {:reject, message}
      e -> {:error, e}
    end
  end
--- a/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
@ -23,8 +23,6 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  alias Pleroma.Web.Endpoint
  alias Pleroma.Web.Router

-  require Logger
-
  @users_page_size 50

  plug(
@ -68,6 +66,12 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
    when action in [:list_user_statuses, :list_instance_statuses]
  )

+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["read:chats"], admin: true}
+    when action in [:list_user_chats]
+  )
+
  plug(
    OAuthScopesPlug,
    %{scopes: ["read"], admin: true}
@ -256,6 +260,20 @@ def list_user_statuses(%{assigns: %{user: admin}} = conn, %{"nickname" => nickna
    end
  end

+  def list_user_chats(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname} = _params) do
+    with %User{id: user_id} <- User.get_cached_by_nickname_or_id(nickname, for: admin) do
+      chats =
+        Pleroma.Chat.for_user_query(user_id)
+        |> Pleroma.Repo.all()
+
+      conn
+      |> put_view(AdminAPI.ChatView)
+      |> render("index.json", chats: chats)
+    else
+      _ -> {:error, :not_found}
+    end
+  end
+
  def user_toggle_activation(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
    user = User.get_cached_by_nickname(nickname)

--- a/lib/pleroma/web/admin_api/controllers/chat_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/chat_controller.ex
@ -0,0 +1,85 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.ChatController do
+  use Pleroma.Web, :controller
+
+  alias Pleroma.Activity
+  alias Pleroma.Chat
+  alias Pleroma.Chat.MessageReference
+  alias Pleroma.ModerationLog
+  alias Pleroma.Pagination
+  alias Pleroma.Plugs.OAuthScopesPlug
+  alias Pleroma.Web.AdminAPI
+  alias Pleroma.Web.CommonAPI
+  alias Pleroma.Web.PleromaAPI.Chat.MessageReferenceView
+
+  require Logger
+
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["read:chats"], admin: true} when action in [:show, :messages]
+  )
+
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["write:chats"], admin: true} when action in [:delete_message]
+  )
+
+  action_fallback(Pleroma.Web.AdminAPI.FallbackController)
+
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.ChatOperation
+
+  def delete_message(%{assigns: %{user: user}} = conn, %{
+        message_id: message_id,
+        id: chat_id
+      }) do
+    with %MessageReference{object: %{data: %{"id" => object_ap_id}}} = cm_ref <-
+           MessageReference.get_by_id(message_id),
+         ^chat_id <- to_string(cm_ref.chat_id),
+         %Activity{id: activity_id} <- Activity.get_create_by_object_ap_id(object_ap_id),
+         {:ok, _} <- CommonAPI.delete(activity_id, user) do
+      ModerationLog.insert_log(%{
+        action: "chat_message_delete",
+        actor: user,
+        subject_id: message_id
+      })
+
+      conn
+      |> put_view(MessageReferenceView)
+      |> render("show.json", chat_message_reference: cm_ref)
+    else
+      _e ->
+        {:error, :could_not_delete}
+    end
+  end
+
+  def messages(conn, %{id: id} = params) do
+    with %Chat{} = chat <- Chat.get_by_id(id) do
+      cm_refs =
+        chat
+        |> MessageReference.for_chat_query()
+        |> Pagination.fetch_paginated(params)
+
+      conn
+      |> put_view(MessageReferenceView)
+      |> render("index.json", chat_message_references: cm_refs)
+    else
+      _ ->
+        conn
+        |> put_status(:not_found)
+        |> json(%{error: "not found"})
+    end
+  end
+
+  def show(conn, %{id: id}) do
+    with %Chat{} = chat <- Chat.get_by_id(id) do
+      conn
+      |> put_view(AdminAPI.ChatView)
+      |> render("show.json", chat: chat)
+    end
+  end
+end
--- a/lib/pleroma/web/admin_api/controllers/instance_document_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/instance_document_controller.ex
@ -0,0 +1,41 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.InstanceDocumentController do
+  use Pleroma.Web, :controller
+
+  alias Pleroma.Plugs.InstanceStatic
+  alias Pleroma.Plugs.OAuthScopesPlug
+  alias Pleroma.Web.InstanceDocument
+
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
+  action_fallback(Pleroma.Web.AdminAPI.FallbackController)
+
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.InstanceDocumentOperation
+
+  plug(OAuthScopesPlug, %{scopes: ["read"], admin: true} when action == :show)
+  plug(OAuthScopesPlug, %{scopes: ["write"], admin: true} when action in [:update, :delete])
+
+  def show(conn, %{name: document_name}) do
+    with {:ok, url} <- InstanceDocument.get(document_name),
+         {:ok, content} <- File.read(InstanceStatic.file_path(url)) do
+      conn
+      |> put_resp_content_type("text/html")
+      |> send_resp(200, content)
+    end
+  end
+
+  def update(%{body_params: %{file: file}} = conn, %{name: document_name}) do
+    with {:ok, url} <- InstanceDocument.put(document_name, file.path) do
+      json(conn, %{"url" => url})
+    end
+  end
+
+  def delete(conn, %{name: document_name}) do
+    with :ok <- InstanceDocument.delete(document_name) do
+      json(conn, %{})
+    end
+  end
+end
--- a/lib/pleroma/web/admin_api/views/chat_view.ex
+++ b/lib/pleroma/web/admin_api/views/chat_view.ex
@ -0,0 +1,30 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.ChatView do
+  use Pleroma.Web, :view
+
+  alias Pleroma.Chat
+  alias Pleroma.User
+  alias Pleroma.Web.MastodonAPI
+  alias Pleroma.Web.PleromaAPI
+
+  def render("index.json", %{chats: chats} = opts) do
+    render_many(chats, __MODULE__, "show.json", Map.delete(opts, :chats))
+  end
+
+  def render("show.json", %{chat: %Chat{user_id: user_id}} = opts) do
+    user = User.get_by_id(user_id)
+    sender = MastodonAPI.AccountView.render("show.json", user: user, skip_visibility_check: true)
+
+    serialized_chat = PleromaAPI.ChatView.render("show.json", opts)
+
+    serialized_chat
+    |> Map.put(:sender, sender)
+    |> Map.put(:receiver, serialized_chat[:account])
+    |> Map.delete(:account)
+  end
+
+  def render(view, opts), do: PleromaAPI.ChatView.render(view, opts)
+end
--- a/lib/pleroma/web/admin_api/views/status_view.ex
+++ b/lib/pleroma/web/admin_api/views/status_view.ex
@ -8,6 +8,7 @@ defmodule Pleroma.Web.AdminAPI.StatusView do
  require Pleroma.Constants

  alias Pleroma.Web.AdminAPI
+  alias Pleroma.Web.CommonAPI
  alias Pleroma.Web.MastodonAPI

  defdelegate merge_account_views(user), to: AdminAPI.AccountView
@ -17,7 +18,7 @@ def render("index.json", opts) do
  end

  def render("show.json", %{activity: %{data: %{"object" => _object}} = activity} = opts) do
-    user = MastodonAPI.StatusView.get_user(activity.data["actor"])
+    user = CommonAPI.get_user(activity.data["actor"])

    MastodonAPI.StatusView.render("show.json", opts)
    |> Map.merge(%{account: merge_account_views(user)})
--- a/lib/pleroma/web/api_spec/operations/admin/chat_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/admin/chat_operation.ex
@ -0,0 +1,96 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Admin.ChatOperation do
+  alias OpenApiSpex.Operation
+  alias Pleroma.Web.ApiSpec.Schemas.Chat
+  alias Pleroma.Web.ApiSpec.Schemas.ChatMessage
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def delete_message_operation do
+    %Operation{
+      tags: ["admin", "chat"],
+      summary: "Delete an individual chat message",
+      operationId: "AdminAPI.ChatController.delete_message",
+      parameters: [
+        Operation.parameter(:id, :path, :string, "The ID of the Chat"),
+        Operation.parameter(:message_id, :path, :string, "The ID of the message")
+      ],
+      responses: %{
+        200 =>
+          Operation.response(
+            "The deleted ChatMessage",
+            "application/json",
+            ChatMessage
+          )
+      },
+      security: [
+        %{
+          "oAuth" => ["write:chats"]
+        }
+      ]
+    }
+  end
+
+  def messages_operation do
+    %Operation{
+      tags: ["admin", "chat"],
+      summary: "Get the most recent messages of the chat",
+      operationId: "AdminAPI.ChatController.messages",
+      parameters:
+        [Operation.parameter(:id, :path, :string, "The ID of the Chat")] ++
+          pagination_params(),
+      responses: %{
+        200 =>
+          Operation.response(
+            "The messages in the chat",
+            "application/json",
+            Pleroma.Web.ApiSpec.ChatOperation.chat_messages_response()
+          )
+      },
+      security: [
+        %{
+          "oAuth" => ["read:chats"]
+        }
+      ]
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["chat"],
+      summary: "Create a chat",
+      operationId: "AdminAPI.ChatController.show",
+      parameters: [
+        Operation.parameter(
+          :id,
+          :path,
+          :string,
+          "The id of the chat",
+          required: true,
+          example: "1234"
+        )
+      ],
+      responses: %{
+        200 =>
+          Operation.response(
+            "The existing chat",
+            "application/json",
+            Chat
+          )
+      },
+      security: [
+        %{
+          "oAuth" => ["read"]
+        }
+      ]
+    }
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/admin/instance_document_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/admin/instance_document_operation.ex
@ -0,0 +1,115 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Admin.InstanceDocumentOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Admin", "InstanceDocument"],
+      summary: "Get the instance document",
+      operationId: "AdminAPI.InstanceDocumentController.show",
+      security: [%{"oAuth" => ["read"]}],
+      parameters: [
+        Operation.parameter(:name, :path, %Schema{type: :string}, "The document name",
+          required: true
+        )
+        | Helpers.admin_api_params()
+      ],
+      responses: %{
+        200 => document_content(),
+        400 => Operation.response("Bad Request", "application/json", ApiError),
+        403 => Operation.response("Forbidden", "application/json", ApiError),
+        404 => Operation.response("Not Found", "application/json", ApiError)
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["Admin", "InstanceDocument"],
+      summary: "Update the instance document",
+      operationId: "AdminAPI.InstanceDocumentController.update",
+      security: [%{"oAuth" => ["write"]}],
+      requestBody: Helpers.request_body("Parameters", update_request()),
+      parameters: [
+        Operation.parameter(:name, :path, %Schema{type: :string}, "The document name",
+          required: true
+        )
+        | Helpers.admin_api_params()
+      ],
+      responses: %{
+        200 => Operation.response("InstanceDocument", "application/json", instance_document()),
+        400 => Operation.response("Bad Request", "application/json", ApiError),
+        403 => Operation.response("Forbidden", "application/json", ApiError),
+        404 => Operation.response("Not Found", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp update_request do
+    %Schema{
+      title: "UpdateRequest",
+      description: "POST body for uploading the file",
+      type: :object,
+      required: [:file],
+      properties: %{
+        file: %Schema{
+          type: :string,
+          format: :binary,
+          description: "The file to be uploaded, using multipart form data."
+        }
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["Admin", "InstanceDocument"],
+      summary: "Get the instance document",
+      operationId: "AdminAPI.InstanceDocumentController.delete",
+      security: [%{"oAuth" => ["write"]}],
+      parameters: [
+        Operation.parameter(:name, :path, %Schema{type: :string}, "The document name",
+          required: true
+        )
+        | Helpers.admin_api_params()
+      ],
+      responses: %{
+        200 => Operation.response("InstanceDocument", "application/json", instance_document()),
+        400 => Operation.response("Bad Request", "application/json", ApiError),
+        403 => Operation.response("Forbidden", "application/json", ApiError),
+        404 => Operation.response("Not Found", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp instance_document do
+    %Schema{
+      title: "InstanceDocument",
+      type: :object,
+      properties: %{
+        url: %Schema{type: :string}
+      },
+      example: %{
+        "url" => "https://example.com/static/terms-of-service.html"
+      }
+    }
+  end
+
+  defp document_content do
+    Operation.response("InstanceDocumentContent", "text/html", %Schema{
+      type: :string,
+      example: "<h1>Instance panel</h1>"
+    })
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/chat_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/chat_operation.ex
@ -184,7 +184,8 @@ def post_chat_message_operation do
            "application/json",
            ChatMessage
          ),
-        400 => Operation.response("Bad Request", "application/json", ApiError)
+        400 => Operation.response("Bad Request", "application/json", ApiError),
+        422 => Operation.response("MRF Rejection", "application/json", ApiError)
      },
      security: [
        %{
--- a/lib/pleroma/web/api_spec/operations/status_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/status_operation.ex
@ -55,7 +55,7 @@ def create_operation do
            "application/json",
            %Schema{oneOf: [Status, ScheduledStatus]}
          ),
-        422 => Operation.response("Bad Request", "application/json", ApiError)
+        422 => Operation.response("Bad Request / MRF Rejection", "application/json", ApiError)
      }
    }
  end
--- a/lib/pleroma/web/common_api/common_api.ex
+++ b/lib/pleroma/web/common_api/common_api.ex
@ -48,6 +48,9 @@ def post_chat_message(%User{} = user, %User{} = recipient, content, opts \\ [])
              local: true
            )} do
      {:ok, activity}
+    else
+      {:common_pipeline, {:reject, _} = e} -> e
+      e -> e
    end
  end

@ -550,4 +553,21 @@ def hide_reblogs(%User{} = user, %User{} = target) do
  def show_reblogs(%User{} = user, %User{} = target) do
    UserRelationship.delete_reblog_mute(user, target)
  end
+
+  def get_user(ap_id, fake_record_fallback \\ true) do
+    cond do
+      user = User.get_cached_by_ap_id(ap_id) ->
+        user
+
+      user = User.get_by_guessed_nickname(ap_id) ->
+        user
+
+      fake_record_fallback ->
+        # TODO: refactor (fake records is never a good idea)
+        User.error_user(ap_id)
+
+      true ->
+        nil
+    end
+  end
 end
--- a/lib/pleroma/web/instance_document.ex
+++ b/lib/pleroma/web/instance_document.ex
@ -0,0 +1,62 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.InstanceDocument do
+  alias Pleroma.Config
+  alias Pleroma.Web.Endpoint
+
+  @instance_documents %{
+    "terms-of-service" => "/static/terms-of-service.html",
+    "instance-panel" => "/instance/panel.html"
+  }
+
+  @spec get(String.t()) :: {:ok, String.t()} | {:error, atom()}
+  def get(document_name) do
+    case Map.fetch(@instance_documents, document_name) do
+      {:ok, path} -> {:ok, path}
+      _ -> {:error, :not_found}
+    end
+  end
+
+  @spec put(String.t(), String.t()) :: {:ok, String.t()} | {:error, atom()}
+  def put(document_name, origin_path) do
+    with {_, {:ok, destination_path}} <-
+           {:instance_document, Map.fetch(@instance_documents, document_name)},
+         :ok <- put_file(origin_path, destination_path) do
+      {:ok, Path.join(Endpoint.url(), destination_path)}
+    else
+      {:instance_document, :error} -> {:error, :not_found}
+      error -> error
+    end
+  end
+
+  @spec delete(String.t()) :: :ok | {:error, atom()}
+  def delete(document_name) do
+    with {_, {:ok, path}} <- {:instance_document, Map.fetch(@instance_documents, document_name)},
+         instance_static_dir_path <- instance_static_dir(path),
+         :ok <- File.rm(instance_static_dir_path) do
+      :ok
+    else
+      {:instance_document, :error} -> {:error, :not_found}
+      {:error, :enoent} -> {:error, :not_found}
+      error -> error
+    end
+  end
+
+  defp put_file(origin_path, destination_path) do
+    with destination <- instance_static_dir(destination_path),
+         {_, :ok} <- {:mkdir_p, File.mkdir_p(Path.dirname(destination))},
+         {_, {:ok, _}} <- {:copy, File.copy(origin_path, destination)} do
+      :ok
+    else
+      {error, _} -> {:error, error}
+    end
+  end
+
+  defp instance_static_dir(filename) do
+    [:instance, :static_dir]
+    |> Config.get!()
+    |> Path.join(filename)
+  end
+end
--- a/lib/pleroma/web/mastodon_api/views/status_view.ex
+++ b/lib/pleroma/web/mastodon_api/views/status_view.ex
@ -55,23 +55,6 @@ defp get_replied_to_activities(activities) do
    end)
  end

-  def get_user(ap_id, fake_record_fallback \\ true) do
-    cond do
-      user = User.get_cached_by_ap_id(ap_id) ->
-        user
-
-      user = User.get_by_guessed_nickname(ap_id) ->
-        user
-
-      fake_record_fallback ->
-        # TODO: refactor (fake records is never a good idea)
-        User.error_user(ap_id)
-
-      true ->
-        nil
-    end
-  end
-
  defp get_context_id(%{data: %{"context_id" => context_id}}) when not is_nil(context_id),
    do: context_id

@ -119,7 +102,7 @@ def render("index.json", opts) do
          # Note: unresolved users are filtered out
          actors =
            (activities ++ parent_activities)
-            |> Enum.map(&get_user(&1.data["actor"], false))
+            |> Enum.map(&CommonAPI.get_user(&1.data["actor"], false))
            |> Enum.filter(& &1)

          UserRelationship.view_relationships_option(reading_user, actors, subset: :source_mutes)
@ -138,7 +121,7 @@ def render(
        "show.json",
        %{activity: %{data: %{"type" => "Announce", "object" => _object}} = activity} = opts
      ) do
-    user = get_user(activity.data["actor"])
+    user = CommonAPI.get_user(activity.data["actor"])
    created_at = Utils.to_masto_date(activity.data["published"])
    activity_object = Object.normalize(activity)

@ -211,7 +194,7 @@ def render(
  def render("show.json", %{activity: %{data: %{"object" => _object}} = activity} = opts) do
    object = Object.normalize(activity)

-    user = get_user(activity.data["actor"])
+    user = CommonAPI.get_user(activity.data["actor"])
    user_follower_address = user.follower_address

    like_count = object.data["like_count"] || 0
@ -265,7 +248,7 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}

    reply_to = get_reply_to(activity, opts)

-    reply_to_user = reply_to && get_user(reply_to.data["actor"])
+    reply_to_user = reply_to && CommonAPI.get_user(reply_to.data["actor"])

    content =
      object
--- a/lib/pleroma/web/metadata/restrict_indexing.ex
+++ b/lib/pleroma/web/metadata/restrict_indexing.ex
@ -10,7 +10,9 @@ defmodule Pleroma.Web.Metadata.Providers.RestrictIndexing do
  """

  @impl true
-  def build_tags(%{user: %{local: false}}) do
+  def build_tags(%{user: %{local: true, discoverable: true}}), do: []
+
+  def build_tags(_) do
    [
      {:meta,
       [
@ -19,7 +21,4 @@ def build_tags(%{user: %{local: false}}) do
       ], []}
    ]
  end
-
-  @impl true
-  def build_tags(%{user: %{local: true}}), do: []
 end
--- a/lib/pleroma/web/pleroma_api/controllers/chat_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/chat_controller.ex
@ -90,6 +90,16 @@ def post_chat_message(
      conn
      |> put_view(MessageReferenceView)
      |> render("show.json", chat_message_reference: cm_ref)
+    else
+      {:reject, message} ->
+        conn
+        |> put_status(:unprocessable_entity)
+        |> json(%{error: message})
+
+      {:error, message} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: message})
    end
  end

@ -146,11 +156,8 @@ def index(%{assigns: %{user: %{id: user_id} = user}} = conn, _params) do
    blocked_ap_ids = User.blocked_users_ap_ids(user)

    chats =
-      from(c in Chat,
-        where: c.user_id == ^user_id,
-        where: c.recipient not in ^blocked_ap_ids,
-        order_by: [desc: c.updated_at]
-      )
+      Chat.for_user_query(user_id)
+      |> where([c], c.recipient not in ^blocked_ap_ids)
      |> Repo.all()

    conn
--- a/lib/pleroma/web/pleroma_api/views/scrobble_view.ex
+++ b/lib/pleroma/web/pleroma_api/views/scrobble_view.ex
@ -10,14 +10,14 @@ defmodule Pleroma.Web.PleromaAPI.ScrobbleView do
  alias Pleroma.Activity
  alias Pleroma.HTML
  alias Pleroma.Object
+  alias Pleroma.Web.CommonAPI
  alias Pleroma.Web.CommonAPI.Utils
  alias Pleroma.Web.MastodonAPI.AccountView
-  alias Pleroma.Web.MastodonAPI.StatusView

  def render("show.json", %{activity: %Activity{data: %{"type" => "Listen"}} = activity} = opts) do
    object = Object.normalize(activity)

-    user = StatusView.get_user(activity.data["actor"])
+    user = CommonAPI.get_user(activity.data["actor"])
    created_at = Utils.to_masto_date(activity.data["published"])

    %{
--- a/lib/pleroma/web/rich_media/parser.ex
+++ b/lib/pleroma/web/rich_media/parser.ex
@ -20,35 +20,60 @@ def parse(url) do
      with {:ok, data} <- get_cached_or_parse(url),
           {:ok, _} <- set_ttl_based_on_image(data, url) do
        {:ok, data}
-      else
-        {:error, {:invalid_metadata, data}} = e ->
-          Logger.debug(fn -> "Incomplete or invalid metadata for #{url}: #{inspect(data)}" end)
-          e
-
-        error ->
-          Logger.error(fn -> "Rich media error for #{url}: #{inspect(error)}" end)
-          error
      end
    end

    defp get_cached_or_parse(url) do
-      case Cachex.fetch!(:rich_media_cache, url, fn _ -> {:commit, parse_url(url)} end) do
+      case Cachex.fetch(:rich_media_cache, url, fn ->
+             case parse_url(url) do
+               {:ok, _} = res ->
+                 {:commit, res}
+
+               {:error, reason} = e ->
+                 # Unfortunately we have to log errors here, instead of doing that
+                 # along with ttl setting at the bottom. Otherwise we can get log spam
+                 # if more than one process was waiting for the rich media card
+                 # while it was generated. Ideally we would set ttl here as well,
+                 # so we don't override it number_of_waiters_on_generation
+                 # times, but one, obviously, can't set ttl for not-yet-created entry
+                 # and Cachex doesn't support returning ttl from the fetch callback.
+                 log_error(url, reason)
+                 {:commit, e}
+             end
+           end) do
+        {action, res} when action in [:commit, :ok] ->
+          case res do
            {:ok, _data} = res ->
              res

-        {:error, :body_too_large} = e ->
+            {:error, reason} = e ->
+              if action == :commit, do: set_error_ttl(url, reason)
              e
+          end

-        {:error, {:content_type, _}} = e ->
-          e
+        {:error, e} ->
+          {:error, {:cachex_error, e}}
+      end
+    end
+
+    defp set_error_ttl(_url, :body_too_large), do: :ok
+    defp set_error_ttl(_url, {:content_type, _}), do: :ok

    # The TTL is not set for the errors above, since they are unlikely to change
    # with time
-        {:error, _} = e ->
+
+    defp set_error_ttl(url, _reason) do
      ttl = Pleroma.Config.get([:rich_media, :failure_backoff], 60_000)
      Cachex.expire(:rich_media_cache, url, ttl)
-          e
+      :ok
    end
+
+    defp log_error(url, {:invalid_metadata, data}) do
+      Logger.debug(fn -> "Incomplete or invalid metadata for #{url}: #{inspect(data)}" end)
+    end
+
+    defp log_error(url, reason) do
+      Logger.warn(fn -> "Rich media error for #{url}: #{inspect(reason)}" end)
    end
  end

--- a/lib/pleroma/web/router.ex
+++ b/lib/pleroma/web/router.ex
@ -178,9 +178,14 @@ defmodule Pleroma.Web.Router do
    get("/users", AdminAPIController, :list_users)
    get("/users/:nickname", AdminAPIController, :user_show)
    get("/users/:nickname/statuses", AdminAPIController, :list_user_statuses)
+    get("/users/:nickname/chats", AdminAPIController, :list_user_chats)

    get("/instances/:instance/statuses", AdminAPIController, :list_instance_statuses)

+    get("/instance_document/:name", InstanceDocumentController, :show)
+    patch("/instance_document/:name", InstanceDocumentController, :update)
+    delete("/instance_document/:name", InstanceDocumentController, :delete)
+
    patch("/users/confirm_email", AdminAPIController, :confirm_email)
    patch("/users/resend_confirmation_email", AdminAPIController, :resend_confirmation_email)

@ -214,6 +219,10 @@ defmodule Pleroma.Web.Router do
    get("/media_proxy_caches", MediaProxyCacheController, :index)
    post("/media_proxy_caches/delete", MediaProxyCacheController, :delete)
    post("/media_proxy_caches/purge", MediaProxyCacheController, :purge)
+
+    get("/chats/:id", ChatController, :show)
+    get("/chats/:id/messages", ChatController, :messages)
+    delete("/chats/:id/messages/:message_id", ChatController, :delete_message)
  end

  scope "/api/pleroma/emoji", Pleroma.Web.PleromaAPI do
--- a/priv/static/index.html
+++ b/priv/static/index.html
@ -1 +1 @@
-<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1,user-scalable=no"><title>Pleroma</title><!--server-generated-meta--><link rel=icon type=image/png href=/favicon.png><link href=/static/css/app.77b1644622e3bae24b6b.css rel=stylesheet><link href=/static/fontello.1599568314856.css rel=stylesheet></head><body class=hidden><noscript>To use Pleroma, please enable JavaScript.</noscript><div id=app></div><script type=text/javascript src=/static/js/vendors~app.90c4af83c1ae68f4cd95.js></script><script type=text/javascript src=/static/js/app.55d173dc5e39519aa518.js></script></body></html>
+<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1,user-scalable=no"><title>Pleroma</title><!--server-generated-meta--><link rel=icon type=image/png href=/favicon.png><link href=/static/css/app.77b1644622e3bae24b6b.css rel=stylesheet><link href=/static/fontello.1600365488745.css rel=stylesheet></head><body class=hidden><noscript>To use Pleroma, please enable JavaScript.</noscript><div id=app></div><script type=text/javascript src=/static/js/vendors~app.90c4af83c1ae68f4cd95.js></script><script type=text/javascript src=/static/js/app.826c44232e0a76bbd9ba.js></script></body></html>
--- a/priv/static/static/font/fontello.1599568314856.woff
+++ b/priv/static/static/font/fontello.1599568314856.woff
--- a/priv/static/static/font/fontello.1599568314856.woff2
+++ b/priv/static/static/font/fontello.1599568314856.woff2
--- a/priv/static/static/font/fontello.1600365488745.eot
+++ b/priv/static/static/font/fontello.1600365488745.eot
--- a/priv/static/static/font/fontello.1600365488745.svg
+++ b/priv/static/static/font/fontello.1600365488745.svg
@ -92,6 +92,8 @@

 <glyph glyph-name="block" unicode="&#xe82a;" d="M732 359q0 90-48 164l-421-420q76-50 166-50 62 0 118 25t96 65 65 97 24 119z m-557-167l421 421q-75 50-167 50-83 0-153-40t-110-111-41-153q0-91 50-167z m682 167q0-88-34-168t-91-137-137-92-166-34-167 34-137 92-91 137-34 168 34 167 91 137 137 91 167 34 166-34 137-91 91-137 34-167z" horiz-adv-x="857.1" />

+<glyph glyph-name="megaphone" unicode="&#xe82b;" d="M929 500q29 0 50-21t21-51-21-50-50-21v-214q0-29-22-50t-50-22q-233 194-453 212-32-10-51-36t-17-57 22-51q-11-19-13-37t4-32 19-31 26-28 35-28q-17-32-63-46t-94-7-73 31q-4 13-17 49t-18 53-12 50-9 56 2 55 12 62h-68q-36 0-63 26t-26 63v107q0 37 26 63t63 26h268q243 0 500 215 29 0 50-22t22-50v-214z m-72-337v532q-220-168-428-191v-151q210-23 428-190z" horiz-adv-x="1000" />
+
 <glyph glyph-name="spin3" unicode="&#xe832;" d="M494 857c-266 0-483-210-494-472-1-19 13-20 13-20l84 0c16 0 19 10 19 18 10 199 176 358 378 358 107 0 205-45 273-118l-58-57c-11-12-11-27 5-31l247-50c21-5 46 11 37 44l-58 227c-2 9-16 22-29 13l-65-60c-89 91-214 148-352 148z m409-508c-16 0-19-10-19-18-10-199-176-358-377-358-108 0-205 45-274 118l59 57c10 12 10 27-5 31l-248 50c-21 5-46-11-37-44l58-227c2-9 16-22 30-13l64 60c89-91 214-148 353-148 265 0 482 210 493 473 1 18-13 19-13 19l-84 0z" horiz-adv-x="1000" />

 <glyph glyph-name="spin4" unicode="&#xe834;" d="M498 857c-114 0-228-39-320-116l0 0c173 140 428 130 588-31 134-134 164-332 89-495-10-29-5-50 12-68 21-20 61-23 84 0 3 3 12 15 15 24 71 180 33 393-112 539-99 98-228 147-356 147z m-409-274c-14 0-29-5-39-16-3-3-13-15-15-24-71-180-34-393 112-539 185-185 479-195 676-31l0 0c-173-140-428-130-589 31-134 134-163 333-89 495 11 29 6 50-12 68-11 11-27 17-44 16z" horiz-adv-x="1001" />
--- a/priv/static/static/font/fontello.1600365488745.ttf
+++ b/priv/static/static/font/fontello.1600365488745.ttf
--- a/priv/static/static/font/fontello.1600365488745.woff
+++ b/priv/static/static/font/fontello.1600365488745.woff
--- a/priv/static/static/font/fontello.1600365488745.woff2
+++ b/priv/static/static/font/fontello.1600365488745.woff2
--- a/priv/static/static/fontello.1600365488745.css
+++ b/priv/static/static/fontello.1600365488745.css
@ -1,11 +1,11 @@
@font-face {
 	font-family: "Icons";
-	src: url("./font/fontello.1599568314856.eot");
-	src: url("./font/fontello.1599568314856.eot") format("embedded-opentype"),
-		url("./font/fontello.1599568314856.woff2") format("woff2"),
-		url("./font/fontello.1599568314856.woff") format("woff"),
-		url("./font/fontello.1599568314856.ttf") format("truetype"),
-		url("./font/fontello.1599568314856.svg") format("svg");
+	src: url("./font/fontello.1600365488745.eot");
+	src: url("./font/fontello.1600365488745.eot") format("embedded-opentype"),
+		url("./font/fontello.1600365488745.woff2") format("woff2"),
+		url("./font/fontello.1600365488745.woff") format("woff"),
+		url("./font/fontello.1600365488745.ttf") format("truetype"),
+		url("./font/fontello.1600365488745.svg") format("svg");
 	font-weight: normal;
 	font-style: normal;
 }
@ -156,3 +156,5 @@ .icon-music::before { content: "\e828"; }
 .icon-doc::before { content: "\e829"; }

 .icon-block::before { content: "\e82a"; }
+
+.icon-megaphone::before { content: "\e82b"; }
--- a/priv/static/static/fontello.json
+++ b/priv/static/static/fontello.json
@ -405,6 +405,12 @@
      "css": "block",
      "code": 59434,
      "src": "fontawesome"
+    },
+    {
+      "uid": "3e674995cacc2b09692c096ea7eb6165",
+      "css": "megaphone",
+      "code": 59435,
+      "src": "fontawesome"
    }
  ]
 }
--- a/priv/static/static/js/2.c92f4803ff24726cea58.js.map
+++ b/priv/static/static/js/2.c92f4803ff24726cea58.js.map
--- a/priv/static/static/js/2.e852a6b4b3bba752b838.js
+++ b/priv/static/static/js/2.e852a6b4b3bba752b838.js
--- a/priv/static/static/js/2.e852a6b4b3bba752b838.js.map
+++ b/priv/static/static/js/2.e852a6b4b3bba752b838.js.map
--- a/priv/static/static/js/app.55d173dc5e39519aa518.js
+++ b/priv/static/static/js/app.55d173dc5e39519aa518.js
--- a/priv/static/static/js/app.55d173dc5e39519aa518.js.map
+++ b/priv/static/static/js/app.55d173dc5e39519aa518.js.map
--- a/priv/static/static/js/app.826c44232e0a76bbd9ba.js
+++ b/priv/static/static/js/app.826c44232e0a76bbd9ba.js
--- a/priv/static/static/js/app.826c44232e0a76bbd9ba.js.map
+++ b/priv/static/static/js/app.826c44232e0a76bbd9ba.js.map
--- a/priv/static/sw-pleroma.js
+++ b/priv/static/sw-pleroma.js
@ -1,4 +1,4 @@
-var serviceWorkerOption = {"assets":["/static/fontello.1599568314856.css","/static/font/fontello.1599568314856.eot","/static/font/fontello.1599568314856.svg","/static/font/fontello.1599568314856.ttf","/static/font/fontello.1599568314856.woff","/static/font/fontello.1599568314856.woff2","/static/img/nsfw.74818f9.png","/static/css/app.77b1644622e3bae24b6b.css","/static/js/app.55d173dc5e39519aa518.js","/static/js/vendors~app.90c4af83c1ae68f4cd95.js","/static/css/2.0778a6a864a1307a6c41.css","/static/js/2.c92f4803ff24726cea58.js","/static/css/3.b2603a50868c68a1c192.css","/static/js/3.7d21accf4e5bd07e3ebf.js","/static/js/4.5719922a4e807145346d.js","/static/js/5.cf05c5ddbdbac890ae35.js","/static/js/6.ecfd3302a692de148391.js","/static/js/7.dd44c3d58fb9dced093d.js","/static/js/8.636322a87bb10a1754f8.js","/static/js/9.6010dbcce7b4d7c05a18.js","/static/js/10.46fbbdfaf0d4800f349b.js","/static/js/11.708cc2513c53879a92cc.js","/static/js/12.b3bf0bc313861d6ec36b.js","/static/js/13.adb8a942514d735722c4.js","/static/js/14.d015d9b2ea16407e389c.js","/static/js/15.19866e6a366ccf982284.js","/static/js/16.38a984effd54736f6a2c.js","/static/js/17.9c25507194320db2e85b.js","/static/js/18.94946caca48930c224c7.js","/static/js/19.233c81ac2c28d55e9f13.js","/static/js/20.818c38d27369c3a4d677.js","/static/js/21.ce4cda179d888ca6bc2a.js","/static/js/22.2ea93c6cc569ef0256ab.js","/static/js/23.a57a7845cc20fafd06d1.js","/static/js/24.35eb55a657b5485f8491.js","/static/js/25.5a9efe20e3ae1352e6d2.js","/static/js/26.cf13231d524e5ca3b3e6.js","/static/js/27.fca8d4f6e444bd14f376.js","/static/js/28.e0f9f164e0bfd890dc61.js","/static/js/29.0b69359f0fe5c0785746.js","/static/js/30.fce58be0b52ca3e32fa4.js"]};
+var serviceWorkerOption = {"assets":["/static/fontello.1600365488745.css","/static/font/fontello.1600365488745.eot","/static/font/fontello.1600365488745.svg","/static/font/fontello.1600365488745.ttf","/static/font/fontello.1600365488745.woff","/static/font/fontello.1600365488745.woff2","/static/img/nsfw.74818f9.png","/static/css/app.77b1644622e3bae24b6b.css","/static/js/app.826c44232e0a76bbd9ba.js","/static/js/vendors~app.90c4af83c1ae68f4cd95.js","/static/css/2.0778a6a864a1307a6c41.css","/static/js/2.e852a6b4b3bba752b838.js","/static/css/3.b2603a50868c68a1c192.css","/static/js/3.7d21accf4e5bd07e3ebf.js","/static/js/4.5719922a4e807145346d.js","/static/js/5.cf05c5ddbdbac890ae35.js","/static/js/6.ecfd3302a692de148391.js","/static/js/7.dd44c3d58fb9dced093d.js","/static/js/8.636322a87bb10a1754f8.js","/static/js/9.6010dbcce7b4d7c05a18.js","/static/js/10.46fbbdfaf0d4800f349b.js","/static/js/11.708cc2513c53879a92cc.js","/static/js/12.b3bf0bc313861d6ec36b.js","/static/js/13.adb8a942514d735722c4.js","/static/js/14.d015d9b2ea16407e389c.js","/static/js/15.19866e6a366ccf982284.js","/static/js/16.38a984effd54736f6a2c.js","/static/js/17.9c25507194320db2e85b.js","/static/js/18.94946caca48930c224c7.js","/static/js/19.233c81ac2c28d55e9f13.js","/static/js/20.818c38d27369c3a4d677.js","/static/js/21.ce4cda179d888ca6bc2a.js","/static/js/22.2ea93c6cc569ef0256ab.js","/static/js/23.a57a7845cc20fafd06d1.js","/static/js/24.35eb55a657b5485f8491.js","/static/js/25.5a9efe20e3ae1352e6d2.js","/static/js/26.cf13231d524e5ca3b3e6.js","/static/js/27.fca8d4f6e444bd14f376.js","/static/js/28.e0f9f164e0bfd890dc61.js","/static/js/29.0b69359f0fe5c0785746.js","/static/js/30.fce58be0b52ca3e32fa4.js"]};
        
        !function(t){var e={};function n(r){if(e[r])return e[r].exports;var o=e[r]={i:r,l:!1,exports:{}};return t[r].call(o.exports,o,o.exports,n),o.l=!0,o.exports}n.m=t,n.c=e,n.d=function(t,e,r){n.o(t,e)||Object.defineProperty(t,e,{enumerable:!0,get:r})},n.r=function(t){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(t,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(t,"__esModule",{value:!0})},n.t=function(t,e){if(1&e&&(t=n(t)),8&e)return t;if(4&e&&"object"==typeof t&&t&&t.__esModule)return t;var r=Object.create(null);if(n.r(r),Object.defineProperty(r,"default",{enumerable:!0,value:t}),2&e&&"string"!=typeof t)for(var o in t)n.d(r,o,function(e){return t[e]}.bind(null,o));return r},n.n=function(t){var e=t&&t.__esModule?function(){return t.default}:function(){return t};return n.d(e,"a",e),e},n.o=function(t,e){return Object.prototype.hasOwnProperty.call(t,e)},n.p="/",n(n.s=196)}([function(t,e){var n=Array.isArray;t.exports=n},function(t,e,n){t.exports=n(53)},function(t,e,n){var r=n(32),o="object"==typeof self&&self&&self.Object===Object&&self,i=r||o||Function("return this")();t.exports=i},function(t,e,n){var r=n(97),o=n(100);t.exports=function(t,e){var n=o(t,e);return r(n)?n:void 0}},function(t,e,n){var r=n(8),o=n(64),i=n(65),a="[object Null]",s="[object Undefined]",c=r?r.toStringTag:void 0;t.exports=function(t){return null==t?void 0===t?s:a:c&&c in Object(t)?o(t):i(t)}},function(t,e){t.exports=function(t){return null!=t&&"object"==typeof t}},function(t,e){var n;n=function(){return this}();try{n=n||new Function("return this")()}catch(t){"object"==typeof window&&(n=window)}t.exports=n},function(t,e,n){var r=n(31),o=n(20);t.exports=function(t){return null!=t&&o(t.length)&&!r(t)}},function(t,e,n){var r=n(2).Symbol;t.exports=r},function(t,e){t.exports=function(t){var e=typeof t;return null!=t&&("object"==e||"function"==e)}},function(t,e,n){var r=n(4),o=n(5),i="[object Symbol]";t.exports=function(t){return"symbol"==typeof t||o(t)&&r(t)==i}},function(t,e,n){var r=n(72),o=n(78),i=n(7);t.exports=function(t){return i(t)?r(t):o(t)}},function(t,e,n){var r=n(87),o=n(88),i=n(89),a=n(90),s=n(91);function c(t){var e=-1,n=null==t?0:t.length;for(this.clear();++e<n;){var r=t[e];this.set(r[0],r[1])}}c.prototype.clear=r,c.prototype.delete=o,c.prototype.get=i,c.prototype.has=a,c.prototype.set=s,t.exports=c},function(t,e,n){var r=n(24);t.exports=function(t,e){for(var n=t.length;n--;)if(r(t[n][0],e))return n;return-1}},function(t,e,n){var r=n(3)(Object,"create");t.exports=r},function(t,e,n){var r=n(109);t.exports=function(t,e){var n=t.__data__;return r(e)?n["string"==typeof e?"string":"hash"]:n.map}},function(t,e,n){var r=n(10),o=1/0;t.exports=function(t){if("string"==typeof t||r(t))return t;var e=t+"";return"0"==e&&1/t==-o?"-0":e}},function(t,e){t.exports=function(t){return t}},function(t,e,n){var r=n(42),o=n(164),i=n(37),a=n(0);t.exports=function(t,e){return(a(t)?r:o)(t,i(e,3))}},function(t,e){t.exports=function(t){return t.webpackPolyfill||(t.deprecate=function(){},t.paths=[],t.children||(t.children=[]),Object.defineProperty(t,"loaded",{enumerable:!0,get:function(){return t.l}}),Object.defineProperty(t,"id",{enumerable:!0,get:function(){return t.i}}),t.webpackPolyfill=1),t}},function(t,e){var n=9007199254740991;t.exports=function(t){return"number"==typeof t&&t>-1&&t%1==0&&t<=n}},function(t,e){t.exports=function(t,e){for(var n=-1,r=null==t?0:t.length,o=Array(r);++n<r;)o[n]=e(t[n],n,t);return o}},function(t,e,n){var r=n(74),o=n(5),i=Object.prototype,a=i.hasOwnProperty,s=i.propertyIsEnumerable,c=r(function(){return arguments}())?r:function(t){return o(t)&&a.call(t,"callee")&&!s.call(t,"callee")};t.exports=c},function(t,e){var n=9007199254740991,r=/^(?:0|[1-9]\d*)$/;t.exports=function(t,e){var o=typeof t;return!!(e=null==e?n:e)&&("number"==o||"symbol"!=o&&r.test(t))&&t>-1&&t%1==0&&t<e}},function(t,e){t.exports=function(t,e){return t===e||t!=t&&e!=e}},function(t,e,n){var r=n(3)(n(2),"Map");t.exports=r},function(t,e,n){var r=n(101),o=n(108),i=n(110),a=n(111),s=n(112);function c(t){var e=-1,n=null==t?0:t.length;for(this.clear();++e<n;){var r=t[e];this.set(r[0],r[1])}}c.prototype.clear=r,c.prototype.delete=o,c.prototype.get=i,c.prototype.has=a,c.prototype.set=s,t.exports=c},function(t,e,n){var r=n(0),o=n(10),i=/\.|\[(?:[^[\]]*|(["'])(?:(?!\1)[^\\]|\\.)*?\1)\]/,a=/^\w*$/;t.exports=function(t,e){if(r(t))return!1;var n=typeof t;return!("number"!=n&&"symbol"!=n&&"boolean"!=n&&null!=t&&!o(t))||a.test(t)||!i.test(t)||null!=e&&t in Object(e)}},function(t,e){
 /*!
--- a/test/fixtures/custom_instance_panel.html
+++ b/test/fixtures/custom_instance_panel.html
@ -0,0 +1 @@
+<h2>Custom instance panel</h2>
--- a/test/fixtures/tesla_mock/admin@mastdon.example.org.json
+++ b/test/fixtures/tesla_mock/admin@mastdon.example.org.json
@ -1,5 +1,8 @@
 {
-  "@context": ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1", {
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers",
      "sensitive": "as:sensitive",
      "movedTo": "as:movedTo",
@ -14,7 +17,8 @@
        "@id": "as:alsoKnownAs",
        "@type": "@id"
      }
-  }],
+    }
+  ],
  "id": "http://mastodon.example.org/users/admin",
  "type": "Person",
  "following": "http://mastodon.example.org/users/admin/following",
@ -23,6 +27,7 @@
  "outbox": "http://mastodon.example.org/users/admin/outbox",
  "preferredUsername": "admin",
  "name": null,
+  "discoverable": "true",
  "summary": "\u003cp\u003e\u003c/p\u003e",
  "url": "http://mastodon.example.org/@admin",
  "manuallyApprovesFollowers": false,
@ -34,7 +39,8 @@
    "owner": "http://mastodon.example.org/users/admin",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtc4Tir+3ADhSNF6VKrtW\nOU32T01w7V0yshmQei38YyiVwVvFu8XOP6ACchkdxbJ+C9mZud8qWaRJKVbFTMUG\nNX4+6Q+FobyuKrwN7CEwhDALZtaN2IPbaPd6uG1B7QhWorrY+yFa8f2TBM3BxnUy\nI4T+bMIZIEYG7KtljCBoQXuTQmGtuffO0UwJksidg2ffCF5Q+K//JfQagJ3UzrR+\nZXbKMJdAw4bCVJYs4Z5EhHYBwQWiXCyMGTd7BGlmMkY6Av7ZqHKC/owp3/0EWDNz\nNqF09Wcpr3y3e8nA10X40MJqp/wR+1xtxp+YGbq/Cj5hZGBG7etFOmIpVBrDOhry\nBwIDAQAB\n-----END PUBLIC KEY-----\n"
  },
-  "attachment": [{
+  "attachment": [
+    {
      "type": "PropertyValue",
      "name": "foo",
      "value": "bar"
@ -58,5 +64,7 @@
    "mediaType": "image/png",
    "url": "https://cdn.niu.moe/accounts/headers/000/033/323/original/850b3448fa5fd477.png"
  },
-  "alsoKnownAs": ["http://example.org/users/foo"]
+  "alsoKnownAs": [
+    "http://example.org/users/foo"
+  ]
 }
--- a/test/fixtures/tesla_mock/https___osada.macgirvin.com_channel_mike.json
+++ b/test/fixtures/tesla_mock/https___osada.macgirvin.com_channel_mike.json
@ -8,6 +8,7 @@
    "preferredUsername": "mike",
    "name": "Mike Macgirvin (Osada)",
    "updated": "2018-08-29T03:09:11Z",
+    "discoverable": "true",
    "icon": {
        "type": "Image",
        "mediaType": "image/jpeg",
--- a/test/support/factory.ex
+++ b/test/support/factory.ex
@ -31,6 +31,7 @@ def user_factory do
      nickname: sequence(:nickname, &"nick#{&1}"),
      password_hash: Pbkdf2.hash_pwd_salt("test"),
      bio: sequence(:bio, &"Tester Number #{&1}"),
+      discoverable: true,
      last_digest_emailed_at: NaiveDateTime.utc_now(),
      last_refreshed_at: NaiveDateTime.utc_now(),
      notification_settings: %Pleroma.User.NotificationSetting{},
--- a/test/tasks/config_test.exs
+++ b/test/tasks/config_test.exs
@ -40,6 +40,19 @@ test "error if file with custom settings doesn't exist" do
      on_exit(fn -> Application.put_env(:quack, :level, initial) end)
    end

+    @tag capture_log: true
+    test "config migration refused when deprecated settings are found" do
+      clear_config([:media_proxy, :whitelist], ["domain_without_scheme.com"])
+      assert Repo.all(ConfigDB) == []
+
+      Mix.Tasks.Pleroma.Config.migrate_to_db("test/fixtures/config/temp.secret.exs")
+
+      assert_received {:mix_shell, :error, [message]}
+
+      assert message =~
+               "Migration is not allowed until all deprecation warnings have been resolved."
+    end
+
    test "filtered settings are migrated to db" do
      assert Repo.all(ConfigDB) == []

--- a/test/user_search_test.exs
+++ b/test/user_search_test.exs
@ -25,6 +25,14 @@ test "excludes invisible users from results" do
      assert found_user.id == user.id
    end

+    test "excludes users when discoverable is false" do
+      insert(:user, %{nickname: "john 3000", discoverable: false})
+      insert(:user, %{nickname: "john 3001"})
+
+      users = User.search("john")
+      assert Enum.count(users) == 1
+    end
+
    test "excludes service actors from results" do
      insert(:user, actor_type: "Application", nickname: "user1")
      service = insert(:user, actor_type: "Service", nickname: "user2")
--- a/test/web/activity_pub/pipeline_test.exs
+++ b/test/web/activity_pub/pipeline_test.exs
@ -26,7 +26,7 @@ test "when given an `object_data` in meta, Federation will receive a the origina
        {
          Pleroma.Web.ActivityPub.MRF,
          [],
-          [filter: fn o -> {:ok, o} end]
+          [pipeline_filter: fn o, m -> {:ok, o, m} end]
        },
        {
          Pleroma.Web.ActivityPub.ActivityPub,
@ -51,7 +51,7 @@ test "when given an `object_data` in meta, Federation will receive a the origina
                 Pleroma.Web.ActivityPub.Pipeline.common_pipeline(activity, meta)

        assert_called(Pleroma.Web.ActivityPub.ObjectValidator.validate(activity, meta))
-        assert_called(Pleroma.Web.ActivityPub.MRF.filter(activity))
+        assert_called(Pleroma.Web.ActivityPub.MRF.pipeline_filter(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.ActivityPub.persist(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.SideEffects.handle(activity, meta))
        refute called(Pleroma.Web.Federator.publish(activity))
@ -68,7 +68,7 @@ test "it goes through validation, filtering, persisting, side effects and federa
        {
          Pleroma.Web.ActivityPub.MRF,
          [],
-          [filter: fn o -> {:ok, o} end]
+          [pipeline_filter: fn o, m -> {:ok, o, m} end]
        },
        {
          Pleroma.Web.ActivityPub.ActivityPub,
@ -93,7 +93,7 @@ test "it goes through validation, filtering, persisting, side effects and federa
                 Pleroma.Web.ActivityPub.Pipeline.common_pipeline(activity, meta)

        assert_called(Pleroma.Web.ActivityPub.ObjectValidator.validate(activity, meta))
-        assert_called(Pleroma.Web.ActivityPub.MRF.filter(activity))
+        assert_called(Pleroma.Web.ActivityPub.MRF.pipeline_filter(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.ActivityPub.persist(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.SideEffects.handle(activity, meta))
        assert_called(Pleroma.Web.Federator.publish(activity))
@ -109,7 +109,7 @@ test "it goes through validation, filtering, persisting, side effects without fe
        {
          Pleroma.Web.ActivityPub.MRF,
          [],
-          [filter: fn o -> {:ok, o} end]
+          [pipeline_filter: fn o, m -> {:ok, o, m} end]
        },
        {
          Pleroma.Web.ActivityPub.ActivityPub,
@ -131,7 +131,7 @@ test "it goes through validation, filtering, persisting, side effects without fe
                 Pleroma.Web.ActivityPub.Pipeline.common_pipeline(activity, meta)

        assert_called(Pleroma.Web.ActivityPub.ObjectValidator.validate(activity, meta))
-        assert_called(Pleroma.Web.ActivityPub.MRF.filter(activity))
+        assert_called(Pleroma.Web.ActivityPub.MRF.pipeline_filter(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.ActivityPub.persist(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.SideEffects.handle(activity, meta))
      end
@ -148,7 +148,7 @@ test "it goes through validation, filtering, persisting, side effects without fe
        {
          Pleroma.Web.ActivityPub.MRF,
          [],
-          [filter: fn o -> {:ok, o} end]
+          [pipeline_filter: fn o, m -> {:ok, o, m} end]
        },
        {
          Pleroma.Web.ActivityPub.ActivityPub,
@ -170,7 +170,7 @@ test "it goes through validation, filtering, persisting, side effects without fe
                 Pleroma.Web.ActivityPub.Pipeline.common_pipeline(activity, meta)

        assert_called(Pleroma.Web.ActivityPub.ObjectValidator.validate(activity, meta))
-        assert_called(Pleroma.Web.ActivityPub.MRF.filter(activity))
+        assert_called(Pleroma.Web.ActivityPub.MRF.pipeline_filter(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.ActivityPub.persist(activity, meta))
        assert_called(Pleroma.Web.ActivityPub.SideEffects.handle(activity, meta))
      end
--- a/test/web/admin_api/controllers/admin_api_controller_test.exs
+++ b/test/web/admin_api/controllers/admin_api_controller_test.exs
@ -1510,6 +1510,56 @@ test "excludes reblogs by default", %{conn: conn, user: user} do
    end
  end

+  describe "GET /api/pleroma/admin/users/:nickname/chats" do
+    setup do
+      user = insert(:user)
+      recipients = insert_list(3, :user)
+
+      Enum.each(recipients, fn recipient ->
+        CommonAPI.post_chat_message(user, recipient, "yo")
+      end)
+
+      %{user: user}
+    end
+
+    test "renders user's chats", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/chats")
+
+      assert json_response(conn, 200) |> length() == 3
+    end
+  end
+
+  describe "GET /api/pleroma/admin/users/:nickname/chats unauthorized" do
+    setup do
+      user = insert(:user)
+      recipient = insert(:user)
+      CommonAPI.post_chat_message(user, recipient, "yo")
+      %{conn: conn} = oauth_access(["read:chats"])
+      %{conn: conn, user: user}
+    end
+
+    test "returns 403", %{conn: conn, user: user} do
+      conn
+      |> get("/api/pleroma/admin/users/#{user.nickname}/chats")
+      |> json_response(403)
+    end
+  end
+
+  describe "GET /api/pleroma/admin/users/:nickname/chats unauthenticated" do
+    setup do
+      user = insert(:user)
+      recipient = insert(:user)
+      CommonAPI.post_chat_message(user, recipient, "yo")
+      %{conn: build_conn(), user: user}
+    end
+
+    test "returns 403", %{conn: conn, user: user} do
+      conn
+      |> get("/api/pleroma/admin/users/#{user.nickname}/chats")
+      |> json_response(403)
+    end
+  end
+
  describe "GET /api/pleroma/admin/moderation_log" do
    setup do
      moderator = insert(:user, is_moderator: true)
--- a/test/web/admin_api/controllers/chat_controller_test.exs
+++ b/test/web/admin_api/controllers/chat_controller_test.exs
@ -0,0 +1,219 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.ChatControllerTest do
+  use Pleroma.Web.ConnCase
+
+  import Pleroma.Factory
+
+  alias Pleroma.Chat
+  alias Pleroma.Chat.MessageReference
+  alias Pleroma.Config
+  alias Pleroma.ModerationLog
+  alias Pleroma.Object
+  alias Pleroma.Repo
+  alias Pleroma.Web.CommonAPI
+
+  defp admin_setup do
+    admin = insert(:user, is_admin: true)
+    token = insert(:oauth_admin_token, user: admin)
+
+    conn =
+      build_conn()
+      |> assign(:user, admin)
+      |> assign(:token, token)
+
+    {:ok, %{admin: admin, token: token, conn: conn}}
+  end
+
+  describe "DELETE /api/pleroma/admin/chats/:id/messages/:message_id" do
+    setup do: admin_setup()
+
+    test "it deletes a message from the chat", %{conn: conn, admin: admin} do
+      user = insert(:user)
+      recipient = insert(:user)
+
+      {:ok, message} =
+        CommonAPI.post_chat_message(user, recipient, "Hello darkness my old friend")
+
+      object = Object.normalize(message, false)
+
+      chat = Chat.get(user.id, recipient.ap_id)
+      recipient_chat = Chat.get(recipient.id, user.ap_id)
+
+      cm_ref = MessageReference.for_chat_and_object(chat, object)
+      recipient_cm_ref = MessageReference.for_chat_and_object(recipient_chat, object)
+
+      result =
+        conn
+        |> put_req_header("content-type", "application/json")
+        |> delete("/api/pleroma/admin/chats/#{chat.id}/messages/#{cm_ref.id}")
+        |> json_response_and_validate_schema(200)
+
+      log_entry = Repo.one(ModerationLog)
+
+      assert ModerationLog.get_log_entry_message(log_entry) ==
+               "@#{admin.nickname} deleted chat message ##{cm_ref.id}"
+
+      assert result["id"] == cm_ref.id
+      refute MessageReference.get_by_id(cm_ref.id)
+      refute MessageReference.get_by_id(recipient_cm_ref.id)
+      assert %{data: %{"type" => "Tombstone"}} = Object.get_by_id(object.id)
+    end
+  end
+
+  describe "GET /api/pleroma/admin/chats/:id/messages" do
+    setup do: admin_setup()
+
+    test "it paginates", %{conn: conn} do
+      user = insert(:user)
+      recipient = insert(:user)
+
+      Enum.each(1..30, fn _ ->
+        {:ok, _} = CommonAPI.post_chat_message(user, recipient, "hey")
+      end)
+
+      chat = Chat.get(user.id, recipient.ap_id)
+
+      result =
+        conn
+        |> get("/api/pleroma/admin/chats/#{chat.id}/messages")
+        |> json_response_and_validate_schema(200)
+
+      assert length(result) == 20
+
+      result =
+        conn
+        |> get("/api/pleroma/admin/chats/#{chat.id}/messages?max_id=#{List.last(result)["id"]}")
+        |> json_response_and_validate_schema(200)
+
+      assert length(result) == 10
+    end
+
+    test "it returns the messages for a given chat", %{conn: conn} do
+      user = insert(:user)
+      other_user = insert(:user)
+      third_user = insert(:user)
+
+      {:ok, _} = CommonAPI.post_chat_message(user, other_user, "hey")
+      {:ok, _} = CommonAPI.post_chat_message(user, third_user, "hey")
+      {:ok, _} = CommonAPI.post_chat_message(user, other_user, "how are you?")
+      {:ok, _} = CommonAPI.post_chat_message(other_user, user, "fine, how about you?")
+
+      chat = Chat.get(user.id, other_user.ap_id)
+
+      result =
+        conn
+        |> get("/api/pleroma/admin/chats/#{chat.id}/messages")
+        |> json_response_and_validate_schema(200)
+
+      result
+      |> Enum.each(fn message ->
+        assert message["chat_id"] == chat.id |> to_string()
+      end)
+
+      assert length(result) == 3
+    end
+  end
+
+  describe "GET /api/pleroma/admin/chats/:id" do
+    setup do: admin_setup()
+
+    test "it returns a chat", %{conn: conn} do
+      user = insert(:user)
+      other_user = insert(:user)
+
+      {:ok, chat} = Chat.get_or_create(user.id, other_user.ap_id)
+
+      result =
+        conn
+        |> get("/api/pleroma/admin/chats/#{chat.id}")
+        |> json_response_and_validate_schema(200)
+
+      assert result["id"] == to_string(chat.id)
+      assert %{} = result["sender"]
+      assert %{} = result["receiver"]
+      refute result["account"]
+    end
+  end
+
+  describe "unauthorized chat moderation" do
+    setup do
+      user = insert(:user)
+      recipient = insert(:user)
+
+      {:ok, message} = CommonAPI.post_chat_message(user, recipient, "Yo")
+      object = Object.normalize(message, false)
+      chat = Chat.get(user.id, recipient.ap_id)
+      cm_ref = MessageReference.for_chat_and_object(chat, object)
+
+      %{conn: conn} = oauth_access(["read:chats", "write:chats"])
+      %{conn: conn, chat: chat, cm_ref: cm_ref}
+    end
+
+    test "DELETE /api/pleroma/admin/chats/:id/messages/:message_id", %{
+      conn: conn,
+      chat: chat,
+      cm_ref: cm_ref
+    } do
+      conn
+      |> put_req_header("content-type", "application/json")
+      |> delete("/api/pleroma/admin/chats/#{chat.id}/messages/#{cm_ref.id}")
+      |> json_response(403)
+
+      assert MessageReference.get_by_id(cm_ref.id) == cm_ref
+    end
+
+    test "GET /api/pleroma/admin/chats/:id/messages", %{conn: conn, chat: chat} do
+      conn
+      |> get("/api/pleroma/admin/chats/#{chat.id}/messages")
+      |> json_response(403)
+    end
+
+    test "GET /api/pleroma/admin/chats/:id", %{conn: conn, chat: chat} do
+      conn
+      |> get("/api/pleroma/admin/chats/#{chat.id}")
+      |> json_response(403)
+    end
+  end
+
+  describe "unauthenticated chat moderation" do
+    setup do
+      user = insert(:user)
+      recipient = insert(:user)
+
+      {:ok, message} = CommonAPI.post_chat_message(user, recipient, "Yo")
+      object = Object.normalize(message, false)
+      chat = Chat.get(user.id, recipient.ap_id)
+      cm_ref = MessageReference.for_chat_and_object(chat, object)
+
+      %{conn: build_conn(), chat: chat, cm_ref: cm_ref}
+    end
+
+    test "DELETE /api/pleroma/admin/chats/:id/messages/:message_id", %{
+      conn: conn,
+      chat: chat,
+      cm_ref: cm_ref
+    } do
+      conn
+      |> put_req_header("content-type", "application/json")
+      |> delete("/api/pleroma/admin/chats/#{chat.id}/messages/#{cm_ref.id}")
+      |> json_response(403)
+
+      assert MessageReference.get_by_id(cm_ref.id) == cm_ref
+    end
+
+    test "GET /api/pleroma/admin/chats/:id/messages", %{conn: conn, chat: chat} do
+      conn
+      |> get("/api/pleroma/admin/chats/#{chat.id}/messages")
+      |> json_response(403)
+    end
+
+    test "GET /api/pleroma/admin/chats/:id", %{conn: conn, chat: chat} do
+      conn
+      |> get("/api/pleroma/admin/chats/#{chat.id}")
+      |> json_response(403)
+    end
+  end
+end
--- a/test/web/admin_api/controllers/instance_document_controller_test.exs
+++ b/test/web/admin_api/controllers/instance_document_controller_test.exs
@ -0,0 +1,106 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.InstanceDocumentControllerTest do
+  use Pleroma.Web.ConnCase, async: true
+  import Pleroma.Factory
+  alias Pleroma.Config
+
+  @dir "test/tmp/instance_static"
+  @default_instance_panel ~s(<p>Welcome to <a href="https://pleroma.social" target="_blank">Pleroma!</a></p>)
+
+  setup do
+    File.mkdir_p!(@dir)
+    on_exit(fn -> File.rm_rf(@dir) end)
+  end
+
+  setup do: clear_config([:instance, :static_dir], @dir)
+
+  setup do
+    admin = insert(:user, is_admin: true)
+    token = insert(:oauth_admin_token, user: admin)
+
+    conn =
+      build_conn()
+      |> assign(:user, admin)
+      |> assign(:token, token)
+
+    {:ok, %{admin: admin, token: token, conn: conn}}
+  end
+
+  describe "GET /api/pleroma/admin/instance_document/:name" do
+    test "return the instance document url", %{conn: conn} do
+      conn = get(conn, "/api/pleroma/admin/instance_document/instance-panel")
+
+      assert content = html_response(conn, 200)
+      assert String.contains?(content, @default_instance_panel)
+    end
+
+    test "it returns 403 if requested by a non-admin" do
+      non_admin_user = insert(:user)
+      token = insert(:oauth_token, user: non_admin_user)
+
+      conn =
+        build_conn()
+        |> assign(:user, non_admin_user)
+        |> assign(:token, token)
+        |> get("/api/pleroma/admin/instance_document/instance-panel")
+
+      assert json_response(conn, :forbidden)
+    end
+
+    test "it returns 404 if the instance document with the given name doesn't exist", %{
+      conn: conn
+    } do
+      conn = get(conn, "/api/pleroma/admin/instance_document/1234")
+
+      assert json_response_and_validate_schema(conn, 404)
+    end
+  end
+
+  describe "PATCH /api/pleroma/admin/instance_document/:name" do
+    test "uploads the instance document", %{conn: conn} do
+      image = %Plug.Upload{
+        content_type: "text/html",
+        path: Path.absname("test/fixtures/custom_instance_panel.html"),
+        filename: "custom_instance_panel.html"
+      }
+
+      conn =
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> patch("/api/pleroma/admin/instance_document/instance-panel", %{
+          "file" => image
+        })
+
+      assert %{"url" => url} = json_response_and_validate_schema(conn, 200)
+      index = get(build_conn(), url)
+      assert html_response(index, 200) == "<h2>Custom instance panel</h2>"
+    end
+  end
+
+  describe "DELETE /api/pleroma/admin/instance_document/:name" do
+    test "deletes the instance document", %{conn: conn} do
+      File.mkdir!(@dir <> "/instance/")
+      File.write!(@dir <> "/instance/panel.html", "Custom instance panel")
+
+      conn_resp =
+        conn
+        |> get("/api/pleroma/admin/instance_document/instance-panel")
+
+      assert html_response(conn_resp, 200) == "Custom instance panel"
+
+      conn
+      |> delete("/api/pleroma/admin/instance_document/instance-panel")
+      |> json_response_and_validate_schema(200)
+
+      conn_resp =
+        conn
+        |> get("/api/pleroma/admin/instance_document/instance-panel")
+
+      assert content = html_response(conn_resp, 200)
+      assert String.contains?(content, @default_instance_panel)
+    end
+  end
+end
--- a/test/web/admin_api/search_test.exs
+++ b/test/web/admin_api/search_test.exs
@ -177,5 +177,14 @@ test "it returns unapproved user" do
      assert total == 3
      assert count == 1
    end
+
+    test "it returns non-discoverable users" do
+      insert(:user)
+      insert(:user, discoverable: false)
+
+      {:ok, _results, total} = Search.user()
+
+      assert total == 2
+    end
  end
 end
--- a/test/web/common_api/common_api_test.exs
+++ b/test/web/common_api/common_api_test.exs
@ -217,6 +217,17 @@ test "it reject messages over the local limit" do

      assert message == :content_too_long
    end
+
+    test "it reject messages via MRF" do
+      clear_config([:mrf_keyword, :reject], ["GNO"])
+      clear_config([:mrf, :policies], [Pleroma.Web.ActivityPub.MRF.KeywordPolicy])
+
+      author = insert(:user)
+      recipient = insert(:user)
+
+      assert {:reject, "[KeywordPolicy] Matches with rejected keyword"} ==
+               CommonAPI.post_chat_message(author, recipient, "GNO/Linux")
+    end
  end

  describe "unblocking" do
@ -1193,4 +1204,24 @@ test "respects visibility=private" do
      assert Visibility.get_visibility(activity) == "private"
    end
  end
+
+  describe "get_user/1" do
+    test "gets user by ap_id" do
+      user = insert(:user)
+      assert CommonAPI.get_user(user.ap_id) == user
+    end
+
+    test "gets user by guessed nickname" do
+      user = insert(:user, ap_id: "", nickname: "mario@mushroom.kingdom")
+      assert CommonAPI.get_user("https://mushroom.kingdom/users/mario") == user
+    end
+
+    test "fallback" do
+      assert %User{
+               name: "",
+               ap_id: "",
+               nickname: "erroruser@example.com"
+             } = CommonAPI.get_user("")
+    end
+  end
 end
--- a/test/web/mastodon_api/views/account_view_test.exs
+++ b/test/web/mastodon_api/views/account_view_test.exs
@ -68,7 +68,7 @@ test "Represent a user account" do
        sensitive: false,
        pleroma: %{
          actor_type: "Person",
-          discoverable: false
+          discoverable: true
        },
        fields: []
      },
@ -166,7 +166,7 @@ test "Represent a Service(bot) account" do
        sensitive: false,
        pleroma: %{
          actor_type: "Service",
-          discoverable: false
+          discoverable: true
        },
        fields: []
      },
--- a/test/web/metadata/metadata_test.exs
+++ b/test/web/metadata/metadata_test.exs
@ -16,7 +16,14 @@ test "for remote user" do
    end

    test "for local user" do
-      user = insert(:user)
+      user = insert(:user, discoverable: false)
+
+      assert Pleroma.Web.Metadata.build_tags(%{user: user}) =~
+               "<meta content=\"noindex, noarchive\" name=\"robots\">"
+    end
+
+    test "for local user set to discoverable" do
+      user = insert(:user, discoverable: true)

      refute Pleroma.Web.Metadata.build_tags(%{user: user}) =~
               "<meta content=\"noindex, noarchive\" name=\"robots\">"
@ -24,11 +31,19 @@ test "for local user" do
  end

  describe "no metadata for private instances" do
-    test "for local user" do
+    test "for local user set to discoverable" do
      clear_config([:instance, :public], false)
-      user = insert(:user, bio: "This is my secret fedi account bio")
+      user = insert(:user, bio: "This is my secret fedi account bio", discoverable: true)

      assert "" = Pleroma.Web.Metadata.build_tags(%{user: user})
    end
+
+    test "search exclusion metadata is included" do
+      clear_config([:instance, :public], false)
+      user = insert(:user, bio: "This is my secret fedi account bio", discoverable: false)
+
+      assert ~s(<meta content="noindex, noarchive" name="robots">) ==
+               Pleroma.Web.Metadata.build_tags(%{user: user})
+    end
  end
 end
--- a/test/web/metadata/restrict_indexing_test.exs
+++ b/test/web/metadata/restrict_indexing_test.exs
@ -14,8 +14,14 @@ test "for remote user" do

    test "for local user" do
      assert Pleroma.Web.Metadata.Providers.RestrictIndexing.build_tags(%{
-               user: %Pleroma.User{local: true}
+               user: %Pleroma.User{local: true, discoverable: true}
             }) == []
    end
+
+    test "for local user when discoverable is false" do
+      assert Pleroma.Web.Metadata.Providers.RestrictIndexing.build_tags(%{
+               user: %Pleroma.User{local: true, discoverable: false}
+             }) == [{:meta, [name: "robots", content: "noindex, noarchive"], []}]
+    end
  end
 end
--- a/test/web/pleroma_api/controllers/chat_controller_test.exs
+++ b/test/web/pleroma_api/controllers/chat_controller_test.exs
@ -100,7 +100,7 @@ test "it fails if there is no content", %{conn: conn, user: user} do
        |> post("/api/v1/pleroma/chats/#{chat.id}/messages")
        |> json_response_and_validate_schema(400)

-      assert result
+      assert %{"error" => "no_content"} == result
    end

    test "it works with an attachment", %{conn: conn, user: user} do
@ -126,6 +126,23 @@ test "it works with an attachment", %{conn: conn, user: user} do

      assert result["attachment"]
    end
+
+    test "gets MRF reason when rejected", %{conn: conn, user: user} do
+      clear_config([:mrf_keyword, :reject], ["GNO"])
+      clear_config([:mrf, :policies], [Pleroma.Web.ActivityPub.MRF.KeywordPolicy])
+
+      other_user = insert(:user)
+
+      {:ok, chat} = Chat.get_or_create(user.id, other_user.ap_id)
+
+      result =
+        conn
+        |> put_req_header("content-type", "application/json")
+        |> post("/api/v1/pleroma/chats/#{chat.id}/messages", %{"content" => "GNO/Linux"})
+        |> json_response_and_validate_schema(422)
+
+      assert %{"error" => "[KeywordPolicy] Matches with rejected keyword"} == result
+    end
  end

  describe "DELETE /api/v1/pleroma/chats/:id/messages/:message_id" do