seaweedfs/weed
Sebastian Kurfuerst 10404c4275 FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
- one JWT for reading and one for writing, analogous to how the JWT
  between Master and Volume Server works
- I did not implement IP `whiteList` parameter on the filer

Additionally, because http_util.DownloadFile now sets the JWT,
the `download` command should now work when `jwt.signing.read` is
configured. By looking at the code, I think this case did not work
before.

## Docs to be adjusted after a release

Page `Amazon-S3-API`:

```
# Authentication with Filer

You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as
explained in [Security-Configuration](Security-Configuration) -
controlled by the `grpc.*` configuration in `security.toml`.

Starting with version XX, it is also possible to authenticate the HTTP
operations between the S3-API-Proxy and the Filer (especially
uploading new files). This is configured by setting
`filer_jwt.signing.key` and `filer_jwt.signing.read.key` in
`security.toml`.

With both configurations (gRPC and JWT), it is possible to have Filer
and S3 communicate in fully authenticated fashion; so Filer will reject
any unauthenticated communication.
```

Page `Security Overview`:

```
The following items are not covered, yet:

- master server http REST services

Starting with version XX, the Filer HTTP REST services can be secured
with a JWT, by setting `filer_jwt.signing.key` and
`filer_jwt.signing.read.key` in `security.toml`.

...

Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer.
Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).**

...

# Securing Filer HTTP with JWT

To enable JWT-based access control for the Filer,

1. generate `security.toml` file by `weed scaffold -config=security`
2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string
3. copy the same `security.toml` file to the filers and all S3 proxies.

If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`.

If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`.

The S3 API Gateway reads the above JWT keys and sends authenticated
HTTP requests to the filer.
```

Page `Security Configuration`:

```
(update scaffold file)

...

[filer_jwt.signing]
key = "blahblahblahblah"

[filer_jwt.signing.read]
key = "blahblahblahblah"
```

Resolves: #158
2021-12-30 14:45:27 +01:00
..
cluster add brokers 2021-11-09 08:50:55 -08:00
command FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client 2021-12-30 14:45:27 +01:00
filer filer server: add "datacenter, rack and datanode" for path specific configuration 2021-12-23 23:25:05 +08:00
filesys use proper chunk size limit option 2021-12-24 22:52:18 -08:00
ftpd go fmt 2021-09-14 10:37:06 -07:00
glog add log level possible values 2021-04-12 21:15:51 -07:00
iamapi adjust client name 2021-11-08 17:47:56 -08:00
images refactor: move from io/ioutil to io and os package 2021-10-14 12:27:58 +08:00
messaging upgrade protoc to 3.17.3 2021-12-05 00:42:25 -08:00
notification Do reconnect to RabbitMQ 2021-02-11 13:59:36 +05:00
operation filer server: allow upload file to specific dataNode 2021-12-22 21:57:26 +08:00
pb filer server: add "datacenter, rack and datanode" for path specific configuration 2021-12-23 23:25:05 +08:00
query go fmt 2019-10-09 00:03:18 -07:00
remote_storage remote.mount: print out metadata sync errors 2021-11-06 11:29:50 -07:00
replication revert 2021-11-28 23:35:22 -08:00
s3api FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client 2021-12-30 14:45:27 +01:00
security FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client 2021-12-30 14:45:27 +01:00
sequence removing etcd sequencer 2021-10-10 19:10:46 -07:00
server FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client 2021-12-30 14:45:27 +01:00
shell filer server: add "datacenter, rack and datanode" for path specific configuration 2021-12-23 23:25:05 +08:00
static S 2021-07-15 17:43:38 -07:00
stats handle ipv6 addresses 2021-09-07 16:43:54 -07:00
storage remove debug messages 2021-12-16 00:58:15 -08:00
topology Merge pull request #2527 from banjiaojuhao/master-assign-by-datanode 2021-12-21 08:56:51 -08:00
util FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client 2021-12-30 14:45:27 +01:00
wdclient add lock messages 2021-12-10 13:24:38 -08:00
Makefile make install 2021-11-09 12:34:57 -08:00
weed.go Add autocomplete 2021-08-03 02:56:52 +09:00