Commit graph

26 commits

Author SHA1 Message Date
chrislu 77362700e1 S3: fail fast when "X-Amz-Copy-Source" is a folder
fix #2593
2022-01-18 12:04:40 -08:00
chrislu c87b8f4c30 S3: fail fast when "X-Amz-Copy-Source" is a folder
fix https://github.com/chrislusf/seaweedfs/issues/2593
2022-01-17 23:09:37 -08:00
Sebastian Kurfuerst 10404c4275 FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
- one JWT for reading and one for writing, analogous to how the JWT
  between Master and Volume Server works
- I did not implement IP `whiteList` parameter on the filer

Additionally, because http_util.DownloadFile now sets the JWT,
the `download` command should now work when `jwt.signing.read` is
configured. By looking at the code, I think this case did not work
before.

## Docs to be adjusted after a release

Page `Amazon-S3-API`:

```
# Authentication with Filer

You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as
explained in [Security-Configuration](Security-Configuration) -
controlled by the `grpc.*` configuration in `security.toml`.

Starting with version XX, it is also possible to authenticate the HTTP
operations between the S3-API-Proxy and the Filer (especially
uploading new files). This is configured by setting
`filer_jwt.signing.key` and `filer_jwt.signing.read.key` in
`security.toml`.

With both configurations (gRPC and JWT), it is possible to have Filer
and S3 communicate in fully authenticated fashion; so Filer will reject
any unauthenticated communication.
```

Page `Security Overview`:

```
The following items are not covered, yet:

- master server http REST services

Starting with version XX, the Filer HTTP REST services can be secured
with a JWT, by setting `filer_jwt.signing.key` and
`filer_jwt.signing.read.key` in `security.toml`.

...

Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer.
Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).**

...

# Securing Filer HTTP with JWT

To enable JWT-based access control for the Filer,

1. generate `security.toml` file by `weed scaffold -config=security`
2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string
3. copy the same `security.toml` file to the filers and all S3 proxies.

If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`.

If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`.

The S3 API Gateway reads the above JWT keys and sends authenticated
HTTP requests to the filer.
```

Page `Security Configuration`:

```
(update scaffold file)

...

[filer_jwt.signing]
key = "blahblahblahblah"

[filer_jwt.signing.read]
key = "blahblahblahblah"
```

Resolves: #158
2021-12-30 14:45:27 +01:00
Chris Lu 2ba08afed1
Merge pull request #2498 from kmlebedev/s3_audit_log 2021-12-07 09:35:48 -08:00
Konstantin Lebedev 4ec8715f20 audit log 2021-12-07 12:15:48 +05:00
chrislu 5c3b783310 s3: copy object API needs to escape special characters
fix https://github.com/chrislusf/seaweedfs/issues/2482
2021-12-04 23:24:53 -08:00
Chris Lu f17fa400d5 refactoring 2021-10-31 18:05:34 -07:00
Chris Lu 006b4ec0f8 refactoring 2021-10-31 18:02:08 -07:00
Chris Lu 71175461ef add glog for s3 handlers 2021-09-19 00:18:59 -07:00
Chris Lu e5fc35ed0c change server address from string to a type 2021-09-12 22:47:52 -07:00
Chris Lu f62df7f695 fix wrong srcPath checking 2021-09-05 23:37:40 -07:00
Chris Lu 491f7636f8 s3: CopyObject return http Status 400 Bad Request for non-existing source
fix https://github.com/chrislusf/seaweedfs/issues/2306
2021-09-03 17:38:39 -07:00
Chris Lu 5a0f92423e use grpc and jwt 2021-08-12 21:40:33 -07:00
Chris Lu 8b382a8209 refactor 2021-06-10 21:50:21 -07:00
Chris Lu 33b87244ef refactoring 2021-06-10 16:54:36 -07:00
Chris Lu b1a86cf808 s3: copy object to itself
fix https://github.com/chrislusf/seaweedfs/issues/1922
2021-03-19 01:31:49 -07:00
Chris Lu 86cdb2a3e5 adjust logs 2020-11-11 02:01:24 -08:00
Chris Lu b9c1f3e9de s3: fixes for list multipart upload 2020-09-21 10:51:24 -07:00
Chris Lu 5b40a2690a refactoring 2020-09-19 14:09:58 -07:00
Chris Lu 4fc0bd1a81 return http response directly 2020-09-09 03:53:09 -07:00
Chris Lu 54cd582653 refactor 2020-07-26 12:58:58 -07:00
popstk feb52400be fix s3api copy object handler SerializationError 2020-07-25 15:52:31 +08:00
Chris Lu 35dde56711 refactoring 2020-02-25 12:58:45 -08:00
Chris Lu cf5064d702 properly close http response 2020-02-14 09:09:15 -08:00
Chris Lu b2743afaee remove println 2020-01-31 00:21:18 -08:00
Chris Lu a80ecbfe84 s3: add s3 copy
fix https://github.com/chrislusf/seaweedfs/issues/1190
2020-01-31 00:11:12 -08:00