gitlab-com-seaweedfs/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2024-01-19 02:48:24 +00:00
Code Issues Projects Releases Wiki Activity

add section about using 128 bit needle id

Chris Lu 2018-07-31 23:40:19 -07:00
parent e25aefb1cd
commit 9c7322c7cc
1 changed files with 14 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
Optimization.md

@ -76,10 +76,10 @@ The file id generation is actually pretty trivial and you could use your own way
A file key has 3 parts: A file key has 3 parts:
- volume id: a volume with free spaces - volume id: a volume with free spaces
- file id: a monotonously increasing and unique number - needle id: a monotonously increasing and unique number
- file cookie: a random number, you can customize it in whichever way you want - file cookie: a random number, you can customize it in whichever way you want
You can directly ask master server to assign a file key, and replace the file id part to your own unique id, e.g., user id. You can directly ask master server to assign a file key, and replace the needle id part to your own unique id, e.g., user id.
Also you can get each volume's free space from the server status. Also you can get each volume's free space from the server status.
@ -91,7 +91,17 @@ Once you are sure about the volume free spaces, you can use your own file ids. J
The assigned file cookie can also be customized. The assigned file cookie can also be customized.
Customizing the file id and/or file cookie is an acceptable behavior. "strict monotonously increasing" is not necessary, but keeping file id in a "mostly" increasing order is expected in order to keep the in memory data structure efficient. Customizing the needle id and/or file cookie is an acceptable behavior. "strict monotonously increasing" is not necessary, but keeping file id in a "mostly" increasing order is expected in order to keep the in memory data structure efficient.
## Customize Needle Id to 128 bit
By default, the needle id is a 64-bit number. But sometimes a 128-bit UUID is desired as a needle id. Now it is also possible to use a 128-bit number as the needle id. But you would need to build the binary with a special tag "128BitNeedleId".
```bash
go install -tags 128BitNeedleId
```
Running this mode, the needle id generated from the master is still 64 bit. You are expected to replace the needle id with a hex encoded UUID without all the dash signs, "-".
## Upload large files ## Upload large files
@ -137,4 +147,4 @@ When going to production, you will want to collect the logs. SeaweedFS uses glog
```bash ```bash
weed -v=2 master weed -v=2 master
weed -log_dir=. volume weed -log_dir=. volume
``` ```