Skip to content

Commit 3648c00

Browse files
committed
docs: document limits
Taken more or less verbatim from Consul.
1 parent 0781eeb commit 3648c00

File tree

2 files changed

+79
-2
lines changed

2 files changed

+79
-2
lines changed

website/source/docs/configuration/index.html.md

Lines changed: 54 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,57 @@ testing.
177177
server agents if it is expected that a terminated server instance will never
178178
join the cluster again.
179179

180+
- `limits` - Available in Nomad 0.10.3 and later, this is a nested object that
181+
configures limits that are enforced by the agent. The following parameters
182+
are available:
183+
184+
- `https_handshake_timeout` `(string: "5s")` - Configures the limit for how
185+
long the HTTPS server in both client and server agents will wait for a
186+
client to complete a TLS handshake. This should be kept conservative as it
187+
limits how many connections an unauthenticated attacker can open if
188+
[`tls.http = true`][tls] is being used (strongly recommended in
189+
production). Default value is `5s`. `0` disables HTTP handshake timeouts.
190+
191+
- `http_max_conns_per_client` `(int: 100)` - Configures a limit of how many
192+
concurrent TCP connections a single client IP address is allowed to open to
193+
the agent's HTTP server. This affects the HTTP servers in both client and
194+
server agents. Default value is `100`. `0` disables HTTP connection limits.
195+
196+
- `rpc_handshake_timeout` `(string: "5s")` - Configures the limit for how
197+
long servers will wait after a client TCP connection is established before
198+
they complete the connection handshake. When TLS is used, the same timeout
199+
applies to the TLS handshake separately from the initial protocol
200+
negotiation. All Nomad clients should perform this immediately on
201+
establishing a new connection. This should be kept conservative as it
202+
limits how many connections an unauthenticated attacker can open if
203+
TLS is being using to authenticate clients (strongly recommended in
204+
production). When `tls.rpc` is true on servers, this limits how long the
205+
connection and associated goroutines will be held open before the client
206+
successfully authenticates. Default value is `5s`. `0` disables RPC handshake
207+
timeouts.
208+
209+
- `rpc_max_conns_per_client` `(int: 100)` - Configures a limit of how
210+
many concurrent TCP connections a single source IP address is allowed
211+
to open to a single server. Client agents do not accept RPC TCP connections
212+
directly and therefore are not affected. It affects both clients connections
213+
and other server connections. Nomad clients multiplex many RPC calls over a
214+
single TCP connection, except for streaming endpoints such as [log
215+
streaming][log-api] which require their own connection when routed through
216+
servers. A server needs at least 2 TCP connections (1 Raft, 1 RPC) per peer
217+
server locally and in any federated region. Servers also need a TCP connection
218+
per routed streaming endpoint concurrently in use. Only operators use streaming
219+
endpoints; as of 0.10.3 Nomad client code does not. A reasonably low limit
220+
significantly reduces the ability of an unauthenticated attacker to consume
221+
unbounded resources by holding open many connections. You may need to
222+
increase this if WAN federated servers connect via proxies or NAT gateways
223+
or similar causing many legitimate connections from a single source IP.
224+
Default value is `100` which is designed to support the majority of users.
225+
`0` disables RPC connection limits. `26` is the minimum as `20` connections
226+
are always reserved for non-streaming connections (Raft and RPC) to ensure
227+
streaming RPCs do not prevent normal server operation. This minimum may be
228+
lowered in the future when streaming RPCs no longer require their own TCP
229+
connection.
230+
180231
- `log_level` `(string: "INFO")` - Specifies the verbosity of logs the Nomad
181232
agent will output. Valid log levels include `WARN`, `INFO`, or `DEBUG` in
182233
increasing order of verbosity.
@@ -250,7 +301,7 @@ testing.
250301
- `syslog_facility` `(string: "LOCAL0")` - Specifies the syslog facility to
251302
write to. This has no effect unless `enable_syslog` is true.
252303
253-
- `tls` `(`[`TLS`]`: nil)` - Specifies configuration for TLS.
304+
- `tls` `(`[`TLS`][tls]`: nil)` - Specifies configuration for TLS.
254305
255306
- `vault` `(`[`Vault`]`: nil)` - Specifies configuration for
256307
connecting to Vault.
@@ -283,7 +334,8 @@ http_api_response_headers {
283334
[`Plugin`]: /docs/configuration/plugin.html "Nomad Agent Plugin Configuration"
284335
[`Sentinel`]: /docs/configuration/sentinel.html "Nomad Agent sentinel Configuration"
285336
[`Server`]: /docs/configuration/server.html "Nomad Agent server Configuration"
286-
[`TLS`]: /docs/configuration/tls.html "Nomad Agent tls Configuration"
337+
[tls]: /docs/configuration/tls.html "Nomad Agent tls Configuration"
287338
[`Vault`]: /docs/configuration/vault.html "Nomad Agent vault Configuration"
288339
[go-sockaddr/template]: https://godoc.org/github.com/hashicorp/go-sockaddr/template
340+
[log-api]: /api/client.html#stream-logs
289341
[hcl]: https://github.com/hashicorp/hcl "HashiCorp Configuration Language"

website/source/guides/upgrade/upgrade-specific.html.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,30 @@ details provided for their upgrades as a result of new features or changed
1515
behavior. This page is used to document those details separately from the
1616
standard upgrade flow.
1717

18+
## Nomad 0.10.3
19+
20+
### Connection Limits Added
21+
22+
Nomad 0.10.3 introduces the [limits][limits] agent configuration parameters for
23+
mitigating denial of service attacks from users who are not authenticated via
24+
mTLS. The default limits stanza is:
25+
26+
```hcl
27+
limits {
28+
https_handshake_timeout = "5s"
29+
http_max_conns_per_client = 100
30+
rpc_handshake_timeout = "5s"
31+
rpc_max_conns_per_client = 100
32+
}
33+
```
34+
35+
If your Nomad agent's endpoints are protected from unauthenticated users via
36+
other mechanisms these limits may be safely disabled by setting them to `0`.
37+
38+
However the defaults were chosen to be safe for a wide variety of Nomad
39+
deployments and may protect against accidental abuses of the Nomad API that
40+
could cause unintended resource usage.
41+
1842
## Nomad 0.10.2
1943

2044
### Preemption Panic Fixed
@@ -385,6 +409,7 @@ deleted and then Nomad 0.3.0 can be launched.
385409
[dangling-containers]: /docs/drivers/docker.html#dangling-containers
386410
[gh-6787]: https://github.com/hashicorp/nomad/issues/6787
387411
[hcl2]: https://github.com/hashicorp/hcl2
412+
[limits]: /docs/configuration/index.html#limits
388413
[lxc]: /docs/drivers/external/lxc.html
389414
[migrate]: /docs/job-specification/migrate.html
390415
[plugins]: /docs/drivers/external/index.html

0 commit comments

Comments
 (0)