diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst index 335c1d00f8..2b8479243c 100644 --- a/doc/sphinx/reference/index.rst +++ b/doc/sphinx/reference/index.rst @@ -10,7 +10,9 @@ The VCL language .. toctree:: :maxdepth: 1 - VCL - Varnish Configuration Language + VCL - The Varnish Configuration Language + VCL backend configuration + VCL backend health probe states.rst VCL Design Patterns diff --git a/doc/sphinx/reference/vcl-backend.rst b/doc/sphinx/reference/vcl-backend.rst new file mode 100644 index 0000000000..8d54721444 --- /dev/null +++ b/doc/sphinx/reference/vcl-backend.rst @@ -0,0 +1,157 @@ +.. role:: ref(emphasis) + +.. _vcl-backend(7): + +============ +VCL-backends +============ + +-------------------- +Configuring Backends +-------------------- + +:Manual section: 7 + +.. _backend_definition: + +Backend definition +------------------ + +A backend declaration creates and initialises a named backend object. +A declaration start with the keyword ``backend`` followed by the name of the +backend. The actual declaration is in curly brackets, in a key/value fashion.:: + + backend name { + .attribute1 = value; + .attribute2 = value; + [...] + } + +If there is a backend named ``default`` it will be used unless another +backend is explicitly set. If no backend is named ``default`` the first +backend in the VCL program becomes the default. + +If you only use dynamic backends created by VMODs, an empty, always failing +(503) backend can be specified:: + + backend default none; + +A backend must be specified with either a ``.host`` or a ``.path`` attribute, but +not both. All other attributes have default values. + +Attribute ``.host`` +------------------- + +To specify a networked backend ``.host`` takes either a numeric +IPv4/IPv6 address or a domain name which resolves to *at most* +one IPv4 and one IPv6 address:: + + .host = "127.0.0.1"; + + .host = "[::1]:8080"; + + .host = "example.com:8081"; + +The TCP port number can be specied as part of ``.host`` as a bove +or separately using the ``.port`` attribute:: + + .port = 8081; + +Attribute ``.path`` +------------------- + +The absolute path to a Unix(4) domain socket of a local backend:: + + .path = "/var/run/http.sock"; + +A warning will be issued if the uds-socket does not exist when the +VCL is loaded. This makes it possible to start the UDS-listening peer, +or set the socket file's permissions afterwards. + +If the uds-socket socket does not exist or permissions deny access, +connection attempts will fail. + +Attribute ``.host_header`` +-------------------------- + +A host header to add to probes and regular backend requests if they have no such header:: + + .host_header = "Host: example.com"; + +Timeout Attributes +------------------ + +These attributes control how patient `varnishd` is during backend fetches:: + + .connect_timeout = 1.4s; + .first_byte_timeout = 20s; + .between_bytes_timeout = 10s; + +The default values comes parameters with the same names, see :ref:`varnishd(1)`. + +Attribute ``.max_connections`` +------------------------------ + +Limit how many simultaneous connections varnish can open to the backend:: + + .max_connections = 1000; + +Attribute ``.proxy_header`` +--------------------------- + +Send a PROXY protocol header to the backend with the ``client.ip`` and +``server.ip`` values:: + + .proxy_header = 2; + +Legal values are one and two, depending which version of the PROXY protocol you want. + +*Notice* this setting will lead to backend connections being used +for a single request only (subject to future improvements). Thus, +extra care should be taken to avoid running into failing backend +connections with EADDRNOTAVAIL due to no local ports being +available. Possible options are: + + * Use additional backend connections to extra IP addresses or TCP ports + + * Increase the number of available ports (Linux sysctl ``net.ipv4.ip_local_port_range``) + + * Reuse backend connection ports early (Linux sysctl ``net.ipv4.tcp_tw_reuse``) + +Attribute ``.preamble`` +----------------------- + +Send a BLOB on all newly opened connections to the backend:: + + .preamble = :SGVsbG8gV29ybGRcbgo=:; + +Attribute ``.probe`` +-------------------- + +Please see :ref:`vcl-probe(7)`. + +SEE ALSO +-------- + +* :ref:`varnishd(1)` +* :ref:`vcl(7)` +* :ref:`vcl-probe(7)` +* :ref:`vmod_directors(3)` +* :ref:`vmod_std(3)` + +HISTORY +------- + +VCL was developed by Poul-Henning Kamp in cooperation with Verdens +Gang AS, Redpill Linpro and Varnish Software. This manual page is +written by Per Buer, Poul-Henning Kamp, Martin Blix Grydeland, +Kristian Lyngstøl, Lasse Karstensen and others. + +COPYRIGHT +--------- + +This document is licensed under the same license as Varnish +itself. See LICENSE for details. + +* Copyright (c) 2006 Verdens Gang AS +* Copyright (c) 2006-2021 Varnish Software AS diff --git a/doc/sphinx/reference/vcl-probe.rst b/doc/sphinx/reference/vcl-probe.rst new file mode 100644 index 0000000000..4268807c80 --- /dev/null +++ b/doc/sphinx/reference/vcl-probe.rst @@ -0,0 +1,183 @@ +.. role:: ref(emphasis) + +.. _vcl-probe(7): + +========= +VCL-probe +========= + +--------------------------------- +Configuring Backend Health Probes +--------------------------------- + +:Manual section: 7 + +.. _reference-vcl_probes: + +Backend health probes +--------------------- + +Varnish can be configured to periodically send a request to test if a +backend is answering and thus "healthy". + +Probes can be configured per backend:: + + backend foo { + [...] + .probe = { + [...] + } + } + +They can be named and shared between backends:: + + probe light { + [...] + } + + backend foo { + .probe = light; + } + + backend bar { + .probe = light; + } + +Or a ``default`` probe can be defined, which applies to all backends +without a specific ``.probe`` configured:: + + probe default { + [...] + } + +The basic syntax is the same as for backends:: + + probe name { + .attribute1 = value; + .attribute2 = "value"; + [...] + } + +There are no mandatory attributes, they all have defaults. + +Attribute ``.url`` +------------------ + +The URL to query. Defaults to ``/``:: + + .url = "/health-probe"; + +Attribute ``.request`` +---------------------- + +Can be used to specify a full HTTP/1.1 request to be sent:: + + .request = "GET / HTTP/1.1" + "Host: example.com" + "X-Magic: We're fine with this." + "Connection: close"; + +Each of the strings will have ``CRNL`` appended and a final HTTP +header block terminating ``CRNL`` will be appended as well. + +Because connection shutdown is part of the health check, +``Connection: close`` is mandatory. + +Attribute ``.expected_response`` +-------------------------------- + +The expected HTTP status, defaults to ``200``:: + + .expected_response = 418; + +Attribute ``.timeout`` +---------------------- + +How fast the probe must succeed, default is two seconds:: + + .timeout = 10s; + +Attribute ``.interval`` +----------------------- + +Time between probes, default is five seconds:: + + .interval = 1m; + +The backend health shift register +--------------------------------- + +Backend health probes uses a 64 stage shift register to remember the +most recent health probes and to evaluate the total health of the backend. + +In the CLI, a good backend health status looks like this: + +.. code-block:: text + + varnish> backend.list -p boot.backend + Backend name Admin Probe Health Last change + boot.backend probe 5/5 healthy Wed, 13 Jan 2021 10:31:50 GMT + Current states good: 5 threshold: 4 window: 5 + Average response time of good probes: 0.000793 + Oldest ================================================== Newest + 4444444444444444444444444444444444444444444444444444444444444444 Good IPv4 + XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit + RRRRRRRRRRRRRRRRRRRRRRR----RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR Good Recv + HHHHHHHHHHHHHHHHHHHHHHH--------HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH Happy + +Starting from the bottom, the last line shows that this backend has been +declared "Happy" for most the 64 health probes, but there were some +trouble some while ago. + +However, in this case the ``.window`` is configured to five, and the +``.threshold`` is set to four, so at this point in time, the backend +is considered fully healthy. + +An additional ``.initial`` fills that many "happy" entries in the +shift register when the VCL is loaded, so that backends can quickly +become healthy, even if their health is normally considered over +many samples:: + + .interval = 1s; + .window = 60; + .threshold = 45; + .initial = 43; + +This backend will be considered healthy if three out of four health +probes in the last minute were good, but it becomes healthy as soon +as two good probes have happened after the VCL was loaded. + +The default values are: + +* ``.window`` = 8 + +* ``.threshold`` = 3 + +* ``.initial`` = one less than ``.threshold`` + + +SEE ALSO +======== + +* :ref:`varnishd(1)` +* :ref:`vcl(7)` +* :ref:`vcl-backend(7)` +* :ref:`vmod_directors(3)` +* :ref:`vmod_std(3)` + +HISTORY +======= + +VCL was developed by Poul-Henning Kamp in cooperation with Verdens +Gang AS, Redpill Linpro and Varnish Software. This manual page is +written by Per Buer, Poul-Henning Kamp, Martin Blix Grydeland, +Kristian Lyngstøl, Lasse Karstensen and others. + +COPYRIGHT +========= + +This document is licensed under the same license as Varnish +itself. See LICENSE for details. + +* Copyright (c) 2006 Verdens Gang AS +* Copyright (c) 2006-2021 Varnish Software AS diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index d6f468e04c..97fe80466b 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -214,157 +214,10 @@ Example:: */ } -.. _backend_definition: - -Backend definition ------------------- - -A backend declaration creates and initialises a named backend object. A -declaration start with the keyword ``backend`` followed by the name of the -backend. The actual declaration is in curly brackets, in a key/value fashion.:: - - backend name { - .attribute = "value"; - } - -One of the attributes ``.host`` or ``.path`` is mandatory (but not -both). The attributes will inherit their defaults from the global -parameters. The following attributes are available: - - ``.host`` - The host to be used. IP address or a hostname that resolves to a - single IP address. This attribute is mandatory, unless ``.path`` - is declared. - - ``.path`` (``VCL >= 4.1``) - - The absolute path of a Unix domain socket at which a backend is - listening. If the file at that path does not exist or is not - accessible to Varnish at VCL load time, then the VCL compiler - issues a warning, but does not fail. This makes it possible to - start the UDS-listening peer, or set the socket file's - permissions, after starting Varnish or loading VCL with a UDS - backend. But the socket file must exist and have necessary - permissions before the first connection is attempted, otherwise - fetches will fail. If the file does exist and is accessible, then - it must be a socket; otherwise the VCL load fails. One of - ``.path`` or ``.host`` must be declared (but not both). ``.path`` - may only be used in VCL since version 4.1. - - ``.port`` - The port on the backend that Varnish should connect to. Ignored if - a Unix domain socket is declared in ``.path``. - - ``.host_header`` - A host header to add to probes and regular backend requests if they have no - such header. - - ``.connect_timeout`` - Timeout for connections. - - Default: ``connect_timeout`` parameter, see :ref:`varnishd(1)` - - ``.first_byte_timeout`` - Timeout for first byte. - - Default: ``first_byte_timeout`` parameter, see :ref:`varnishd(1)` - - ``.between_bytes_timeout`` - Timeout between bytes. - - Default: ``between_bytes_timeout`` parameter, see :ref:`varnishd(1)` - - ``.probe`` - Attach a probe to the backend. See `Probes`_ - - ``.proxy_header`` - The PROXY protocol version Varnish should use when connecting to - this backend. Allowed values are ``1`` and ``2``. - - *Notice* this setting will lead to backend connections being used - for a single request only (subject to future improvements). Thus, - extra care should be taken to avoid running into failing backend - connections with EADDRNOTAVAIL due to no local ports being - available. Possible options are: - - * Use additional backend connections to extra IP addresses or TCP - ports - - * Increase the number of available ports (Linux sysctl - ``net.ipv4.ip_local_port_range``) - - * Reuse backend connection ports early (Linux sysctl - ``net.ipv4.tcp_tw_reuse``) - - ``.max_connections`` - Maximum number of open connections towards this backend. If - Varnish reaches the maximum Varnish it will start failing - connections. - -Empty backends can also be defined using the following syntax.:: - - backend name none; - -An empty backend will always return status code 503 as if it is sick. - -Backends can be used with *directors*. Please see the -:ref:`vmod_directors(3)` man page for more information. - -.. _reference-vcl_probes: - -Probes ------- - -Probes will query the backend for status on a regular basis and mark -the backend as down it they fail. A probe is defined as this:: - - probe name { - .attribute = "value"; - } - -The probe named ``default`` is special and will be used for all backends -which do not explicitly reference a probe. - -There are no mandatory options. These are the options you can set: - - ``.url`` - The URL to query. Defaults to ``/``. - Mutually exclusive with ``.request`` - - ``.request`` - Specify a full HTTP request using multiple strings. ``.request`` will - have ``\r\n`` automatically inserted after every string. - Mutually exclusive with ``.url``. - - *Note* that probes require the backend to complete sending the - response and close the connection within the specified timeout, so - ``.request`` will, for ``HTTP/1.1``, most likely need to contain a - ``"Connection: close"`` string. - - ``.expected_response`` - The expected HTTP response code. Defaults to ``200``. - - ``.timeout`` - The timeout for the probe. Default is ``2s``. - - ``.interval`` - How often the probe is run. Default is ``5s``. - - ``.initial`` - How many of the polls in ``.window`` are considered good when Varnish - starts. Defaults to the value of ``.threshold`` - 1. In this case, the - backend starts as sick and requires one single poll to be - considered healthy. - - ``.window`` - How many of the latest polls we examine to determine backend health. - Defaults to ``8``. - - ``.threshold`` - How many of the polls in ``.window`` must have succeeded to - consider the backend to be healthy. - Defaults to ``3``. +Backends and health probes +-------------------------- +Please see :ref:`vcl-backend(7)` and :ref:`vcl-probe(7)` Access Control List (ACL) ------------------------- @@ -542,6 +395,8 @@ SEE ALSO ======== * :ref:`varnishd(1)` +* :ref:`vcl-backend(7)` +* :ref:`vcl-probe(7)` * :ref:`vmod_directors(3)` * :ref:`vmod_std(3)` diff --git a/man/Makefile.am b/man/Makefile.am index 15bf7c9fca..bae5513703 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -4,6 +4,8 @@ dist_man_MANS = \ varnish-cli.7 \ varnish-counters.7 \ vcl.7 \ + vcl-backend.7 \ + vcl-probe.7 \ vsl.7 \ vsl-query.7 \ varnishadm.1 \ @@ -48,6 +50,12 @@ vcl.7: $(top_builddir)/doc/sphinx/reference/vcl.rst \ $(top_srcdir)/bin/varnishd/builtin.vcl $(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vcl.rst $@ +vcl-backend.7: $(top_builddir)/doc/sphinx/reference/vcl-backend.rst + $(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vcl-backend.rst $@ + +vcl-probe.7: $(top_builddir)/doc/sphinx/reference/vcl-probe.rst + $(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vcl-probe.rst $@ + vsl.7: $(top_builddir)/doc/sphinx/reference/vsl.rst \ $(top_builddir)/doc/sphinx/include/vsl-tags.rst $(BUILD_MAN) $(top_builddir)/doc/sphinx/reference/vsl.rst $@