ifqld
is an HTTP server for running IFQL queries to one or more InfluxDB
servers.
ifqld
runs on port 8093
by default
Here is the rough design specification for details until we get documentation up: http://bit.ly/ifql-spec
-
Upgrade to InfluxDB >= 1.4.1 https://portal.influxdata.com/downloads
-
Update the InfluxDB configuration file to enable IFQL processing; restart the InfluxDB server. InfluxDB will open port
8082
to accept IFQL queries.
This port has no authentication.
[ifql]
enabled = true
log-enabled = true
bind-address = ":8082"
-
Download
ifqld
and install from https://github.com/influxdata/ifql/releases -
Start
ifqld
with the InfluxDB host and port of8082
. To run in federated mode (see below), add the--host
option for each InfluxDB host.
ifqld --verbose --host localhost:8082
- To run a query POST an IFQL query string to
/query
as theq
parameter:
curl -XPOST --data-urlencode \
'q=from(db:"telegraf")
|> filter(fn: (r) => r["_measurement"] == "cpu" AND r["_field"] == "usage_user")
|> range(start:-170h)
|> sum()' \
http://localhost:8093/query
To spin up a testing environment you can run:
docker-compose up
Inside the root
directory. It will spin up an influxdb
and ifqld
daemon
ready to be used. influxd
is exposed on port 8086
and port 8082
.
Metrics are exposed on /metrics
.
ifqld
records the number of queries and the number of different functions within IFQL queries
By passing the --host
option multiple times ifqld
will query multiple
InfluxDB servers.
For example:
ifqld --host influxdb1:8082 --host influxdb2:8082
The results from multiple InfluxDB are merged together as if there was one server.
IFQL constructs a query by starting with a table of data and passing the table through transformations steps to describe the desired query operations.
Transformations are represented as functions which take a table of data as an input argument and return a new table that has been transformed.
There is a special function from
which is a source function, meaning it does not accept a table as input, but rather produces a table.
All other transformation functions accept at least one table and return a table as a result.
For example to get the last point for each series in a database you start by creating a table using from
and then pass that table into the limit
function.
// Select the last point per series in the telegraf database.
limit(table:from(db:"telegraf"), n:1)
Since it is common to chain long lists of transformations together the pipe forward operator |>
can be used to make reading the code easier.
These two expressions are equivalent:
// Select the last point per series in the telegraf database.
limit(table:from(db:"telegraf"), n:1)
// Same as above, but uses the pipe forward operator to indicate the flow of data.
from(db:"telegraf") |> limit(n:1)
Long list of functions can thus be chained together:
// Get the first point per host from the last minute of data.
from(db:"telegraf") |> range(start:-1m) |> group(by:["host"]) |> first()
Below is a list of supported functions.
Starting point for all queires. Get data from the specified database.
Example: from(db:"telegraf")
-
db
stringfrom(db:"telegraf")
-
hosts
array of stringsfrom(db:"telegraf", hosts:["host1", "host2"])
Counts the number of results
Example: from(db:"telegraf") |> count()
Returns the first result of the query
Example: from(db:"telegraf") |> first()
Groups results by a user-specified set of tags
by
array of strings Group by these specific tag names Cannot be used withexcept
option
Example: from(db: "telegraf") |> range(start: -30m) |> group(by: ["tag_a", "tag_b"])
keep
array of strings Keep specific tag keys that were not inby
in the results
Example: from(db: "telegraf") |> range(start: -30m) |> group(by: ["tag_a", "tag_b"], keep:["tag_c"])
except
array of strings Group by all but these tag keys Cannot be used withby
option
Example: from(db: "telegraf") |> range(start: -30m) |> group(except: ["tag_a"], keep:["tag_b", "tag_c"])
Join two time series together on time and the list of on
keys.
Example:
cpu = from(db: "telegraf") |> filter(fn: (r) => r["_measurement"] == "cpu" and r["_field"] == "usage_user") |> range(start: -30m)
mem = from(db: "telegraf") |> filter(fn: (r) => r["_measurement"] == "mem" and r["_field"] == "used_percent") |> range(start: -30m)
join(tables:{cpu:cpu, mem:mem}, on:["host"], fn: (tables) => tables.cpu["_value"] + tables.mem["_value"])
-
tables
map of tables Map of tables to join. Currently only two tables are allowed. -
on
array of strings List of tag keys that when equal produces a result set. -
fn
Defines the function that merges the values of the tables.
The function must defined to accept a single parameter.
The parameter is a map, which uses the same keys found in the tables
map.
The function is called for each joined set of records from the tables.
Returns the last result of the query
Example: from(db: "telegraf") |> last()
Restricts the number of rows returned in the results.
Example: from(db: "telegraf") |> limit(n: 10)
Applies a function to each row of the table.
fn
function
Function to apply to each row. The return value of the function may be a single value or an object.
Example:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"]=="cpu" AND
r["_field"] == "usage_system" AND
r["service"] == "app-server")
|> range(start:-12h)
// Square the value
|> map(fn: (r) => r._value * r._value)
Example:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"]=="cpu" AND
r["_field"] == "usage_system" AND
r["service"] == "app-server")
|> range(start:-12h)
// Square the value and keep the original value
|> map(fn: (r) => ({value: r._value, value2:r._value * r._value}))
Returns the max value within the results
Example:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"]=="cpu" AND
r["_field"] == "usage_system" AND
r["service"] == "app-server")
|> range(start:-12h)
|> window(every:10m)
|> max()
Returns the mean of the values within the results
Example:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"] == "mem" AND
r["_field"] == "used_percent")
|> range(start:-12h)
|> window(every:10m)
|> mean()
Returns the min value within the results
Example:
from(db:"foo")
|> filter(fn: (r) => r[ "_measurement"] == "cpu" AND
r["_field" ]== "usage_system")
|> range(start:-12h)
|> window(every:10m, period: 5m)
|> min()
Filters the results by time boundaries
Example:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"] == "cpu" AND
r["_field"] == "usage_system")
|> range(start:-12h, stop: -15m)
-
start duration Specifies the oldest time to be included in the results
-
stop duration or timestamp Specifies exclusive upper time bound Defaults to "now"
Example to sample every fifth point starting from the second element:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"] == "cpu" AND
r["_field"] == "usage_system")
|> range(start:-1d)
|> sample(n: 5, pos: 1)
n
Sample every Nth elementpos
Position offset from start of results to begin samplingpos
must be less thann
Ifpos
less than 0, a random offset is used. Default is -1 (random offset)
Add tag of key and value to set
Example: from(db: "telegraf") |> set(key: "mykey", value: "myvalue")
key
stringvalue
string
Skew of the results
Example: from(db: "telegraf") |> range(start: -30m, stop: -15m) |> skew()
Sorts the results by the specified columns Default sort is ascending
Example:
from(db:"telegraf")
|> filter(fn: (r) => r["_measurement"] == "system" AND
r["_field"] == "uptime")
|> range(start:-12h)
|> sort(cols:["region", "host", "value"])
cols
array of strings List of columns used to sort; precedence from left to right. Default is["value"]
For example, this sorts by uptime descending to find the longest running instances.
from(db:"telegraf")
|> filter(fn: (r) => r["_measurement"] == "system" AND
r["_field"] == "uptime")
|> range(start:-12h)
|> sort(desc: true)
desc
bool Sort results descending
Difference between min and max values
Example: from(db: "telegraf") |> range(start: -30m) |> spread()
Standard Deviation of the results
Example: from(db: "telegraf") |> range(start: -30m, stop: -15m) |> stddev()
Sum of the results
Example: from(db: "telegraf") |> range(start: -30m, stop: -15m) |> sum()
Filters the results using an expression
Example:
from(db:"foo")
|> filter(fn: (r) => r["_measurement"]=="cpu" AND
r["_field"] == "usage_system" AND
r["service"] == "app-server")
|> range(start:-12h)
|> max()
fn
function(record) bool
Function to when filtering the records. The function must accept a single parameter which will be the records and return a boolean value. Records which evaluate to true, will be included in the results.
Partitions the results by a given time range
every
duration Duration of time between windows
Defaults to period
's value
from(db:"foo")
|> range(start:-12h)
|> window(every:10m)
|> max()
period
duration Duration of the windowed parition
from(db:"foo")
|> range(start:-12h)
|> window(every:10m)
|> max()
Default to every
's value
-
start
time The time of the initial window parition. -
round
duration Rounds a window's bounds to the nearest duration
Example:
from(db:"foo")
|> range(start:-12h)
|> window(every:10m)
|> max()
IFQL also allows the user to define their own functions. The function syntax is:
(parameter list) => <function body>
The list of parameters is simply a list of identifiers with optional default values. The function body is either a single expression which is returned or a block of statements. Functions may be assigned to identifiers to given them a name.
Examples:
// Define a simple addition function
add = (a,b) => a + b
// Define a helper function to get data from a telegraf measurement.
// By default the database is expected to be named "telegraf".
telegrafM = (measurement, db="telegraf") =>
from(db:db)
|> filter(fn: (r) => r._measurement == measurement)
// Define a helper function for a common join operation
// Use block syntax since we have more than a single expression
abJoin = (measurementA, measurementB, on) => {
a = telegrafM(measurement:measurementA)
b = telegrafM(measurement:measurementB)
return join(
tables:{a:a, b:b},
on:on,
// Return a map from the join fn,
// this creates a table with a column for each key in the map.
// Note the () around the map to indicate a single map expression instead of function block.
fn: (t) => ({
a: t.a._value,
b: t.b._value,
}),
)
}
Functions may also declare that an argument can be piped into from an pipe forward operator by specifing a special default value:
// Define add function which accepts `a` as the piped argument.
add = (a=<-, b) => a + b
// Call add using the pipe forward syntax.
1 |> add(b:3) // 4
// Define measurement function which accepts table as the piped argument.
measurement = (m, table=<-) => table |> filter(fn: (r) => r._measurement == m)
// Define field function which accepts table as the piped argument
field = (field, table=<-) => table |> filter(fn: (r) => r._field == field)
// Query usage_idle from the cpu measurement and the telegraf database.
// Using the measurement and field functions.
from(db:"telegraf")
|> measurement(m:"cpu")
|> field(field:"usage_idle")