Skip to content

Commit fa169cc

Browse files
charislamLoquacity
andauthored
[UDA] Note that action signature is mandatory (github#1304)
You must include an action signature for your job, even if you don't need to use the `job_id` and `config` arguments. Also removed some repetition that is hard to maintain. (The UDA examples have already been moved into how-to guides.) Left the UDA page in concepts for now, but these will also be migrated over time. Also some drive-by formatting changes to test the automated formatting set-up. Co-authored-by: Lana Brindley <[email protected]>
1 parent 72a45be commit fa169cc

File tree

3 files changed

+35
-248
lines changed

3 files changed

+35
-248
lines changed

timescaledb/how-to-guides/user-defined-actions/about-user-defined-actions.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ tags: [user-defined actions, background jobs, scheduled jobs, automation framewo
66
---
77

88
# About user-defined actions
9+
910
With user-defined actions, you can write custom functions and procedures, and
1011
schedule them to run periodically. TimescaleDB natively includes some
1112
job-scheduling policies, such as:
13+
1214
* [Continuous aggregate policies][caggs] to automatically refresh continuous
1315
aggregates
1416
* [Compression policies][compressing] to compress historical data

timescaledb/how-to-guides/user-defined-actions/create-and-register.md

Lines changed: 30 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -6,46 +6,57 @@ tags: [user-defined actions, scheduled jobs, background jobs, automation framewo
66
---
77

88
# Create and register user-defined actions
9+
910
Adding a user-defined action to your database is a 2-step process:
11+
1012
1. [Define a function or procedure](#define-a-function-or-procedure)
1113
1. [Register your action with the job scheduler](#register-an-action)
1214

1315
## Define a function or procedure
16+
1417
To create an action, first write the commands you want your
1518
[function][postgres-createfunction] or [procedure][postgres-createprocedure] to
1619
execute. Then wrap your commands in a `CREATE OR REPLACE` statement. You can
1720
also define the language of your commands in this statement. The example below
1821
uses PLPGSQL.
1922

2023
The outer statement contains the action signature: `(job_id INT, config JSONB)`.
21-
Include the signature literally. Don't replace with an actual job ID or config
22-
object. You define `config` when you register your job. `job_id` is also
23-
automatically generated then.
24+
You must include this signature even if you don't need to use those arguments in
25+
your action. Include it literally, don't replace it with an actual job ID or config
26+
object. When you register the job, you can define the config.
27+
The job ID is automatically generated.
2428

25-
The following example defines a simple procedure that raises a notice. Replace
26-
the `RAISE NOTICE` with the commands you want to run.
29+
This example defines a simple procedure that raises a notice. Replace
30+
the `RAISE NOTICE` command with the commands you want to run.
2731

2832
<procedure>
2933

3034
## Defining a procedure
31-
1. Write the commands you want your job to run.
35+
36+
1. Write the commands you want your job to run. For example:
37+
3238
```sql
3339
BEGIN
3440
RAISE NOTICE 'Executing job % with config %', job_id, config;
3541
END
3642
```
37-
2. Wrap the commands in a `CREATE OR REPLACE PROCEDURE` statement.
43+
44+
1. Wrap the commands in a `CREATE OR REPLACE PROCEDURE` statement. For example:
45+
3846
```sql
39-
CREATE OR REPLACE PROCEDURE user_defined_action(job_id INT, config JSONB) LANGUAGE PLPGSQL AS
40-
$$
41-
BEGIN
42-
RAISE NOTICE 'Executing job % with config %', job_id, config;
43-
END
44-
$$;
47+
CREATE OR REPLACE PROCEDURE user_defined_action(job_id INT, config JSONB)
48+
LANGUAGE PLPGSQL AS
49+
$$
50+
BEGIN
51+
RAISE NOTICE 'Executing job % with config %', job_id, config;
52+
END
53+
$$;
54+
```
4555

4656
</procedure>
4757

4858
## Register an action
59+
4960
To make the job scheduler run your action, you need to register it. Use the
5061
[`add_job`][api-add_job] function. Supply the name of your action, the schedule
5162
you want it to run on, and the content of your config. If your job needs no
@@ -54,10 +65,13 @@ parameters, use a NULL config.
5465
<procedure>
5566

5667
## Registering an action
57-
1. Add a job for your user-defined action.
68+
69+
1. Add a job for your user-defined action. For example:
70+
5871
```sql
5972
SELECT add_job('user_defined_action', '1h', config => '{"hypertable":"metr"}');
6073
```
74+
6175
1. The `add_job` call returns a `job_id`. It stores the `job_id` and `config`
6276
in the TimescaleDB catalog.
6377
1. The action runs on schedule. It also runs when you manually start it by
@@ -67,14 +81,15 @@ parameters, use a NULL config.
6781
</procedure>
6882

6983
## List current jobs
84+
7085
To list all currently registered jobs, query
7186
[`timescaledb_information.jobs`][api-timescaledb_information-jobs].
87+
7288
```sql
7389
SELECT * FROM timescaledb_information.jobs;
7490
```
7591

7692
[api-add_job]: /api/:currentVersion:/actions/add_job
77-
[api-alter_job]: /api/:currentVersion:/actions/alter_job
7893
[api-run_job]: /api/:currentVersion:/actions/run_job
7994
[api-timescaledb_information-jobs]: /api/:currentVersion:/informational-views/jobs/
8095
[postgres-createfunction]: https://www.postgresql.org/docs/current/sql-createfunction.html

timescaledb/overview/core-concepts/user-defined-actions.md

Lines changed: 3 additions & 233 deletions
Original file line numberDiff line numberDiff line change
@@ -10,238 +10,8 @@ tags: [user-defined actions, background jobs, scheduled jobs]
1010
User-defined actions allow you to run functions and procedures implemented in a
1111
language of your choice on a schedule within TimescaleDB. This allows
1212
automatic periodic tasks that are not covered by existing policies, or the
13-
ability to enhance existing policies with additional functionality.
13+
ability to enhance existing policies with additional features.
1414

15-
User-defined actions have allow free-form configuration through a JSONB object
16-
which allows endless flexibility and reusability.
15+
To learn more about user-defined actions, see the [how-to guide][how-to].
1716

18-
## Examples
19-
20-
The following section provides a number of examples of user-defined actions
21-
that you can specify and subsequently schedule as part of TimescaleDB's
22-
automation framework.
23-
24-
### Generic retention
25-
26-
Create a generic data retention policy that applies to ALL hypertables, as opposed
27-
to just a single one as required by `add_retention_policy`.
28-
The policy could be further refined with additional filters, by adding a WHERE
29-
clause to the PERFORM query in the procedure definition.
30-
31-
```sql
32-
CREATE OR REPLACE PROCEDURE generic_retention (job_id int, config jsonb)
33-
LANGUAGE PLPGSQL
34-
AS $$
35-
DECLARE
36-
drop_after interval;
37-
BEGIN
38-
SELECT jsonb_object_field_text (config, 'drop_after')::interval INTO STRICT drop_after;
39-
40-
IF drop_after IS NULL THEN
41-
RAISE EXCEPTION 'Config must have drop_after';
42-
END IF;
43-
44-
PERFORM drop_chunks(format('%I.%I', table_schema, table_name), older_than => drop_after)
45-
FROM timescaledb_information.hypertables;
46-
END
47-
$$;
48-
```
49-
50-
Register job to run daily dropping chunks on all hypertables that are older
51-
than 12 months.
52-
53-
```sql
54-
SELECT add_job('generic_retention','1d', config => '{"drop_after":"12 month"}');
55-
```
56-
57-
### Tiered storage
58-
59-
Action that moves chunks older than a certain time to a different tablespace.
60-
61-
```sql
62-
CREATE OR REPLACE PROCEDURE move_chunks (job_id int, config jsonb)
63-
LANGUAGE PLPGSQL
64-
AS $$
65-
DECLARE
66-
ht REGCLASS;
67-
lag interval;
68-
destination name;
69-
chunk REGCLASS;
70-
tmp_name name;
71-
BEGIN
72-
SELECT jsonb_object_field_text (config, 'hypertable')::regclass INTO STRICT ht;
73-
SELECT jsonb_object_field_text (config, 'lag')::interval INTO STRICT lag;
74-
SELECT jsonb_object_field_text (config, 'tablespace') INTO STRICT destination;
75-
76-
IF ht IS NULL OR lag IS NULL OR destination IS NULL THEN
77-
RAISE EXCEPTION 'Config must have hypertable, lag and destination';
78-
END IF;
79-
80-
FOR chunk IN
81-
SELECT show.oid
82-
FROM show_chunks(ht, older_than => lag)
83-
SHOW (oid)
84-
INNER JOIN pg_class pgc ON pgc.oid = show.oid
85-
INNER JOIN pg_tablespace pgts ON pgts.oid = pgc.reltablespace
86-
WHERE pgts.spcname != destination
87-
LOOP
88-
RAISE NOTICE 'Moving chunk: %', chunk::text;
89-
EXECUTE format('ALTER TABLE %s SET TABLESPACE %I;', chunk, destination);
90-
END LOOP;
91-
END
92-
$$;
93-
```
94-
95-
Register job to run daily moving chunks older than 12 months on hypertable
96-
metrics to tablespace old_chunks.
97-
98-
```sql
99-
SELECT add_job('move_chunks','1d', config => '{"hypertable":"metrics","lag":"12 month","tablespace":"old_chunks"}');
100-
```
101-
102-
The above action uses the simpler `ALTER TABLE ... SET TABLESPACE` for moving
103-
a chunk, but it could alternatively be written in terms of TimescaleDB's
104-
[`move_chunk`][api-move_chunk]. The `move_chunk` function also requires an
105-
index as input, but performs data re-ordering as part of the move (for faster
106-
subsequent queries) and requires lower lock levels, so the chunk remains available
107-
for reads during the move.
108-
109-
### Downsample and compress
110-
111-
Action that downsamples and compresses chunks on hypertable `metrics`
112-
older than a certain age. The example query computes a simple `avg` over
113-
hourly data for downsampling, but this query can be arbitrarily complex.
114-
115-
```sql
116-
CREATE OR REPLACE PROCEDURE downsample_compress (job_id int, config jsonb)
117-
LANGUAGE PLPGSQL
118-
AS $$
119-
DECLARE
120-
lag interval;
121-
chunk REGCLASS;
122-
tmp_name name;
123-
BEGIN
124-
SELECT jsonb_object_field_text (config, 'lag')::interval INTO STRICT lag;
125-
126-
IF lag IS NULL THEN
127-
RAISE EXCEPTION 'Config must have lag';
128-
END IF;
129-
130-
FOR chunk IN
131-
SELECT show.oid
132-
FROM show_chunks('metrics', older_than => lag) SHOW (oid)
133-
INNER JOIN pg_class pgc ON pgc.oid = show.oid
134-
INNER JOIN pg_namespace pgns ON pgc.relnamespace = pgns.oid
135-
INNER JOIN timescaledb_information.chunks chunk ON chunk.chunk_name = pgc.relname
136-
AND chunk.chunk_schema = pgns.nspname
137-
WHERE chunk.is_compressed::bool = FALSE
138-
LOOP
139-
RAISE NOTICE 'Processing chunk: %', chunk::text;
140-
141-
-- build name for temp table
142-
SELECT '_tmp' || relname
143-
FROM pg_class
144-
WHERE oid = chunk INTO STRICT tmp_name;
145-
146-
-- copy downsampled chunk data into temp table
147-
EXECUTE format($sql$ CREATE UNLOGGED TABLE %I AS
148-
SELECT time_bucket('1h', time), device_id, avg(value) FROM %s GROUP BY 1, 2;
149-
$sql$, tmp_name, chunk);
150-
151-
-- clear original chunk
152-
EXECUTE format('TRUNCATE %s;', chunk);
153-
154-
-- copy downsampled data back into chunk
155-
EXECUTE format('INSERT INTO %s(time, device_id, value) SELECT * FROM %I;', chunk, tmp_name);
156-
157-
-- drop temp table
158-
EXECUTE format('DROP TABLE %I;', tmp_name);
159-
160-
PERFORM compress_chunk (chunk);
161-
162-
COMMIT;
163-
END LOOP;
164-
END
165-
$$;
166-
```
167-
168-
Register job to run daily downsampling and compressing chunks older than
169-
12 months.
170-
171-
```sql
172-
SELECT add_job('downsample_compress','1d', config => '{"lag":"12 month"}');
173-
```
174-
175-
### Set up a table and procedure
176-
177-
Set up a simple table and procedure:
178-
```sql
179-
CREATE TABLE IF NOT EXISTS times(time timestamptz);
180-
181-
CREATE OR REPLACE PROCEDURE add_time(job_id int, config jsonb)
182-
LANGUAGE PLPGSQL
183-
AS $$
184-
BEGIN
185-
INSERT INTO times VALUES (now());
186-
END
187-
$$;
188-
189-
CALL add_time(null, '{}');
190-
191-
SELECT * from times;
192-
```
193-
194-
Returns:
195-
```sql
196-
|time |
197-
|-----------------------------|
198-
|2021-12-27 10:50:28.449 -0600|
199-
```
200-
201-
Register a job to run every 5 seconds and return a `job_id`:
202-
```sql
203-
SELECT add_job('add_time', '5 secs');
204-
```
205-
206-
Returns:
207-
```sql
208-
|add_job|
209-
|-------|
210-
|1000 |
211-
```
212-
213-
Wait for some jobs to run, then view the results:
214-
```sql
215-
SELECT * from times;
216-
```
217-
218-
Returns:
219-
```sql
220-
|time |
221-
|-----------------------------|
222-
|2021-12-27 10:50:28.449 -0600|
223-
|2021-12-27 10:50:38.825 -0600|
224-
|2021-12-27 10:50:44.164 -0600|
225-
|2021-12-27 10:50:49.207 -0600|
226-
|...|
227-
```
228-
229-
See running jobs:
230-
```sql
231-
SELECT job_id, total_runs, total_failures, total_successes
232-
FROM timescaledb_information.job_stats;
233-
```
234-
235-
Returns:
236-
```sql
237-
|job_id|total_runs|total_failures|total_successes|
238-
|------|----------|--------------|---------------|
239-
|1000 |9 |0 |9 |
240-
```
241-
242-
Stop or delete the job:
243-
```sql
244-
SELECT delete_job(1000);
245-
```
246-
247-
[api-move_chunk]: /api/:currentVersion:/hypertable/move_chunk
17+
[how-to]: /timescaledb/:currentVersion:/how-to-guides/user-defined-actions/

0 commit comments

Comments
 (0)