Skip to content

Commit 7103ebb

Browse files
alvherrealvherre
committed
Add support for MERGE SQL command
MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <[email protected]> Author: Álvaro Herrera <[email protected]> Author: Amit Langote <[email protected]> Author: Simon Riggs <[email protected]> Reviewed-by: Peter Eisentraut <[email protected]> Reviewed-by: Andres Freund <[email protected]> (earlier versions) Reviewed-by: Peter Geoghegan <[email protected]> (earlier versions) Reviewed-by: Robert Haas <[email protected]> (earlier versions) Reviewed-by: Japin Li <[email protected]> Reviewed-by: Justin Pryzby <[email protected]> Reviewed-by: Tomas Vondra <[email protected]> Reviewed-by: Zhihong Yu <[email protected]> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/[email protected]
1 parent ae63017 commit 7103ebb

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

95 files changed

+8726
-167
lines changed

‎contrib/test_decoding/expected/ddl.out

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,52 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
192192
COMMIT
193193
(33 rows)
194194

195+
-- MERGE support
196+
BEGIN;
197+
MERGE INTO replication_example t
198+
USING (SELECT i as id, i as data, i as num FROM generate_series(-20, 5) i) s
199+
ON t.id = s.id
200+
WHEN MATCHED AND t.id < 0 THEN
201+
UPDATE SET somenum = somenum + 1
202+
WHEN MATCHED AND t.id >= 0 THEN
203+
DELETE
204+
WHEN NOT MATCHED THEN
205+
INSERT VALUES (s.*);
206+
COMMIT;
207+
/* display results */
208+
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
209+
data
210+
--------------------------------------------------------------------------------------------------------------------------------------------------
211+
BEGIN
212+
table public.replication_example: INSERT: id[integer]:-20 somedata[integer]:-20 somenum[integer]:-20 zaphod1[integer]:null zaphod2[integer]:null
213+
table public.replication_example: INSERT: id[integer]:-19 somedata[integer]:-19 somenum[integer]:-19 zaphod1[integer]:null zaphod2[integer]:null
214+
table public.replication_example: INSERT: id[integer]:-18 somedata[integer]:-18 somenum[integer]:-18 zaphod1[integer]:null zaphod2[integer]:null
215+
table public.replication_example: INSERT: id[integer]:-17 somedata[integer]:-17 somenum[integer]:-17 zaphod1[integer]:null zaphod2[integer]:null
216+
table public.replication_example: INSERT: id[integer]:-16 somedata[integer]:-16 somenum[integer]:-16 zaphod1[integer]:null zaphod2[integer]:null
217+
table public.replication_example: UPDATE: id[integer]:-15 somedata[integer]:-15 somenum[integer]:-14 zaphod1[integer]:null zaphod2[integer]:null
218+
table public.replication_example: UPDATE: id[integer]:-14 somedata[integer]:-14 somenum[integer]:-13 zaphod1[integer]:null zaphod2[integer]:null
219+
table public.replication_example: UPDATE: id[integer]:-13 somedata[integer]:-13 somenum[integer]:-12 zaphod1[integer]:null zaphod2[integer]:null
220+
table public.replication_example: UPDATE: id[integer]:-12 somedata[integer]:-12 somenum[integer]:-11 zaphod1[integer]:null zaphod2[integer]:null
221+
table public.replication_example: UPDATE: id[integer]:-11 somedata[integer]:-11 somenum[integer]:-10 zaphod1[integer]:null zaphod2[integer]:null
222+
table public.replication_example: UPDATE: id[integer]:-10 somedata[integer]:-10 somenum[integer]:-9 zaphod1[integer]:null zaphod2[integer]:null
223+
table public.replication_example: UPDATE: id[integer]:-9 somedata[integer]:-9 somenum[integer]:-8 zaphod1[integer]:null zaphod2[integer]:null
224+
table public.replication_example: UPDATE: id[integer]:-8 somedata[integer]:-8 somenum[integer]:-7 zaphod1[integer]:null zaphod2[integer]:null
225+
table public.replication_example: UPDATE: id[integer]:-7 somedata[integer]:-7 somenum[integer]:-6 zaphod1[integer]:null zaphod2[integer]:null
226+
table public.replication_example: UPDATE: id[integer]:-6 somedata[integer]:-6 somenum[integer]:-5 zaphod1[integer]:null zaphod2[integer]:null
227+
table public.replication_example: UPDATE: id[integer]:-5 somedata[integer]:-5 somenum[integer]:-4 zaphod1[integer]:null zaphod2[integer]:null
228+
table public.replication_example: UPDATE: id[integer]:-4 somedata[integer]:-4 somenum[integer]:-3 zaphod1[integer]:null zaphod2[integer]:null
229+
table public.replication_example: UPDATE: id[integer]:-3 somedata[integer]:-3 somenum[integer]:-2 zaphod1[integer]:null zaphod2[integer]:null
230+
table public.replication_example: UPDATE: id[integer]:-2 somedata[integer]:-2 somenum[integer]:-1 zaphod1[integer]:null zaphod2[integer]:null
231+
table public.replication_example: UPDATE: id[integer]:-1 somedata[integer]:-1 somenum[integer]:0 zaphod1[integer]:null zaphod2[integer]:null
232+
table public.replication_example: DELETE: id[integer]:0
233+
table public.replication_example: DELETE: id[integer]:1
234+
table public.replication_example: DELETE: id[integer]:2
235+
table public.replication_example: DELETE: id[integer]:3
236+
table public.replication_example: DELETE: id[integer]:4
237+
table public.replication_example: DELETE: id[integer]:5
238+
COMMIT
239+
(28 rows)
240+
195241
CREATE TABLE tr_unique(id2 serial unique NOT NULL, data int);
196242
INSERT INTO tr_unique(data) VALUES(10);
197243
ALTER TABLE tr_unique RENAME TO tr_pkey;

‎contrib/test_decoding/sql/ddl.sql

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,22 @@ COMMIT;
9393
/* display results */
9494
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1', 'include-sequences', '0');
9595

96+
-- MERGE support
97+
BEGIN;
98+
MERGE INTO replication_example t
99+
USING (SELECT i as id, i as data, i as num FROM generate_series(-20, 5) i) s
100+
ON t.id = s.id
101+
WHEN MATCHED AND t.id < 0 THEN
102+
UPDATE SET somenum = somenum + 1
103+
WHEN MATCHED AND t.id >= 0 THEN
104+
DELETE
105+
WHEN NOT MATCHED THEN
106+
INSERT VALUES (s.*);
107+
COMMIT;
108+
109+
/* display results */
110+
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
111+
96112
CREATE TABLE tr_unique(id2 serial unique NOT NULL, data int);
97113
INSERT INTO tr_unique(data) VALUES(10);
98114
ALTER TABLE tr_unique RENAME TO tr_pkey;

‎doc/src/sgml/libpq.sgml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4125,9 +4125,11 @@ char *PQcmdTuples(PGresult *res);
41254125
<structname>PGresult</structname>. This function can only be used following
41264126
the execution of a <command>SELECT</command>, <command>CREATE TABLE AS</command>,
41274127
<command>INSERT</command>, <command>UPDATE</command>, <command>DELETE</command>,
4128-
<command>MOVE</command>, <command>FETCH</command>, or <command>COPY</command> statement,
4129-
or an <command>EXECUTE</command> of a prepared query that contains an
4130-
<command>INSERT</command>, <command>UPDATE</command>, or <command>DELETE</command> statement.
4128+
<command>MERGE</command>, <command>MOVE</command>, <command>FETCH</command>,
4129+
or <command>COPY</command> statement, or an <command>EXECUTE</command> of a
4130+
prepared query that contains an <command>INSERT</command>,
4131+
<command>UPDATE</command>, <command>DELETE</command>,
4132+
or <command>MERGE</command> statement.
41314133
If the command that generated the <structname>PGresult</structname> was anything
41324134
else, <xref linkend="libpq-PQcmdTuples"/> returns an empty string. The caller
41334135
should not free the return value directly. It will be freed when

‎doc/src/sgml/mvcc.sgml

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,37 @@ COMMIT;
422422
<literal>11</literal>, which no longer matches the criteria.
423423
</para>
424424

425+
<para>
426+
<command>MERGE</command> allows the user to specify various
427+
combinations of <command>INSERT</command>, <command>UPDATE</command>
428+
or <command>DELETE</command> subcommands. A <command>MERGE</command>
429+
command with both <command>INSERT</command> and <command>UPDATE</command>
430+
subcommands looks similar to <command>INSERT</command> with an
431+
<literal>ON CONFLICT DO UPDATE</literal> clause but does not
432+
guarantee that either <command>INSERT</command> or
433+
<command>UPDATE</command> will occur.
434+
If MERGE attempts an <command>UPDATE</command> or
435+
<command>DELETE</command> and the row is concurrently updated but
436+
the join condition still passes for the current target and the
437+
current source tuple, then <command>MERGE</command> will behave
438+
the same as the <command>UPDATE</command> or
439+
<command>DELETE</command> commands and perform its action on the
440+
updated version of the row. However, because <command>MERGE</command>
441+
can specify several actions and they can be conditional, the
442+
conditions for each action are re-evaluated on the updated version of
443+
the row, starting from the first action, even if the action that had
444+
originally matched appears later in the list of actions.
445+
On the other hand, if the row is concurrently updated or deleted so
446+
that the join condition fails, then <command>MERGE</command> will
447+
evaluate the condition's <literal>NOT MATCHED</literal> actions next,
448+
and execute the first one that succeeds.
449+
If <command>MERGE</command> attempts an <command>INSERT</command>
450+
and a unique index is present and a duplicate row is concurrently
451+
inserted, then a uniqueness violation is raised.
452+
<command>MERGE</command> does not attempt to avoid the
453+
error by executing an <command>UPDATE</command>.
454+
</para>
455+
425456
<para>
426457
Because Read Committed mode starts each command with a new snapshot
427458
that includes all transactions committed up to that instant,
@@ -924,7 +955,8 @@ ERROR: could not serialize access due to read/write dependencies among transact
924955

925956
<para>
926957
The commands <command>UPDATE</command>,
927-
<command>DELETE</command>, and <command>INSERT</command>
958+
<command>DELETE</command>, <command>INSERT</command>, and
959+
<command>MERGE</command>
928960
acquire this lock mode on the target table (in addition to
929961
<literal>ACCESS SHARE</literal> locks on any other referenced
930962
tables). In general, this lock mode will be acquired by any

‎doc/src/sgml/plpgsql.sgml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1388,7 +1388,7 @@ EXECUTE format('SELECT count(*) FROM %I '
13881388
Another restriction on parameter symbols is that they only work in
13891389
optimizable SQL commands
13901390
(<command>SELECT</command>, <command>INSERT</command>, <command>UPDATE</command>,
1391-
<command>DELETE</command>, and certain commands containing one of these).
1391+
<command>DELETE</command>, <command>MERGE</command>, and certain commands containing one of these).
13921392
In other statement
13931393
types (generically called utility statements), you must insert
13941394
values textually even if they are just data values.
@@ -1666,7 +1666,8 @@ GET DIAGNOSTICS integer_var = ROW_COUNT;
16661666
</listitem>
16671667
<listitem>
16681668
<para>
1669-
<command>UPDATE</command>, <command>INSERT</command>, and <command>DELETE</command>
1669+
<command>UPDATE</command>, <command>INSERT</command>, <command>DELETE</command>,
1670+
and <command>MERGE</command>
16701671
statements set <literal>FOUND</literal> true if at least one
16711672
row is affected, false if no row is affected.
16721673
</para>

‎doc/src/sgml/ref/allfiles.sgml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@ Complete list of usable sgml source files in this directory.
158158
<!ENTITY listen SYSTEM "listen.sgml">
159159
<!ENTITY load SYSTEM "load.sgml">
160160
<!ENTITY lock SYSTEM "lock.sgml">
161+
<!ENTITY merge SYSTEM "merge.sgml">
161162
<!ENTITY move SYSTEM "move.sgml">
162163
<!ENTITY notify SYSTEM "notify.sgml">
163164
<!ENTITY prepare SYSTEM "prepare.sgml">

‎doc/src/sgml/ref/create_policy.sgml

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,8 @@ CREATE POLICY <replaceable class="parameter">name</replaceable> ON <replaceable
5555
</para>
5656

5757
<para>
58-
For <command>INSERT</command> and <command>UPDATE</command> statements,
58+
For <command>INSERT</command>, <command>UPDATE</command>, and
59+
<command>MERGE</command> statements,
5960
<literal>WITH CHECK</literal> expressions are enforced after
6061
<literal>BEFORE</literal> triggers are fired, and before any actual data
6162
modifications are made. Thus a <literal>BEFORE ROW</literal> trigger may
@@ -281,7 +282,9 @@ CREATE POLICY <replaceable class="parameter">name</replaceable> ON <replaceable
281282
<listitem>
282283
<para>
283284
Using <literal>INSERT</literal> for a policy means that it will apply
284-
to <literal>INSERT</literal> commands. Rows being inserted that do
285+
to <literal>INSERT</literal> commands and <literal>MERGE</literal>
286+
commands that contain <literal>INSERT</literal> actions.
287+
Rows being inserted that do
285288
not pass this policy will result in a policy violation error, and the
286289
entire <literal>INSERT</literal> command will be aborted.
287290
An <literal>INSERT</literal> policy cannot have
@@ -305,7 +308,9 @@ CREATE POLICY <replaceable class="parameter">name</replaceable> ON <replaceable
305308
to <literal>UPDATE</literal>, <literal>SELECT FOR UPDATE</literal>
306309
and <literal>SELECT FOR SHARE</literal> commands, as well as
307310
auxiliary <literal>ON CONFLICT DO UPDATE</literal> clauses of
308-
<literal>INSERT</literal> commands. Since <literal>UPDATE</literal>
311+
<literal>INSERT</literal> commands.
312+
<literal>MERGE</literal> commands containing <literal>UPDATE</literal>
313+
actions are affected as well. Since <literal>UPDATE</literal>
309314
involves pulling an existing record and replacing it with a new
310315
modified record, <literal>UPDATE</literal>
311316
policies accept both a <literal>USING</literal> expression and
@@ -435,7 +440,7 @@ CREATE POLICY <replaceable class="parameter">name</replaceable> ON <replaceable
435440
<entry>&mdash;</entry>
436441
</row>
437442
<row>
438-
<entry><command>INSERT</command></entry>
443+
<entry><command>INSERT</command> / <command>MERGE ... THEN INSERT</command></entry>
439444
<entry>&mdash;</entry>
440445
<entry>New row</entry>
441446
<entry>&mdash;</entry>
@@ -459,7 +464,7 @@ CREATE POLICY <replaceable class="parameter">name</replaceable> ON <replaceable
459464
<entry>&mdash;</entry>
460465
</row>
461466
<row>
462-
<entry><command>UPDATE</command></entry>
467+
<entry><command>UPDATE</command> / <command>MERGE ... THEN UPDATE</command></entry>
463468
<entry>
464469
Existing &amp; new rows <footnoteref linkend="rls-select-priv"/>
465470
</entry>
@@ -613,6 +618,14 @@ AND
613618
(see <link linkend="sql-createview"><command>CREATE VIEW</command></link>).
614619
</para>
615620

621+
<para>
622+
No separate policy exists for <command>MERGE</command>. Instead, the policies
623+
defined for <command>SELECT</command>, <command>INSERT</command>,
624+
<command>UPDATE</command>, and <command>DELETE</command> are applied
625+
while executing <command>MERGE</command>, depending on the actions that are
626+
performed.
627+
</para>
628+
616629
<para>
617630
Additional discussion and practical examples can be found
618631
in <xref linkend="ddl-rowsecurity"/>.

‎doc/src/sgml/ref/insert.sgml

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -589,6 +589,13 @@ INSERT <replaceable>oid</replaceable> <replaceable class="parameter">count</repl
589589
is a partition, an error will occur if one of the input rows violates
590590
the partition constraint.
591591
</para>
592+
593+
<para>
594+
You may also wish to consider using <command>MERGE</command>, since that
595+
allows mixing <command>INSERT</command>, <command>UPDATE</command>, and
596+
<command>DELETE</command> within a single statement.
597+
See <xref linkend="sql-merge"/>.
598+
</para>
592599
</refsect1>
593600

594601
<refsect1>
@@ -759,7 +766,9 @@ INSERT INTO distributors (did, dname) VALUES (10, 'Conrad International')
759766
Also, the case in
760767
which a column name list is omitted, but not all the columns are
761768
filled from the <literal>VALUES</literal> clause or <replaceable>query</replaceable>,
762-
is disallowed by the standard.
769+
is disallowed by the standard. If you prefer a more SQL standard
770+
conforming statement than <literal>ON CONFLICT</literal>, see
771+
<xref linkend="sql-merge"/>.
763772
</para>
764773

765774
<para>

0 commit comments

Comments
 (0)