Skip to content

Chapter 21.1 · Resource Management · 28 min read

Resource Groups in Apache Cloudberry, Explained

How resource groups use cgroups to enforce real CPU, memory and IO limits, the capability set that replaced the old names, and what the views report.

By Tushar Pednekar · · Verified against the Apache Cloudberry source tree, read September 2026

Resource groups are the strategic direction and the harder thing to deploy. This chapter is honest about both.

A single-node PostgreSQL server that meets a greedy query loses some throughput. An MPP cluster that meets one loses a host: the statement fans out into a gang on every segment, each process reads at full speed, and the neighbours on the same machine simply stop making progress. Resource management is the machinery that says how much of the cluster one statement may take, and Cloudberry ships two independent implementations of it. Resource groups are the modern one, and the subject of this section; resource queues are the legacy one, and the subject of §21.2.

The distinction worth carrying through the whole section is that a resource group enforces on two entirely different planes at once. Concurrency and memory are the database’s business — counters in shared memory, a wait queue, a number handed to the executor. CPU cores and disk bandwidth are the kernel’s business — Linux control groups, written as plain text into /sys/fs/cgroup. A group is a single catalog row that configures both planes, and most of what makes the implementation interesting is the seam between them.

21.1.1 Two enforcement planes, and the state of this cluster

Start from the shape of the problem. One statement on this cluster is not one process; it is a QD plus one QE per segment per slice (§1.3, §1.4, §13.3). “Limit the query” therefore has to mean “limit a set of processes, on every host, simultaneously” — and the two planes divide that job differently. The slot is taken once, by the QD, at transaction start (xact.c:2817). The cgroup attachment happens N+1 times: the QD adds its own pid (resgroup.c:1604), and each QE adds its own when it receives the dispatched group snapshot (resgroup.c:1777).

database plane  ·  one statement takes  exactly one slot  from ResGroupData.nRunning, on the coordinator only  ·  resgroup.c:1138
host A  ·  QD
postgres pid 4171
writes pid into
gpdb.service/16385/cgroup.procs
host B  ·  QE seg0
postgres pid 5220
writes pid into
gpdb.service/16385/cgroup.procs
host C  ·  QE seg1
postgres pid 5221
writes pid into
gpdb.service/16385/cgroup.procs
host D  ·  QE seg2
postgres pid 5222
writes pid into
gpdb.service/16385/cgroup.procs
kernel plane  ·  every capability except concurrency and memory_quota is enforced by a cgroup that exists separately on each host, which is why cpu_max_percent means “percent of this machine” and never “percent of the cluster”. On the demo cluster used throughout this book all four processes happen to share one machine; the figure shows the general case.
One slot, N+1 cgroup memberships. The database plane counts once; the kernel plane must be told once per process, on each host.

Which manager is in force is decided by a single string GUC. It is not an enum — hence the empty enumvals below — and it is PGC_POSTMASTER, defined at guc_gp.c:4939 with a check/assign/show hook trio (guc.h) whose assign side is gpvars_assign_gp_resource_manager_policy (cdbvars.c:536). That function accepts exactly three spellings, and choosing either group spelling also quietly sets gp_enable_resqueue_priority = false (cdbvars.c:545, :550): the two managers are mutually exclusive by construction, not by policy.

The manager selector on this cluster — a string GUC, postmaster-only.

SELECT name, setting, context, vartype, enumvals, extra_desc
  FROM pg_settings WHERE name = 'gp_resource_manager';

SET gp_resource_manager = 'group-v2';
        name         | setting |  context   | vartype | enumvals |                      extra_desc
---------------------+---------+------------+---------+----------+-------------------------------------------------------
 gp_resource_manager | queue | postmaster | string  |          | Only support "queue", "group" and "group-v2" for now.
(1 row)

ERROR:  parameter "gp_resource_manager" cannot be changed without restarting the server

Read this before the rest of the section. gp_resource_manager is queue here, and it cannot be changed without restarting the cluster, which is out of scope for this book’s demos. So resource groups on this machine are fully inspectable but not enforcing: the catalogs are real, the DDL runs, the views exist, the source is the source — but no slot is ever taken and no cgroup is ever written. Everything below is honest about which side of that line it comes from. Enabling groups would require exactly three things: gp_resource_manager = ‘group’ or ‘group-v2’ in postgresql.conf on every host, a restart, and a pre-created cgroup hierarchy that the gpadmin user can write — for v2 a directory named by gp_resource_group_cgroup_parent (here gpdb.service) directly under the cgroup2 mount. The second condition is unsatisfiable on this box, which is the subject of §21.1.3.

Why ‘group-v2’ could not work here even after a restart: a CentOS 7 kernel with no unified hierarchy.

$ grep cgroup /proc/self/mounts
$ grep -c cgroup2 /proc/self/mounts
$ ls -d /sys/fs/cgroup/*/gpdb*
$ uname -r
tmpfs  /sys/fs/cgroup          tmpfs  rw,nosuid,nodev,noexec,relatime,mode=755 0 0
cgroup /sys/fs/cgroup/systemd  cgroup rw,...,name=systemd 0 0
cgroup /sys/fs/cgroup/blkio    cgroup rw,...,blkio 0 0
cgroup /sys/fs/cgroup/cpuacct,cpu cgroup rw,...,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/cpuset   cgroup rw,...,cpuset 0 0
cgroup /sys/fs/cgroup/memory   cgroup rw,...,memory 0 0
... (12 cgroup mounts, all mnt_type "cgroup")

0                            <-- no cgroup2 mount anywhere

ls: cannot access '/sys/fs/cgroup/*/gpdb*': No such file or directory

3.10.0-1160.119.1.el7.x86_64

21.1.2 The group model: one row of capabilities

A resource group is a name plus seven capabilities. Three built-in groups exist in every cluster and exist here, under queue mode, untouched: default_group for ordinary roles, admin_group for superusers and the predefined pg_* roles, and system_group with concurrency = 0, which is the group auxiliary backends are accounted to and which can admit nothing. The catalog is a pair of tables — pg_resgroup holds the name, pg_resgroupcapability holds one row per capability keyed by a small integer — and gp_toolkit.gp_resgroup_config pivots the pair back into something readable.

The three built-in groups, first pivoted by gp_resgroup_config and then in the raw two-table form.

SELECT * FROM gp_toolkit.gp_resgroup_config;

SELECT r.rsgname, c.reslimittype, c.value
  FROM pg_resgroup r JOIN pg_resgroupcapability c ON c.resgroupid = r.oid
 WHERE r.rsgname = 'default_group' ORDER BY c.reslimittype;
 groupid |   groupname   | concurrency | cpu_max_percent | cpu_weight | cpuset | memory_quota | min_cost | io_limit
---------+---------------+-------------+-----------------+------------+--------+--------------+----------+----------
    6437 | default_group | 20          | 20              | 100        | -1     | -1           | 500    | -1
    6438 | admin_group   | 10          | 10              | 100        | -1     | -1           | 500      | -1
    6448 | system_group  | 0           | 10              | 100        | -1     | -1           | 500      | -1
(3 rows)

    rsgname    | reslimittype | value
---------------+--------------+-------
 default_group |            1 | 20        <-- concurrency
 default_group |            2 | 20        <-- cpu_max_percent
 default_group |            3 | 100       <-- cpu_weight
 default_group |            4 | -1        <-- cpuset
 default_group |            5 | -1        <-- memory_quota
 default_group |            6 | 500       <-- min_cost
 default_group |            7 | -1        <-- io_limit
(7 rows)

Those small integers are ResGroupLimitType (pg_resgroup.h:51), and they are the reason the catalog is stable across releases while the names migrate. This capability set is the modern Cloudberry one; older Greenplum material describes a different vocabulary (memory_limit, memory_shared_quota, memory_spill_ratio) that this tree no longer uses at all. In C the same seven values live in ResGroupCaps (resgroup.h:71), which is copied by value into every slot and dispatched to every QE.

CapabilityreslimittypeWhat it governs (per segment host)
concurrency1How many statements the group may run at once. A transaction takes one slot on the coordinator before it does anything else. 0 admits nothing — that is system_group. Range is [0, max_connections] (resgroupcmds.c:928).
cpu_max_percent2A hard CPU ceiling, as a percentage of the host, written to cpu.cfs_quota_us under cgroup v1 (cgroup-ops-linux-v1.c:979) or cpu.max under v2 (cgroup-ops-linux-v2.c:680). Range [1, 100], or -1 to disable the ceiling.
cpu_weight3The proportional share used when CPU is contended and no ceiling applies, written to cpu.shares (v1, :999) or cpu.weight (v2, :702) after scaling by 1024/100. Range [1, 500], default 100.
cpuset4An explicit core list such as '0-1', written to the cpuset controller. Setting it forces cpu_max_percent to -1 (resgroupcmds.c:1015) — a group is pinned or throttled, never both.
memory_quota5The group’s memory budget in MB. Divided by concurrency to give each statement its query_mem (resgroup.c:3630). -1, the default, means “no group budget — use statement_mem”.
min_cost6A planner-cost floor. A plan cheaper than this is released from the group entirely instead of consuming a slot. Default 500.
io_limit7Per-tablespace read/write bandwidth and IOPS caps, written to io.max. cgroup v2 only. -1 means unlimited.

Two host-wide knobs sit above the per-group ones and are worth naming here because they are what make cpu_max_percent mean something absolute. gp_resource_group_cpu_limit (0.9 here) is the fraction of the machine the whole database may consume — it sizes the quota of the parent cgroup (cgroup-ops-linux-v1.c:422, cgroup-ops-linux-v2.c:214) — and gp_resource_group_cpu_priority (10 here) is the multiplier applied to that parent’s weight (v1:446, v2:226), i.e. how Cloudberry as a whole competes with everything else on the box. A group asking for cpu_max_percent = 20 is therefore asking for 20 % of the database’s 90 %, not 20 % of the machine.

memory_quota is the group answer to a question §18.1 deliberately left open. That chapter showed the EXPLAIN ANALYZE footer — Memory used: 128000kB / Memory wanted: 14040kB — and said Chapter 21 owns the granting side. Under groups the granting side is ResourceGroupGetQueryMemoryLimit (resgroup.c:3600): if memory_quota is -1 it simply returns statement_mem, and otherwise it returns memory_quota * 1024 * 1024 / concurrency, floored at statement_mem if the user asked for more. On this cluster the number comes from the queue path instead, which §21.2 traces end to end; the fork between the two is a single if and appears at the close of this section.

min_cost deserves a moment, because it is the capability most likely to surprise. Before a statement is allowed to occupy a slot for its whole transaction, the plan’s total_cost is compared against it (can_bypass_based_on_plan_cost, resgroup.c:3841); a cheaper plan gives its slot back and runs unmanaged. The default is 500. That is a very small number in planner units, as a look at real plans on a 200 000-row table makes clear.

What min_cost = 500 actually excludes. All costs are GPORCA’s; ratios on one machine, not benchmarks.

CREATE TABLE m211_t (id int, v text) DISTRIBUTED BY (id);
INSERT INTO m211_t SELECT g, 'x'||g FROM generate_series(1,200000) g;
ANALYZE m211_t;

EXPLAIN SELECT count(*) FROM m211_t;
EXPLAIN SELECT a.id FROM m211_t a JOIN m211_t b ON a.v = b.v ORDER BY a.id LIMIT 10;
                                     QUERY PLAN
------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=0.00..432.77 rows=1 width=8)
   ->  Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..432.77 rows=1 width=8)
         ->  Partial Aggregate  (cost=0.00..432.77 rows=1 width=8)
               ->  Seq Scan on m211_t  (cost=0.00..432.65 rows=66667 width=1)
 Optimizer: GPORCA
(5 rows)

                                     QUERY PLAN
------------------------------------------------------------------------------------
 Limit  (cost=0.00..925.74 rows=10 width=4)
   ->  Gather Motion 3:1  (slice1; segments: 3)  (cost=0.00..925.74 rows=10 width=4)
         ...
               ->  Sort  (cost=0.00..925.74 rows=66667 width=4)
                     ->  Hash Join  (cost=0.00..901.51 rows=66667 width=4)
                           ->  Redistribute Motion 3:3  (slice2; segments: 3)  ...
                           ->  Hash  (cost=434.98..434.98 rows=66667 width=7)
 Optimizer: GPORCA
(16 rows)

A full three-segment scan and aggregate over 200 000 rows costs 432.77 — under the default min_cost of 500, so it would take no slot at all. The two-way hash join with a sort costs 925.74, so it would. The lesson is not that the default is wrong; it is that min_cost is a planner-cost threshold, and planner cost is only loosely correlated with the thing you actually want to ration. Tune it against real plans from your own workload, the way the two EXPLAINs above were read, rather than against intuition about “small” queries.

21.1.3 The cgroup abstraction layer

Everything on the kernel plane goes through one indirection, and it is the most interesting piece of code in the subsystem. CGroupOpsRoutine (cgroup.h:250) is a struct of 26 function pointers covering the entire lifetime of a group’s kernel-side state: name it, probe it, check it, initialise it, adjust GUCs for it, create and destroy its directories, attach and detach processes, lock and unlock it, set and read CPU limits, weights, usage and core sets, and — added for v2 — parse, apply, free, read, dump and clear IO limits.

src/include/utils/cgroup.h:250

typedef struct CGroupOpsRoutine
{
	getcgroupname_function 	getcgroupname;
	probecgroup_function	probecgroup;
	checkcgroup_function 	checkcgroup;
	initcgroup_function 	initcgroup;
	adjustgucs_function 	adjustgucs;
	createcgroup_function 	createcgroup;
	/* ... attach/detach, lock/unlock, cpu, cpuset, memory ... */
	parseio_function		parseio;
	setio_function			setio;
	getiostat_function		getiostat;
} CGroupOpsRoutine;      /* 26 function pointers in all */
gp_resource_manager (PGC_POSTMASTER, guc_gp.c:4939) 'queue' this cluster 'group' Linux cgroup v1 'group-v2' Linux cgroup v2 non-Linux build the #else branch initCgroup never called postmaster.c:1671 gate is false get_group_routine_v1 cgroup-ops-linux-v1.c:1203 → :1170 get_group_routine_v2 cgroup-ops-linux-v2.c:954 → :921 get_cgroup_routine_dummy cgroup-ops-dummy.c:294 → :261 cgroupOpsRoutine resgroup.c:188 — one global pointer, NULL when the manager is 'queue' cgroupOpsRoutine->attachcgroup(self->groupId, MyProcPid, ...) resgroup.c:1604 on the QD · resgroup.c:1777 on every QE, on every host v1 writes the pid to /sys/fs/cgroup/cpu/gpdb/ <groupOid>/tasks one path per controller v2 writes the pid to /sys/fs/cgroup/gpdb.service/ <groupOid>/cgroup.procs one unified path dummy emits WARNING: resource group is not supported on this system The generation is not detected at run time. It is declared by the administrator through gp_resource_manager, and the server merely verifies the declaration — probecgroup() plus the mount-type test in cgroup.c:602 / :620. The v1 base directory is hardcoded as /gpdb under each controller (cgroup.c:150); only v2 honours gp_resource_group_cgroup_parent.
Dispatch through CGroupOpsRoutine. The administrator's GUC value picks one of three vtables at postmaster start; every kernel-plane operation thereafter is a call through the single global pointer.

Pause on the shape rather than the content, because the book has now shown this trick three times. §18.1.6 introduced TupleTableSlotOps, the vtable that lets one executor call slot_getsomeattrs without knowing whether the tuple underneath is heap, minimal, virtual or AO-columnar. The table access method is the same idea one level down, letting one ExecScan walk a heap or an append-optimised relation. CGroupOpsRoutine is the same idea again, one level out, letting one resource manager write CPU limits without knowing which generation of Linux control groups is mounted. Three unrelated subsystems, one pattern: put the variation behind a struct of function pointers, pick the struct once during startup, and let every call site stay ignorant. Naming it is worth more than any of the three instances individually.

And now the part that is genuinely surprising. One would expect a layer like this to probe — look at /proc/self/mounts, notice a cgroup2 filesystem, choose v2. It does not. initCgroup (resgroup.c:400) keys entirely off the GUC, and the else branch means the two Linux vtables are selected by an administrator’s configuration string, not by anything the kernel says:

src/backend/utils/resgroup/resgroup.c:402

#ifdef __linux__
	if (Gp_resource_manager_policy == RESOURCE_MANAGER_POLICY_GROUP)
	{
		cgroupOpsRoutine = get_group_routine_v1();
		cgroupSystemInfo = get_cgroup_sysinfo_v1();
	}
	else
	{
		cgroupOpsRoutine = get_group_routine_v2();
		cgroupSystemInfo = get_cgroup_sysinfo_v2();
	}
#else

Verification comes second, and only as a yes/no on the declaration already made. probecgroup() runs immediately after the assignment (resgroup.c:418) and a false answer is a hard elog(ERROR, "The control group is not well configured...") at :420 — in the postmaster, so the server does not start. Both implementations of the probe do the same two things: find the mount directory, then check the permissions the group needs (cgroup-ops-linux-v1.c:537, cgroup-ops-linux-v2.c:285). It is getCgroupMountDir (cgroup.c:584) that finally consults the kernel — and even there the GUC is what decides which mnt_type string counts as a match, "cgroup" at :602 versus "cgroup2" at :620, with v1 additionally stripping the trailing controller component so that /sys/fs/cgroup/cpu becomes /sys/fs/cgroup. Declare v2 on a v1 host and the loop finds nothing, the probe fails, and the postmaster refuses to come up. There is no fallback.

On this cluster the whole chain is short-circuited one step earlier. initCgroup is called from exactly one place, postmaster.c:1671, behind if (IsResGroupEnabled()) — and IsResGroupEnabled() (resource_manager.h:28) is false under queue. So cgroupOpsRoutine is never assigned and stays at its initial NULL (resgroup.c:188). That NULL is directly observable, because the two capabilities that must talk to the kernel at DDL time test for it and refuse:

The NULL global, surfaced through DDL. Both errors are the same fact wearing two hats.

CREATE RESOURCE GROUP m211_io
  WITH (cpu_max_percent=10, io_limit='pg_default:rbps=100,wbps=200,riops=1000,wiops=max');

ALTER RESOURCE GROUP m211_rg SET cpuset '0-1';
ERROR:  resource group must be enabled to use io limit feature
        -- resgroupcmds.c:1022:  if (cgroupOpsRoutine == NULL)

ERROR:  resource group must be enabled to use cpuset feature
        -- resgroup.c:3185, reached from EnsureCpusetIsAvailable

That is a complete causal chain, verified end to end and worth rereading as one sentence: gp_resource_manager = ‘queue’IsResGroupEnabled() false (resource_manager.h:28) → initCgroup() skipped (postmaster.c:1671) → cgroupOpsRoutine stays NULL (resgroup.c:188) → io_limit and cpuset DDL rejected (resgroupcmds.c:1022, resgroup.c:3185). Note the corollary: the io_limit string is never even handed to its parser here, because the NULL check sits before cgroupOpsRoutine->parseio(). The grammar of §21.1.6 is therefore read from source, not exercised.

21.1.4 DDL, and what the catalog means when nothing is enforcing

The DDL lives in resgroupcmds.c: CreateResourceGroup at :91, DropResourceGroup at :263, AlterResourceGroup at :364, with GetResGroupIdForRole at :714 doing the lookup that maps a login to a group through pg_authid.rolresgroup. All three are dispatched to the segments as utility statements (CdbDispatchUtilityStatement) so that every host’s catalog agrees, and all three then branch: if groups are activated they also touch shared memory and the cgroup filesystem, and if not they emit a warning and stop. That warning is the single most instructive output in this section.

Group DDL under queue mode: the catalog write succeeds, the enforcement does not, and the server says so.

CREATE RESOURCE GROUP m211_rg
  WITH (concurrency=4, cpu_max_percent=20, memory_quota=512, min_cost=2000);

SELECT groupid, groupname, concurrency, cpu_max_percent, cpu_weight,
       cpuset, memory_quota, min_cost, io_limit
  FROM gp_toolkit.gp_resgroup_config WHERE groupname = 'm211_rg';
WARNING:  resource group is disabled
HINT:  To enable set gp_resource_manager=group
CREATE RESOURCE GROUP

 groupid | groupname | concurrency | cpu_max_percent | cpu_weight | cpuset | memory_quota | min_cost | io_limit
---------+-----------+-------------+-----------------+------------+--------+--------------+----------+----------
   50932 | m211_rg   | 4           | 20              | 100        | -1     | 512          | 2000     | -1
(1 row)

Read that carefully, because it describes a design decision and not a bug. The catalog is authoritative and portable; enforcement is a runtime mode. A group definition written today on a queue cluster survives pg_dump, survives an upgrade, and starts enforcing the moment somebody flips the GUC and restarts — no re-authoring required. The warning comes from resgroupcmds.c:255, in the else arm of if (IsResGroupActivated()). Validation, by contrast, is not deferred: checkResgroupCapLimit (resgroupcmds.c:924) and parseStmtOptions (:985) run regardless of mode, so a nonsensical group can never reach the catalog in the first place.

Validation runs in both modes; role assignment shows both managers writing to the same CREATE ROLE.

CREATE RESOURCE GROUP m211_bad WITH (concurrency=4, cpu_max_percent=0);
CREATE RESOURCE GROUP m211_bad WITH (cpu_max_percent=20, cpu_weight=99999);
CREATE RESOURCE GROUP m211_bad WITH (cpu_max_percent=20, foo=1);

ALTER RESOURCE GROUP m211_rg SET concurrency 8;
CREATE ROLE m211_u LOGIN RESOURCE GROUP m211_rg;
SELECT rrrolname, rrrsgname FROM gp_toolkit.gp_resgroup_role
 WHERE rrrolname LIKE 'm21%';
ERROR:  cpu_max_percent range is [1, 100] or equals to -1
ERROR:  cpu_weight range is [1, 500]
ERROR:  option "foo" not recognized

ALTER RESOURCE GROUP

NOTICE:  resource queue required -- using default resource queue "pg_default"
WARNING:  resource group is disabled
HINT:  To enable set gp_resource_manager=group
CREATE ROLE

 rrrolname |   rrrsgname
-----------+---------------
 m212_u    | default_group
 m211_u    | m211_rg
(2 rows)

The double diagnostic on that CREATE ROLE is the clearest possible picture of the two managers coexisting. A NOTICE says the new role was given the default resource queue, because queues are the active manager; a WARNING says the resource group it explicitly asked for was recorded but is disabled. Both statements are true, both catalogs were written, and every role in the cluster carries a rolresgroup whether or not it will ever be honoured — gp_resgroup_role above is a plain join over pg_authid. (m212_u belongs to §21.2’s demo; it appears here only because the view covers every role.) Dropping is likewise catalog-aware rather than mode-aware:

A group in use cannot be dropped, in either mode. Then the m211_* objects are cleaned up.

DROP RESOURCE GROUP m211_rg;
ALTER ROLE m211_u RESOURCE GROUP default_group;
DROP RESOURCE GROUP m211_rg;
DROP ROLE m211_u;
DROP TABLE m211_t;
ERROR:  resource group is used by at least one role

WARNING:  resource group is disabled
HINT:  To enable set gp_resource_manager=group
ALTER ROLE

DROP RESOURCE GROUP
DROP ROLE
DROP TABLE

21.1.5 Admission control at run time

Everything in this subsection is read from source, not observed — no slot is ever taken on this cluster. The entry point is AssignResGroupOnMaster (resgroup.c:1540), called from StartTransaction (xact.c:2817) after the transaction is under way, because deciding which group a session belongs to means reading pg_authid. Its mirror is UnassignResGroup (:1619), which runs at commit or abort. The slot itself is the smallest interesting object here: ResGroupSlotData (resgroup.c:136) is little more than a group pointer, a process count, a free-list link, and a by-value copy of the seven capabilities — a snapshot, so that an ALTER RESOURCE GROUP mid-flight cannot change the rules a running statement is playing by.

src/backend/utils/resgroup/resgroup.c:1135

	caps = &group->caps;

	/* First check if the concurrency limit is reached */
	if (group->nRunning >= caps->concurrency)
		return NULL;

	/* Now actually get a free slot */
	slot = slotpoolAllocSlot();
	initSlot(slot, group);
	group->nRunning++;

	return slot;
Session BResGroupData in shared memorySession ASession BResGroupData in shared memorySession Aattachcgroup adds the QD pid to the group cgroup [resgroup.c:1604]waitOnGroup sleeps — wait event ResourceGroup [resgroup.c:1785]groupAcquireSlot at resgroup.c:1266nRunning below concurrency, slot grantedgroupAcquireSlotno slot, groupGetSlot returns NULLgroupWaitQueuePush on waitProcs [resgroup.c:1303]commit calls UnassignResGroup then groupPutSlotwakeupSlots loop at resgroup.c:1352 to 1371groupWaitQueuePop, sets waitProc resSlot, procWakeupwakes already holding a granted slot
Slot acquisition, queueing and hand-off. Everything happens under ResGroupLock in exclusive mode; the waiter sleeps outside it.

The wait queue is strictly FIFO and is a dclist of PGPROC links hanging directly off the group (ResGroupData.waitProcs, resgroup.c:159). Its five operations — groupWaitQueuePush (:2448), groupWaitQueuePop (:2470), groupWaitQueueErase (:2496), groupWaitQueueIsEmpty (:2517) and the assertion-only groupWaitQueueValidate (:2387) — are all called with ResGroupLock held exclusively. Note the hand-off discipline in wakeupSlots: the waker takes the slot on the sleeper’s behalf and stores it in waitProc->resSlot before waking it, so a woken process can never lose a race to a newcomer. gp_resource_group_queuing_timeout (resgroup.c:1840) bounds the sleep; 0, the value here, waits forever. A separate path, ResGroupMoveQuery (:3508) behind the SQL function pg_resgroup_move_query(sessionId, groupName), can move an already-running session into a different group, bounded by gp_resource_group_move_timeout (:3447).

The segment side closes the N+1 loop from §21.1.1 and reuses the dispatch machinery of §13.3 rather than inventing anything. Having acquired the slot, the QD serialises the group id and the capability snapshot into the 'M' dispatch message (SerializeResGroupInfo, cdbdisp_query.c:919); each QE unpacks it in its main loop and calls SwitchResGroupOnSegment (postgres.c:6072, implemented at resgroup.c:1680), which allocates a segment-local slot and attaches its own pid to the group’s cgroup at :1777. The slot count is a coordinator-side truth; the cgroup membership is a per-host truth; the same capability struct drives both.

21.1.6 io_limit gets its own grammar

Six of the seven capabilities are integers. The seventh, io_limit, is a small language — a semicolon-separated list of per-tablespace clauses, each naming a tablespace (by name, by OID, or * for all) and then a comma-separated set of the four keys rbps, wbps, riops, wiops, whose values are either integers or the literal max. Rather than hand-rolling a parser, Cloudberry gives the clause a dedicated bison grammar and flex scanner, prefixed so they can coexist with the main SQL parser in one binary: io_limit_gram.y (213 lines, generating 1835) and io_limit_scanner.l (124 lines, generating 2483), with the runtime in cgroup_io_limit.c.

src/backend/utils/resgroup/io_limit_gram.y:1

%define api.pure true
%define api.prefix {io_limit_yy}
%error-verbose
...
%union {
	char *str;
	uint64 integer;
	IOconfig *ioconfig;
	TblSpcIOLimit *tblspciolimit;
	List *list;
}
...
%parse-param { IOLimitParserContext *context }

The scanner is worth a glance too, because it is a stateful one: seeing a : it does BEGIN ts_param (io_limit_scanner.l:25-29), and only inside that state does the pattern [wr](b|io)ps produce an IO_KEY token (:31) or the word max produce VALUE_MAX (:37). Outside it, the same characters are just an identifier — which is how a tablespace legitimately named rbps remains parseable. Errors carry a caret: io_limit_yyerror (:181) reports errhint(" %s\n %*c", line, io_limit_yycolumn, '^'), pointing at the offending column of the original string.

io_limit  =>  pg_default:rbps=100,wbps=200,riops=1000,wiops=max
ID or NUMBER or STAR  ·  IO_KEY, only inside scanner state ts_param  ·  NUMBER or VALUE_MAX
▼  io_limit_parse  →  List *
TblSpcIOLimit
cgroup_io_limit.h:57
Oid  tablespace_oid  = 1663
List *bdi_list  → { makedev 8,16 }
IOconfig *ioconfig  
A tablespace is a directory; the directory may span several disks, so fill_bdi_list resolves it to the set of block-device ids the kernel wants.
IOconfig
cgroup_io_limit.h:31  ·  four uint64, read by offset
uint64 rbps   = 100
uint64 wbps   = 200
uint64 riops  = 1000
uint64 wiops  = PG_UINT64_MAX (max)
All four are uint64 on purpose: the grammar stores a key by array offset into the struct (IOconfigFields, cgroup_io_limit.c:44), which is also how it detects a duplicated key.
setio_v2 then writes one line per device  →  8:16 rbps=104857600 wbps=209715200 riops=1000 wiops=max  into  /sys/fs/cgroup/gpdb.service/<groupOid>/io.max  (cgroup-ops-linux-v2.c:890-891)
What the grammar produces: a List of TblSpcIOLimit, each pairing a tablespace with an IOconfig of four uint64 fields, plus the block devices that tablespace actually sits on.

Two boundaries this figure implies. First, io_limit is cgroup v2 only — every IO entry in the v1 vtable is a stub that raises WARNING: resource group io limit only can be used in cgroup v2. (cgroup-ops-linux-v1.c:1154-1169), because v1’s blkio controller has no equivalent of io.max. Second, the bandwidth keys are validated in range [2, ULLONG_MAX/1024/1024] and the IOPS keys in [2, UINT_MAX] (io_limit_value_validate, cgroup_io_limit.c:311) — and note the units: rbps/wbps are given in MB/s and multiplied up on the way to io.max, which is why the figure’s rbps=100 becomes 104857600. The matching read-back path is getiostat, feeding gp_toolkit.gp_resgroup_iostats_per_host.

21.1.7 Bypass valves, and the observability surface

A concurrency limit of 20 would be crippling if SHOW work_mem or a psql tab-completion query consumed a slot for the duration of its transaction. Groups therefore have escape valves, applied at two moments. Before planning, shouldBypassQuery (resgroup.c:2621) re-parses the statement and lets SET/RESET/SHOW and catalog-only SELECTs through. After planning, check_and_unassign_from_resgroup (:3651) gets a second chance with the finished plan in hand, and gives a held slot back if any of three tests passes: cost below min_cost, or a pure catalog plan with gp_resource_group_bypass_catalog_query on, or a single-segment direct-dispatch plan with gp_resource_group_bypass_direct_dispatch on (:3677-3678). Explicit BEGIN … END blocks are excluded, because a slot released mid-block would never be reacquired.

Both of those two GUCs default to on, and it is worth being concrete about why that matters rather than treating it as a detail. EXPLAIN SELECT * FROM m211_t WHERE id = 42 in §21.1.2 produced Gather Motion 1:1 (slice1; segments: 1) — two slices, the second direct-dispatched to exactly one segment. That is precisely the shape can_bypass_direct_dispatch_plan (resgroup.c:3860) matches. So on a group-managed cluster, a high-rate OLTP workload of single-key lookups is by default not admission-controlled at all, which is usually what you want (those queries touch one segment and finish in a millisecond) and is occasionally a nasty surprise (ten thousand of them per second are not free). Turning it off is a session-level SET.

GUCContext · value hereWhat it does
gp_resource_group_cgroup_parentpostmaster · gpdb.serviceName of the top-level cgroup v2 directory under the cgroup2 mount. v1 ignores it entirely and hardcodes /gpdb beneath each controller (cgroup.c:150).
gp_resource_group_cpu_limitpostmaster · 0.9Fraction of the host’s CPU the whole database may use; sizes the parent cgroup’s quota.
gp_resource_group_cpu_prioritypostmaster · 10Multiplier on the parent cgroup’s weight — how Cloudberry competes with other cgroups on the host.
gp_resource_group_bypassuser · offBlanket session escape hatch: every statement runs unmanaged. Its check hook (guc_gp.c:5499) rejects the SET while a slot is held — so on this cluster, where nothing is ever assigned, it always succeeds.
gp_resource_group_bypass_catalog_queryuser · onRelease the slot for plans whose range table is entirely catalog relations.
gp_resource_group_bypass_direct_dispatchuser · onRelease the slot for single-segment direct-dispatch plans.
gp_resource_group_queuing_timeoutuser · 0Milliseconds to wait for a slot before erroring out. 0 waits forever.
gp_resource_group_move_timeoutuser · 30000Milliseconds pg_resgroup_move_query may wait for the target session to accept the move.

The observability surface is a family of gp_toolkit views over shared memory rather than over catalogs, and that is exactly why most of them are empty here: gp_resgroup_status is a join of pg_resgroup against the set-returning function pg_resgroup_get_status (resgroup_helper.c:206), which reads the shared-memory group array that was never populated. (0 rows) is the honest answer, not a missing feature. Two views in the family are not empty, though, and they are the two that read from somewhere else: gp_resgroup_role is a plain catalog join, and resgroup_session_level_memory_consumption reads the per-segment SessionState vmem accounting that runs regardless of manager (§18.3).

The whole view family, as it really answers under queue mode. Empty where empty is the truth.

SELECT * FROM gp_toolkit.gp_resgroup_status;
SELECT * FROM gp_toolkit.gp_resgroup_status_per_host;
SELECT * FROM gp_toolkit.gp_resgroup_iostats_per_host;

SELECT sess_id, rsgid, rsgname, usename, segid, vmem_mb
  FROM gp_toolkit.resgroup_session_level_memory_consumption
 WHERE usename = 'gpadmin' AND datname = 'postgres' ORDER BY sess_id, segid;
 groupid | groupname | num_running | num_queueing | num_queued | num_executed | total_queue_duration
---------+-----------+-------------+--------------+------------+--------------+----------------------
(0 rows)

 groupid | groupname | hostname | cpu_usage | memory_usage
---------+-----------+----------+-----------+--------------
(0 rows)

 rsgname | hostname | tablespace | rbps | wbps | riops | wiops
---------+----------+------------+------+------+-------+-------
(0 rows)

 sess_id | rsgid | rsgname  | usename | segid | vmem_mb
---------+-------+----------+---------+-------+---------
     236 |     0 | unknown | gpadmin |    -1 |      19
     236 |     0 | unknown  | gpadmin |     0 |      14
     236 |     0 | unknown  | gpadmin |     1 |      14
     236 |     0 | unknown  | gpadmin |     2 |      14
(4 rows)

That last result is a small gift. The memory accounting is per segment and it is live — one row for the QD at segid = -1 and one for each QE — so the machinery §18.3 described is visibly running even with no group to attribute it to, which is what rsgid = 0 / rsgname = unknown means. Under groups, this view is how you would watch a group’s statements approach their memory_quota share and, if runaway_vmem_mb were configured, see the runaway detector arm.

Which brings this section back to where §18.1 left off. That chapter’s Memory used: / Memory wanted: footer is produced by whatever the resource manager granted, and the choice between the two managers is one if in memquota.c:906: IsResQueueEnabled() routes to ResourceQueueGetQueryMemoryLimit, IsResGroupEnabled() routes to ResourceGroupGetQueryMemoryLimit, and a cluster with neither simply returns statement_mem. Under groups the answer would be memory_quota / concurrency — a group budget divided among its own slots, so that raising a group’s concurrency automatically shrinks each statement’s share. On this cluster the queue branch is taken, statement_mem is 128000 kB, and the number in that footer comes from gp_resqueue_memory_policy = eager_free; §21.2 traces that path end to end and closes the loop for real. It also explains why the manager that ships as the default here is the one called legacy.

More in Resource Management