Greenplum 6 Changes

© Copyright 2019 Pivotal Software, Inc. All rights Reserved. Version 1.0
Greenplum 6
Changes
Cyrille LINTZ
April 2019

Agenda
● Querying
● Data Types
● Storage / DDL
● Extensions
● Administration
● Backup / Restore
● Security
● Catalog & gp_toolkit
● Parameters
● Expansion
● Removed features
● Other items

Overview
● Merge with PostgreSQL 9.4 (so includes changes related to all
releases between PG 8.4 and PG 9.4)
● Greenplum 6 provides a single client and loader tool package that
you can download and install on a client system. Previous
Greenplum releases provided separate client and loader
packages.

Order by - Collate - 1/2
● Collation support to specify sort order and character classification
behavior for data at the column level (PostgreSQL 9.1).
● Ability to define a COLLATE clause to a column (if collatable data
type).
● If not specified, the column data type's default collation is used.
● GPORCA supports collation only when all columns in the query
use the same collation. If columns in the query use different
collations, then Greenplum uses the Postgres query planner.

Order by - Collate - 2/2
create table test_order
(f_default_collate text
,f_collate_c text collate "C"
,f_description text);
select * from test_order order by f_default_collate;
f_default_collate | f_collate_c | f_description
-------------------+-------------+------------------
1 | 1 | only the value 1
2 | 2 | space + value 2
Order by field with no collate
select * from test_order order by f_collate_c;
-------------------+-------------+------------------
Order by field with collate
select * from test_order order by f_default_collate collate "C";
-------------------+-------------+------------------
Order by field with no collate, but adding collate

Recursive Query - 1/3
create table public.places
(place text,is_in text)
distributed by (place);
COPY public.places from stdin;
U.S.A N
California U.S.A
Florida U.S.A
Massachusetts U.S.A
San Francisco California
Los Angeles California
San Diego California
Miami Florida
Orlando Florida
Tampa Florida
United Kingdom N
London United Kingdom
Leeds United Kingdom
.
demo=# WITH RECURSIVE q AS
(SELECT place, is_in
FROM places
WHERE is_in is null
UNION ALL
SELECT m.place, m.is_in
FROM places m
JOIN q ON q.place = m.is_in)
SELECT place, is_in FROM q;
place | is_in
----------------+----------------
U.S.A |
Massachusetts | U.S.A
Florida | U.S.A
California | U.S.A
Orlando | Florida
Tampa | Florida
Miami | Florida
Los Angeles | California
San Diego | California
San Francisco | California
United Kingdom |
Leeds | United Kingdom
London | United Kingdom
(13 rows)
Example 1

demo=# WITH RECURSIVE q (place, is_in, depth, path, cycle)
AS ( SELECT place, is_in,1 as depth ,ARRAY[place], false as cycle FROM places
where is_in is null
UNION ALL
SELECT m.place, m.is_in,q.depth + 1 as depth ,path || m.place, m.place = ANY(path) FROM places m
JOIN q ON q.place = m.is_in
where not cycle)
SELECT place, is_in, depth, array_to_string(path,' ==> '), cycle FROM q;
place | is_in | depth | array_to_string | cycle
----------------+----------------+-------+----------------------------------------+-------
United Kingdom | | 1 | United Kingdom | f
Leeds | United Kingdom | 2 | United Kingdom ==> Leeds | f
London | United Kingdom | 2 | United Kingdom ==> London | f
U.S.A | | 1 | U.S.A | f
Massachusetts | U.S.A | 2 | U.S.A ==> Massachusetts | f
Florida | U.S.A | 2 | U.S.A ==> Florida | f
California | U.S.A | 2 | U.S.A ==> California | f
Orlando | Florida | 3 | U.S.A ==> Florida ==> Orlando | f
Tampa | Florida | 3 | U.S.A ==> Florida ==> Tampa | f
Miami | Florida | 3 | U.S.A ==> Florida ==> Miami | f
Los Angeles | California | 3 | U.S.A ==> California ==> Los Angeles | f
San Diego | California | 3 | U.S.A ==> California ==> San Diego | f
San Francisco | California | 3 | U.S.A ==> California ==> San Francisco | f
(13 rows)
Example 2

Error Handling: custom Detail and Hint
CREATE FUNCTION sp_test_raise_specific_mess
(p_param text)
RETURNS text
AS $$
BEGIN
if p_param = 'test'
then
return 'OK';
else
raise exception 'the parameter % is not the
expected one (e.g. test)', p_param
using ERRCODE='22012';
end if;
EXCEPTION
WHEN OTHERS THEN
RAISE EXCEPTION '%,%', sqlerrm,sqlstate
using DETAIL='detail custo',
HINT='hint custo';
END;
$$
LANGUAGE plpgsql;
demo=# select sp_test_raise_specific_mess('test');
sp_test_raise_specific_mess
-----------------------------
OK
(1 row)
DDL Function Output when no error
demo=# select sp_test_raise_specific_mess('test1');
ERROR: the parameter test1 is not the expected one
(e.g. test),22012
DETAIL: detail custo
HINT: hint custo
Output when error

JSONB (PostgreSQL 9.4)
● The data types json and jsonb are almost identical
○ json data is stored as an exact copy of the JSON input text
○ jsonb stores data in a decomposed binary form
▪ slightly slower input (due to added conversion overhead),
▪ Possibly more disk space than plain json (due to a larger table
footprint)
▪ certain queries (especially aggregate ones) may be slower
(due to the lack of statistics)
▪ more efficiency
▪ significantly faster to process
▪ supports indexing (GIN, btree, and hash)
▪ simpler schema designs (replacing entity-attribute-value
tables)
Benefits Drawbacks

JSONB (PostgreSQL 9.4) - Filter result
SELECT count(*) FROM books_json
WHERE data->'published' = 'false';
ERROR: operator does not exist: json = unknown
LINE 2: WHERE data->'published' = 'false';
^
HINT: No operator matches the given name and
argument type(s). You might need to add explicit
type casts.
SELECT count(*) FROM books_jsonb
WHERE data->'published' = 'false';
count
-------
1
JSON JSONB
SELECT count(*) FROM books_json
WHERE data->>'published' = 'false';
count
-------
1

JSONB (PostgreSQL 9.4) - Check containment &
Existence
SELECT '["A", "B", "C"]'::json
@> '["A", "B"]'::json as check;
ERROR: operator does not exist: json @> json
LINE 2: @> '["A", "B"]'::json as check;
^
type casts.
SELECT '["A", "B", "C"]'::jsonb
@> '["A", "B"]'::jsonb as check;
check
-------
t
JSON JSONB
SELECT '["A", "B", "C"]'::json ? 'A' as check;
ERROR: operator does not exist: json ? unknown
LINE 1: SELECT '["A", "B", "C"]'::json ? 'A' as
check;
^
type casts.
SELECT '["A", "B", "C"]'::jsonb ? 'A' as check;
check
-------
t

JSONB (PostgreSQL 9.4) - Indexes
CREATE TABLE table_json
(id integer
, data json
) distributed by (id);
CREATE INDEX idx_btree
ON table_json ((data->'published'));
ERROR: data type json has no default operator class
for access method "btree"
HINT: You must specify an operator class or define
a default operator class for the data type.
CREATE INDEX idx_gin
ON table_json using gin((data->'published'));
ERROR: data type json has no default operator class
for access method "gin"
HINT: You must specify an operator class or define
a default operator class for the data type.
CREATE TABLE table_jsonb
(id integer
, data jsonb
) distributed by (id);
CREATE INDEX idx_btree
ON table_jsonb ((data->'published'));
CREATE INDEX
CREATE INDEX idx_gin
ON table_jsonb using gin((data->'published'));
CREATE INDEX
JSON JSONB

CREATE Table - appendoptimized
● appendonly replaced by appendoptimized (both syntaxes work)
CREATE TABLE table_appendonly (a int)
with (appendonly=true) distributed randomly;
CREATE TABLE
With appendonly
CREATE TABLE table_appendoptimized (a int)
with (appendoptimized=true) distributed randomly;
CREATE TABLE
With appendoptimized
select relname,relkind,relstorage,reloptions from pg_class
where relname like 'table_append%';
relname | relkind | relstorage | reloptions
-----------------------+---------+------------+-------------------
table_appendoptimized | r | a | {appendonly=true}
table_appendonly | r | a | {appendonly=true}
Content of pg_class

Replicated Tables - 1/3
● The DISTRIBUTED REPLICATED clause replicates the entire table to all Greenplum
Database segment instances.
● Useful when it is necessary to execute user-defined functions on segments when the functions
require access to all rows in the table
● Useful to improve query performance by preventing broadcast and redistribution motions.
● Replicated tables can have both PRIMARY KEY and UNIQUE column constraints.

create table table_replicated (a int , b text)
distributed replicated;
insert into table_replicated
select id, 'val ' || id
from generate_series (1,10000) id;
select pg_relation_size('table_replicated');
pg_relation_size
------------------
917504
create table table_non_replicated (a int , b text)
distributed randomly;
insert into table_non_replicated
select id, 'val ' || id
from generate_series (1,10000) id;
select pg_relation_size('table_non_replicated');
pg_relation_size
------------------
458752
With Non Replicated table With Replicated Tables
Size is multiplied by the
number of primaries
select gp_segment_id, count(*) from table_replicated
group by 1;
ERROR: column "gp_segment_id" does not exist
LINE 1: select gp_segment_id, count(*) from ...
^
select gp_segment_id, count(*) from
table_non_replicated group by 1;
gp_segment_id | count
---------------+-------
0 | 5011
1 | 4989 The field gp_segment_id doesn't
exist in replicated tables

explain select count(*) from table_fact f inner join table_replicated d on f.a = d.a;
QUERY PLAN
----------------------------------------------------------------------------------------------------
Aggregate (cost=0.00..874.73 rows=1 width=8)
-> Gather Motion 2:1 (slice1; segments: 2) (cost=0.00..874.73 rows=1 width=8)
-> Aggregate (cost=0.00..874.73 rows=1 width=8)
-> Hash Join (cost=0.00..874.73 rows=50000 width=1)
Hash Cond: (table_fact.a = table_replicated.a)
-> Seq Scan on table_fact (cost=0.00..432.15 rows=50000 width=4)
-> Hash (cost=431.23..431.23 rows=10000 width=4)
-> Seq Scan on table_replicated (cost=0.00..431.23 rows=10000 width=4)
Optimizer: PQO version 3.29.0
explain select count(*) from table_fact f inner join table_non_replicated d on f.a = d.a;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------
Aggregate (cost=0.00..874.31 rows=1 width=8)
-> Gather Motion 2:1 (slice3; segments: 2) (cost=0.00..874.31 rows=1 width=8)
-> Aggregate (cost=0.00..874.31 rows=1 width=8)
-> Hash Join (cost=0.00..874.31 rows=50000 width=1)
Hash Cond: (table_fact.a = table_non_replicated.a)
-> Redistribute Motion 2:2 (slice1; segments: 2) (cost=0.00..433.15 rows=50000 width=4)
Hash Key: table_fact.a
-> Seq Scan on table_fact (cost=0.00..432.15 rows=50000 width=4)
-> Hash (cost=431.22..431.22 rows=5000 width=4)
-> Redistribute Motion 2:2 (slice2; segments: 2) (cost=0.00..431.22 rows=5000 width=4)
Hash Key: table_non_replicated.a
-> Seq Scan on table_non_replicated (cost=0.00..431.12 rows=5000 width=4)
Optimizer: PQO version 3.29.0
WithNonReplicatedtable
1 slice vs 3 slices
No redistribution
WithReplicatedtable

Unlogged Tables
● Data written to unlogged tables is not written to the write-ahead (WAL) log, which makes them
considerably faster than ordinary tables.
● However, the contents of an unlogged table are not replicated to mirror segment instances.
Also an unlogged table is not crash-safe.
● Any indexes created on an unlogged table are automatically unlogged as well.
● For a unlogged table, the field relpersistence of pg_class has the value 'u'
create unlogged table table_unlogged
(a int , b text)
Possibly useful for Temp tables and staging tables (Truncate, Insert)

Zstandard Compression Algorithm
● New compression type: Zstandard
● ZStandard can provide for:
○ either good compression ratio or speed, depending on compression level
○ or a good compromise on both.
● The compression rate is good:
CREATE TABLE table_zstd5(id int4, t text)
WITH (appendoptimized=true, compresstype=zstd, compresslevel=5);
Compression type Table size (B) Ratio vs Heap Size
Heap 8,847,360
Zlib - 3 1,215,304 7
Zstd - 1 132,608 67
Zstd - 19 91,976 96

CREATE TABLE - LIKE
create table t_like
(a int ,
b text default 'a',
c text not null)
with (appendoptimized=true
, orientation=column
, compresstype=quicklz)
distributed by (a);
create index idx_t_like on t_like(a);
comment on column t_like.c
is 'comment field c';
Example
● With LIKE syntax, ability to copy:
○ Default values
○ Indexes (New)
○ Constraints
○ Comments (New)
○ Storage (New)

create table table_unique (a int, b int, c int
, constraint test_uniq unique (b,c))
demo-# distributed by (b,c);
CREATE TABLE
distributed by (b);
CREATE TABLE
distributed by (c);
CREATE TABLE
CREATE TABLE - Unique constraint
distributed by (b,c);
NOTICE: CREATE TABLE / UNIQUE will create implicit
index "table_unique_b_key" for table "table_unique"
CREATE TABLE
distributed by (b);
CREATE TABLE
distributed by (c);
ERROR: UNIQUE constraint and DISTRIBUTED BY
definitions incompatible
HINT: When there is both a UNIQUE constraint, and a
DISTRIBUTED BY clause, the DISTRIBUTED BY clause
must be equal to or a left-subset of the UNIQUE
columns
Before GPDB 6 From GPDB 6
No more require
to be a left-subset

CREATE TABLE
alter table table_unique set distributed by (b);
ALTER TABLE
alter table table_unique set distributed by (c);
ALTER TABLE
CREATE TABLE - Unique constraint - Alter DK
CREATE TABLE
alter table table_unique set distributed by (b);
ALTER TABLE
alter table table_unique set distributed by (c);
ALTER TABLE
Samer behavior

● Ability to specify the hash function used to distribute data across segment instances.
CREATE TABLE - Distribution
Review the example:
https://gpdb.docs.pivotal.io/6-0Beta/admin_guide/ddl/ddl-table.html#top
It contains a bug (already raised)
ORCA doesn't work with custom DK hash functions
More information:
https://github.com/greenplum-
db/gpdb/commit/242783ae9f573bb03da3d89dc689c6743dfce510

● The ROWS and RANGE SQL keywords have changed from reserved to unreserved, and may
be used as table or column names without quoting.
create table t_reserved_words
(rows int
, range int)
CREATE TABLE
select rows,range from t_reserved_words ;
rows | range
------+-------
(0 rows)
Field - Reserved words
(rows int,
range int)
ERROR: syntax error at or near "rows"
LINE 2: (rows int,
^
-- Workaround using ""
demo-# ("rows" int,
demo(# "range" int)
demo-# distributed randomly;
CREATE TABLE

Oracle Extension
● The Oracle Compatibility Functions are now available in Greenplum Database as an extension,
based on the PostgreSQL orafce project at https://github.com/orafce/orafce.
CREATE EXTENSION orafce;
⇒ Create 13 schemas including oracle
cd $GPHOME/share/postgresql/contrib
psql -d demo -f orafunc.sql
⇒ Create a single schema oracomp
Impact on upgrade if the source DB uses oracompat.

Reindex of shared catalog tables (like pg_shdepend)
demo=# reindex table pg_shdepend;
REINDEX
demo=# reindex table pg_database;
REINDEX
demo=# reindex table pg_shdepend ;
ERROR: shared table "pg_shdepend" can only be
reindexed in stand-alone mode
demo=# reindex table pg_database ;
ERROR: shared table "pg_database" can only be
reindexed in stand-alone mode
Workaround: Follow the KB https://pvtl.force.com/s/article/Reindex-fails-
with-ERROR--shared-table-xxx-can-only-be-reindexed-in-stand-alone-
mode
This method requires to stop Greenplum
reindexdb -s takes now into account the shared tables

Changes on VACUUM and vacuumdb
● VACUUM can more easily skip pages it cannot lock. Now VACUUM skips a block it cannot lock and
retries the block later.
● VACUUM rechecks block visibility after it has removed dead tuples. If all remaining tuples in the block
are visible to current and future transactions, the block is marked as all-visible.
● The partitions with no data are age-frozen. so that they do not have to be vacuumed separately and
do not affect calculation of the number of remaining transaction IDs before wraparound occurs.
These tables include the root and intermediate tables in the partition hierarchy and, if they are
append-optimized, their associated meta-data tables.
● vacuumdb utility has a new -F (--freeze) option to freeze row transaction information.
Reduce the frequency of a vacuum appearing to be "stuck," which occurs when VACUUM
waits to lock a block for cleanup and another session has held a lock on the block for a
long time
Reduce the number of required VACUUM and not increase the number of transactions

Parallel backup/restore tools
● gpcrondump & gpdbrestore are removed
● gpbackup & gprestore are separated from Greenplum DB installer.
● The pg_dump and pg_dumpall command-line utilities have a new option --lock-wait-timeout=timeout (PostgreSQL
8.4). When specified, instead of waiting indefinitely the dump fails if the utility cannot acquire shared table locks
within the specified number of milliseconds.
● The -d and -D command-line options are removed from the pg_dump and pg_dumpall utilities. The corresponding
long versions, --inserts and --column-inserts are still supported. A new --binary-upgrade option is added, for use by
in-place upgrade utilities.
● The -w (--no-password) option was added to the pg_dump, pg_dumpall, and pg_restore utilities.
● The pg_restore utility has a new option -j (--number-of-jobs) parameter. This option can reduce time to restore a large
database by running tasks such as loading data, creating indexes, and creating constraints concurrently.
The only tools for parallel backup/restore are gpbackup / gprestore
Separate binaries in Pivotal Network

Serial backup/restore tools
● pg_dump and pg_dumpall
○ New option --lock-wait-timeout=timeout
When specified, instead of waiting indefinitely the dump fails if the utility cannot acquire shared table
locks within the specified number of milliseconds.
○ The -d and -D options are removed.
The corresponding long versions --inserts and --column-inserts are still supported.
○ New --binary-upgrade option, for use by in-place upgrade utilities.
○ New -w (--no-password) option
● pg_restore
○ New option -j (--number-of-jobs)
This option can reduce time to restore a large database by running tasks such as loading data, creating
indexes, and creating constraints concurrently.
○ New -w (--no-password) option

Column level privileges - 1/2
create table t_grant_column (a int, b int, c int) distributed randomly;
insert into t_grant_column select id, id*2,id * 3 from generate_series(1,100) id;
dp t_grant_column
Access privileges
--------+----------------+-------+-------------------+--------------------------
public | t_grant_column | table | |
grant select (b) on table t_grant_column to user1;
dp t_grant_column
Access privileges
--------+----------------+-------+-------------------+--------------------------
public | t_grant_column | table | | b: +
| | | | user1=r/gpadmin
As gpadmin
select * from t_grant_column limit 1;
ERROR: permission denied for relation t_grant_column
select b from t_grant_column limit 1;
b
----
12
(1 rows)
As user1

Column level privileges - 1/2
grant select on table t_grant_column to user2;
dp t_grant_column
Access privileges
--------+----------------+-------+-------------------------+--------------------------
public | t_grant_column | table | gpadmin=arwdDxt/gpadmin+| b: +
| | | user2=r/gpadmin | user1=r/gpadmin
revoke select (b) on table t_grant_column from user2;
NOTICE: no privileges could be revoked
As gpadmin
select * from t_grant_column limit 1;
a | b | c
---+----+----
6 | 12 | 18
(1 row)
As user2
Unable to revoke a single column if a grant is set on the table level
Require to grant each wanted column individually

Catalog & gp_toolkit
Greenplum 6

pg_class
Field Description Status Comment
reloftype OID of an entry in pg_type for a composite type. New
relallvisible Number of all-visible blocks New this value may not be up-to-date
relpersistence The type of object persistence New
p = heap or AO table
u = unlogged temporary table
t = temporary table
relispopulated New
relreplident New
relminmxid New
relrefs Unused Removed
reltoastidxid For a TOAST table, the OID of its index. Removed
relkind The type of object
Changed
(New values)
f = foreign table
b = append-only block directory
M = append-only visibility map

pg_attribute
attcollation
The defined collation of the column, or zero if the is not
of a collatable data type
New
attacl
Column-level access privileges, if any have been
granted specifically on this column
New
attoptions Attribute-level options, as "keyword=value" strings. New
attfdwoptions
Attribute-level foreign data wrapper options, as
"keyword=value" strings.
New

gp_segment_configuration
datadir Segment instance data directory. New
No more need to join with the table
pg_filespace_entry (removed)
replication_port
The TCP port the file block replication process is using
to keep primary and mirror segments synchronized.
Removed
The new field datadir is useful and simplify the ident of directory storing
the segment

pg_db_role_setting
● pg_db_role_setting records the default values of server configuration
settings for each role and database combination.
select datname, setconfig
from pg_database
Inner join pg_db_role_setting
on oid = setdatabase;
datname | setconfig
---------+------------------------
demo | {statement_timeout=1h}
Query for Database
select rolname, setconfig
from pg_authid
Inner join pg_db_role_setting
on oid = setrole;
rolname | setconfig
---------+---------------------
user1 | {log_lock_waits=on}
Query for Roles
demo=# drds
List of settings
Role | Database | Settings
-------+----------+----------------------
user1 | | log_lock_waits=on
| demo | statement_timeout=1h
With psql metacommand

pg_database
datcollate LC_COLLATE for this database. New
datctype LC_CTYPE for this database. New
datminmxid
A Multixact ID is used to support row locking by multiple
transactions. All multixact IDs before this one have been
replaced with a transaction ID in this database.
New
This is used to track whether the database
needs to be vacuumed in order to prevent
multixact ID wraparound or to allow
pg_multixact to be shrunk. It is the minimum of
the per-table pg_class.relminmxid values.
datconfig
Session defaults for user-settable server configuration
parameters
Removed
Database config is now stored in
pg_db_role_setting
● CREATE DATABASE SQL command has new parameters LC_COLLATE and LC_CTYPE to specify the
collation order and character classification for the new database.
● createdb command-line utility has new options -l (--locale), --lc-collate, and --lc-ctype to specify the locale and
character classification for the database

pg_authid
rolreplication Role is a replication role. New
If Yes, this role can initiate streaming replication
and set/unset the system backup mode using
pg_start_backup and pg_stop_backup.
rolcreaterexthdfs
Privilege to create read external tables with the gphdfs
protocol
Removed
rolcreatewexthdfs
Privilege to create write external tables with the gphdfs
protocol
Removed
rolconfig Session defaults for server configuration parameters Removed
Role config is now stored in pg_db_role_setting
● CREATE ROLE and ALTER ROLE SQL commands have new parameters REPLICATION/NOREPLICATION
These clauses determine whether a role is allowed to initiate streaming replication or put the system in and
out of backup mode. A role having the REPLICATION attribute is a very highly privileged role, and should
only be used on roles actually used for replication. If not specified, NOREPLICATION is the default .

pg_proc
protransform
Calls to this function can be simplified by this other
function
New
proiswindow Function is a window function Renamed Old name = proiswin
proleakproof
The function has no side effects. No information about
the arguments is conveyed except via the return value.
New
Any function that might throw an error
depending on the values of its arguments is not
leak-proof.
proexeclocation Where the function executes when it is invoked New
m - master only
a - any segment instance
s - all segment instances.
● CREATE FUNCTION SQL command has a new parameter EXECUTE ON . It specifies where (master or
segment instance) a function executes when it is invoked during the query execution process.
EXECUTE ON ANY (the default) indicates that the function can be executed on the master, or any segment
instance, and it returns the same result regardless of where it is executed. Greenplum determines where the
function executes.
EXECUTE ON MASTER indicates that the function must execute only on the master instance.
EXECUTE ON ALL SEGMENTS indicates that the function must execute on all primary segment instances,
but not the master, for each invocation. The overall result of the function is the UNION ALL of the results
from all segment instances.

gp_distribution_policy
policytype Table distribution policy New
p - Partitioned policy. Table data is distributed
among segment instances.
r - Replicated policy. Table data is replicated on
each segment instance.
numsegments
The number of segment instances on which the table
data is distributed.
New
distkey The column number(s) of the distribution column(s). New It replaces the old field attrnums
distclass
The operator class identifier(s) of the distribution
column(s).
New
attrnums The column number(s) of the distribution column(s) Removed
● gp_distribution_policy now contains more information about Greenplum Database tables and the policy for
distributing table data across the segments including the operator class of the distribution hash functions.

gp_toolkit.gp_resgroup_status_per_host
Field Description Comment
rsgname The name of the resource group.
groupid OID of the resource group
hostname
cpu Real-time CPU usage of the RG on the host.
memory_used Real-time memory usage of the RG on the host.
It includes RG fixed, RG shared memory and
global shared memory used by the RG
memory_available Unused fixed and shared memory for the RG available on the host. not include available RG global shared memory
memory_quota_used Real-time fixed memory usage for the RG on the host.
memory_quota_available Fixed memory available to the RG on the host.
memory_quota_proposed Total fixed memory allotted to the RG on the host.
memory_shared_used Group shared memory used by the RG on the host
memory_shared_granted Portion of group shared memory allotted to the RG on the host.
memory_shared_proposed
Total amount of group shared memory requested by the RG on the
host
● This view allows to see current memory (MBs) and CPU usage and allocation for each resource group on a
per-host basis.

gp_toolkit.gp_resgroup_status_per_segment
Field Description Comment
rsgname The name of the resource group.
groupid OID of the resource group
hostname
segment_id
cpu Real-time CPU usage of the RG by the segment
memory_used Real-time memory usage of the RG by the segment
It includes RG fixed, RG shared memory and
global shared memory used by the RG
memory_available
Unused fixed and shared memory for the RG available by the
segment.
not include available RG global shared memory
memory_quota_used Real-time fixed memory usage for the RG by the segment.
memory_quota_available Fixed memory available to the RG by the segment
memory_quota_proposed Total fixed memory allotted to the RG by the segment
memory_shared_used Group shared memory used by the RG by the segment
memory_shared_granted Portion of group shared memory allotted to the RG by the segment
memory_shared_proposed
Total amount of group shared memory requested by the RG by the
segment
● Same as gp_toolkit.gp_resgroup_status_per_host, but on a per-host and per-segment basis.

New parameters - 1/5
Removed features Default Comment
autovacuum_multixact_freeze_max_age New Not documented
autovacuum_work_mem New Not documented
bonjour New Not documented
create_restartpoint_on_ckpt_record_replay Not documented
default_transaction_deferrable Read-only parameter
dynamic_shared_memory_type Not documented
effective_io_concurrency Not documented
enable_indexonlyscan Not documented
enable_material Not documented
event_source Not documented
exit_on_error Not documented
external_pid_file Not documented
geqo_seed Not documented
gin_fuzzy_search_limit Not documented

gp_enable_global_deadlock_detector off
If off (the default), Greenplum Database executes concurrent update and
delete operations on a heap table serially.
If on, concurrent updates are permitted and the Global Deadlock Detector
determines when a deadlock exists, and breaks the deadlock by cancelling
one or more backend processes associated with the youngest transaction(s)
involved.
gp_enable_minmax_optimization Not documented
gp_fts_mark_mirror_down_grace_period Not documented
gp_fts_probe_retries 5
Specifies the number of times the fault detection process (ftsprobe) attempts
to connect to a segment before reporting segment failure.
gp_global_deadlock_detector_period 120 sec
Specifies the executing interval (in seconds) of the global deadlock detector
backend process.
hot_standby Not documented
hot_standby_feedback Not documented
huge_pages Not documented
lo_compat_privileges Not documented

lock_timeout 0 msec
Abort any statement that waits longer than the specified number of
milliseconds while attempting to acquire a lock on a table, index, row, or other
database object. The time limit applies separately to each lock acquisition
attempt. The limit applies both to explicit locking requests (such as LOCK
TABLE, or SELECT FOR UPDATE without NOWAIT) and to implicitly-
acquired locks. If log_min_error_statement is set to ERROR or lower,
Greenplum Database logs the statement that timed out. A value of zero (the
default) turns off this lock wait monitoring.
log_file_mode Not documented
max_pred_locks_per_transaction Not documented
max_replication_slots Not documented
max_standby_archive_delay Not documented
max_standby_streaming_delay Not documented
max_wal_senders Not documented
max_worker_processes Not documented
quote_all_identifiers Not documented
restart_after_crash Not documented
segment_size Not documented

session_preload_libraries Not documented
ssl_ca_file Not documented
ssl_cert_file Not documented
ssl_crl_file Not documented
ssl_ecdh_curve Not documented
ssl_key_file Not documented
ssl_prefer_server_ciphers Not documented
synchronous_standby_names Not documented
temp_file_limit Not documented
temp_tablespaces Empty
temp_tablespaces specifies tablespaces in which to create temporary objects
(temp tables and indexes on temp tables) when a CREATE command does
not explicitly specify a tablespace.
trace_recovery_messages Not documented
track_functions Not documented
track_io_timing Not documented
transaction_deferrable Not documented

vacuum_defer_cleanup_age Not documented
vacuum_freeze_table_age Not documented
vacuum_multixact_freeze_min_age Not documented
vacuum_multixact_freeze_table_age Not documented
wal_block_size Not documented
wal_keep_segments Not documented
wal_level Not documented
wal_log_hints Not documented
wal_receiver_timeout Not documented
wal_segment_size Not documented
wal_sender_timeout Not documented

Changed parameters - 1/2
Removed features Status Comment
bytea_output Default value
Old default value: escape
New default value: hex
debug_pretty_print Default value
Old default value: off
New default value: on
effective_cache_size Default value
Old default value: 512MB
New default value: 16GB
gp_recursive_cte Renamed Replacing gp_recursive_cte_prototype
gp_workfile_compress Renamed
Replacing gp_workfile_compress_algorithm
When workfile compression is enabled, Greenplum Database uses Zstandard
compression.
log_rotation_size Default value
Old default value: 0
New default value: 1GB
temp_buffers Default value
New default value: 32MB
track_activity_query_size Renamed Replacing pgstat_track_activity_query_size
unix_socket_directories Default value
Old default value:
New default value: /tmp
unix_socket_permissions Default value
New default value: 0777

Changed parameters - 2/2
wal_buffers Default value
Old default value: 256kB
New default value: 4000kB

Removed parameters - 1/2
add_missing_from Removed
custom_variable_classes Removed
Gptext uses this parameter. But it seems that it is not anymore useful from
GPDB6 http://docs-gptext-develop-
staging.cfapps.io/320/topics/installing.html#gpconfig_gptext
explain_pretty_print Removed
gp_analyze_relative_error Removed
gp_backup_directIO* Removed
gp_connections_per_thread Removed
gp_email_* Removed Impact on customer sending emails from Greenplum (Vente Privée, SKY, …)
gp_enable_fallback_plan Removed
gp_enable_sequential_window_plans Removed Possible impact on upgrade at IVECO
gp_filerep_* Removed
gp_idf_deduplicate Removed
gp_hadoop_* Removed
gp_idf_deduplicate Removed
gp_max_csv_line_length Removed
gp_max_databases Removed

Removed parameters - 2/2
gp_max_filespaces Removed
gp_max_tablespaces Removed
gp_num_contents_in_cluster Removed
gp_workfile_checksumming Removed
gp_workfile_compress_algorithm Renamed Replaced by gp_workfile_compress_algorithm
krb_srvname Removed
log_count_recovered_files_batch Removed
max_fsm_* Removed
pgstat_track_activity_query_size Renamed Replaced by track_activity_query_size
regex_flavor Removed
replication_timeout Removed
ssl_renegotiation_limit Removed

Expansion changes
● Greenplum 6 uses a new jump consistent hash algorithm to map hashed data values to Greenplum segments.
The new algorithm ensures that, after new segments are added to the Greenplum 6 cluster, only those
rows that hash to the new segment need to be moved.
Greenplum 6 hashing has performance characteristics similar to earlier Greenplum releases, but should
enable faster database expansion.
Note that the new algorithm is more CPU intensive than the previous algorithm, so COPY performance
may degrade somewhat on CPU-bound systems.
● The -D option is removed from the gpexpand utility. The expansion schema will be created in the postgres
database.
● The gpstate utility has a new -x option, which displays details of an in-progress system expansion.
gpstate -s and gpstate with no options specified also report if a system expansion is in progress.

Removed features
Removed features Type Replaced by Comment
gptransfer Utility gpcopy Possible impact on upgrade
gpcrondump Utility gpbackup Possible impact on upgrade
gpdbrestore Utility gprestore Possible impact on upgrade
gpmfr Utility ???
gphdfs Ext table protocol pxf
SNMP alerts or send email notifications ???
Veritas NetBackup for backup N/A
use of direct I/O for backup N/A
Direct/IO was previously used to bypass the buffering
of memory within the file system cache
SSLv3 Security protocol N/A

Other changes inherited from PostgreSQL - 1/2
● The SELECT and VALUES SQL commands support the SQL 2008 OFFSET and FETCH syntax (PostgreSQL
8.4). These clauses provide an alternative syntax for limiting the results returned by a query.
● The LOCK SQL command has an optional ONLY keyword (PostgreSQL 8.4). When specified, the table is
locked without locking any tables that inherit from it.
● Support for user-defined I/O conversion casts. (PostgreSQL 8.4).

Other changes inherited from PostgreSQL - 2/2
● INTERVAL Data Type Handling: PostgreSQL 8.4 improves the parsing of INTERVAL literals to align with SQL
standards.
select INTERVAL '1' year;
interval
----------
1 year
select INTERVAL '1' month;
interval
----------
1 mon
select INTERVAL '1' week;
week
----------
00:00:01
select INTERVAL '1' year;
interval
----------
00:00:00
select INTERVAL '1' month;
interval
----------
00:00:00
select INTERVAL '1' week;
week
----------
00:00:01

To check and test
● PL/pgSQL
○ The RETURN QUERY EXECUTE statement, which specifies a query to execute
dynamically (PostgreSQL 8.4). ==> have a look here
https://www.postgresql.org/docs/8.4/plpgsql-control-structures.html
○ Conditional execution using the CASE statement (PostgreSQL 8.4).
● Support for the unnest() array function, which expands an array to a set of rows (PostgreSQL
8.4). It seems that it already exists in GPDB5. So not new
● Support for GIN index method (PostgreSQL (8.3).
● DELETE, INSERT, and UPDATE supports the WITH clause, CTE (common table expression)
(PostgreSQL 9.1).

To check and test
● Queries that use SELECT DISTINCT and UNION/INTERSECT/EXCEPT no longer necessarily return sorted
output. Previously these queries always removed duplicate rows by using Sort/Unique processing. They now
implement hashing to conform to behavior introduced in PostgreSQL 8.4; this method does not produce sorted
output. If your application requires sorted output for these queries, alter the queries to use an explicit ORDER
BY clause. Note that SELECT DISTINCT ON never uses hashing, so its behavior is unchanged from previous
versions.
● Specifying the index name in the CREATE INDEX SQL command is now optional. Greenplum Database
constructs a default index name from the table name and indexed columns.
● In Greenplum 6, a query on an external table with descendants will by default recurse into the descendant
tables. This is a change from previous Greenplum Database versions, which never recursed into descendants.
To get the previous behavior in Greenplum 6, you must include the ONLY keyword in the query to restrict the
query to the parent table.

Greenplum 6 Changes

More Related Content

What's hot (20)

Similar to Greenplum 6 Changes (20)

More from VMware Tanzu (20)

Recently uploaded (20)

Greenplum 6 Changes