Introducing Data Redaction - an enabler to data security in EDB Postgres Advanced Server

Data Redaction
Presented by:
Amul Sul
Principal Software Engineer, EDB

© Copyright EnterpriseDB Corporation, 2020. All rights reserved.2
Welcome
• This webinar is being recorded.
• We will be sharing the slides and recording with you after the session.
• Please submit your questions via Zoom Q&A. All questions will be answered
at the end of the presentation.

Agenda
Data Redaction
• Why & what Data Redaction ?
• What is EDB Data Redaction ?
• How to limit sensitive data exposure in EPAS ?
• Provision for the Oracle compatibility in EPAS ?
• Demo.

Why & What Data Redaction ?
A technique that limits sensitive data exposure.
A GDPR (General Data Protection Regulation)-compliant implementation requires the use of many
technical capabilities, such as authentication, authorization, access control, virtual database, and
encryption.
One of the techniques often considered is data redaction to limits sensitive data exposure by
dynamically changing data as it is displayed for specific users.
Data redaction in EPAS version prior v11 and PostgreSQL -- See Creating a Data Redaction
Capability to Meet GDPR Requirements Using EDB Postgres blog, shows how we can use the
PostgreSQL search_path, user defined functions and views to add data redaction protection.

What is EDB Data Redaction ?
Limits sensitive data exposure by dynamically changing data as it is displayed for specific users.
Data Policy Other User
Privileged User
Custom Data
Masking logic

What is EDB Data Redaction ?
Limits sensitive data exposure by dynamically changing data as it is displayed for specific users.
Policy
Name SSN
Sally Sample 020-78-9345
Jane Doe 123-33-9345
Emp Table Other
User
Privileged
User
Name SSN
Sally Sample xxx-xx-9345
Jane Doe xxx-xx-9345
Name SSN
Sally Sample 020-78-9345
Jane Doe 123-33-9345

How to limit sensitive data exposure in EPAS ?
Using Native Data Redaction Capability of EDB Postgres Advanced Server.
Redaction functionPolicy
Scope and exception options Policy expression
Redaction policies allow a user to
choose redaction behavior via
redaction function.
More than one redaction policy can
be created on the same table, but a
column can only be associated with
one policy.
Flexibility to choose when actual
redaction should apply and
exemptions on columns in the query
via the scope and exception options.
Boolean expression for the policy;
determines how the policy is to be
applied. The redaction occurs if this
policy expression is evaluated to TRUE.

Policy
Create policy:
CREATE REDACTION POLICY name ON table_name
[ FOR ( expression ) ]
[ ADD column_name USING
redaction_function()
[ WITH OPTIONS ( redaction_options ) ] ]
[, … ];
Alter Policy:
ALTER REDACTION POLICY name ON table_name ...
➔Rename policy, enable or disable the policy
➔Change policy expression
➔Add more column or remove existing one
➔Change redaction_function and redaction_options

Policy
Create policy on emp table:
CREATE REDACTION POLICY emp_protect ON emp
ADD COLUMN ssn USING redact_ssn(ssn);
And the table description will be:

Redaction
function
CREATE FUNCTION redact_ssn (ssn varchar(11))
RETURNS varchar(11) AS
$$
SELECT overlay (ssn placing 'xxx-xx' from 1);
$$
LANGUAGE SQL;
Note : Return type of the redaction function should be same as the column type.
Policy
Other
User
Name SSN
Sally Sample xxx-xx-9345
Jane Doe xxx-xx-9345

Scope &
exception
options
Previously seen table description:
➢ SCOPE: Identified the query part where redaction to be applied
for the column.
○ Values: query, top_tlist, top_tlist_or_error.
➢ EXCEPTION: Identified the query part where redaction to be
exempted.
○ Values: none, equal, leakproof.

Policy
expression
ALTER REDACTION POLICY emp_protect ON emp
FOR (SESSION_USER <> 'privileged_user');
OR
CREATE REDACTION POLICY emp_protect ON emp
FOR (SESSION_USER <> 'privileged_user')
ADD COLUMN ssn USING redact_ssn(ssn);

Oracle compatibility Provision in EPAS ?
DBMS_REDACT package
Redaction functionPolicy
Policy expression
Like Oracle, provides various
redaction type and supporting
functions.
DBMS_REDACT package provides
Oracle like procedure to add, alter,
enable, disable or drop the policy.
Same as the native support, the
redaction occurs if this policy
expression is evaluated to TRUE.
Scope and exception options
No provision, but user can use
native alter syntax to set scope and
exception.

Policy
BEGIN
DBMS_REDACT.add_policy (
object_schema => 'public',
object_name => 'emp',
policy_name => 'emp_protect',
policy_description => 'policy for emp table ...',
column_name => 'ssn',
function_type => DBMS_REDACT.partial,
function_parameters => DBMS_REDACT.REDACT_US_SSN_F5,
expression => '1=1',
enable => true);
END;

Policy
BEGIN
enable => true);
END;

Function_type and Function_parameters:
BEGIN
enable => true);
END;
Redaction
function

Parameters: function_type
Redaction
function
NONE No redaction.
FULL Full redaction, redacts full values of the column data.
PARTIAL
Partial redaction, redacts a portion of the column data.
function_parameters needed.
RANDOM
Random redaction, each query results in a different random
value depending on the datatype of the column.
REGEXP
Regular Expression based redaction, searches for the
pattern of data to redact. regexp_pattern,
regexp_replace_string, regexp_position,
regexp_occurence, regexp_match_parameter
needed.
CUSTOM Custom redaction type.

Parameters: function_parameters needed for PARTIAL type.
1. REDACT_US_SSN_F5
- Redacts the first 5 numbers of SSN.
- Example: The number 123-45-6789 becomes XXX-
XX-6789
1. REDACT_NA_PHONE_NUMBER
- Redacts the North American phone number by 0 leaving
the area code.
- Example: 1234567890 becomes 1230000000.
1. REDACT_DATE_MILLENNIUM
- Redacts a date that is in the DD-MM-YY format.
- Example: Redacts all date to 01-JAN-2000.
So on…
Partial redaction supports only Character, Number and Date types.
Redaction
function

Existing function_parameters constants not matching my
requirement, then?
Answer: No problem, you can use your function_parameters.
Here are the previously seen function_parameters constants for
Character, Number and Date type respectively and its internal
definition.
1. REDACT_US_SSN_F5 => 'VVVFVVFVVVV,VVV-VV-VVVV,X,1,5'
- input_fmt,output_fmt,mask_char,start,end
1. REDACT_NA_PHONE_NUMBER => '0,4,10'
- mask_digit,start,end
1. REDACT_DATE_MILLENNIUM => 'm1d1y2000'
- monthDigit,dayDigit,yearDigit
- You can replace hours, minutes and seconds too.
Redaction
function

Policy expression:
BEGIN
enable => true);
END;
Policy
expression

Alter
Policy
Action parameter of ALTER_POLICY() decides what
to alter:
1. Add column to the existing policy,
- action => ADD_COLUMN
1. Modify/Drop column redaction method,
- action => MODIFY_COLUMN
1. Modify policy expression,
- action => MODIFY_EXPRESSION
1. Set policy description, and
- action => SET_POLICY_DESCRIPTION
1. Set column description
- action => SET_COLUMN_DESCRIPTION

Alter
Policy
Alter policy to add another column:
BEGIN
DBMS_REDACT.alter_policy (
action =>
DBMS_REDACT.add_column,
column_name => 'salary',
function_type =>
DBMS_REDACT.full);
END;
e.g:

1. A sample data set with employee IDs, names, social security numbers, salary
etc. is created in the table employees in the mycompany database.
2. A data redaction policy for ssn and salary column will be applied whenever user other than
privilegeduser tries to access the employees table data
Demo
Step-by-step walkthrough for the complete demo:

Step 1: Create database
DROP DATABASE IF EXISTS mycompany;
CREATE DATABASE mycompany
WITH OWNER = enterprisedb;
Step 2: Connect to the new database
psql -d mycompany -U enterprisedb
psql (11.6.13)
Type "help" for help.
mycompany=>
Demo
A sample data set with employee IDs, names, social security numbers, salary etc. is created in the
table employees in the mycompany database.
Step 3: Create table
CREATE TABLE employees (
id SERIAL PRIMARY KEY,
name VARCHAR(40) NOT NULL,
SSN VARCHAR(11) NOT NULL,
salary MONEY);
Step 4: Add sample data
INSERT INTO employees (name, ssn, salary)
VALUES ('Sally Sample', '020-78-9345', 51234.34),
('Jane Doe', '123-33-9345', 62500.00),
('Bill Foo', '123-89-9345', 45350.00);

CREATE ROLE privilegeduser LOGIN PASSWORD 'password';
GRANT ALL ON employees TO privilegeduser;
CREATE ROLE non_privilegeduser LOGIN PASSWORD 'password';
GRANT ALL ON employees TO non_privilegeduser;
Demo
Create privileged and non-privileged user and grant the necessary access.

BEGIN
object_name => 'employees',
policy_name => 'emp_data_protect',
policy_description => 'hide sensitive info of the
employees',
function_parameters => 'VVVFVVFVVVV,VVV-VV-VVVV,#,1,5',
expression => 'SESSION_USER <>
''privilegeduser''',
enable => true);
END;
Demo
Create policy and on ssn and salary column for non-privileged users.

BEGIN
DBMS_REDACT.alter_policy (
object_name => 'employees',
policy_name => 'emp_data_protect',
action => DBMS_REDACT.add_column,
column_name => 'salary',
function_type => DBMS_REDACT.full);
END;
Demo
Add salary column to emp_data_protect policy.

mycompany=> c - privilegeduser
You are now connected to database "mycompany" as user "privilegeduser".
mycompany=> SELECT * FROM employees;
id | name | ssn | salary
----+--------------+-------------+------------
1 | Sally Sample | 020-78-9345 | $51,234.34
2 | Jane Doe | 123-33-9345 | $62,500.00
3 | Bill Foo | 123-89-9345 | $45,350.00
(3 rows)
Demo
By default table owner and super user can see un-redacted data.
Also, privilegeduser can see un-redacted data to whom we have exempted from the policy.

mycompany=> c - non_privilegeduser
You are now connected to database "mycompany" as user
"non_privilegeduser".
mycompany=> SELECT * FROM employees;
----+--------------+-------------+--------
1 | Sally Sample | ###-##-9345 | $0.00
2 | Jane Doe | ###-##-9345 | $0.00
3 | Bill Foo | ###-##-9345 | $0.00
(3 rows)
Demo
When a user other than privilegeduser tries to access the employee table will see redacted data for
ssn and salary column.

mycompany=> c - privilegeduser
mycompany=> EXPLAIN VERBOSE SELECT * FROM employees;
QUERY PLAN
---------------------------------------------------------------------
Seq Scan on public.employees (cost=0.00..14.50 rows=450 width=150)
Output: id, name, ssn, salary
(2 rows)
mycompany=> EXPLAIN VERBOSE SELECT * FROM employees;
QUERY PLAN
--------------------------------------------------------------------------
Output: id, name, redact_partial_str(ssn, ...), redact_full_num(salary)
(2 rows)
Demo
Explain plan of the privilegeduser and non-privilegeduser user’s query.

You are now connected to database "mycompany" as user
"non_privilegeduser".
mycompany=> SELECT * FROM employees WHERE salary > 60000::money;
----+----------+-------------+--------
2 | Jane Doe | ###-##-9345 | $0.00
(1 row)
How to restrict this ?
Demo
By default for the Oracle compatibility if policy created using DBMS_REDACT package procedure
the scope is “top_tlist” -- So what?

mycompany=> c - enterprisedb
mycompany=> ALTER REDACTION POLICY emp_data_protect ON employees
MODIFY COLUMN salary WITH OPTIONS (SCOPE query);
ALTER REDACTION POLICY
----+------+-----+--------
(0 rows)
Demo
Use native syntax to tweak scope and exception, since no provision in DBMS_REDACT package for
that.

-- scope: top_tlist
mycompany=> EXPLAIN VERBOSE SELECT * FROM employees WHERE salary > 60000::money;
QUERY PLAN
--------------------------------------------------------------------------
Filter: (employees.salary > (60000)::money)
(3 rows)
-- scope: query
mycompany=> EXPLAIN VERBOSE SELECT * FROM employees WHERE salary > 60000::money;
QUERY PLAN
--------------------------------------------------------------------------
Filter: (redact_full_num(employees.salary) > (60000)::money)
(3 rows)
Demo
Explain plan of the query when scope “top_tlist” vs “query”.

MODIFY COLUMN salary WITH OPTIONS (SCOPE top_tlist_or_error);
ALTER REDACTION POLICY
ERROR: redacted column is allowed only in top targetlist
Demo
Use SCOPE for the strictness.

MODIFY COLUMN ssn WITH OPTIONS (SCOPE top_tlist_or_error, EXCEPTION equal);
mycompany=> SELECT * FROM employees WHERE ssn = '123-89-9345';
----+----------+-------------+--------
3 | Bill Foo | ###-##-9345 | $0.00
(1 row)
mycompany=> SELECT * FROM employees WHERE ssn like '123-89%';
ERROR: redacted column is allowed only in top targetlist
Demo
Some reasons you want to show information if the non_privilegeduser has exact column value, but
your scope is top_tlist_or_error, then?
--

Who is EDB?

The largest dedicated PostgreSQL company
EDB acquires 2ndQuadrant in Sept 2020
• More customers: Than any dedicated PostgreSQL
company
• More experts: Leading PostgreSQL contributors
• More innovation: Positioned to lead in enterprise
PostgreSQL and hybrid cloud
+

EDB supercharges PostgreSQL

Blog:
● Native Data Redaction Capability in EDB Postgres Advanced Server 11
● Creating a Data Redaction Capability to Meet GDPR Requirements
Document:
• EDB Postgres Advanced Server : Security : Data Redaction
• EDB Postgres Advanced Server : Built-In Packages : DBMS_REDACT
Learn more about EDB data redaction:
--

Introducing Data Redaction - an enabler to data security in EDB Postgres Advanced Server

More Related Content

What's hot (20)

Similar to Introducing Data Redaction - an enabler to data security in EDB Postgres Advanced Server (20)

More from EDB (20)

Recently uploaded (20)

Introducing Data Redaction - an enabler to data security in EDB Postgres Advanced Server