Change Data Capture for VSAM on IBM z/OS

For VSAM there is no native logging. A logging subsystem stores changed records in a log using standard system logging services performed in the VSAM journaling (JRNAD) exit. The logging subsystem can be used with any job that updates a VSAM cluster, including batch or CICS. It is non-intrusive, fully reentrant, and resides in CSA (common storage area). Logging is enabled for each data set to be monitored by associating it with the logging subsystem in a DD statement. The subsystem collects records that are written or deleted in a log stream. Records stored in the logs are extended with a transaction ID, time stamp and update/delete indicator. The logs are read by the VSAM adapter using an additional synonym for the log.

Using DataMigrator, a direct load flow is created to read the log records and write the changes to a target table. While there is no indication in the log to distinguish between a new and updated record, the data flow matches the key(s) against the target table and performs an update or insert as required. As with Change Data Capture for relational databases, a checkpoint file is used to record the last record processed.

The CDC solution consists of a permanently running z/OS VSAM logging subsystem and a logstream utilizing the standard z/OS system logger to collect changes to the monitored VSAM clusters.

In the illustration below:

Change Data Capture flow chart

To view an example JCL for the setup processes that are done once by a system administrator, see:

To view an example of the JCL change that is needed for each program that updates VSAM clusters, see Associating VSAM Clusters With the CDC Subsystem.


Top of page

Example: Creating the Logstream Data Set

The logstream is the data set that stores the collected VSAM data records. A system administrator must define the logstream, IBI.CDCVSAM.LGSTREAM, with JCL. This job should be run by a system administrator.

 //jobname JOB (user),REGION=0M, 
// MSGCLASS=Q,NOTIFY=user,CLASS=A, 
// MSGLEVEL=(1,1) //LOGR EXEC PGM=IXCMIAPU 
//SYSPRINT DD SYSOUT=* 
//SYSIN DD *DATA TYPE(LOGR) REPORT(NO) 
DEFINE STRUCTURE NAME(IBI_CDC_CFSTRUCT) LOGSNUM(1) 
MAXBUFSIZE( 65532 ) AVGBUFSIZE( 32766 ) 
DEFINE LOGSTREAM NAME(IBI.CDCVSAM.LGSTREAM) 
STRUCTNAME( IBI_CDC_CFSTRUCT ) 
LOWOFFLOAD( 40 ) 
HIGHOFFLOAD( 95 ) 
STG_DUPLEX(YES) 
DUPLEXMODE(COND) 
AUTODELETE(YES) 
/*

The output from the job looks like this:

ADMINISTRATIVE DATA UTILITY: INPUT 															DATA TYPE = LOGR
________________________________________________________________________________
LINE # 					CONTROL CARDS
 
					1 					DATA TYPE(LOGR) REPORT(NO) 
					2 					DEFINE STRUCTURE NAME(IBI_CDC_CFSTRUCT) LOGSNUM(1)
					3 					MAXBUFSIZE(65532)	AVGBUFSIZE(32766)
					4 					DEFINE LOGSTREAM NAME(IBI.CDCVSAM.LGSTREAM)
					5						STRUCTNAME(IBI_CDC_CFSTRUCT)
					6 					LOWOFFLOAD(40)
					7 					HIGHOFFLOAD(95)
					8 					STG_DUPLEX(YES)
    	9 					DUPLEXMODE(COND) 
					10 				AUTODELETE(YES)

In addition the jobstream data set requires the following RACF profile:

 RESOURCE(IBI.CDCVSAM.LGSTREAM) CLASS(LOGSTREAM) READ, ALTER

Top of page

Example: Activating the Logging Subsystem

The logging subsystem must be activated to support VSAM CDC processing. The VSAM I/O operations are logged within this subsystem.

In the following example, a mainframe job initiates the subsystem.

 //jobname JOB 
//* JOB to START/SWAP/INFO/DEACTIVATE Subsystem * 
// SET P1='ICDC,S,CDCSSVT ,,,,,' 
// SET P2='M727703B,300 ,IBI.CDCVSAM.LGSTREAM ' 
//T EXEC PGM=CDCSSLDR, PARM='&P1&P2' 
//STEPLIB DD DSN=qualif.HOME.LOAD,DISP=SHR 
//CDCREPRT DD SYSOUT=* 
//

The example JCL includes the following:

CDCSSLDR Parameters

This is the main CDC program.

All parameters must be specified, separated by commas. Parameters are of fixed length and should be padded with spaces to the specified length.

Position

Length

Description

1

4

Subsystem name (for example ICDC).

2

1

Action:

S
Start (brings up subsystem).
W
Swap (substitutes active subsystem with the new one).
I
Info (subsystem activities info).
D
Deactivate (brings subsystem down).

3

8

Subsystem implementing authorized module name: CDCSSVT.

4

8

Reserved for future use (CPF prefix for Sysplex).

5

 

Reserved for IBI use.

6

 

Reserved for IBI use.

7

 

Reserved for IBI use.

8

8

Server version identification, for example M727703B.

9

8

Maximum number of VSAM data sets serviced by the subsystem.

10

26

Logstream name, for example: IBI.CDCVSAM.LGSTREAM

STEPLIB

The STEPLIB DD statement identifies the location of the load library for the CDC subsystem. It must be set as authorized.

The default locations where qualif is the location of the IBI Server data sets are:

HFS Deployment

 //STEPLIB DD DSN=qualif.HOME.LOAD,DISP=SHR

PDS Deployment

 //STEPLIB DD DSN=qualif.P.HOME.LOAD,DISP=SHR


Example: Seeing CDC Subsystem Activities Information

To see a report of subsystem activities information, submit a job like this example, with parameters as described above.

 //jobname JOB 
//* Subsystem information job * 
// SET P1='ICDC,I,CDCSSVT ,,,,,' 
// SET P2='M727703B,300 ,IBI.CDCVSAM.LGSTREAM ' 
//T EXEC PGM=CDCSSLDR, PARM='&P1&P2' 
//STEPLIB DD DSN=qualif.HOME.LOAD,DISP=SHR 
//CDCREPRT DD SYSOUT=* 
//

The subsystem information report looks like this:

 S u b s y s t e m   C D C   V S A M 
Date : 09/15/2011                          Time : 13:02:17 
Subsystem Name                    : ICDC 
Subsystem Status                  : Active 
Subsystem Version                 : M727703B 
Module Name                       : CDCSSVT 
Subtable Address                  : 0x108DFDE8 
Log Stream Name                   : IBI.CDCVSAM.LGSTREAM 
ENF Logger Exit Status            : Active 
Number of Dsn's Available         : 300 
Number of Dsn's Activated         : 3 
   T a b l e   L o g g e r   C o n d i t i o n s 
------------------------------------------------------ 
| Logger Status |  Counter  | Last  RC  | Last  RSN  | 
------------------------------------------------------ 
| Error         | 0         | 0         | 0          | 
------------------------------------------------------ 
| ENF Events    | 0         | 0         | 0          | 
------------------------------------------------------ 
| Warnings      | 0         | 0         | 0          | 
------------------------------------------------------ 
  T a b l e   V S A M   C D C   S t a t i s t i c s 
---------------------------------------------------------------------- 
| NNN |  D a t a   S e t   N a m e   |  Upd. Counts  |  Del. Counts  | 
----------------------------------------------------------------------
| 1   | EDASYS.EDAPORT.CUST2.CLUSTER |       6       |       6       | 
---------------------------------------------------------------------- 
| 2   | EDASYS.EDAPORT.CUST3.CLUSTER |       2       |       1       | 
---------------------------------------------------------------------- 
| 3   | EDASYS.EDAPORT.CUST5.CLUSTER |       1       |       1       | 
----------------------------------------------------------------------

iWay Software