Using Release(deallocate) and Painful Lessons to be learned on DB2 locking
3 likes2,963 views
This document discusses thread reuse using the RELEASE(DEALLOCATE) bind option in DB2, considerations for lock avoidance, and lessons learned on DB2 locking. It provides primers on thread reuse, the RELEASE bind option, lock avoidance techniques like commit log sequence numbers and possibly uncommitted bits, and the ramifications of lock avoidance for SQL. It recommends using programming techniques to avoid data currency exposures when using lock avoidance, and outlines how to identify packages that can safely be rebound with CURRENTDATA(NO).
Using Release(deallocate) and Painful Lessons to be learned on DB2 locking
- 1. #IDUG Using RELEASE(DEALLOCATE) and Painful Lessons to be Learned on DB2 Locking John Campbell DB2 for z/OS Development Session Code: 6007
- 2. #IDUG 2 Agenda • Thread reuse using RELEASE(DEALLOCATE) • Primer on thread reuse with RELEASE bind option • Considerations and limitations • Painful Lessons to be Learned on DB2 Locking • Primer on lock avoidance • CURRENTDATA(YES) versus CURRENTDATA(NO) • Ramifications of lock avoidance for SQL • What should an application programmer do • Finding packages to safely rebind with CURRENTDATA(NO)
- 3. #IDUG 3 “Resources”: Static SQL • Static statements • Packages and statements • Parent locks • Index look-aside buffer, dynamic prefetch tracking via sequential detection
- 4. #IDUG 4 RELEASE - BIND and REBIND Option • Determines when to release the resources that a program uses • RELEASE(COMMIT) - Releases resources at commit point • RELEASE(DEALLOCATE) - Releases resources when thread terminates (child page/row locks still released at commit) • RELEASE(INHERITFROMPLAN) – Support added by APAR PM07087 • Default behavior • BIND PLAN - COMMIT • BIND PACKAGE - plan value • REBIND PLAN/PACKAGE existing value • DB2Binder Utility for JDBC and SQLJ • DEALLOCATE is default in DB2 10 • COMMIT is default in DB2 9 and earlier releases
- 5. #IDUG Package Allocation Program A : “Insert into table 1” Thread allocation Sign on, authorization checking Locate SKCT and SKPT Allocate CT/PT and STMT1 Start Insert STMT1 Lock table space 1 Lock data page End STMT1 “insert into table 2” Allocate STMT2 Start Insert STMT2 Lock table space 2 Lock data page End STMT2 COMMIT ; Program A Stmt 1: Insert into table1 Stmt 2: Insert into table2 SPT01 EDM_SKELTON_POOL SKCT/SKPT V10 Thread storage above the bar V9 EDM thread pool below the bar CT PT SQL Stmt1 SQL Stmt2 I/O
- 6. #IDUG 6 Package Allocation and Commit Program A Stmt 1: Insert into table1 Stmt 2: Insert into table2 Commit ; SPT01 EDM_SKELTON_POOL SKCT/SKPT V10 Thread storage above the bar V9 EDM thread pool below the bar CT PT SQL Stmt1 SQL Stmt2 I/O Expensive Operation! Lock TS 1 Lock table1 data page Lock TS 2 Lock table2 data page RELEASE PT, statements Unlock data pages Unlock TS1, TS2 COMMIT
- 7. #IDUG 7 Thread Reuse and RELEASE(DEALLOCATE) • Thread reuse eliminates CPU cost of DB2 thread allocation and deallocation • CICS • Protected ENTRYs • Organic reuse of ENTRYs (or POOL) • IMS/TM • Fast Path (IFP) regions • Wait-For-Input (WFI) regions • Pseudo Wait-For-Input (PWFI) regions • DDF • High Performance DBATs • WebSphere Type 2 local connections • Batch with intermediate commits • Use of RELEASE(DEALLOCATE) coupled with effective thread reuse i.e., thread persistence • Further reduces the CPU cost with potential for significant savings (up to 10% plus) • Resources are held until thread deallocation • Without thread reuse RELEASE(COMMIT) vs. RELEASE(DEALLOCATE) is a moot point for discussion
- 8. #IDUG 8 Limitations and Considerations • Virtual and real storage • DB2 9 and earlier - DBM1 virtual storage below the 2G bar and real storage • Package information is stored in EDM pool below the bar • DB2 10 after REBIND - real storage usage • Package information is stored in thread storage above the bar in DB2 9 and earlier releases • Accumulated DB2 object control blocks • Virtual, real, potentially CPU cost for scanning the objects built up under the thread • Recommendations • Design for thread reuse for • High volume simple transactions • Complex transactions at a reasonably high rate • Selectively use RELEASE(DEALLOCATE) on high use packages – use % of Total Acctg Class 7 CPU • DBM1 31-bit virtual storage constraint (DB2 9) • Real storage constraint (DB2 10) • Use CICS or WebSphere parameter to periodically clean up and rejuvenate threads (thread deallocation)
- 9. #IDUG 9 Considerations for Clean Up • REUSELIMIT (0-10,000) in CICS TS V4R2 - default 1000 • Number of times a thread can be reused before it is terminated • Use default and monitor DB2 storage usage and adjust the number if needed • WebSphere Type 2 connection Aged Timeout - default 0 • The interval in seconds before a physical connection is discarded • Consider setting WAS “aged timeout” to less than 5 min, recommend using 120 secs to reduce exposure of long living threads • DB2 10 High Performance DBATs (threads) • Thread will go inactive every 200 commits • No user control for this value • DB2 11 optimizes RELEASE(DEALLOCATE) execution so that it is consistently better performing than RELEASE(COMMIT)
- 10. #IDUG 10 Considerations - Concurrency • More persistent threads with RELEASE(DEALLOCATE) is also trade off with concurrency • BIND/REBIND • SQL DDL • Online REORG which invalidates packages • For RELEASE(DEALLOCATE) some locks are held beyond commit until thread termination • Mass delete locks (SQL DELETE without WHERE clause) • Gross level lock acquired on behalf of a SQL LOCK TABLE • Table space defined with LOCKSIZE TABLESPACE|TABLE • Note: no longer a problem for gross level lock acquired by lock escalation • DO YOUR HOMEWORK BEFORE USING PERSISTENT THREADS WITH RELEASE(DEALLOCATE) BIND OPTION
- 11. #IDUG 11 Primer on lock avoidance • Combination of techniques used by DB2 to try to avoid taking a S page/row locks when processing for read only SQL whilst preventing the retrieval of uncommitted data by the application • (1) Page latching (and page p-lock in data sharing) controlled by DB2 to ensure physical consistency of the page • (2) Commit log sequence number (CLSN) – at the page level • DB2 tracks "time" of last update to page (on page) (A) • DB2 tracks "time" of oldest uncommitted activity on every pageset/partition (B) • Non Data Sharing • CLSN = lowest uncommitted RBA for all active transactions for a given pageset • Data Sharing • For non-GBP-dependent page sets, each member uses a local CLSN = lowest uncommitted LRSN for all active transactions for a given pageset • For GBP-dependent page sets, a Global CLSN value is maintained for the entire data sharing group = lowest CLSN value across all members across all page sets (GBP-dependent or not) • If (A) < (B) everything on the page is guaranteed to be committed • Else, check Possibly UNCommitted bits (PUNC bits)
- 12. #IDUG 12 Primer on lock avoidance … • Combination of techniques to prevent retrieval of uncommitted data … • (3) Possibly UNCommitted bits (PUNC bits) – at the row level • On each row, a PUNC bit is set when the row is updated • PUNC bits are periodically reset • If successful CLSN check and more than 25% of the rows have the PUNC bit ON • RR scanner • REORG TABLESPACE • If the PUNC bit is not ON, the row/key is guaranteed to be committed
- 13. #IDUG 13 CURRENTDATA(YES) versus CURRENTDATA(NO) … • Plans and packages have a better chance for lock avoidance if they are bound with ISOLATION(CS) and CURRENTDATA(NO) • What is CURRENTDATA? • Helps to determine the currency and stability of data returned to an application cursor • Only applies to applications bound with ISOLATION(CS) • What is isolation CURSOR STABILITY(CS)? • Data fetched by a cursor is committed, but if the application process returns to the same page, another application might have since updated, deleted, or inserted qualifying rows • If the cursor is defined as FOR UPDATE OF • Data returned by the cursor is stable and it may not be updated by another transaction while the updatable cursor is positioned on it • If the cursor is defined as FOR READ|FETCH ONLY, or it is implicitly read only (or it is ambiguous) • ISOLATION(CS) ensures that the data returned is committed and the stability of the cursor is determined by the CURRENTDATA option
- 14. #IDUG 14 Primer on lock avoidance … • Benefits of lock avoidance • Improved concurrency through less lock collisions • Decrease in lock and unlock activity requests, with an associated decrease in CPU resource consumption and data sharing overhead • V8 improvements • Lock avoidance for non-cursor ‘singleton’ SELECT • In V7, ISOLATION(CS) CURRENTDATA(YES) acquires S page/row lock on the qualified row • In V8, DB2 no longer acquires and hold S page/row lock on the qualified row for ISOLATION(CS) CURRENTDATA(YES or NO) • Overflow lock avoidance when the update of a variable length row in a data page results in a new row that cannot fit in that page i.e., indirect reference • In V7, no lock avoidance on both pointer and overflow • In V8, lock on pointer only • Need to distinguish carefully between eligibility for lock avoidance and actually achieving it
- 15. #IDUG 15 Primer on lock avoidance … • BIND option ISOLATION(CS) with CURRENTDATA(NO) could reduce # Lock/Unlock requests dramatically • High Unlock requests/commit could also be possible from • Large number of relocated rows after update of compressed or VL row • Lock/Unlock of pointer record (or page) • Large number of pseudo-deleted entries in unique indexes • Lock/Unlock of data (page or row) in insert to unique index when pseudo-deleted entries exist • Both can be eliminated by REORG Field Name Description QTXALOCK LOCK REQUESTS QTXAUNLK UNLOCK REQUESTS LOCKING ACTIVITY QUANTITY /SECOND /THREAD /COMMIT ------------------------ -------- ------- ------- ------- ... LOCK REQUESTS 521.0M 24.2K 3134.34 1050.75 UNLOCK REQUESTS 478.1M 22.2K 2876.06 964.16 Lock avoidance may not be working effectively if Unlock requests/commit is high, e.g. >5 ROT
- 16. #IDUG 16 Primer on lock avoidance … • Effective lock avoidance is very important in data sharing • Global locks propagated beyond IRLM to Coupling Facility are relatively expensive • Global management of lock contention/resolution is very expensive • Effective Lock avoidance is critical to achieving good performance and lower data sharing overhead • Any long-running UR(s) can reduce the effectiveness of lock avoidance by stopping the Global CLSN value from moving forward • Recommendation: Aggressively monitor long-running URs • 'First cut' ROTs: • URs running for a long time without committing: zparm URCHKTH<=5 • Message DSNR035I • URs performing massive update activity without committing: zparm URLGWTH=10(K) • Message DSNJ031I • Need management ownership and process for getting rogue applications fixed up so that they commit frequently based on • Elapsed time and/or • CPU time (no. of SQL update statements)
- 17. #IDUG 17 CURRENTDATA(YES) versus CURRENTDATA(NO) … • Switching from CURRENTDATA(YES) to CURRENTDATA(NO) • In general, can be done without any adverse effect to applications • But consider the following scenario: 1. Program A bound with ISOLATION(CS) CURRENTDATA(YES) fetches a row from a cursor with ORDER BY – and the row is read from the base table and not from a work file 2. Program B tries to update the row just read by Program A • The access is not allowed because the CURRENTDATA(YES) option for Program A caused a lock to be acquired on the page/row in the base table 3. Program A then issues a searched UPDATE of the row just fetched from step 1. • In the above example CURRENTDATA(YES) protects the integrity of the data read by Program A in between the FETCH and the searched UPDATE • If Program A was to be rebound with CURRENTDATA(NO) then 1. Program B may (should) be able to access the row just fetched by Program A • This assumes correct timing and that lock avoidance is effective 2. When Program A issues the searched UPDATE, it could wipe out the changes just made by Program B
- 18. #IDUG 18 CURRENTDATA(YES) versus CURRENTDATA(NO) … • Switching from CURRENTDATA(YES) to CURRENTDATA(NO) … • Now reconsider the same scenario: 1. Program A bound with ISOLATION(CS) CURRENTDATA(YES) fetches a row from a cursor with ORDER BY, but the row is now read from a work file 2. Program B tries to update the row just read by Program A 1. Access is now allowed as there is no lock on the page/row in the base table 3. Program A then issues a searched UPDATE of the row just fetched from step 1 • In the above example CURRENTDATA(YES) does NOT protects the integrity of the data read by Program A in between the FETCH and the searched UPDATE
- 19. #IDUG 19 Ramifications of lock avoidance for SQL • Data may be returned without a lock • All data returned will be committed at the time it is taken from page • But may change by the time the application sees it • DB2 only attempts lock avoidance for ISOLATION(CS) transactions when BOTH of the following are true: • The plan/package is bound CURRENTDATA(NO) • The cursor (if cursor-based FETCH) is declared FOR READ|FETCH ONLY, or has some other attribute (e.g., ORDER BY) that makes it a read-only cursor • Applications that depend on the stability of what is under a read only cursor, should use CURRENTDATA(YES) • Default is CURRENTDATA(NO) • Now lets look at another example
- 20. #IDUG 20 Ramifications of lock avoidance for SQL … • Example DECLARE CURSOR AS ... SELECT THIRD_COL ... WHERE CLUST_COL = 'ABC' AND ANOTHER_COL = 'DEF' ORDER BY CLUST_COL; <--- makes cursor read only FETCH ... <--- say this returns row ABC.DEF.GHI without taking a lock UPDATE SET … WHERE CLUST_COL = 'ABC' AND ANOTHER_COL = 'DEF' AND THIRD_COL = 'GHI‘ • What is the issue? • Row may no longer be ABC.DEF.GHI by the time update executes • Another transaction may have gotten in and updated it to ABC.DEF.XYZ • So update will receive "row not found"
- 21. #IDUG 21 What should an application programmer do • Use programming techniques which avoid possible data currency exposures and access path dependencies (these sometimes occur regardless of the CURRENTDATA option) • Define cursors with their intended use • Easier to read application code • Number of ambiguous cursors can be reduced • Some queries will not allow the use of a FOR UPDATE cursor due to possible work file usage e.g., with ORDER BY • Use predicates on searched updates to enforce data currency • When a FOR UPDATE cursor cannot be used with a query it is common for application programmers to use a read only with a subsequent searched UPDATE or DELETE • It is important to use WHERE predicates to ensure that data has not changed since your cursor first selected the data • Include all columns which logically determine if the update is necessary, instead of updating based solely on the key • This is an exposure regardless of the CURRENTDATA option, but the use of CURRENTDATA(NO) can increase the exposure opportunity • Consider the use of a timestamp or a version number • Add a timestamp or version number column to a table to record the last update • Select the timestamp or version number column with the cursor • Use it as a predicate on the searched UPDATE or DELETE to ensure that an application is updating the same row
- 22. #IDUG 22 Finding packages to safely rebind with CURRENTDATA(NO) • Must recognise that access path dependencies will inevitably occur • CURRENTDATA(YES) will not stop access path dependency and associated exposures • Access path independence and CURRENTDATA(NO) should be the goal for all applications • Can safely rebind packages that use only true read-only cursors with CURRENTDATA(NO)
- 23. #IDUG 23
- 24. #IDUG John Campbell DB2 for z/OS Development [email protected] Session 6007 Using RELEASE(DEALLOCATE) and Painful Lessons to be Learned on DB2 Locking