IBM InfoSphere Data Replication (IIDR)

Posted by Frank Fillmore on April 10, 2013 under DB2 for i, DB2 for Linux Unix Windows, DB2 for z/OS, IBM Information Management Software Sales, Information on Demand Conference, InfoSphere, Q-Replication. Tags: , , .

IBM has bundled its replication technology into a single package: IBM InfoSphere Data Replication (IIDR).  IIDR combines three components:

  • SQL Replication (heritage DataPropagator- Relational)
    • Easy to set up
    • Staging tables
  • InfoSphere Replication Server (IRS – heritage Q Replication)
    • High volume, low latency
    • Native Oracle and DB2 sources and targets
    • WebSphere MQ transport layer
  • InfoSphere Change Data Capture (ICDC – heritage DataMirror)
    • Broadest set of heterogeneous sources and targets
    • TCP/IP transport layer

There are a few of important take-aways.

  1. IBM substituted an implementation decision for a buying decision.  For quite a while SQL Replication was bundled free with DB2 for LUW.  ICDC and IRS were separately purchasable technologies with a lot of functional overlap.  In 2010 my colleague, Kim May, delivered an IBM Information on Demand (IOD) presentation distinguishing between the three.  Now you purchase the IIDR bundle and determine which technology is best suited for a particular use case.  As with most IBM software on distributed platforms, the cost is based on Processor Value Units (PVUs).
  2. New feature/functionality will be built into IIDR rather than the former heritage ICDC and IRS packaging.
  3. There will be a convergence of the technologies over time.  Many of the prospective changes are still IBM Confidential, but IBM is looking to consolidate components where it makes sense to do so.  There is a long-term roadmap that I hope IBM be sharing shortly.
  4. Upgrading and migration paths are a work-in-progress.  If you currently own ICDC or IRS and want to move up to IIDR, contact The Fillmore Group for pricing and implementation assistance.

Free Webinar: What Your IBM Competitive Sales Specialist Wants You To Know

Posted by Kim May on February 25, 2013 under Baltimore Washington DB2 Users Group, Big Data, DB2 Education, DB2 for Linux Unix Windows, DB2 Migrations, IBM DB2 Services, IBM Information Management Software Sales, IBM Mid Market Customers, IBM Pure Systems, International DB2 Users Group (IDUG), Netezza, Optim, Oracle, Q-Replication, TFG Blog. Tags: , , , , , , , .

Join IBM DB2 Competitive Sales Specialist Bill Kincaid and me for The Mid-Atlantic Information Management Virtual Users Group webinar on February 28th.  The Mid-Atlantic Information Management Virtual Users Group is focused on delivering information and education to DB2 users in the Mid-Atlantic region.  Given the topic and the terrific speaker lined up, and since the session will be delivered via webinar, anyone interested in IBM database solutions is welcome to join us.
 
In the session current DB2 customers will learn how IBM has enhanced DB2 administrative tool functionality, added new features, and is offering new product bundles that deliver better value at a lower cost.  Customers considering DB2 will learn how IBM is providing better assessment tools and easier migration processes to simplify the transition to DB2. 
 
Is DB2 the best?  Is moving to DB2 a no-brainer?  Join us, ask questions, and learn more about today’s DB2. 
 
Date:  Thursday, February 28, 2013
Time:  11am – 12:00pm Eastern
Registration is required.  Please register here
 
Bill Kincaid
Bill Kincaid is the IBM DB2 Competitive Sales Specialist for the Mid-Atlantic Region.  Bill has worked for IBM, Microsoft and Oracle, as well as data virtualization solution developer xkoto.  Bill’s specialty is helping database users – customers currently using DB2 and customers considering migrating to DB2 – to fully understand the value of DB2.  Bill earned his BA at the University of North Carolina – Charlotte and his MBA from the University of Georgia.  He lives in Charlotte and works with IBM customers in the Eastern US.
 
I hope you will join us.

Parallel asntdiff Dramatically Improves Performance for Q Rep on z/OS

Posted by Frank Fillmore on February 11, 2013 under DB2 for z/OS, InfoSphere, Q-Replication. Tags: , , .

The asntdiff utility is used to compare the contents of two tables, primarily in the context of SQL and Q Replication.  The problem has been that very large tables can take a very long time to compare.  Moreover, in a dynamic replication environment, the data you are comparing might rapidly be changing.  There were attempts at fixing this problem, primarily by using the asntdiff -f parameter to provide a smaller range of rows to be compared.  Using the -f input file, SQL queries subset the data on a key value (e.g. account number) or transaction timestamp.  The bottom line is that many customers abandoned using asntdiff against production tables with millions or billions of rows.

Parallel asntdiff provides orders-of-magnitude improvements in performance.  The key differences between the processing of the legacy asntdiff and the parallel version are as follows:

  • Parallel processing: partitioning, difference detection, and reporting all in parallel
  • Compare multiple partitions in parallel (source and target table) and retrieve rows from each partition also in parallel from source and target
  • Minimize traffic (only send data needed)
  • Exploit computational capabilities of systems (checksum calculations, etc.)
  • Use a Stored Procedure (ASNTDSP) which runs at source and target systems
  • Avoid storing intermediate results

The results are dramatic.  A European bank killed a legacy asntdiff compare utility run after 17 hours; it might have run for 30.  Parallel asntdiff completed in 5 minutes.

There are some restrictions and advisories:

  1. The source and target tables must have the same code page and collation sequences.
  2. For optimal performance, increase the system resource limit for application threads and set NUMTHREADS to 21.  The minimal requirement is 6 threads.
  3. Since asntdiff can determine the parameter NUMBLOCKS automatically, grant the SELECT privilege on the following catalog tables: SYSIBM.SYSTABLES, SYSIBM.SYSKEYS and SYSIBM.SYSINDEXES.

There are others, but the topic is well worth exploring to speed up asntdiff – or to be able to use it at all.

Thanks to Jayanti Mahapatra and Xiao Li of IBM’s Silicon Valley Lab for some of the material presented here.

IBM Big Data, Integration and Governance Forum

Posted by Kim May on January 23, 2013 under Baltimore Washington DB2 Users Group, Big Data, DB2 Education, DB2 Gold Consultants, Frank Fillmore, IBM Information Management Software Sales, IBM Mid Market Customers, IBM Pure Systems, InfoSphere, Netezza, Q-Replication, TFG Blog. Tags: , , , , .

The Fillmore Group – and a special guest to-be-announced soon, will be be participating in the IBM Big Data, Integration and Governance Forum at the Sheraton Hotel in Reston, VA on March 5th.  The forum begins at 8am with a complimentary breakfast and sessions end at 1:15pm, followed by lunch provided by IBM.   There is no charge to attend the forum.

Great speakers are lined up to explain how best to leverage innovative techniques and gain a rock-solid understanding of how to:

  • Enhance the quality, availability and integrity of your data
  • Gain real-time insight into performance
  • Optimize and improve critical decision making
  • Reduce data management cost and complexity issues
  • Enforce data quality standards and stewardship policies
  • Effectively manage regulatory compliance demands
  • Leverage support for master data management (MDM) programs

Read More…

“Updating Netezza with Data from Q Replication”

Posted by Frank Fillmore on January 2, 2013 under DB2 for Linux Unix Windows, DB2 for z/OS, IDAA, InfoSphere, Netezza, Q-Replication.

In cooperation with IBM, The Fillmore Group performed a proof-of-concept for a large, well-known midwestern retailer in 2011 to demonstrate the use of IBM InfoSphere Data Replication (IIDR) Event Publisher (EP) feature to feed OLTP data into a Netezza appliance.  There is now a native changed data capture Apply component for Netezza built on technology IBM acquired from Data Mirror.  This provides for heterogeneous near-real-time OLTP replication to a Netezza appliance.  However – and this is a significant “however” – your shop might already be using Q Replication.  If you are replicating from a DB2 OLTP database, does it make sense to introduce a different replication technology just to keep your Netezza appliance current?

The answer is found in the developerworks post “Updating Netezza with Data from Q Replication“.  Built on the template developed by The Fillmore Group, IBM has published a java consumer that will provide mini-batches of comma delimited data files created by IIDR Event Publisher for ingestion into Netezza using native Netezza utilities.  Note that this approach works for both the new PureData System for Analytics as well as IBM DB2 Analytics Accelerator (IDAA) which are built on Netezza technology.

Kudos to IBMer Cecile Madsen of Silicon Valley Lab for her work on this technique.

First Half 2013 Classes Scheduled

Posted by Kim May on December 19, 2012 under Authorized Training Partner, DB2 Connect, DB2 Education, DB2 for Linux Unix Windows, DB2 for VSE&VM, DB2 for z/OS, InfoSphere, Q-Replication, TFG Blog. Tags: , , , , , .

The Fillmore Group has been delivering IBM Authorized Training classes in our Towson, Maryland classroom for the past several years and believe we offer some of the best DB2 training around – in a comfortable facility at an affordable travel location.  Our first half of 2013 schedule has recently been posted on the IBM website and a complete list of classes is available here.

In addition to the public, open enrollment classes we host on IBM’s behalf, we deliver an extended group of IBM classes directly to organizations as private classes.  The complete list of private classes available is attached here

If you or anyone on your team is looking to upgrade your DB2 skills in 2013 please let me know how I can assist.   Email me directly and I will help!

Unused 2012 Budget? Invest in DB2 Now!

Posted by Kim May on November 26, 2012 under Authorized Training Partner, DB2 Education, DB2 for Linux Unix Windows, DB2 for z/OS, DB2 Gold Consultants, DB2 Migrations, Frank Fillmore, IBM DB2 Services, InfoSphere, Netezza, Optim, Oracle, Q-Replication, TFG Blog. Tags: , , , , , .

Don’t wait – act now!  Invest in DB2 before year end.
 
With 2012 rapidly drawing to a close now is the time to review unused budget funds and invest in DB2 training and services.  The Fillmore Group has our DBA on Demand support option and IBM Authorized Training classes scheduled that can be paid for in advance and used to ensure your team is prepared for a successful 2013.
 
DBA on Demand offers expert consulting services, either onsite or remote, at a discounted hourly rate.  Our “150 for 150” offer delivers 150 hours of support at a discounted hourly rate of $150 per hour.
 
There are still seats available for the December 10th CL412/CL442 DB2 for LUW Performance Tuning and Monitoring Workshop to be delivered by DB2 Gold Consultant Frank C. Fillmore, Jr., in our Towson, Maryland classroom.  Additional TFG classes scheduled are available on the IBM Learning Services website.
 
Please feel free to contact me with any questions on how The Fillmore Group can help preserve your unused funds and help your team have a successful 2013 with DB2!

DB2 December Renewal? Stop! Evaluate AESE First!

Posted by Kim May on November 15, 2012 under DB2 Education, DB2 for Linux Unix Windows, DB2 Gold Consultants, Frank Fillmore, IBM Champion, IBM DB2 Services, IBM Information Management Software Sales, Q-Replication, TFG Blog. Tags: , , , , , , , .

Given that so many IBM customers make their initial software purchases in the fourth quarter of the calendar year (when the IBM pricing gets really aggressive), the number of DB2 customers whose annual software license subscription and support is due each December is higher than in any other month of the year.  For this reason we’ve decided to deliver another webinar on the DB2 Advanced Enterprise Server Edition (AESE) bundle in hopes of persuading renewing DB2 customers to consider the upgrade.  AESE has provided substantial value to every upgrading customer we’ve worked with, and enabled their IT organization to better monitor, utilize, and extend the use of their database. 

Frank Fillmore will deliver the webinar on Monday, November 19th from 11am – 12pm EST.  Register HERE.

The invitation, in case you didn’t receive one directly from me (!) –

If your organization’s DB2 license Subscription and Support is scheduled to renew in December, now is the time to learn about the DB2 Advanced Enterprise Server Edition (AESE) product bundle and how you can finesse the upgrade JUST BEFORE paying IBM the annual Subscription and Support fee to make it VERY affordable. Read More…

We Know You Deserve It, But…

Posted by Kim May on November 2, 2012 under DB2 Stars, IBM DB2 Services, Information on Demand Conference, Optim, Q-Replication, TFG Blog. Tags: , , .

I submitted this blog post to the IOD blog (IOD blog) as I was leaving the conference:

At this year’s Information On Demand conference I had the pleasure to catch up with three IBMers who have endorsed, supported and yes, tolerated The Fillmore Group in various ways for many years. Each of these three wonderful IBMers announced their participation in the IBM “Road to Retirement” program and shared some of their post-IBM plans. I wish them all the very best, while at the same time am saddened to see such talented and dedicated experts leave IBM.

Sandy Ovard has been working as Business Partner liaison to Lab Services and has helped us navigate the increasingly complex Lab Services partner engagement process. As we depend on Lab Services for a portion of our services business, Sandy has had the frequently unpleasant job of soliciting Business Partner feedback and addressing issues. Her patience and consistent willingness to pitch in will be missed.

Holly Hayes is one of the SVL development superstars who has worked with a number of technologies. Holly is one of the few “surf and turf” people who fully understand the most technical of technical subjects and has the communication skills to explain them to those non-techies, like me, who need analogies and pictures to understand. I am have been surprised – and amazed – on more than one occasion when I hear about a customer that struggled with a new product implementation and hear, “and then I got Holly involved,” and invariably, things smooth out.

Finally, David Tolleson. David has been the replication and federation Product Manager and a tireless promoter of these solutions. David has blogged, responded to endless emails, and enthusiastically supported a niche technology that enables some of the world’s largest IT shops to meet those 24X7 uptime SLA’s. He has worked with a team that is not only technically capable, but fun to work with.

Good luck to you all!

Q Replication Virtual Users Group – Over 100 Registered!

Posted by Kim May on August 3, 2012 under Authorized Training Partner, DB2 Education, Frank Fillmore, Q-Replication, TFG Blog. Tags: , , , .

We have over 100 Q Replication users registered to attend the upcoming Virtual Users Group session, scheduled for August 9th (see the post below).  The Fillmore Group, as an IBM Authorized Training Partner, delivers the IBM public training class for Q Rep, when it runs, which is hardly EVER!  

Great products deserve great support.  I am happy we can help Q Rep users gain better skills.

If you would like to join us the registration link is in the post below.