IBM Data Replication 2020: In a Nutshell

IBM replication is based on two unique technical solutions: InfoSphere Change Data Capture, the heritage DataMirror product, and Q-Replication, the IBM-developed replication solution that uses MQ messaging.

IBM replication supports numerous sources and targets. For heterogeneous environments, that is, where either the source or target are not Db2, IBM recommends InfoSphere Change Data Capture. For Db2 to Db2 replication, IBM recommends Q Replication.

The currently available sources are targets are below. New offerings are always being added; for a current list visit the IBM replication homepage.

New offering highlight: Mainframe replication for Db2 for z and VSAM featuring “Remote Capture”, with the replication engine installed off the mainframe. The new solution delivers easier installation and administration as well as lower MIPs consumption to support replication from System z.

Coming soon: Refreshed monitoring tools built on Watson Knowledge Catalog, enabling expanded governance and integration with replication monitoring.

Implementation support, healthchecks and education:  Contact The Fillmore Group. We have several Quickstart offerings and have been delivering the IBM replication education offerings for decades.

Replication Sources: Db2 (LUW, Z/OS, iSeries), Oracle, SQL Server, PostgreSQL, VSAM, IMS, Informix, Sybase, MySQL.

Replication Targets: All above RDBMSs plus Teradata, Kafka, Hadoop, Files, Information Server (DataStage/QualityStage), Cloudant, AWS RedShift, S3, Azure SQL DB, JDBC targets (MongoDb, etc.), IBM IAS/ Db2WH/Db2WHoC, Netezza/Cloud Pak 4 Data System, and more.

Remote Capture Quickstart

Postgres Capture Quickstart

Kafka Replication Quickstart

Several people who registered for The Fillmore Group’s Columbus IDAA class in July were unable to attend, so we’ve decided to borrow the “Db2 Night Show” format, with Frank hosting the “IDAA Afternoon Show” Wednesday, August 28^th from 4:00 p.m. – 5:30 p.m. EDT.

Frank will deliver a fast-paced, content-rich 90-minute webinar covering:

1. A review of capabilities included in recent IDAA v7.1.x code drops
2. The return of the High Performance Storage Saver (HPSS)
3. Integrated Synchronization
4. The closer-to-real Hybrid Transaction Analytic Processing (HTAP) capability
5. Disaster Recovery and High Availability considerations

And as an added bonus we will step all registered attendees through The Fillmore Group’s exclusive IDAA form-factor Buying Guide to compare:

• IBM Integrated Analytics Systems (IIAS)
• Docker container running on System z
• Docker container running on LinuxONE

We’ll do this via webinar, so come-one-come-all.  We’ll upload the materials we used in the Columbus class so anyone who did not register for class but attends the webinar can access them.

I hope you will join us if your schedule allows, and invite any colleagues who may be interested in the material.

Registration is required.  Please register HERE.

Join us for this new and improved for 2019 class to be delivered by IBM Gold Consultant Frank Fillmore. Contents include IDAA v7 early experiences, best practices, and v5 to v7 migration tips. Attendees will learn how an IDAA v7 (aka “Sailfish”) can extend System z’s processing capabilities to enable Hybrid Transaction/Analytic Processing (HTAP).  For IBM System z customers considering an IDAA purchase, this class provides an overview of the architecture, use cases delivering value, and insight into the administration of the accelerator.

Class is free to IBM System z customers considering an IDAA purchase.  The Fillmore Group will provide breakfast and lunch each day; attendees are responsible for their own travel and lodging expenses.

Registration is required – Register HERE

Dates (2 days):            Thursday and Friday, July 18 and 19, 2019

Location:                      IBM, 6060 Britton Parkway, Building 1, Dublin, Ohio, 43016

Time:                           9:00am – 4:00pm EDT

Cost:                            Class is *free* to pre-registered attendees

Instructor:                    Frank Fillmore

Topics:

• An introduction to the V7.1 architecture and Integrated Analytics System (IAS).
• The heterogeneous accelerator environment: what works well today.
• Assessment tools to support acquisition: Workload Assessment, Client Value Engagement, PoC.
• Positioning IDAA within your organization with the most valuable use case.
• Migrating from the Netezza-based IDAA to v7.1.

Lab 1: Basic IDAA Activities

• Data Studio / IBM Db2 Analytics Accelerator Studio GUI
• Table acceleration
• Query processing and analysis

Enhancing Value to the IDAA with the IBM Tools Ecosystem

• Assessment: What is the workload that would benefit from being accelerated?
• Optimization: How do I monitor and optimize a workload to take advantage of the Accelerator?
• Administration: Can I manage the Accelerator more effectively?

Lab 2: Advanced IDAA Exploitation

• Incremental Update: keeping accelerated data in sync with DB2 for z/OS OLTP
• Accelerator-only Tables (AoTs): intermediate tables and ETL processing

On July 23 and 24, 2018 The Fillmore Group delivered a hands-on IBM Db2 Analytics Accelerator (IDAA) v7 Workshop to 12 students across 5 different enterprises at the IBM Dublin, Ohio (USA) Technical Education Center.  This was the first IDAA v7 hands-on Workshop delivered to customers anywhere.  Many thanks to Atlee, Bill, Bob, Dom, Emmi, Larry, Marshall, Mohammed, Naga, Ruth, Shawn, and Toby for all of the great questions.  Kudos to the IBM Poughkeepsie Benchmark Center for standing up the IDAA v7 environment in record time with just-released functionality like Incremental Update.  And thanks to IBMers Rich Gast, Mike Hood, Sandro Ramirez (aka “Nicky’s Dad”), Julie Efthymiou, and Rudy Benvenuto for all of their support.

Finally – and most importantly – the Workshop’s flawless execution from scheduling, invitations and outreach to customers, to welcome packets for travelers, to food service, and overall logistics was the direct result of many months of hard work on the part of my colleague, Kim May.  I really appreciate it!

The presentation materials are here:IDAA Workshop 2018-07

Since IDAA v7 is so radically different from it’s predecessors, the Workshop generated lots of questions which will be incorporated into future sessions.  Watch this space.

Gold Consultant and Z Systems Champion Frank Fillmore will deliver a free, hands-on IBM Db2 Analytics Accelerator/IDAA class on Monday and Tuesday, July 23^rd and 24^th, at the IBM Dublin Campus near Columbus, Ohio.

Class has been updated to showcase the new IDAA V7.1 (“Sailfish”) and covers the new architecture, sizing considerations for the new generation of IDAA appliances, and recommendations for organizations upgrading from the Mako (V5) to Sailfish.

The fine print:  attendees must be IBM customers considering an IDAA purchase in 2018.  While class is free (and we will treat attendees to breakfast and lunch both days), attendees are responsible for their own travel and lodging expenses.

We have 20 available class seats, available on a first-come-first-served basis.  The registration link is here and the course outline is below. I will send travel information and recommendations to everyone registering (I’m working on them now!)

Finally, if there is any additional information needed please feel free to email me at kim.may@thefillmoregroup.com.

IBM Db2 Analytics Accelerator Training

Dates (2 days):  Monday and Tuesday, July 23rd and 24th, 2018

Class Times:  9:00am – 4:00pm

Cost:  Class is *free* to pre-registered attendees

Instructor:  Frank Fillmore

Join us for this refreshed class that includes material specific to the latest IDAA “Sailfish” releases, including V7.1, and the M4002, aka “Maverick”. Attendees will learn how and why an IDAA can extend your System z’s processing capabilities to enable Hybrid Transaction/Analytic Processing (HTAP).

Topics:

• An introduction to the new V7.1 architecture and IBM Integrated Analytics System (IIAS).
• The heterogeneous accelerator environment:  what works well today.
• Assessment tools to support acquisition:  Workload Assessment, CVE, PoC.
• Positioning IDAA within your organization with the most valuable use case.
• Migrating from the Netezza-based IDAA to V7.1.

Lab 1: Basic IDAA Activities
– Data Studio / IBM Db2 Analytics Accelerator Studio GUI
– Table acceleration
– Query processing and analysis

Enhancing Value to the IDAA with the IBM Tools Ecosystem
– Assessment: What is the workload that would benefit from being accelerated?
– Optimization: How do I monitor and optimize a workload to take advantage of the Accelerator?
– Administration: Can I manage the Accelerator more effectively?

Lab 2: Advanced IDAA Exploitation
– Incremental Update: keeping accelerated data in sync with DB2 for z/OS OLTP
– Accelerator-only Tables (AoTs): intermediate tables and ETL processing

On July 23, 2010 I attended the announcement of the IBM Smart Analytics Optimizer (ISAO).  This was IBM’s first iteration of heterogeneous scale-out to enable mixed OLTP and OLAP workloads to run on the same platform.  Subsequently IBM acquired Netezza and the Netezza appliance became the foundation for the IBM Db2 Analytics Accelerator (IDAA).  In IDAA workshops I would refer to the ISAO as IDAA v0.5.

Fast forward to today when IBM announced IDAA v7.1.  “What is old is new again.”  Think of this delivery of IDAA as “ISAO v4.0” (there was some good-natured debate about the numbering).  Why do I frame it those terms?

1. IDAA has received a “brain transplant”.  Instead of Postgres, the database engine for IDAA will be Db2 Warehouse.  This will give IDAA greater compatibility with Db2 for z/OS, including broader SQL and datatype support, TIMESTAMP consistency, and more.  By exploiting Db2 with BLU Acceleration features such as in-memory columnar (rather than row-organized) data, Single Instruction, Multiple Data (SIMD) chips, actionable compression, and “data skipping” (avoiding extents that don’t contain data of interest) IDAA v7.1 provides a robust platform for better performance with a roadmap for continuous improvement and enhancement.  IBM has also consolidated its data repository development efforts by focusing on fewer, higher-value initiatives.
2. IDAA has received a “heart transplant”.  The IBM Integrated Analytics System (IIAS) appliance on which IDAA v7.1 is based, no longer depends on Field Programmable Gate Arrays (FPGA) that were the “secret sauce” of the Netezza (aka PureData System for Analytics – PDA).  FPGAs might be reintroduced at a later time.

So, a few questions – and a fewer answers.

• How long will the legacy Netezza/PDA “Mako” N3001 be supported?  Until 2023.  So the installed base of existing IDAAs will be protected for five years.
• Will the Database Partitioning Feature (DPF) – IBM’s MPP shared-nothing data warehouse scaling technology – be exploited by IDAA v7.1?  Dunno.
• Wither the IDAA cloud deployment introduced in v6?  Nary a mention today.

Topics for a future blog post:

• “True HTAP” – Hybrid Transaction / Analytic Processing
• IDAA deployment on IBM z – using Docker images sitting on Integrated Facility for Linux (IFL) specialty processors.

When I started working for The Fillmore Group over ten years ago, the world of DB2 was far different.  The State of Maryland had probably over a dozen IBM mainframes running state government agencies.  We occasionally delivered training and support, but often struggled with the procurement processes, as we weren’t an approved vendor.

As I learned, the master vendor list for IT vendors to the State of Maryland adds new vendors only once every 8 to 10 years, as the qualification process (we are talking government here!) is onerous and terribly time consuming for all involved.  So I waited and finally this past year it was time again for open enrollment.  The qualification process WAS onerous and time consuming and required the assistance of Maryland Governor Hogan’s office to make the insurance requirements attainable for small businesses like The Fillmore Group.

Many thanks Governor Hogan!  Today The Fillmore Group received the fully executed contract to deliver IT services to the State of Maryland.   There aren’t as many mainframes as there were ten years ago and efforts have been underway to consolidate their workloads, but we can help.  I think a private cloud running System z with a couple shared IDAA’s would be cost effective, efficient, and modern.  Can’t wait to get started!

Please join us Thursday, July 28th, from 11am-12pm EDT for a free webinar with an update on IBM’s Q Replication.

Having worked with Q Replication customers since 2006, DB2 Gold Consultant and IBM Champion Frank Fillmore has collected useful tips and techniques for Q Replication implementation and administration. Join Frank as he shares his notes and thoughts on the topics below and delivers a mid-2016 checkpoint on the evolving IBM InfoSphere Data Replication (IIDR) product development roadmap.

In this session Frank will cover the topics below.  Please register for the session here.

1. Cold Start a Q Replication topology.
   1. When and why would you need to consider a Cold Start.
   2. What are the steps?
2. Oracle Native Q Apply – replication directly to an Oracle database (including RAC and Exadata).
   1. Comparison to Federated Apply.
   2. Co-existence and migration.
3. Tooling – what tool do I use when?
   1. Q Replication Dashboard.
   2. ASNMON.
   3. Replication Center.
   4. ASNCLP.
4. ASNCONSUMER – used to replicate to PureData System for Analytics (Netezza).
5. Pricing models.

About Frank Fillmore:

DB2 Gold Consultant and IBM Champion Frank C. Fillmore, Jr. is the Founder and President of The Fillmore Group, Inc., a Premier IBM Business Partner that specializes in IBM Analytics software sales, consulting, and training.  Frank has been providing IBM database-oriented solutions to government, industry, and not-for-profit enterprises as varied as The World Bank, J.P. Morgan Chase, Northrop Grumman, and the FBI for over 29 years.  Frank’s current practice centers on Hybrid Transaction and Analytic Processing (HTAP) and Oracle to DB2 migrations as well as information governance and integration.  Frank kayaks and cheers for the Baltimore Orioles.

There are still seats available for the June 6th and 7th *FREE* hands-on IDAA training class, scheduled for delivery in the IBM Chicago offices.  DB2 Gold Consultant Frank Fillmore will be delivering this class.  Frank has delivered this class several times and the feedback has been consistently positive, as the class helps customers understand the broad range of deployment options and use cases, as well as how it works.

If you are interested in joining us please review the course outline and register HERE.  Please note:  this class is available to prospective IDAA customers only and registration is required.  If you have any questions on the course content or logistics, please contact Kim May.  I hope to see you in Chicago!

DB2 Gold Consultant Frank Fillmore will deliver a free, hands-on IBM DB2 Analytics Accelerator (IDAA) class at the IBM offices in Chicago at 71 South Wacker Drive on Monday and Tuesday, June 6th and 7th.  The course outline and registration link are HERE.   The Welcome Packet with travel and lodging recommendations will be sent once the registration is accepted.

Class is free, however students are responsible for their own travel and lodging expenses.  The Fillmore Group will provide breakfast and lunch each day to students in class.

The IDAA is a high-performance appliance that integrates IBM Netezza and zEnterprise technologies. The solution delivers extremely fast results for complex and data-intensive DB2 queries on data warehousing, business intelligence and analytic workloads.  You can:

– Query data at high speeds-to significantly improve response times for unpredictable, complex and long-running query workloads.

– Extend the capabilities of DB2 for z/OS-to support a cost-effective analytics solution for data warehousing, business intelligence and predictive analytics.

– Lower operating costs-by reducing System z disk requirements and offloading query workloads to a high-performance platform.

Please note:  this class is offered exclusively to IBM z Systems customers considering an IDAA purchase.  Registration is required.

I hope you will join us and learn more about the IDAA! If you have any questions on the class, please feel free to contact me at kim.may@thefillmoregroup.com.