IBM’s flagship enterprise relational database Db2 has just been added to AWS Relational Database Service (RDS). Db2 joins commercial vendor relational database implementations Oracle and MS SQL Server along with open source databases MySQL, MariaDB, and PostreSQL as a fully-managed AWS offering.

Why this matters:

• Deployment I’m currently working with a large commercial vehicle manufacturer. Over two weeks ago I requested that a small Db2 database be deployed for query optimization. The clock is still ticking and the database still isn’t available. With RDS Db2 can be deployed and configured in minutes.
• Horizontal scaling Many of The Fillmore Group’s clients across several different industries have significant variability of demand – days of the week, times of the month, and seasonal variations. For large retailers purchasing ramps up in the summer, staffing in the early fall, sales in late fall and early winter, and returns in January. Managing on-prem capacity in such an environment usually means expensive compute and memory is underutilized for several months in the spring and summer. RDS provides on-demand scale up and scale down.
• Vertical scaling A large industrial client provisioned Db2 to store sensor data that will be analyzed for anomalies. Within six months one such database was at 85 Tb and continues to grow. Capacity growth is both predictable and linear, but even metronomically adding compute, memory, and storage is both budget and labor intensive.

Two more things to consider:

1. IBM invented the relational database. A Relational Model of Data for Large Shared Data Banks was published by Edgar F. Codd, IBM Research, in 1970 in the Communications of the Association of Computing Machinery (ACM). Trillions of dollars of value has been realized by organizations which have deployed IBM Db2 or other relational database variants since they became available in the 1980s.
2. Many enterprises have successfully migrated workloads from other-vendor relational database implementations. Here’s just one example: SmarterQuestions White Paper – Oracle to DB2 Migration Lessons Learned – Final

Greetings after a little while.

For those of you who have been working with IBM’s data virtualization technology – as I have – since the beginning (we miss you DataJoiner, Federation Server, et al), there’s yet another new name: Watson Query a service of Cloud Pak for Data (CP4D).

DataJoiner began as a research project (code name: “Garlic”) led by then-IBMer Nelson Mattos. The goal was to provide heterogeneous data access via a Db2 for LUW “gateway”. Enterprises in the 1980s began deploying lots of discrete data repositories (primarily Oracle, Sybase, and MS SQL Server along with Db2) to serve line-of-business and departmental applications (e.g. sales, accounting, finance, manufacturing, logistics). The ability to access data from multiple disparate servers to provide a comprehensive view of business processes was a significant advance. DataJoiner was not limited to RDBMS repositories, but could also retrieve data from MS Access, MS Excel, CSV and Text files… pretty much any data on persistent storage with a definable structure.

For a while DataJoiner also provided increased performance and reduced expense benefits as well. IBM sold DataJoiner as a front-end to Oracle because (at least in some cases) a query using the cost-based Db2 optimizer embedded in DataJoiner produced a better access path than the, at that time, rules-based Oracle optimizer. Expenses were reduced due to the per-seat Oracle license charges; DataJoiner was just “one seat” which could support hundreds of concurrent users. Over time Oracle has modified both their optimizer and licensing model.

For those of you who geek-out on the roots of the technology that we apply to common data administration challenges in our day jobs – as I do – you’ll find a seminal paper on the IEEE website: DataJoiner: a practical approach to multi-database access

So why the history lesson?

• At its roots Watson Query continues to use much of the original federation “plumbing”: Wrappers, Servers, Nicknames, etc.
• Watson Query no longer relies on point-to-point connections to the back-end data repositories, but exploits a “computational mesh” which provides increased performance and resiliency. This approach leverages third-party data repository vendor advances in parallelism, caching, and compression. All of this is of enormous importance when enterprise data is distributed not just across multiple in-house geographically disparate locations, but perhaps across multiple cloud vendors as well.

• BUT (there’s usually a but), due to engineering and design within CP4D some of the capabilities in IBM’s legacy federation stack have not yet been incorporated into Watson Query. Two are of particular interest – which is why I have posted two Ideas (the successor to IBM Request for Enhancement – RFE):
  1. Formerly for data repositories with Indexes that metadata would automatically be captured for a Remote Table in, say, Oracle and propagated back to the Db2 SYSCAT.INDEXES catalog view for use by the Db2 optimizer to help build the best access path for the Remote Table represented by the Nickname. Right now that’s not being done. Hence Idea ASQL-I-16 “Propagation of Indexes for Virtualized Tables to SYSCAT.INDEXES”.
  2. Federated queries have always used Db2’s SQL dialect. There is a PASSTHRU capability that enables users to employ the native SQL dialect of the back-end data repository. For certain queries this can produce significant performance benefits. See Idea ASQL-I-17 “Federated PASSTHRU Capability in Watson Query (WQ)”.

Please review both of these Ideas and vote!

https://ibm-data-and-ai.ideas.ibm.com/ideas/ASQL-I-16

https://ibm-data-and-ai.ideas.ibm.com/ideas/ASQL-I-17

Thanks in advance.

#IBM #WatsonQuery #DataVirtualization #Federation

IBM Data Replication 2020: In a Nutshell

IBM replication is based on two unique technical solutions: InfoSphere Change Data Capture, the heritage DataMirror product, and Q-Replication, the IBM-developed replication solution that uses MQ messaging.

IBM replication supports numerous sources and targets. For heterogeneous environments, that is, where either the source or target are not Db2, IBM recommends InfoSphere Change Data Capture. For Db2 to Db2 replication, IBM recommends Q Replication.

The currently available sources are targets are below. New offerings are always being added; for a current list visit the IBM replication homepage.

New offering highlight: Mainframe replication for Db2 for z and VSAM featuring “Remote Capture”, with the replication engine installed off the mainframe. The new solution delivers easier installation and administration as well as lower MIPs consumption to support replication from System z.

Coming soon: Refreshed monitoring tools built on Watson Knowledge Catalog, enabling expanded governance and integration with replication monitoring.

Implementation support, healthchecks and education:  Contact The Fillmore Group. We have several Quickstart offerings and have been delivering the IBM replication education offerings for decades.

Replication Sources: Db2 (LUW, Z/OS, iSeries), Oracle, SQL Server, PostgreSQL, VSAM, IMS, Informix, Sybase, MySQL.

Replication Targets: All above RDBMSs plus Teradata, Kafka, Hadoop, Files, Information Server (DataStage/QualityStage), Cloudant, AWS RedShift, S3, Azure SQL DB, JDBC targets (MongoDb, etc.), IBM IAS/ Db2WH/Db2WHoC, Netezza/Cloud Pak 4 Data System, and more.

Remote Capture Quickstart

Postgres Capture Quickstart

Kafka Replication Quickstart

Thanks to my colleagues, Kim May and Ed Lynch, for a terrific IBM InfoSphere Data Replication (IIDR) new features webinar delivered today.  The recording of the webinar is found here.    These are the presentation materials: Data Replication Update 2019-02-07 – Final

The Call for Speakers for the 2019 IDUG North America Conference, scheduled for Charlotte, NC, from June 2 – 6, 2019, is open.  For Db2 users this is an excellent opportunity to share your experience with your peers.  IDUG has a well documented presentation submission format and process, detailed on the IDUG North America webpage.

You may submit an unlimited number of abstracts.  However, it is important that you provide all of the information requested when submitting your presentation(s) for review. Failure to provide all requested information may impact whether or not the abstract is accepted.

One last note:  given the timing, don’t forget to reach out to your local Db2 RUG and offer to deliver your presentation at one of their meetings prior to the conference.  It’s a great opportunity to do a practice run, and also allow your local peers – some of whom may not be attending the conference – to appreciate your work.

From the IDUG webpage:

“IDUG is soliciting presentation abstracts that explore into details of implementation, migrating to a new release, or maximizing performance from any member of the DB2 product family. In recognition of the complex requirements of today’s database implementations, presentation abstracts relating to migration from Oracle to DB2, upgrading within versions of DB2, and new hardware implementations are also being requested. Presentations may cover details of DB2 tools, Big Data and other ancillary DB2 products.

All presentations must be strongly technical in nature, cannot contain marketing themes, and must support and strengthen the information services community by providing the highest quality education and services designed to promote the effective utilization of the DB2 and its family of products.”

As always the holidays are over too soon.  Here at The Fillmore Group we are back to work, starting the year with the usual goal setting and head scratching.  Three initiatives we worked on in 2017 showed a great deal of promise but never seemed to quite gain the traction we had hoped for.  So, unlike in prior years when I would start with a blank page, I am committing myself to pushing these three harder in 2018, as all three offer terrific options to maximize the value of an organization’s data assets.  These are:

Oracle to Db2 database migrations.  Last year IBM offered discounted Db2 licenses to current Oracle customers – and when I say “discounted” I mean practically gave them away.  Frank Fillmore and I delivered several competitive migration presentations and got lots of positive feedback from Oracle customers.  Still, we didn’t see a lot of Db2 adoption from these folks and I’m sorry – for them and us.  This is a head-scratcher, because I’m not sure where we are getting snagged, but we need to figure it out in 2018.

IDAA/IIAS/HTAP vs. “edge systems”.  Admittedly 2017 was a bit of a transition year for the IBM Db2 Analytics Accelerator (aka the IDAA) as IBM completely revamped the design, replacing the Netezza-bolted-to-a-mainframe with 2 Db2 BLU Acceleration-based options to deliver the Hybrid Transaction Analytic Processing (HTAP) capability.  Meanwhile in 2017 analysts were predicting that edge systems are the future – sounding frighteningly like they did in the late 1990’s when the standalone data warehouse was the next big thing.  It makes absolutely no sense to me to build out a completely new system, particularly on the periphery of an organization’s most secure system, to perform analytics when IBM has clearly demonstrated you can leave the data in place and perform both your OLTP and analytic processing in tandem.  We need to continue to educate customers to ensure they understand the overall value – reducing complexity, reducing cost, and keeping data secure.  The new BLU-based version has a new name:  IBM Integrated Analytics System – IIAS.

Supporting the Db2 eco-system.  This is such a no-brainer but one that needs constant attention, effort and input to maintain.  IBM formal classroom education offerings are minimal.  Conferences – IBM and IDUG – are changing and it’s difficult to determine which is most appropriate to attend.  Users groups (including the Balto/Washington Db2 Users Group) are struggling to provide our users with unbiased product information, education and resources.  We aren’t giving up and hope the greater Db2 community continues to work to stay active and involved within the community and within our individual organizations.  If we want to stay relevant we need to stay involved.

Happy 2018!  Off to work!

Sirius/XM already has the Holly channel playing Christmas music and Halloween was just a couple weeks ago. It’s way too early!  But…the clock IS ticking and there really are only 4 solid weeks left in 2017 to close some of those, “I’ll get to them later” deals.

This Thursday The Fillmore Group will deliver a webinar for IBM sellers willing to consider collaborating with a Business Partner.  We understand many sellers have heard about – or experienced – unsuccessful teaming with a partner.  Join us to overcome your fears!

In this session we will *try* to explain that not all partners are alike, and some (like The Fillmore Group) can help you close deals by extending your reach and helping with challenging technical situations (which product is the “right” one?), arcane licensing and packaging research, and using an implementation discussion to address a customer’s concerns about the cost of services and education (yes Oracle takeout opportunities, I’m talking about you!)

IBM program challenges that impact Business Partners will also be explained, so that IBM sellers understand why partners get so antsy about keeping the lines of communication wide open – and getting added to sales records.

All in 30 fast minutes.

If you are an IBM seller who would love to close an extra deal or two before year end, please join us.  If you aren’t an IBM seller, please join us for another webinar; this one is just for sellers.

The session will take place on Thursday, November 16th from 11:00am – 11:30am EST.  Register HERE.

I really enjoyed Lisbon and seeing everyone at the IDUG Conference.  More than one person commented on the turnout (excellent) and the crowd (young!)  There seems to be some new energy in the Db2 community, in part, I believe, as long-time Db2 users are accepting that in spite of IBM’s perpetual “re-invention”, the folks using their tried-and-true technologies – the ones that built up the war chest funding the re-invention – continue to deliver state-of-the-art performance and functionality.  The technology is excellent, users are lucky to work with it, and the experts who have invested their best career years in Db2 are ready to work.

So now what?  With IBM’s current “Oracle Attack” promotion customers can purchase Db2 for as little as 20% of their annual Oracle support cost.  I understand there are several Oracle customers preparing to migrate to Db2.  Does this mean Db2 will (finally) show up on one of those “most popular relational database” graphs?  I hope so!

And, along with these migrations the demand for services to support them is growing, so there are more opportunities for customers to get to know the many skilled Db2 consultants and experts available to support their adoption of Db2.

Independent consultants – like The Fillmore Group’s team – can deliver expert services to support migrations, often at a fraction of the cost of IBM’s.  Many consultants have experience delivering formal classroom training as well as services, and most are actively involved in the Db2 community and happy to introduce Db2 newbies to the local users groups and IDUG.  The consulting community has been promoting the Oracle Attack through webinars (view the replay of my webinar here) and blog posts (read Dave Beulke’s post here), and want to make sure every prospective Db2 customer has access to skilled migration services when they are ready to make the switch.

Call us!  We want to support the growth of the Db2 community, and we’re ready to work.

Thanks to my colleague, Kim May, for another wonderful job leading customers and IBMers through the rewards and challenges of converting Oracle databases to DB2. The recording of the latest webinar is found here.  The presentation materials: Oracle Customers Consider DB2 5.16.17

Finally, The Fillmore Group’s White Paper on Oracle to DB2 migrations: SmarterQuestions White Paper – Oracle to DB2 Migration Lessons Learned – Final

Please reach out to Kim or me if you have any questions.

With IBM’s announcement this week of super-aggressive discounting being offered to current Oracle customers considering a move to DB2, and with the deadline looming for many Oracle customers faced with another year of bloated “investment” in Oracle support, we’ve decided to deliver updated material on the benefits of moving to DB2.

Our upcoming webinar, “Oracle Renewal Looming?  Consider DB2!” will be delivered on Tuesday, May 16th, from 11:30am to 12:30pm EDT.

DB2 Gold Consultant Frank Fillmore will join me for the session, as will IBM’s Tony Mancini, Worldwide Competitive Team Executive.  Tony will give us a brief overview of IBM’s recent announcement, and Frank and I will deliver a fast-paced, interactive 60-minute session that will (we hope!) provide current Oracle customers with an understanding of both the cost savings and incredible database functionality available with DB2.

Please register HERE to join us!