IBM’s flagship enterprise relational database Db2 has just been added to AWS Relational Database Service (RDS). Db2 joins commercial vendor relational database implementations Oracle and MS SQL Server along with open source databases MySQL, MariaDB, and PostreSQL as a fully-managed AWS offering.

Why this matters:

• Deployment I’m currently working with a large commercial vehicle manufacturer. Over two weeks ago I requested that a small Db2 database be deployed for query optimization. The clock is still ticking and the database still isn’t available. With RDS Db2 can be deployed and configured in minutes.
• Horizontal scaling Many of The Fillmore Group’s clients across several different industries have significant variability of demand – days of the week, times of the month, and seasonal variations. For large retailers purchasing ramps up in the summer, staffing in the early fall, sales in late fall and early winter, and returns in January. Managing on-prem capacity in such an environment usually means expensive compute and memory is underutilized for several months in the spring and summer. RDS provides on-demand scale up and scale down.
• Vertical scaling A large industrial client provisioned Db2 to store sensor data that will be analyzed for anomalies. Within six months one such database was at 85 Tb and continues to grow. Capacity growth is both predictable and linear, but even metronomically adding compute, memory, and storage is both budget and labor intensive.

Two more things to consider:

1. IBM invented the relational database. A Relational Model of Data for Large Shared Data Banks was published by Edgar F. Codd, IBM Research, in 1970 in the Communications of the Association of Computing Machinery (ACM). Trillions of dollars of value has been realized by organizations which have deployed IBM Db2 or other relational database variants since they became available in the 1980s.
2. Many enterprises have successfully migrated workloads from other-vendor relational database implementations. Here’s just one example: SmarterQuestions White Paper – Oracle to DB2 Migration Lessons Learned – Final

Greetings after a little while.

For those of you who have been working with IBM’s data virtualization technology – as I have – since the beginning (we miss you DataJoiner, Federation Server, et al), there’s yet another new name: Watson Query a service of Cloud Pak for Data (CP4D).

DataJoiner began as a research project (code name: “Garlic”) led by then-IBMer Nelson Mattos. The goal was to provide heterogeneous data access via a Db2 for LUW “gateway”. Enterprises in the 1980s began deploying lots of discrete data repositories (primarily Oracle, Sybase, and MS SQL Server along with Db2) to serve line-of-business and departmental applications (e.g. sales, accounting, finance, manufacturing, logistics). The ability to access data from multiple disparate servers to provide a comprehensive view of business processes was a significant advance. DataJoiner was not limited to RDBMS repositories, but could also retrieve data from MS Access, MS Excel, CSV and Text files… pretty much any data on persistent storage with a definable structure.

For a while DataJoiner also provided increased performance and reduced expense benefits as well. IBM sold DataJoiner as a front-end to Oracle because (at least in some cases) a query using the cost-based Db2 optimizer embedded in DataJoiner produced a better access path than the, at that time, rules-based Oracle optimizer. Expenses were reduced due to the per-seat Oracle license charges; DataJoiner was just “one seat” which could support hundreds of concurrent users. Over time Oracle has modified both their optimizer and licensing model.

For those of you who geek-out on the roots of the technology that we apply to common data administration challenges in our day jobs – as I do – you’ll find a seminal paper on the IEEE website: DataJoiner: a practical approach to multi-database access

So why the history lesson?

• At its roots Watson Query continues to use much of the original federation “plumbing”: Wrappers, Servers, Nicknames, etc.
• Watson Query no longer relies on point-to-point connections to the back-end data repositories, but exploits a “computational mesh” which provides increased performance and resiliency. This approach leverages third-party data repository vendor advances in parallelism, caching, and compression. All of this is of enormous importance when enterprise data is distributed not just across multiple in-house geographically disparate locations, but perhaps across multiple cloud vendors as well.

• BUT (there’s usually a but), due to engineering and design within CP4D some of the capabilities in IBM’s legacy federation stack have not yet been incorporated into Watson Query. Two are of particular interest – which is why I have posted two Ideas (the successor to IBM Request for Enhancement – RFE):
  1. Formerly for data repositories with Indexes that metadata would automatically be captured for a Remote Table in, say, Oracle and propagated back to the Db2 SYSCAT.INDEXES catalog view for use by the Db2 optimizer to help build the best access path for the Remote Table represented by the Nickname. Right now that’s not being done. Hence Idea ASQL-I-16 “Propagation of Indexes for Virtualized Tables to SYSCAT.INDEXES”.
  2. Federated queries have always used Db2’s SQL dialect. There is a PASSTHRU capability that enables users to employ the native SQL dialect of the back-end data repository. For certain queries this can produce significant performance benefits. See Idea ASQL-I-17 “Federated PASSTHRU Capability in Watson Query (WQ)”.

Please review both of these Ideas and vote!

https://ibm-data-and-ai.ideas.ibm.com/ideas/ASQL-I-16

https://ibm-data-and-ai.ideas.ibm.com/ideas/ASQL-I-17

Thanks in advance.

#IBM #WatsonQuery #DataVirtualization #Federation

Over 180 folks attended the “45 Minutes to Understand IBM Replication – Sources & Targets, Lower Latency, Costs” webinar on April 30, 2020 hosted by my colleague @KimMayTFG and me.  Thank you for joining us – along with the excellent questions and feedback.

These are the presentation materials IBM Data Replication Update 2020-04-30

You’ll find the recording of the webinar here.

Extending appreciation to all who participated in the IBM Db2 Analytics Accelerator (IDAA) v7.1 webinar on August 28, 2019.  If you weren’t able to join us, here is the webcast recording.  In a fast-paced 90 minutes we covered:

1. A review of capabilities included in recent IDAA v7.1.x code drops
2. The return of the High Performance Storage Saver (HPSS)
3. Integrated Synchronization
4. The closer-to-real Hybrid Transaction Analytic Processing (HTAP) capability
5. Disaster Recovery and High Availability considerations

The Fillmore Group’s exclusive IDAA Buyer’s Guide compares these IDAA form factors and deployment options:

• IBM Integrated Analytics Systems (IIAS)
• Docker container running on System z
• Docker container running on LinuxONE

And this is the handout: IDAA Webinar 2019-08

Please note: if you would like the exclusive IDAA Buyer’s Guide referenced in the webinar, contact my colleague, Kim May (kim.may@thefillmoregroup.com).

Thanks to my colleagues, Kim May and Ed Lynch, for a terrific IBM InfoSphere Data Replication (IIDR) new features webinar delivered today.  The recording of the webinar is found here.    These are the presentation materials: Data Replication Update 2019-02-07 – Final

I only spoke with Steve Mills a couple of times and attended presentations he delivered at conferences a few others, but he has had an enormous impact on my career.  Steve was the executive sponsor of the DB2 Gold Consultants program, of which I have been a member since 1998.  Steve recognized that specialized, deep technical skills were imperative to enabling the adoption and successful deployment of IBM’s world-class, function-rich – but let’s face it: enormously complex – information technologies.  Through the DB2 Gold Consultants program, Steve cultivated a cadre that were not only among the best DB2 practitioners, but foremost in the world in relational database technology in general.  Folks like Richard Yevich, Bonnie Baker, Frank Ingrassia, Terry Purcell, Jan Henderyckx, and Gerry Hodge.  I am humbled to be numbered among them.

For years IBM would sponsor a week-long DB2 Gold Consultants conference, usually at Silicon Valley or Toronto Lab.  Dozens of presenters including IBM Fellows like Pat Selinger, Don Haderle, Don Chamberlin, Hamid Pirahesh, and Curt Cotner would tell us about broad industry trends and specifics regarding features and functions in vNext, vNext+, and vNext++.  It was an amazing feast of insight which enabled me to better serve my clients.  None of that would have happened without Steve’s leadership.

An even more critical and direct aspect of Steve’s support was his sponsorship of contracting vehicles for DB2 Gold Consultants which enabled us to engage directly with IBM without having to add additional layers of third-part procurement.  He understood the practical problem of small businesses and sole practitioners trying to navigate a large, cumbersome bureaucracy.  One time I explained to Steve that even with the contracting vehicle in place, IBM buyers occasionally refused to use it.  His response was “Sometimes the inmates are running the asylum”.  I didn’t hear anything for a couple of weeks and then I received a call from a senior IBM procurement officer who introduced himself by saying: “Hi, I’m one of the ‘inmates'”.  Of course, the problem was resolved.

Steve Mills, the quintessential old-school “wild duck” IBMer, will be missed.

I saw IBM’s 100th birthday celebration made CNN’s homepage today.  It’s quite a compliment to the people, past and present, who  made IBM successful, and enabled the company to thrive consistently.  IBM has contributed so much great technology to our world over the years. 

As the daughter of a career IBM salesman, Russ May, my family moved around as I was growing up (in the 60’s and 70’s IBM stood for, “I’ve Been Moved”) and finally settled in Baltimore.  At the age of 23 I was working across the street from my dad’s office at 100 E Pratt when I was diagnosed with Stage 4 colon cancer and given three months to live.  The doctors at Johns Hopkins recommended I try, as a last ditch effort, a chemotherapy regimen, and I agreed.  I was scheduled to recieve treatments each morning, Monday through Friday, one week a month.

Read More…