Thanks to all who attended the Optim Virtual Briefing on DB2 Connect.  If you were not one of the 250 attendees last Thursday, the replay and materials are available here.

For more in-depth training, consider the following IBM authorized training classes.

• CF602 “DB2 Connect 9 to DB2 for z/OS DRDA Implementation with TCP/IP” December 6, 2010
• CF632 “DB2 Connect 9 for DB2 for z/OS Problem Determination and Performance” September 20, 2010

Kimberly Madia’s IBM/Optim team is hosting a Virtual Tech Briefing this Thursday:

DB2 Connect for DBAs: A Primer and Look Ahead

Whether you are a DB2 for LUW DBA who would like to access enterprise information or a z/OS DBA wondering how all those Java and .NET programmers are getting to your data, DB2 Connect is the solution.   Frank Fillmore, A DB2 Gold Consultant with an extensive history in training and consulting, will step you through a DB2 Connect primer from end-to-end, including platform architecture, DB2 Connect configuration parameters, and more. Case studies from large scale DB2 Connect health checks will be included.   IBM’s Kevin Foster, who manages the development of the product, will be on hand to discuss licensing, new packaging options such as the DB2 Connect Advanced Edition, which provides pureQuery acceleration in the box, and upcoming changes to the product.

Date: 19 August 2010
Time:  10:00-11:00AM Pacific, 1:00-2:00PM Eastern
Register here: http://ow.ly/2iAlr

I attended IBM’s zEnterprise announcement in New York yesterday.  IBM Senior VP Steve Mills said it was the most important announcement IBM had ever made in its impact on saving customers money.  He also said IBM spent US$1.5 billion dollars on the zEnterprise research and development effort over the past several years.  So, as movie reviewers ask about the latest blockbuster: “Can you see the money up on the screen?”.  The answer comes in a few loosely coupled parts.

1. Oskar Schindler said you must have a “clever accountant”.  Mills made it clear that organizations that can accurately allocate their IT expenses will see the most benefit from zEnterprise.  zEnterprise delivers System z quality of services (QOS) across heterogeneous architectures: the aforementioned System z as well as Power 7 blade servers and (eventually) System x blades.  The problem for most organizations is that System z “mainframe” costs have been capitalized from central IT budgets for over four decades.  As the PC revolution unfolded since the early 1980’s, most of the costs for networking, systems administrator salaries, PCs themselves and the software they run have been expensed out of departmental budgets.  Organizations with the discipline to accurately accumulate these costs certainly will be able to see the benefit of deploying the zEnterprise platform.  Interestingly, the table talk at lunch indicated that some IBMers see the sweet-spot for zEnterprise in the rapidly growing economies of China and Russia.  The reason?  Tight budgetary control and hierarchical, centralized decision-making in state and quasi-state enterprises (think Gazprom) will help them “get it” immediately.  I would not be surprised to see zEnterprise adoption in emerging and growing economies exceed that of North America in the next two years.
2. IBM has been able to run Linux on System z hardware using Virtual Servers and z/VM for a decade.  And the System z has been able to dispatch workloads to specialty engines within System z such as the Integrated Facility for Linux, zIIP, and zAAP for years.  Think of zEnterprise as extending that dispatching capability out of the physical System z box to discrete blade servers.  IBM’s goal is to move away from the “you can do everything on System z” posture - which in reality was a losing, rear-guard action - to embracing disparate architectures and acknowledging that maybe a print server really should run under Linux on an x86 platform.  Yet you can benefit from the centralized management and security of the System z.  This is workload integration at the chip, firmware, hypervisor, and middleware levels.  A pretty neat trick.
3. So what can I do with the zEnterprise?  Here are two relatively simple scenarios.

The Online Travel Portal  One well-known travel reservation site front-ends their expensive Oracle transaction servers with MySQL running on cheap x86 hardware.  While you’re noodling around trying to figure out the best intinerary, all you’re seeing is data replicated from Oracle to MySQL on a near-real-time basis.  When you enter your credit card number and hit “Purchase”, you’re routed to the Oracle OLTP server.  This is called “database tiering”.  I can now architect the same topology on System z with DB2 for z/OS on the back-end and x86 blades running the DB2 Express-C freeware database.  On the zEnterprise platform, these databases will communicate over a 10Gb private, secure network with extraordinarily low latency.  Ever get the “That seat is no longer available” message?  It might be a thing of the past with zEnterprise.

The Hospitality Company  This organization runs their centralized reservation systems on DB2 for z/OS already.  In order to support their frequent-guest affinity program portal, they have WebSphere Application Server running on a separate System p AIX server.  The only problem is that sometimes transactions hang to the point that the JVMs have to be recycled.  The Java programmers say their code is tightly written and the DB2 for z/OS database administrations say that the incoming SQL requests are satisfied sub-second.  While zEnterprise alone would not resolve this problem - see pureQuery and the lyrically named Optim Performance Manager Extended Edition - the application and the database servers will be as tightly coupled as possible while each runs on the optimal platform.  Since the transfer points and the servers themselves are under unified management, an entire layer of complexity (and potential breakage) will be eliminated.

The real buzz in the announcement for me is the IBM Smart Analytics Optimizer (ISAO).  For a generation as a DB2 database administrator, I’ve told my clients that OLTP and ad hoc query workloads should not be intermingled.  The solution has been to make copies of the data using replication technologies - InfoSphere Change Data Capture and Q Replication among them.  This approach has been a boon to DBAs, and software, storage, and server salesmen everywhere.  When it achieves its full promise, ISAO will evaluate incoming database requests and dispatch them along with the data needed to satisfy the request to the appropriate platform server.  DB2 for z/OS will serve as a centralized front-end for all workloads: OLTP, OLAP, ad hoc query, etc.  ISAO will transparently run the workload on the optimal platform and return the result set to the requesting application.  Organizations will be able to dismantle the miasma of extracts, FTPs, and other artifices now necessary to keep analytic workloads from bogging down OLTP.  And they’ll reduce complexity.  And save a ton of cash.

So on whose door will IBM knock first?  Clearly the System z installed base will be getting lots of attention.  But could Facebook or some other enterprise with orders-of-magnitude scaling issues (500 million Facebook users and counting) benefit from zEnterprise?  Surprisingly, the answer is Yes!  Facebook needs to manage lots of unstructured data (pictures, videos, et al) , but they also have the need for complex analytics.  First, to target online advertising ever more precisely, but also to serve larger societal needs.  Let’s say a man declares he needs a reduction in child support because he’s nearly broke.  The local social services agency unleases a smart agent to run against social networking sites and comes up with pictures on Facebook from the man’s recent two-week vacation in Hawaii.  Too big brother-ish?  A topic for another day.

As much as I dislike the relentless repetition that’s part of the nature of the world of blogs and twitter and listserves and email blasts, here I go with a shameless pitch for a TFG special services offering I emailed to several DB2 users earlier today.  I am doing this because, at the end of the day, the rate disparity in today’s DB2 services market baffles me.  Are the ridiculously high rates being charged eroding product adoption?  I am afraid so, which is why we are offering a summer services special. 

Read More…

The developerworks team has announced a free technical webcast next week on pureQuery, based on feedback from the z community.  Like other developerworks webcasts it will be available for replay - as is the recommended prerequisite, pureQuery Part 1.

pureQuery Deep Dive Part 3:  Client Optimization Administration Enhancements for DBA’s

June 24th, at 1pm Eastern, 10am Pacific

Our dynamic duo of client optimization experts is back again!  Patrick Titzler and Chris Farrar, who some of you may remember from pureQuery Deep Dive Part 1, will discuss the enhancements released in Fix Pack 3 in Optim Development Studio and pureQuery Runtime that make the maintenance, security, and management of the client optimization processes better for DBA’s.  These enhancements are driven by real customer experiences and are centered around a relational repository that enhances security, fosters collaboration and streamlines the process of administering a pureQuery-enabled application.  Tooling support for these capabilities in Optim Development Studio will be demonstrated. 

Register today and don’t miss out on your opportunity to hear from the experts!

IBM has moved DB2 Connect support into the Optim group, as it is in some ways the original Optim solution.  Kathy Zeidenstein, who manages community outreach efforts for the Optim team (a weekly e-newsletter, content on the developerworks site, Twitter, etc.) is coordinating a DB2 Connect Virtual Briefing scheduled for August 19th.  The primary purpose of the presentation is to introduce new features in the advanced edition, however, as DB2 Connect can be used in so many ways, and is so often under-utilized, she’s invited Frank to devote some of the time allocated to delivering an overview of what DB2 Connect can do.  More information is on the way…for the moment we have the date and time reserved (mark your calendars - August 19th at 1pm Eastern), and a tentative agenda:

DB2 Connect for DBAs:  A Primer and a Look to the Future

Whether you are a DB2 for LUW DBA who would like to access enterprise information or a z/OS DBA wondering how all those Java and .NET programmers are getting to your data, DB2 Connect is the solution.  Frank Fillmore, A DB2 Gold Consultant with an extensive history in training and consulting, will step you through a DB2 Connect primer from end-to-end, including platform architecture, DB2 Connect configuration parameters, and more.  Case studies from large scale DB2 Connect health checks will be included. Kevin Foster, who manages the development of the product will be on hand to discuss packaging including licensing, new packaging options such as the DB2 Connect Advanced Edition, which provides pureQuery acceleration in the box, and upcoming changes to the product.
What you will learn:
• Why you need DB2 Connect
• How DB2 Connect is packaged and licensed
• Platform architecture
• Configuration and tuning options

As soon as I have registration information I will post it.

If you will be in Baltimore on May 6th please join us!

Do you know the cost of capturing a change in your production database systems?  Does it cost an extra $100,000 per month to capture that change - and then is the change moved to a data warehouse that’s left many users dissatisfied?  We have a solution, InfoSphere Change Data Capture.

The Fillmore Group is teaming with IBM for a *free* lunch session at Orioles Park at Camden Yards, from 11am to 1pm, where we will explain how InfoSphere Change Data Capture can do an intelligent capture on heterogeneous databases to:  1) create an audit trail detailing who did what and when, 2) replace inefficient staging of changes on your mainframe, 3) eliminate the overhead and instability of ad hoc queries, 4) remove the inefficiency of triggers and message queues, and 5) reduce mainframe costs by $1 million per year - reliably feed downstream ETL, MDM, or SOA applications.

If you are interested in attending, click here for more information.  See you at the Yard!

I’ve been seeing a lot of an old friend lately: DB2 Connect.  This is the software which enables Distributed Relational Database Architecture (DRDA) Application Requestor (AR) or “client” applications running on Linux, Unix, and Windows platforms to access DB2 ”host” databases on System z and System i.  More precisely, DB2 Connect accesses an Application Server (AS) which can be DB2 for z/OS, DB2 Server for VSE&VM, or DB2 for i (Wait, shouldn’t that be DB2 for “me”?)

What we know today as DB2 Connect started out almost 20 years ago as Distributed Database Connection Services (DDCS).  Then it was amazing to access host DB2 data using a spreadsheet.  Of course it took an experienced consultant a few weeks to install and configure DDCS and it’s supporting software which relied on the Systems Network Architecture (SNA) communications protocol.  It was so difficult, The Fillmore Group had a services offering called “Client/Server in 10 Days” where we guaranteed to get DDCS working in your shop in two weeks.  Now you can install and configure DB2 Connect using TCP/IP in about an hour.

So what does DB2 Connect actually do?  At its most basic, DB2 Connect is a protocol converter that changes the character coding on the host (Extended Binary Coded Decimal Interchange Code or EBCDIC) into the equivalent used on Linux, Unix, and Windows platforms (American Standard Code for Information Interchange or ASCII).  It does lots of other things and can be a source of lots of troubleshooting and performance tuning, but EBCDIC to ASCII translation is a core function.

So what’s happening lately?  First, after a few years of no publicly available training in DB2 Connect, The Fillmore Group is holding classes: CF602 “DB2 Connect 9 to DB2 for z/OS DRDA Implementation with TCP/IP” and CF632 “DB2 Connect 9 for DB2 for z/OS Problem Determination and Performance”.  The CF602 beginning June 21 is actually nearing capacity, so if this is something you need, sign up to attend now.  CF632 starts May 17.

As a bonus, I’ve attached a DB2 Connect configuration worksheet.  If you know what it all means and can use it right away, great!  Have at it.  If not, we’ll see you in class.

Second, on the consulting side of the house we’re seeing lots of DB2 Connect activity - especially with DB2 for i.  Having a relational database directly integrated into the i5/OS operating system, System i always has been a bit of a land unto itself.  Now we’re seeing lots of activity integrating System i data into eCommerce and Business Intelligence applications.  If this is what you’re doing, let us know how we can help.

IBMer David Tolleson does a terrific job of outlining the configuration options for SQL and Q Replication when the source or target DB2 database is partitioned.  This was formerly know as Data Partitioning Feature (DPF).  Today it is part of InfoSphere Warehouse.

Check out David’s presentation.

As you know, I work with IBM’s Q Replication technology - a lot.  Q Replication functionality is delivered in InfoSphere Replication Server.  The challenges are amplified when working on DB2 for z/OS with *really* large tables.  One financial institution at which I am working has tables with over 1 billion rows and hundreds of partitions.  Of course, DB2 for z/OS can manage tables of that size, but what about the tooling?

Q Replication comes with a utility called ASNTDIFF.  ASNTDIFF compares a checksum of rows in the source and target tables being replicated to validate that there are no discrepancies.  Challenge #1 is that when replicating between DB2 for z/OS subsystems, ASNTDIFF runs under Unix Systems Services (USS) that provides the Unix APIs enabled in z/OS.  There are considerations for USS applications that will form the basis for another post.

Challenge #2 is that ASNTDIFF retrieves the rows from the remote system (Application Server or AS) using a three-part-name query across a Distributed Relational Database Architecture (DRDA) connection.  For example:

SELECT * FROM <location>.<schema>.<tablename>

where location is found in the DRDA Communications Database (CDB) portion of the DB2 for z/OS Application Requestor (AR) catalog tables.  You typically run the ASNTDIFF utility on the replication target DB2 for z/OS server.  That’s because the CDB has probably already been configured to support cursor-based loading of the target tables.  Why is this a challenge?  Well, a three-part-name query across a DRDA pipe against a 1 billion row table ran for about 18 hours.  Ouch!

So the basic problem is: how can I get three-part-name queries running across a DRDA connection between DB2 for z/OS subsystems to run faster?  I asked a couple of IBMers and Jim Pickel pointed me in the direction of exploiting OPTIMZE FOR n ROWS.  There’s a good explanation of this in “Limiting the number of DRDA network transmissions”.  Right now the ASNTDIFF utility adds OPTIMIZE for 1000 ROWS and FOR READ ONLY to every ASNTDIFF query.  We’re experiementing with the recommendations in the “Limiting…” document to see if we need to override this hardcoded parameter.  I’ll keep you posted as to our findings.