RTP Admin: Scenarios DLQ and Handler with IBM MQ

RTP

This article was developed with the intention to help IBM MQ Administrators gain a better understanding of Dead Letter Queues (DLQ) and DLQ Handlers for RTP administration. It provides basic scenarios and explanations within IBM MQ.

Becoming a Real Time Payments (RTP) “participant” has numerous challenges.  For many financial entities, this is their first exposure to IBM MQ as a messaging system, which is a requirement to join the network.

The Clearing House’s RTP represents the next evolution of payments innovation for instant transfer of funds and confirmation. Financial institutions, big and small, are offering Real Time Payments to their customers to stay competitive.

 

IBM MQ and RTP

At its fundamentals, the RTP network was built upon the concept of MQ request/reply.  For example, a participant sends a request message following the ISO 20022 XML standard.

MQ request/reply Example

The overall lifespan of the “transaction” is fifteen (15) seconds, but the “hops” between the participant to RTP to participant and back are two (2) seconds each between its respective queue manager.  The Clearing House’s RTP documentation provides a chart to explain further, and is outside the narrative of this article.

When everything works through “happy path”, it’ll be as shown above: the participant app sends a request message and awaits a response within the 15 second window.

But what happens if something goes wrong with the reply message being placed on RESPONSE.FROM.RTP1?

 

THE DEAD LETTER QUEUE AND THE DEAD LETTER HANDLER

THE DEAD LETTER QUEUE AND THE DEAD LETTER HANDLER

A feature of IBM MQ is the assured delivery of the messages to the destination. There can be number of scenarios where MQ may not be able to deliver messages to the destination queue, and all those are routed to the Dead Letter Queue (DLQ) defined for the queue manager.

Before placing message to the DLQ, IBM MQ attaches Dead Letter Header (DLH) to each message which contains all the required information to identify why the message was placed in DLQ and what was the intended destination. This DLH plays a very important role while handling the messages on DLQ.

THE DEAD LETTER QUEUE AND THE DEAD LETTER HANDLER 2

A feature of IBM MQ is the assured delivery of the messages to the destination. There can be number of scenarios where MQ may not be able to deliver messages to the destination queue, and all those are routed to the Dead Letter Queue (DLQ) defined for the queue manager.

Before placing message to the DLQ, IBM MQ attaches Dead Letter Header (DLH) to each message which contains all the required information to identify why the message was placed in DLQ and what was the intended destination. This DLH plays a very important role while handling the messages on DLQ.

THE DEAD LETTER QUEUE AND THE DEAD LETTER HANDLER 3

IBM has provided a Dead Letter Handler utility called “runmqdlq”.  Its purpose is to handle messages which are placed on the DLQ. With it, the routine takes necessary action on the messages placed on the DLQ.  A rule table can be created with required control data and rules and the same can be used as an input to the Dead Letter Handler.

 

TYPES OF DEAD LETTER MESSAGES

In a nutshell, messages placed on the DLQ are in two categories:
– Replayable Messages
– Non-RePlayable Messages

For Replayable Messages, these are placed on the DLQ due to some temporary issues like the destination queue is full or the destination queue is put disabled etc. These messages can be replayed without any further investigation. Also, re-playable messages can be performed by using dlqhandler.

For Non-Replayable Messages, these are placed on the DLQ due to issues which are not temporary in nature (incorrect destination, data conversion errors, mqrc_unknown_reason). These messages require further investigation to identify the cause on why those were placed on the DLQ. Replaying this category of messages with dlqhandler most likely will again arrive on DLQ. Therefore, they would require an MQ Admin to investigate further.

 

THE DLQ HANDLER SETUP

Replayable messages can be run again (reprocessed) by using the dlqhandler. In short, the dlqhandler is setup as follows:
1) Creating the dlqhandler rule file
2) Starting the dlqhandler

The rule file is a simple text-based flat file with configuration instructions as to how a message should be handled.

 

COMMON SCENARIOS

A number of scenarios are possible, based on the design of MQ infrastructure at an organization, but the most common scenarios are:

 

  • SINGLE-PURPOSE: Replayable messages are “replayed” to their original queue and non-replayable messages are placed on a single designated queue for an MQ admin to investigate

 

  • MULTI-PURPOSE: Replayable messages are “replayed” to their original queue and non-replayable messages are placed on a designated queue for each application for an MQ admin to investigate

 

  • HYBRID-PURPOSE: Replayable and non-replayable messages from specific queues are placed on a designated queue for each application for an MQ admin to investigate, while other replayable messages are “replayed” to their original queue, and non-replayable messages are placed on a single designated queue for an MQ admin to investigate.

 

It is worth noting that since messages being put to the DLQ because of their destination queue being full (MQRC_Q_FULL) or put inhibited (MQRC_PUT_INHIBITIED) will be in constant retry mode.  That said, it’s important to have a monitoring and alerting setup on a queue manager and associated mq objects to send an alert (email, page, etc.) for someone to investigate why a queue is approaching 100% depth or put inhibited.

 

SCENARIO 1: SINGLE PURPOSE

This is a very common requirement. This can be achieved by defining the rule file.

Contents of <scenario1.rul> file:

******=====Retry every 30 seconds, and WAIT for messages =======*****
RETRYINT(30) WAIT(YES)

***** For reason queue full and put disabled *******
**** retry continuously to put the message on the original queue ****
REASON(MQRC_Q_FULL) ACTION(RETRY) RETRY(999999999)
REASON(MQRC_PUT_INHIBITED) ACTION(RETRY) RETRY(999999999)

**** For all other dlq messages, move those to designated queue *****
ACTION(FWD) FWDQ(UNPLAYABLE.DLQ) HEADER(YES)
******=========================================================*******

 

SCENARIO 2: MULTI PURPOSE

Suppose there are 3 applications using the qmgr: APP1, APP2 and APP3. The following are the queues for these applications and dedicated queue to hold dlq messages for each application:
APP1 –
APP1.QL
DLQ – APP1.ERR.DLQ
APP2 –
APP2.QL
DLQ – APP2.ERR.DLQ
APP3 –
APP3.QL
DLQ – APP3.ERR.DLQ

We now have to write rules based on the DESTQ for each application. Below the example rule file for this scenario.
Contents of <scenario2.rul> file:

******=====Retry every 30 seconds, and WAIT for messages =======*****
RETRYINT(30) WAIT(YES)
***** For reason queue full and put disabled *******
**** retry continuously to put the message on the original queue ****
REASON(MQRC_Q_FULL) ACTION(RETRY) RETRY(999999999)
REASON(MQRC_PUT_INHIBITED) ACTION(RETRY) RETRY(999999999)
******* For APP1, forward messages to APP1.ERR.DLQ ******
DESTQ(APP1.QL) ACTION(FWD) FWDQ(APP1.ERR.DLQ) HEADER(YES)

******* For APP2, forward messages to APP2.ERR.DLQ ******
DESTQ(APP2.QL) ACTION(FWD) FWDQ(APP2.ERR.DLQ) HEADER(YES)

******* For APP3, forward messages to APP3.ERR.DLQ ******
DESTQ(APP3.QL) ACTION(FWD) FWDQ(APP3.ERR.DLQ) HEADER(YES)
**** For all other dlq messages, move those to designated queue *****
ACTION(FWD) FWDQ(GENEAL.ERR.DLQ) HEADER(YES)
*********=========================================================******

 

SCENARIO 3: HYBRID PURPOSE 

This is an example of regardless as to why the message was put on the DLQ for APP1.QL, the message needs to be forwarded to APP1.ERR.DLQ.

For other non-APP1 DLQ messages, attempt to replay them to their intended destination queue for queue full and queue inhibited messages.

If the retry interval is > 10 or it’s a non-replayable message, forward that message to the UNPLAYABLE.DLQ.

Contents of <scenario3.rul> file

******=====Retry every 30 seconds, and WAIT for messages =======*****
RETRYINT(30) WAIT(YES)

******* For APP1, forward messages to APP1.ERR.DLQ ******
DESTQ(APP1.QL) ACTION(FWD) FWDQ(APP1.ERR.DLQ) HEADER(YES)

***** For reason queue full and put disabled *******
**** retry 10 times to put the message on the original queue ****
REASON(MQRC_Q_FULL) ACTION(RETRY) RETRY(10)
REASON(MQRC_PUT_INHIBITED) ACTION(RETRY) RETRY(10)
**** For all other dlq messages, move those to designated queue *****
ACTION(FWD) FWDQ(UNPLAYABLE.DLQ) HEADER(YES)
******=========================================================*******

 

STARTING THE DLQHANDLER

Once the rule file is created, you can configure dlqhandler startup in the following ways:

1) Manually starting the dlqhandler and keep it running.
runmqdlq QMGR_DLQ QMGR_NAME < qrulefile.rul

2) Configure dlqhandler as a service in the queue manager
SPECIAL NOTE FOR WINDOWS SERVERS:
DEFINE SERVICE(dlqhandler) +
SERVTYPE(SERVER) +
CONTROL(MANUAL) +
STARTCMD(‘c:\var\bin\mqdlq.bat’) +
DESCR(‘dlqhandler service’) +
STARTARG(‘DLQ QMNAME C:\var\rulefiles\qrulefile.rul’) +
STOPCMD(‘c:\var\bin\stopdlh.bat’) +
STOPARG(‘DLQ QMNAME’) +
STDOUT(‘/path/dlq.log’) +
STDERR(‘/path/dlq_error.log’)

NOTE: For Windows, because of the nature of how it doesn’t handle redirects “<” like Linux does, a wrapper script must be written.

Contents of mqdlq.bat (start command):
echo alt ql(%1) get(enabled) | runmqsc %2
runmqdlq.exe %1 %2 %3

Contents of stopldh.bat (stop command):
echo alt ql(%1) get(disabled) | runmqsc %2

Enabling and disabling the DLQ essentially kills the dead letter handler.

3) Configure triggering at DLQ to start dlq handler whenever first message arrives on the queue.

You can do it by simply following the steps on how to configure triggering on any queue. Please refer the following link for more information on triggering.

Once you setup triggering, dlqhandler will be started by triggering based on the condition that you set. You don’t need to have dlqhandler running all the time as it can be started again by triggering. Due to above reason, you don’t need WAIT(YES) in the rule file, you can change it to WAIT(NO).

 

WANT TO LEARN MORE?

TxMQ delivers an RTP Starter Pack to accelerate participant onboarding.  You can learn more about our starter pack here: https://www.txmq.com/RTP/

Our deep industry experience and subject matter experts are available to solve your most complex challenges.  We deliver solutions and innovations to do business in an ever-changing world, to guide & support your organization through the digital transformation journey.

TxMQ
Imagine. Transform. Engage.
We’re here to help you work smart in the new economy.

This post was authored by John Carr, Principle Integration Architect at TxMQ. You can follow John on LinkedIn here.

IBM MQ v9.2 Release Update

Great news MQ Users! IBM has just released the new IBM MQ v9.2 and it’s ready for download.

The new version has many features that continue to make IBM MQ the industry leader for messaging middleware software. MQ v9.2 is the follow-on long-term support release (LTSR) to MQ v9.1. With the LTSR release, fix packs will only provide fixes that include no new functions. MQ v9.2 includes the features that were delivered in the CD releases of MQ versions v9.1.1 to v9.1.5, along with some minor enhancements. The capabilities being incorporated into MQ v9.2 from previous MQ v9.1.x CD releases and those that are new in MQ v9.2 are detailed in the Description section. For more information, see the IBM MQ FAQ for Long Term Support and Continuous Delivery releases website.

This release encompasses all versions of MQ including the MQ Appliance and MQ on Cloud. Below we’ve outlined a few of the new enhancements included in the release.

MQ v9.2 includes enhancements in the following areas:

  • Simplified, smart management
  • Augmented security
  • Expanded client application options
  • Easier adoption of containers
  • More flexible design and deployment

*For more info on the new release view the IBM MQ 9.2 release document here.

*Find out more about what’s new in Version 9.2.0

*Find out what’s changed in Version 9.2.0

Extended Support for your IBM MQ 

Keep in mind that although IBM has extended support for IBM MQ v8.x through September 30th, now’s the time to get your plans in order. If you need help determining an upgrade path or choosing the right option for End Of Service or End Of Life Support for your out of date software, contact TxMQ for a free discovery call.

If you’re interested in learning more about how TxMQ can help support your MQ, take a closer look at some of our services on our IBM MQ and WebSphere Support page.

Feel free to reach out anytime through the form below, we’d love to help you take the next steps towards upgrading your IBM MQ, or assist in choosing your best support options.

 

IBM MQ v8.x End of Support Options

Are you still running IBM MQ v8.0.x (or alternately, an even earlier build)?  

IBM has announced as of April 30th, 2020 it will end of support for IBM MQ v8.0.x. If you are still using that version it’s recommended that you upgrade to v9.1 to avoid any potential security issues that may occur in earlier, unsupported versions of MQ. 
MQ v9.1 Highlights: 

  • IBM MQ Console
  • The administrative REST API
  • The messaging REST API
  • Improvements in error logging 
  • Connectivity to Salesforce with IBM MQ Bridge to Salesforce
  • Connectivity to Blockchain 

What are your plans for IBM MQ?

I plan to upgrade:
It’s never too late to start planning your upgrade and upgrading to IBM MQ’s newest v9.1 is a great option. There are great new features that help you manage costs better, improve efficiency, and manageability. 
Take a closer look here at some of the enhancements.  
If you are still considering your plans, now’s a great time to speak with our SME Integration Upgrade Team. Reach out to us today to set up a free Discovery Session or contact us directly for any questions

I would like to continue to use v8.0.x (or earlier versions):

It’s ok if you’re not ready for the newest version of MQ just yet. However, it’s important to remember that without support you may not be protected against avoidable security risks and additional support costs. IBM does offer Extended Premium support but be prepared, that option will be very expensive. 
Alternatively, as an IBM Business Partner TxMQ offers expert support options. As a business partner, we have highly specialized skills in IBM software. We can help guide you through issues that may arise at a fraction of the cost with the added benefit of flexibility in services. Check out more on TxMQ’s Extended Support Page. (https://www.txmq.com/end-life-software-support/)

I will support it internally: 

If you have an amazing internal team inhouse, odds are they don’t have much time to spare. Putting the gravity of a big project on your internal team can cut into their productivity. For many organizations, this will limit a team’s ability to focus on innovation and improving customer experience. This will make your competitors happy but your customers and clients definitely won’t be. 
Utilizing a trusted partner like TxMQ can help cut costs and give back some time to your internal team to focus on improvements and not just break/fix maintenance. Reach out to our team and learn how we can help maintain your existing legacy applications and platforms so your star team can focus on innovation again. Reach out and ask how we can help today. 

I don’t know, I still need help!

Reach out to TxMQ today and schedule a free Discovery Session to learn what your best options are! TxMQ.com/Contact/

John Carr Set to Present at IBM’s 2020 Integration Tech Conference

Presentation

One of the Best Events of the year is just around the corner!
IBM’s 2020 Integration Technical Conference is scheduled to be held on March 16th through the 19th this year, at the Renaissance Hotel in Dallas, TX.

What is the Integration Technical Conference? 

The Integration Technical Conference was started in 2019 as a replacement for a previously popular MQ and Integration focused Technical Conference known as MQTC. 
Last year’s inaugural Integration Tech Conference was one of the most highly acclaimed conferences of the year. It was praised for having a strong technical focus, great presentations, and in-depth training opportunities, and not just being another sales conference. 
IBM is working even harder this year to improve on last year’s conference and they have officially confirmed the Partners and Subject Matter Experts that have been chosen to present. 

We are proud to announce TxMQ’s very own John Carr, a Sr. Integration Architect, has been selected to present “Practical IBM MQ Implementations for the Cloud: A Journey of DevOps from an IBM MQ admin”. You may have already caught some of John’s previous presentations including “MQ Upgrade Best Practices” last year at the 2019 conference or through one of our webinars (which you can view here).

This year John will be discussing migrating your IBM MQ network from on-prem to the cloud. For those already undertaking their own modernization efforts, this will be a great topic for discussion. The session will walk through a case where TxMQ helped a Mortgage Services organization migrate their entire self-managed data center into the public cloud. You can learn more from the breakdown of the session abstracts here

If you’re lucky enough to attend your days are going to be full, so plan accordingly! This year John’s presentation will be on Day 3, Wednesday, March 18th at 9:50 AM, and on Thursday, March 19th at 4:40 PM. Both will be held in Salon F/G at the Renaissance Dallas Hotel. Lock it into your calendar so you don’t miss it! 

If you are attending don’t forget to stop by the TxMQ booth for some helpful guides, giveaways, and prizes. Also, please drop us a note at info@txmq.com or give us a call (716.636.0070) to connect at the conference, we’d love to hear from you.

Have a great time! We’ll see you there.

IBM MQ Organize Your Team For An Upgrade

TxMQ Work Smarter Webinar Series

Organizing Your Team For an IBM MQ Large-Scale Upgrade

Presented by TxMQ

Featuring John Carr Sr. Solutions Architect at TxMQ

IBM MQ is the most important part of your business you rarely notice until there’s an issue.

It’s a great tool to keep your business running smoothly and give your customers the service they expect. At TxMQ we know Middleware and we are very passionate about IBM MQ. So much so, that we have dedicated a part of our practice to keeping your MQ up and running the way it should be, reliable and secure.

At TxMQ we are proud to present our second installment in our Webinar Series dedicated to IBM MQ. For Part 2, in continuation of our IBM MQ Upgrade Best Practices theme, we will outline how to organize your team for a large-scale upgrade. (If you missed Part 1 you can catch up here.)

Economies of scale aren’t limited to the cloud. As an MQ administrator, you most likely manage a large, diverse MQ network using traditional commodity or virtual servers on-prem. On top of that, you’re challenged with upgrading to MQ 9.x with limited resources.

How should you approach your upgrade in such economies of scale?

This session walks you through the challenges (from server management to security, to deployment) and gives food-for-thought on how to organize yourself and your team. The discussion is hosted by industry expert and Sr. Solutions Architect with TxMQ, John Carr.


Below you can view the complete recording as well as the associated slide deck. Enjoy, and don’t forget to give us some feedback. We would love to hear any suggestions you have or subjects you would like to see covered in future webinars. Let us know here.


If you would like to talk further or need some help with your MQ, reach out to us here and let us know how we can help.

Which of these MQ mistakes are you making?

IBM MQ Logo

IBM MQ Tips Straight From TxMQ Subject Matter Experts

At TxMQ we have extensive experience helping clients with their IBM MQ deployments including planning, integrations, management, upgrades, & enhancements. Throughout the years we’ve seen just about everything, and we have found there are some common mistakes that are easy to stay away from with a little insight. Here are some a few tips to keep you up and running:IBM MQ is a powerful tool that makes a huge difference in your life every day, but for most of us you only notice it when it’s not working. One small mistake can cause havoc to your entire system.

1. Don’t use MQ as a database. MQ is for moving messages from one application or system to another. Databases are the data repository of choice. MQ resources are most optimized when considering data throughput and message delivery efficiency. IBM MQ is optimized when messages are kept small.

2. Don’t expect assured delivery if you didn’t design for it! Assured one-time message delivery is provided by IBM MQ through setting message persistence and advanced planning in the application integration design process. This plans for poisoned message handling that could otherwise cause issues and failures or worse, unexpected results. Know your business requirements for quality of service for message delivery and make sure your integration design accommodates advanced settings and features as appropriate.

3. Don’t give every application their own Queue Manager! Sometimes yes, sometimes no. Understand how to analyze what is best for your needs. Optimize your messaging architecture for shared messaging to control infrastructure costs and optimize operational support resources.

4. Don’t fall out of support. While TxMQ can offer support for OUT of support products, it’s costly to let support lapse on current projects, and even more so if you have to play catch up!

5. Don’t forget monitoring! MQ is powerful and stable, but if a problem occurs, you want to know about it right away. Don’t wait until your XMITQs and application queues fill up and bring the Queue Manager down before you respond!

In the cloud, on mobile, on premises or in the IoT, IBM MQ simplifies, accelerates and facilitates security-rich data exchange. Keep your MQ running smoothly, reach out to talk with one of our Subject Matter Experts today!

If you like our content and would like to keep up to date with TxMQ don’t forget to sign up for our monthly newsletter.

[button link=”https://www.txmq.com/newsletter-sign-up” type=”big” newwindow=”yes”] Stay Informed – Sign up for TxMQ’s Newsletter[/button]

IBM WebSphere Application Server (WAS) v.7 & v.8, and WebSphere MQ v.7.5 End of Support: April 30, 2018

Are you presently running on WAS versions 7 or 8?
   Are you leveraging WebSphere MQ version 7.5?

Time is running out, IBM WebSphere Application Server (WAS) v.7 & v.8, and WebSphere MQ v.7.5 support ends in less than 6 months. As of April 30th 2018, IBM will discontinue support on all WebSphere Application Server versions 7.0.x & v8.0.x; and WebSphere MQ v7.5.x.

It’s recommended that you migrate to WebSphere Application Server v.9 to avoid potential security issues that may occur on the early, unsupported versions of WAS (and Java).
It’s also recommended that you upgrade to IBM MQ version 9.0.x, to leverage new features, and avoid costly premium support fees from IBM.

Why should you go through an upgrade?

Many security risks can percolate when running back-level software, especially WAS running on older Java versions. If you’re currently running on software versions that are out of support, finding the right support team to put out your unexpected fires can be tricky and might just blow the budget.
Upgrading WAS & MQ to supported versions will allow you to tap into new and expanding capabilities, and updated performance enhancements while also protecting yourself from unnecessary, completely avoidable security risks and added support costs.

WebSphere Application Server v.9 Highlights

WebSphere Application Server v.9.0 offers unparalleled functionality to deliver modern applications and services quickly, securely and efficiently.

When you upgrade to v.9.0, you’ll enjoy several upgrade perks including:
  • Java EE 7 compliant architecture.
  • DevOps workflows.
  • Easy connection between your on-prem apps and IBM Bluemix services (including IBM Watson).
  • Container technology that enables greater development and deployment agility.
  • Deployment on Pivotal Cloud Foundry, Azure, Openshift, Amazon Web Services and Bluemix.
  • Ability to provision workloads to IBM cloud (for VMware customers).
  • Enhancements to WebSphere extreme scale that have improved response times and time-to-configuration.

 

IBM MQ v.9.0.4 Highlights

With the latest update moving to MQ V9.0.4, there are even more substantial updates of useful features for IBM MQ, even beyond what came with versions 8 (z/OS) & 8.5.

When you upgrade to v.9.0.4, enhancements include:
  • Additional commands supported as part of the REST API for admin.
  • Availability of a ‘catch-all’ for MQSC commands as part of the REST API for admin.
  • Ability to use a single MQ V9.0.4 Queue Manager as a single point gateway for REST API based admin of other MQ environments including older MQ versions such as MQ V9 LTS and MQ V8.
  • Ability to use MQ V9.0.4 as a proxy for IBM Cloud Product Insights reporting across older deployed versions of MQ.
  • Availability of an enhanced MQ bridge for Salesforce.
  • Initial availability of a new programmatic REST API for messaging applications.

This upgrade cycle also offers you the opportunity to evaluate the MQ Appliance. Talk to TxMQ to see if the MQ Appliance is a good option for your messaging environment.

What's your WebSphere Migration Plan? Let's talk about it!

Why work with an IBM Business Partner to upgrade your IBM Software?

You can choose to work with IBM directly – we can’t (and won’t) stop you – but your budget just might. Working with a premier IBM business partner allows you to accomplish the same task with the same quality, but at a fraction of the price IBM will charge you, with more personal attention and much speedier response times.
Also, IBM business partners are typically niche players, uniquely qualified to assist in your company’s migration planning and execution. They’ll offer you and your company much more customized and consistent attention. Plus, you’ll probably be working with ex-IBMers anyway, who’ve turned in their blue nametags to find greater opportunities working within the business partner network.

There are plenty of things to consider when migrating your software from outdated versions to more current versions.

TxMQ is a premier IBM business partner that works with customers to oversee and manage software migration and upgrade planning. TxMQ subject matter experts are uniquely positioned with relevant experience, allowing them to help a wide range of customers determine the best solution for their migration needs.
Get in touch with us today to discuss your migration and back-level support options. It’s never too late to begin planning and executing your version upgrades.

To check on your IBM Software lifecycle, simply search your product name and version on this IBM page or, give TxMQ a call…

MQ Technical Conference (MQTC) 2017 Delivers Again on Content & Convenience

TxMQ/TxMQ Canada took part as a gold sponsor for the MQ Technical Conference, also known as MQTC, last week, September 24th through September 27th, at the Kalahari Resorts in Sandusky, Ohio.
This annual event has become a mainstay for our team; we’re proud to be an original and ongoing Gold Sponsor of the conference since it began in 2013. Kudos again to Roger Lacroix at Capitalware, Inc. for his vision and organization of this successful conference series.

MQTC is a collection of educational sessions aimed at administrators, developers and engineers, ranging from novice to expert level in the IBM Middleware/Messaging stack.

In contrast to many, if not most, technical events on the calendar, MQTC leaves behind the “glitz and glam” (and cost) of big sales presentations disguised as technical conferences.

This is not to say that the accommodations at MQTC don’t measure up.

The Kalahari Resort in Sandusky, with a horizontal layout, African motif and indoor waterpark, feels like a Las Vegas Hotel & Convention Center set on the grounds of a Disney Property. In fact, some attendees bring their young families for the 3-day event.
MQTC’s venue choice delivers a setting that’s comfortable enough, and reasonable enough to get to (less than an hour drive from Cleveland Hopkins Airport), without the distractions of many major tech conference settings*.

*IT Managers, take note – your training dollars are spent in the classroom, not “elsewhere”.

There are salespeople in attendance – yours truly included – but even the vendor sessions focus on technical content, and solving real world IT/business problems. There are always presenters brought in from the IBM product labs in Hursley, and there is no shortage of expertise on hand from corporate shops and business partners alike.
For those interested in sampling this year’s content, you can easily download any of the presentations for free right on the MQTC website, including a few from the TxMQ team.
Hope to see you there next year!
Miles Roty's Signature

Upgrading For Federal Reserve Bank MQ Integration

For financial institutions using WebSphere MQ V6.x or older to communicate with the Federal Reserve Bank, your time is running out… and fast. WebSphere MQ V6.x went End of Support on September 30th, 2012 and the Federal Reserve Bank is requiring upgrades to a current, supported version.

WebSphere MQ V7.0.0 and V7.0.1 went End of Support at the end of September 2015. For those of you wondering about WebSphere MQ V7.1, you might want to reconsider. While still technically supported by IBM, V7.1 is likely looking at another upgrade within the next 18 to 24 months.
From a business perspective, I recommend financial institutions upgrade to IBM MQ V8.x. At the very least, you could consider MQ V7.5.x.

IBM MQ V8.x, rebranded from WebSphere MQ is now full equipped with fix pack 3 and has proven to be very stable.

Further, MQ V7.5.0.5 and MQ V8.0.0.3, have deprecated SSLv3 connections due to the Poodle Exploit and reduced the number of supported Cipher Suites. (It is critical you understand the supported connection protocols and Cipher Suites supported by the Federal Reserve Bank.)
But MQ upgrades are only half the story.

If you haven’t been keeping pace with MQ updates, you’ve likely not been keeping pace with OS either. OS updates to supported versions are arguably equally as critical as MQ.

Interested in learning more about your options? Let’s start a conversation. Reach out to TxMQ today.

Why ETL Tools Might Hobble Your Real-Time Reporting & Analytics

Companies report large investments in their data warehousing (DW) or Business Intelligence (BI) systems with a large portion of the software budget spent on extract, transformation and load (ETL) tools that provide ease of populating such warehouses, and the ability to manipulate data to map to the new schemas and data structures.

Companies do need DW or BI for analytics on sales, forecasting, stock management and so much more, and they certainly wouldn’t want to run the additional analytics workloads on top of already taxed core systems, so keeping them isolated is a good idea and a solid architectural decision. Considering, however, the extended infrastructure and support costs involved with managing a new ETL layer, it’s not just the building of them that demands effort. There’s the ongoing scheduling of jobs, on-call support (when jobs fail), the challenge of an ever-shrinking batch window when such jobs are able to run without affecting other production workloads, and other such considerations which make the initial warehouse implementation expense look like penny candy.

So not only do such systems come with extravagant cost, but to make matters worse, the vast majority of jobs run overnight. That means, in most cases, the best possible scenario is that you’re looking at yesterday’s data (not today’s). All your expensive reports and analytics are at least a day later than what you require. What happened to the promised near-realtime information you were expecting?
Contrast the typical BI/DW architecture to the better option of building out your analytics and report processing using realtime routing and transformation with tools such as IBM MQ, DataPower and Integration Bus. Most of the application stacks that process this data in realtime have all the related keys in memory –customer number or ids, order numbers, account details, etc. – and are using them to create or update the core systems. Why duplicate all of that again in your BI/DW ETL layer? If you do, you’re dependent on ETLs going into the core systems to find what happened during that period and extracting all that data again to put it somewhere else.

Alongside this, most organizations are already running application messaging and notifications between applications. lf you have all the data keys in memory, use a DW object, method, function or macro to drop the data as an application message into your messaging layer. The message can then be routed to your DW or BI environment for Transformation and Loading there, no Extraction needed, and you can get rid of your ETL tools.

Simplify your environments and lower the cost of operation. If you have multiple DW or BI environments, use the Pub/Sub capabilities of IBM MQ to distribute the message. You may be exchanging a nominal increase in CPU for eliminating problems, headaches and costs to your DW or BI.

Rethinking your strategy in terms of EAI while removing the whole process and overhead of ETL may indeed bring your whole business analytics to the near-realtime reporting and analytics you expected. Consider that your strategic payoff. Best regards in your architecture endeavors!
Image by Mark Morgan.