{"id":955,"date":"2023-03-08T14:58:25","date_gmt":"2023-03-08T04:58:25","guid":{"rendered":"https:\/\/www.infrasoft.com.au\/?p=955"},"modified":"2024-03-28T16:05:13","modified_gmt":"2024-03-28T06:05:13","slug":"ulinga-for-kafka-now-with-kafka-consumer-support","status":"publish","type":"post","link":"https:\/\/www.infrasoft.com.au\/?p=955","title":{"rendered":"uLinga for Kafka &#8211; Now with Kafka Consumer Support"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"955\" class=\"elementor elementor-955\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e1b594f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e1b594f\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2072def\" data-id=\"2072def\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-108b126 elementor-widget elementor-widget-theme-post-featured-image elementor-widget-image\" data-id=\"108b126\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"theme-post-featured-image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"678\" height=\"381\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/uLinga-for-Kafka-Article-Title-Graphic-02-678x381-1.jpg\" class=\"attachment-large size-large wp-image-957\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/uLinga-for-Kafka-Article-Title-Graphic-02-678x381-1.jpg 678w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/uLinga-for-Kafka-Article-Title-Graphic-02-678x381-1-300x169.jpg 300w\" sizes=\"(max-width: 678px) 100vw, 678px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6df9802 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6df9802\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-03032be\" data-id=\"03032be\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ed1ba3e elementor-widget elementor-widget-text-editor\" data-id=\"ed1ba3e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h3>Introduction<\/h3><p>HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside those large environments where the NonStop platform is often present.\u00a0Kafka is being used by thousands of companies, including 60% of the Fortune 100.\u00a0<\/p><p>\u00a0<\/p><p>These organizations use Kafka to manage \u201cstreams\u201d of data, which have become more common as internet usage massively boosts the amount of data being generated and needing to be processed.\u00a0 Kafka allows these huge volumes of data to be processed in real-time, via a combination of \u201cproducers\u201d and \u201cconsumers\u201d, which work with a Kafka \u201ccluster\u201d \u2013 the main data repository.\u00a0<\/p><p>Many HPE NonStop users need to integrate with Kafka in some way, be it to stream NonStop application data to Kafka, or integrate data from Kafka back into their NonStop applications. While there are Java-based solutions to send (PRODUCE) data to and receive (CONSUME) data from Kafka, these require OSS and may not perform at the level that NonStop users require.<\/p><p>uLinga for Kafka, which has had Kafka Producer functionality for over a year, has recently been enhanced to add Kafka Consumer functionality.\u00a0 This article explains how that functionality works and gives some potential use cases that NonStop users may find interesting.<\/p><div><h3><span style=\"color: inherit; font-family: inherit; font-size: 1.75rem;\">uLinga for Kafka &#8211; Overview<\/span><\/h3><\/div><p><a style=\"color: #7a7a7a; background-color: #f9f9f9;\">uLinga for Kafka takes a unique approach to Kafka integration: it runs as a natively compiled NonStop process<br \/>pair, and supports the Kafka communications protocols directly over TCP\/IP.\u00a0 This removes the need for Java<br \/>libraries or intermediate databases, providing the best possible performance on HPE NonStop.\u00a0 It also allows uLinga for<br \/>Kafka to directly communicate with the Kafka cluster, getting streamed data across as quickly and reliably as possible.<\/a><\/p><p>With the latest release of uLinga for Kafka now available, Kafka Consumer functionality has been added bringing with it the same lightning-fast performance as the existing Producer functionality.<\/p><h3>uLinga for Kafka as Kafka Consumer<\/h3><p>Previous releases of uLinga for Kafka provide a range of options to stream NonStop data to Kafka, via uLinga\u2019s Kafka Producer functionality.\u00a0 These features have been outlined in our <a style=\"background-color: #f9f9f9;\" href=\"https:\/\/connect2nonstop.com\/author\/andrew\/\">previous Connection articles<\/a>.<\/p><p>\u00a0<\/p><p>With the addition of Kafka Consumer functionality, uLinga now supports a range of options to stream Kafka data directly from the Kafka cluster, and apply that data to a NonStop database or send it directly to a NonStop application.\u00a0<span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b018868 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b018868\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8c03b3a\" data-id=\"8c03b3a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fd486b8 elementor-widget elementor-widget-image\" data-id=\"fd486b8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png\" class=\"attachment-large size-large wp-image-958\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-300x169.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-768x432.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1.png 1280w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 1. uLinga for Kafka - Consumer to Enscribe File<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-af5f745 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"af5f745\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4ae1c0d\" data-id=\"4ae1c0d\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c2c1864 elementor-widget elementor-widget-text-editor\" data-id=\"c2c1864\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h3><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">The diagram above depicts uLinga for Kafka acting as Kafka Consumer and writing the data consumed to an Enscribe file.\u00a0 <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">This file could be used, for example, as a database refresh for a NonStop application.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">\u00a0 <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">As events\/records are streamed to the cluster by a remote producer, uLinga\u2019s Consumer logic will retrieve those records, process them, and apply them to the Enscribe file.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">\u00a0 <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">As with uLinga\u2019s Producer functionality, very high throughput (greater than 20,000TPS) is possible, with sub-millisecond latency.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">\u00a0 <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">This allows for support of the largest application requirements.<\/span><\/h3><p>\u00a0<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-77ed71f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"77ed71f\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ce4a7ef\" data-id=\"ce4a7ef\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c8bbc6e elementor-widget elementor-widget-image\" data-id=\"c8bbc6e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-2-1024x576.png\" class=\"attachment-large size-large wp-image-962\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-2-1024x576.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-2-300x169.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-2-768x432.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-2.png 1280w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 2. uLinga for Kafka - Consumer direct to Process or Pathway serverclass<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-eb234c0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"eb234c0\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2f1d5a6\" data-id=\"2f1d5a6\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-af6c64d elementor-widget elementor-widget-text-editor\" data-id=\"af6c64d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">Figure 2 shows another use case for Kafka Consumer functionality.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">&nbsp; <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">In this scenario uLinga for Kafka is forwarding consumed data directly to a NonStop application, either via a Guardian WRITE\/WRITEREAD or a Pathway&nbsp; ERVERCLASS_SEND_ directly to a Pathway Serverclass.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">&nbsp; <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">Once again this configuration supports very high throughput, allowing for any application load to be handled.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">&nbsp; <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">This implementation might be used, for instance, to apply online updates to an application database.<\/span><\/p>\n<p><\/p>\n<p>As our <a href=\"https:\/\/connect2nonstop.com\/ulinga-for-kafka-support-for-custom-code-via-userexits\/\">previous Connection article<\/a> outlined, uLinga for Kafka supports User Exits and these can be invoked in either of the above scenarios to support data manipulation or any other custom processing.<\/p>\n<p class=\"MsoNormal\"><b>Using Kafka for Transactional Messaging<\/b><\/p>\n<p><\/p>\n<p class=\"MsoNormal\">The addition of Consumer functionality to uLinga for Kafka opens up some interesting possibilities where Kafka can be used to handle large-scale transaction messaging.&nbsp; When a produced request message can be linked to a consumed response, you have a model for distributed transaction processing that looks quite similar to some message queuing solutions.&nbsp; Consider the following example configuration:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-924d142 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"924d142\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-940aa75\" data-id=\"940aa75\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ef4114a elementor-widget elementor-widget-image\" data-id=\"ef4114a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-3-1024x576.png\" class=\"attachment-large size-large wp-image-963\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-3-1024x576.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-3-300x169.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-3-768x432.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-3.png 1280w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 3. uLinga for Kafka - Produce-Consume Integration with Linux Application<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8fed491 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8fed491\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fc56b6b\" data-id=\"fc56b6b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9217dfc elementor-widget elementor-widget-text-editor\" data-id=\"9217dfc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The following steps take place:<\/p><ol><li>Guardian application issues WRITEREAD\/Pathway application issues SERVERCLASS_SEND_<\/li><li>uLinga for Kafka PRODUCEs data from WRITEREAD\/SERVERCLASS_SEND_ to Kafka, including correlation data (passed either in the Key or a custom header field)<\/li><li>Linux Consumer logic retrieves data from Kafka, including correlation data, Linux application does some processing of data<\/li><li>Linux Producer logic sends response as a Kafka PRODUCE, including original correlation data<\/li><li>uLinga for Kafka CONSUMEs response, including correlation data<\/li><li>uLinga for Kafka correlates response to original request and completes the WRITEREAD\/SERVERCLASS_SEND_<\/li><\/ol><p>\u00a0<\/p><p>This configuration provides a high-performance, reliable messaging mechanism between HPE NonStop and (in this example) a Linux\/cloud-based application, utilising the extensive Kafka support already available to Linux applications.\u00a0 It is also an extremely simple method, from the point of view of the NonStop application, to communicate via Kafka &#8211; using a straightforward WRITEREAD or SERVERCLASS_SEND_.<\/p><p>Depending on specific application and environmental requirements, this approach may provide better clustering\/fault-tolerance than some establishing queuing solutions.\u00a0 It is also likely to perform at least as well, if not considerably better, than most other alternatives.\u00a0 In particular, the low latency of this approach is impressive \u2013 initial testing by Infrasoft has seen overall latency as low as 1ms with this configuration.<\/p><p>If there is a requirement for different \u201cchannels\u201d for the request and response to be sent on, this can be achieved with a minor adjustment to the configuration depicted in Figure 3, and provides completely asynchronous request\/response handling.<\/p><div>\u00a0<\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5cd381d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5cd381d\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f73a595\" data-id=\"f73a595\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-696358c elementor-widget elementor-widget-image\" data-id=\"696358c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-4-1024x576.png\" class=\"attachment-large size-large wp-image-971\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-4-1024x576.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-4-300x169.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-4-768x432.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-4.png 1280w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 4. Separate Channels for Request and Response Messaging<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f6aa66a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f6aa66a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a61e9e3\" data-id=\"a61e9e3\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1ca8cf1 elementor-widget elementor-widget-text-editor\" data-id=\"1ca8cf1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In this example, the NonStop application would handle context management between request and response messages.<\/p><p><strong>Conclusion<\/strong><\/p><p>uLinga for Kafka provides reliable, high performance options for integrating NonStop applications and data with Kafka.\u00a0 The addition of Kafka Consumer functionality gives customers the ability to stream data from Kafka and easily incorporate it with NonStop databases and applications.\u00a0 With uLinga for Kafka\u2019s unique NonStop interprocess communication (IPC) support, customers have the ability to use Kafka in new ways, be it as a high-performance messaging hub, a feed for application updates or any number of additional use cases.<\/p><p>For more information on uLinga for Kafka, speak to our partners at comforte and TIC Software, or contact us at <a href=\"mailto:productinfo@infrasoft.com.au\">productinfo@infrasoft.com.au<\/a>.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Introduction HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside those large environments where the NonStop platform is often present.\u00a0Kafka is being used by thousands of companies, including 60% of the Fortune 100.\u00a0 \u00a0 These organizations use Kafka to manage \u201cstreams\u201d of data, which have [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-955","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>uLinga for Kafka - Now with Kafka Consumer Support - Infrasoft<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.infrasoft.com.au\/?p=955\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"uLinga for Kafka - Now with Kafka Consumer Support - Infrasoft\" \/>\n<meta property=\"og:description\" content=\"Introduction HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside those large environments where the NonStop platform is often present.\u00a0Kafka is being used by thousands of companies, including 60% of the Fortune 100.\u00a0 \u00a0 These organizations use Kafka to manage \u201cstreams\u201d of data, which have [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.infrasoft.com.au\/?p=955\" \/>\n<meta property=\"og:site_name\" content=\"Infrasoft\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited\" \/>\n<meta property=\"article:published_time\" content=\"2023-03-08T04:58:25+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-28T06:05:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png\" \/>\n<meta name=\"author\" content=\"website\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"website\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955\"},\"author\":{\"name\":\"website\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56\"},\"headline\":\"uLinga for Kafka &#8211; Now with Kafka Consumer Support\",\"datePublished\":\"2023-03-08T04:58:25+00:00\",\"dateModified\":\"2024-03-28T06:05:13+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955\"},\"wordCount\":1058,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png\",\"articleSection\":[\"Uncategorized\"],\"inLanguage\":\"en-AU\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.infrasoft.com.au\/?p=955#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955\",\"url\":\"https:\/\/www.infrasoft.com.au\/?p=955\",\"name\":\"uLinga for Kafka - Now with Kafka Consumer Support - Infrasoft\",\"isPartOf\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png\",\"datePublished\":\"2023-03-08T04:58:25+00:00\",\"dateModified\":\"2024-03-28T06:05:13+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#breadcrumb\"},\"inLanguage\":\"en-AU\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.infrasoft.com.au\/?p=955\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage\",\"url\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1.png\",\"contentUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1.png\",\"width\":1280,\"height\":720},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=955#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.infrasoft.com.au\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"uLinga for Kafka &#8211; Now with Kafka Consumer Support\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#website\",\"url\":\"https:\/\/www.infrasoft.com.au\/\",\"name\":\"Infrasoft\",\"description\":\"High Availability Software Infrastructure\",\"publisher\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.infrasoft.com.au\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-AU\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#organization\",\"name\":\"Infrasoft\",\"url\":\"https:\/\/www.infrasoft.com.au\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg\",\"contentUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg\",\"width\":512,\"height\":512,\"caption\":\"Infrasoft\"},\"image\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56\",\"name\":\"website\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g\",\"caption\":\"website\"},\"sameAs\":[\"https:\/\/www.infrasoft.com.au\/\"],\"url\":\"https:\/\/www.infrasoft.com.au\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"uLinga for Kafka - Now with Kafka Consumer Support - Infrasoft","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.infrasoft.com.au\/?p=955","og_locale":"en_US","og_type":"article","og_title":"uLinga for Kafka - Now with Kafka Consumer Support - Infrasoft","og_description":"Introduction HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside those large environments where the NonStop platform is often present.\u00a0Kafka is being used by thousands of companies, including 60% of the Fortune 100.\u00a0 \u00a0 These organizations use Kafka to manage \u201cstreams\u201d of data, which have [&hellip;]","og_url":"https:\/\/www.infrasoft.com.au\/?p=955","og_site_name":"Infrasoft","article_publisher":"https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited","article_published_time":"2023-03-08T04:58:25+00:00","article_modified_time":"2024-03-28T06:05:13+00:00","og_image":[{"url":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png","type":"","width":"","height":""}],"author":"website","twitter_card":"summary_large_image","twitter_misc":{"Written by":"website","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.infrasoft.com.au\/?p=955#article","isPartOf":{"@id":"https:\/\/www.infrasoft.com.au\/?p=955"},"author":{"name":"website","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56"},"headline":"uLinga for Kafka &#8211; Now with Kafka Consumer Support","datePublished":"2023-03-08T04:58:25+00:00","dateModified":"2024-03-28T06:05:13+00:00","mainEntityOfPage":{"@id":"https:\/\/www.infrasoft.com.au\/?p=955"},"wordCount":1058,"commentCount":0,"publisher":{"@id":"https:\/\/www.infrasoft.com.au\/#organization"},"image":{"@id":"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage"},"thumbnailUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png","articleSection":["Uncategorized"],"inLanguage":"en-AU","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.infrasoft.com.au\/?p=955#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.infrasoft.com.au\/?p=955","url":"https:\/\/www.infrasoft.com.au\/?p=955","name":"uLinga for Kafka - Now with Kafka Consumer Support - Infrasoft","isPartOf":{"@id":"https:\/\/www.infrasoft.com.au\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage"},"image":{"@id":"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage"},"thumbnailUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1-1024x576.png","datePublished":"2023-03-08T04:58:25+00:00","dateModified":"2024-03-28T06:05:13+00:00","breadcrumb":{"@id":"https:\/\/www.infrasoft.com.au\/?p=955#breadcrumb"},"inLanguage":"en-AU","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.infrasoft.com.au\/?p=955"]}]},{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.infrasoft.com.au\/?p=955#primaryimage","url":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1.png","contentUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2023\/03\/2023-03-01-Kafka-Consumer-diags-for-Connection-Fig-1.png","width":1280,"height":720},{"@type":"BreadcrumbList","@id":"https:\/\/www.infrasoft.com.au\/?p=955#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.infrasoft.com.au\/"},{"@type":"ListItem","position":2,"name":"uLinga for Kafka &#8211; Now with Kafka Consumer Support"}]},{"@type":"WebSite","@id":"https:\/\/www.infrasoft.com.au\/#website","url":"https:\/\/www.infrasoft.com.au\/","name":"Infrasoft","description":"High Availability Software Infrastructure","publisher":{"@id":"https:\/\/www.infrasoft.com.au\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.infrasoft.com.au\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-AU"},{"@type":"Organization","@id":"https:\/\/www.infrasoft.com.au\/#organization","name":"Infrasoft","url":"https:\/\/www.infrasoft.com.au\/","logo":{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/","url":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg","contentUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg","width":512,"height":512,"caption":"Infrasoft"},"image":{"@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited"]},{"@type":"Person","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56","name":"website","image":{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g","caption":"website"},"sameAs":["https:\/\/www.infrasoft.com.au\/"],"url":"https:\/\/www.infrasoft.com.au\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts\/955"}],"collection":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=955"}],"version-history":[{"count":14,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts\/955\/revisions"}],"predecessor-version":[{"id":978,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts\/955\/revisions\/978"}],"wp:attachment":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=955"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=955"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=955"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}