{"id":816,"date":"2022-05-19T14:06:29","date_gmt":"2022-05-19T04:06:29","guid":{"rendered":"https:\/\/www.infrasoft.com.au\/?p=816"},"modified":"2022-05-25T12:16:04","modified_gmt":"2022-05-25T02:16:04","slug":"ulinga-for-kafka-high-performance-fault-tolerant-kafka-streaming","status":"publish","type":"post","link":"https:\/\/www.infrasoft.com.au\/?p=816","title":{"rendered":"uLinga for Kafka &#8211; High Performance, Fault-Tolerant Kafka Streaming"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"816\" class=\"elementor elementor-816\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e1b594f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e1b594f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2072def\" data-id=\"2072def\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-108b126 elementor-widget elementor-widget-theme-post-featured-image elementor-widget-image\" data-id=\"108b126\" data-element_type=\"widget\" data-widget_type=\"theme-post-featured-image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/09\/uLinga-for-Kafka-Article-Title-Graphic-01-1024x576.jpg\" class=\"attachment-large size-large wp-image-756\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/09\/uLinga-for-Kafka-Article-Title-Graphic-01-1024x576.jpg 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/09\/uLinga-for-Kafka-Article-Title-Graphic-01-300x169.jpg 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/09\/uLinga-for-Kafka-Article-Title-Graphic-01-768x432.jpg 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/09\/uLinga-for-Kafka-Article-Title-Graphic-01-1536x864.jpg 1536w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/09\/uLinga-for-Kafka-Article-Title-Graphic-01-2048x1152.jpg 2048w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6df9802 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6df9802\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-03032be\" data-id=\"03032be\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ed1ba3e elementor-widget elementor-widget-text-editor\" data-id=\"ed1ba3e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<h3>Introduction<\/h3>\n<p>HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside large enterprise environments.&nbsp; Kafka is now used by thousands of companies, including 60% of the Fortune 100, and many NonStop shops.&nbsp; Well known users in the finance space include Goldman Sachs, Rabobank, Barclays, Jack Henry and PayPal, and household names that use Kafka include Netflix, Oracle, LinkedIn and AirBnB.&nbsp; A full list of Kafka users is available <a href=\"https:\/\/kafka.apache.org\/powered-by#:~:text=Today%2C%20Kafka%20is%20used%20by,strategies%20with%20event%20streaming%20architecture.\">here.<\/a><\/p>\n<p>These organizations use Kafka to manage \u201cstreams\u201d of data, which have become prevalent as internet usage massively boosts the amount of data being generated and needing to be processed.&nbsp;Kafka allows these huge volumes of data to be processed in real-time, via a combination of \u201cproducers\u201d and \u201cconsumers\u201d, which work with a Kafka \u201ccluster\u201d \u2013 the main data repository.<\/p>\n<p><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">NonStop Servers have valuable data on them that users may want to stream to Kafka, and NonStop applications may also have a need to send their data directly to Kafka.&nbsp; In these situations, NonStop users need a reliable, high-performing way to get that data streamed to Kafka.&nbsp; This is where uLinga for Kafka comes in.<\/span><\/p>\n<div>\n<h3><span style=\"color: inherit; font-family: inherit; font-size: 1.75rem;\">uLinga for Kafka &#8211; Overview<\/span><\/h3>\n<\/div>\n<p><a>uLinga for Kafka is the latest addition to Infrasoft\u2019s uLinga product range, a solution suite that has been used by large banks, telcos and manufacturers to provide reliable mission-critical communications infrastructure for many years.&nbsp; uLinga for Kafka brings the same performance, scalability, security and manageability to the Kafka space as users have successfully utilized with the other uLinga products.<\/a><\/p>\n<p>uLinga for Kafka takes a unique approach to Kafka integration: it runs as a natively compiled Guardian process pair, and supports the Kafka communications protocols directly over TCP\/IP.&nbsp; This removes the need for Java libraries or intermediate databases, providing the best possible performance on HPE NonStop.&nbsp; It also allows uLinga for Kafka to directly communicate with the Kafka cluster, getting streamed data across as quickly and reliably as possible.<\/p>\n<p><\/p>\n<p>This approach also allows uLinga to provide some unique fault-tolerance features, as we\u2019ll explore later in the article.<\/p><p><br><\/p>\n<h3>uLinga for Kafka &#8211; Application Integration<\/h3>\n<p><\/p>\n<p><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">uLinga for Kafka (uLinga) supports a range of options to communicate with HPE NonStop applications and read\/process NonStop data.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">&nbsp; <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">Applications can use standard Guardian IPC messages, or Pathsend requests, to send data to uLinga, and HTTP clients can send data via uLinga\u2019s inbuilt HTTP interface.<\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">&nbsp; <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">uLinga can process disk records directly from Enscribe files, and as of the most recent product release, also supports TMF Audit Trails, meaning updates to all TMF-protected files can be streamed by uLinga to Kafka.<\/span><\/p>\n<p><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b018868 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b018868\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8c03b3a\" data-id=\"8c03b3a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fd486b8 elementor-widget elementor-widget-image\" data-id=\"fd486b8\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"360\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png\" class=\"attachment-large size-large wp-image-825\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-300x135.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-768x346.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams.png 1103w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 1. uLinga for Kafka Accessing TMF Audit Trail Data<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-af5f745 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"af5f745\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4ae1c0d\" data-id=\"4ae1c0d\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c2c1864 elementor-widget elementor-widget-text-editor\" data-id=\"c2c1864\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<h3><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-size: 16px;\">Fault Tolerance and Reliability<\/span><\/h3><p>There are a number of features within the uLinga for Kafka implementation that ensures it is one of the most reliable Kafka solutions available.<\/p><p><b>FILEREADER and AUDITREADER Guaranteed Delivery<\/b><\/p><p>When processing Enscribe files or TMF audit trails, uLinga has a unique method of guaranteeing delivery of data to Kafka.\u00a0\u00a0 Whenever an Enscribe or TMF audit trail record on NonStop is read and streamed to Kafka, the record sent by uLinga includes a Kafka header that contains a record key.\u00a0 That record key allows uLinga to locate that record in the original NonStop file (either Enscribe or TMF audit trail).\u00a0 If for some reason streaming to Kafka is interrupted \u2013 because the Kafka cluster is down, or the network connection is unavailable, or even if the uLinga process stops \u2013 uLinga will recommence streaming records whenever it is able to restart.\u00a0 When that restart begins, uLinga will query the Kafka cluster to determine the most recent record that was streamed, and will automatically start reading from that point in the Enscribe file or TMF audit trail.\u00a0 This ensures that all records are sent to Kafka, even following an outage.<\/p><p><b>Kafka EOS Support<\/b><\/p><p>Along with this, uLinga supports Kafka\u2019s exactly-once semantics (EOS).\u00a0 EOS is a framework that allows applications to stream data to Kafka without loss or duplication, ensuring that messages are always sent to Kafka, even if a network connection to the cluster fails.\u00a0<\/p><p><b>Checkpointing<\/b><\/p><p>Finally, like all uLinga products, uLinga for Kafka optionally runs as a NonStop process pair.\u00a0 When run in this manner, it can be configured to checkpoint data at various points in uLinga\u2019s processing.\u00a0 If we consider an application that is writing data to uLinga via uLinga\u2019s IPCSERVER interface, we would see a configuration similar to this:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-77ed71f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"77ed71f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ce4a7ef\" data-id=\"ce4a7ef\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c8bbc6e elementor-widget elementor-widget-image\" data-id=\"c8bbc6e\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"372\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig2.png\" class=\"attachment-large size-large wp-image-827\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig2.png 996w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig2-300x139.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig2-768x357.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 2. uLinga for Kafka's IPCSERVER Interface<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-eb234c0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"eb234c0\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2f1d5a6\" data-id=\"2f1d5a6\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-af6c64d elementor-widget elementor-widget-text-editor\" data-id=\"af6c64d\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p class=\"MsoNormal\">The IPCSERVER resource presents an application interface for a NonStop process to communicate with via $RECEIVE.\u00a0 The KAFKAPRODUCER resource is configured with Kafka-specific information, including broker IP address and port, and Kafka Topic name.\u00a0 Checkpointing can be configured on both the IPCSERVER and the KAFKAPRODUCER resources.\u00a0 IPCSERVER checkpointing will ensure that all data received and sent between the Guardian (or Pathsend process) is checkpointed to the uLinga backup process.\u00a0 KAFKAPRODUCER checkpointing will do the same thing for all data sent to\/received from the Kafka cluster.\u00a0 Depending on the application environment, checkpointing may be required at one or both points.<\/p><p class=\"MsoNormal\"><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight );\">In the example depicted below, uLinga for Kafka is running as a NonStop process pair, with checkpointing configured on the IPCSERVER resource.\u00a0 All data flowing through the IPCSERVER resource is checkpointed to the backup process, as indicated by the arrow.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-924d142 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"924d142\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-940aa75\" data-id=\"940aa75\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9ed4db3 elementor-widget elementor-widget-image\" data-id=\"9ed4db3\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-3-1024x576.png\" class=\"attachment-large size-large wp-image-828\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-3-1024x576.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-3-300x169.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-3-768x432.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-3.png 1280w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 3. uLinga Running as a NonStop Process Pair with Checkpointing<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8fed491 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8fed491\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fc56b6b\" data-id=\"fc56b6b\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9217dfc elementor-widget elementor-widget-text-editor\" data-id=\"9217dfc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<h3><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">In the event of the main process dying, due to a CPU failure, or similar, the backup process will take over, and the operating system will redirect the Guardian process or Pathsend Client to the backup process.\u00a0 <\/span><span style=\"color: var( --e-global-color-text ); font-family: var( --e-global-typography-text-font-family ), Sans-serif; font-weight: var( --e-global-typography-text-font-weight ); font-size: 16px;\">This happens seamlessly, and transparently to the Guardian\/Pathsend process.<\/span><\/h3><div><p>\u00a0<\/p><\/div>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0c8ad6c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0c8ad6c\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-23af5f7\" data-id=\"23af5f7\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b0d1ad3 elementor-widget elementor-widget-image\" data-id=\"b0d1ad3\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-4-1024x576.png\" class=\"attachment-large size-large wp-image-829\" alt=\"\" srcset=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-4-1024x576.png 1024w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-4-300x169.png 300w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-4-768x432.png 768w, https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/Diagrams-for-May-June-2022-Connection-Article-Fig-4.png 1280w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 4. uLinga Backup Process Takeover<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b516349 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b516349\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-43a9c52\" data-id=\"43a9c52\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6349d52 elementor-widget elementor-widget-text-editor\" data-id=\"6349d52\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><b>High Performance<\/b><\/p><p><o:p><\/o:p><\/p><p>Because uLinga for Kafka is written in C, and does not\nrequire any additional libraries or interim servers, it is able to achieve\nimpressive performance figures, with extremely low latency.<o:p><\/o:p><\/p><p>Infrasoft testing has shown that the product is able to\nexceed 25,000 TPS with sub-millisecond latency.&nbsp;\nPerhaps more importantly given most NonStops are used in OLTP\nenvironments, when running at a steady state of 1000TPS, uLinga for Kafka has a\nvery low CPU requirement.&nbsp; This will\nallow uLinga for Kafka to process data from the busiest of NonStop OLTP\napplications with minimal overhead.<o:p><\/o:p><\/p><p>\n\n\n\n\n\n<\/p><p>uLinga for Kafka is now available.&nbsp; Please contact <a href=\"mailto:productinfo@infrasoft.com.au\">productinfo@infrasoft.com.au<\/a> for\nmore information or to arrange a trial.<o:p><\/o:p><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Introduction HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside large enterprise environments.&nbsp; Kafka is now used by thousands of companies, including 60% of the Fortune 100, and many NonStop shops.&nbsp; Well known users in the finance space include Goldman Sachs, Rabobank, Barclays, Jack Henry [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-816","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>uLinga for Kafka - High Performance, Fault-Tolerant Kafka Streaming - Infrasoft<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.infrasoft.com.au\/?p=816\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"uLinga for Kafka - High Performance, Fault-Tolerant Kafka Streaming - Infrasoft\" \/>\n<meta property=\"og:description\" content=\"Introduction HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside large enterprise environments.&nbsp; Kafka is now used by thousands of companies, including 60% of the Fortune 100, and many NonStop shops.&nbsp; Well known users in the finance space include Goldman Sachs, Rabobank, Barclays, Jack Henry [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.infrasoft.com.au\/?p=816\" \/>\n<meta property=\"og:site_name\" content=\"Infrasoft\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited\" \/>\n<meta property=\"article:published_time\" content=\"2022-05-19T04:06:29+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-05-25T02:16:04+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png\" \/>\n<meta name=\"author\" content=\"website\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"website\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816\"},\"author\":{\"name\":\"website\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56\"},\"headline\":\"uLinga for Kafka &#8211; High Performance, Fault-Tolerant Kafka Streaming\",\"datePublished\":\"2022-05-19T04:06:29+00:00\",\"dateModified\":\"2022-05-25T02:16:04+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816\"},\"wordCount\":1081,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png\",\"articleSection\":[\"Uncategorized\"],\"inLanguage\":\"en-AU\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.infrasoft.com.au\/?p=816#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816\",\"url\":\"https:\/\/www.infrasoft.com.au\/?p=816\",\"name\":\"uLinga for Kafka - High Performance, Fault-Tolerant Kafka Streaming - Infrasoft\",\"isPartOf\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png\",\"datePublished\":\"2022-05-19T04:06:29+00:00\",\"dateModified\":\"2022-05-25T02:16:04+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#breadcrumb\"},\"inLanguage\":\"en-AU\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.infrasoft.com.au\/?p=816\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage\",\"url\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams.png\",\"contentUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams.png\",\"width\":1103,\"height\":497},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.infrasoft.com.au\/?p=816#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.infrasoft.com.au\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"uLinga for Kafka &#8211; High Performance, Fault-Tolerant Kafka Streaming\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#website\",\"url\":\"https:\/\/www.infrasoft.com.au\/\",\"name\":\"Infrasoft\",\"description\":\"High Availability Software Infrastructure\",\"publisher\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.infrasoft.com.au\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-AU\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#organization\",\"name\":\"Infrasoft\",\"url\":\"https:\/\/www.infrasoft.com.au\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg\",\"contentUrl\":\"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg\",\"width\":512,\"height\":512,\"caption\":\"Infrasoft\"},\"image\":{\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56\",\"name\":\"website\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-AU\",\"@id\":\"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g\",\"caption\":\"website\"},\"sameAs\":[\"https:\/\/www.infrasoft.com.au\/\"],\"url\":\"https:\/\/www.infrasoft.com.au\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"uLinga for Kafka - High Performance, Fault-Tolerant Kafka Streaming - Infrasoft","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.infrasoft.com.au\/?p=816","og_locale":"en_US","og_type":"article","og_title":"uLinga for Kafka - High Performance, Fault-Tolerant Kafka Streaming - Infrasoft","og_description":"Introduction HPE NonStop users are becoming more familiar with Kafka, as Kafka in turn becomes more and more prevalent inside large enterprise environments.&nbsp; Kafka is now used by thousands of companies, including 60% of the Fortune 100, and many NonStop shops.&nbsp; Well known users in the finance space include Goldman Sachs, Rabobank, Barclays, Jack Henry [&hellip;]","og_url":"https:\/\/www.infrasoft.com.au\/?p=816","og_site_name":"Infrasoft","article_publisher":"https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited","article_published_time":"2022-05-19T04:06:29+00:00","article_modified_time":"2022-05-25T02:16:04+00:00","og_image":[{"url":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png","type":"","width":"","height":""}],"author":"website","twitter_card":"summary_large_image","twitter_misc":{"Written by":"website","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.infrasoft.com.au\/?p=816#article","isPartOf":{"@id":"https:\/\/www.infrasoft.com.au\/?p=816"},"author":{"name":"website","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56"},"headline":"uLinga for Kafka &#8211; High Performance, Fault-Tolerant Kafka Streaming","datePublished":"2022-05-19T04:06:29+00:00","dateModified":"2022-05-25T02:16:04+00:00","mainEntityOfPage":{"@id":"https:\/\/www.infrasoft.com.au\/?p=816"},"wordCount":1081,"commentCount":0,"publisher":{"@id":"https:\/\/www.infrasoft.com.au\/#organization"},"image":{"@id":"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage"},"thumbnailUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png","articleSection":["Uncategorized"],"inLanguage":"en-AU","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.infrasoft.com.au\/?p=816#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.infrasoft.com.au\/?p=816","url":"https:\/\/www.infrasoft.com.au\/?p=816","name":"uLinga for Kafka - High Performance, Fault-Tolerant Kafka Streaming - Infrasoft","isPartOf":{"@id":"https:\/\/www.infrasoft.com.au\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage"},"image":{"@id":"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage"},"thumbnailUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams-1024x461.png","datePublished":"2022-05-19T04:06:29+00:00","dateModified":"2022-05-25T02:16:04+00:00","breadcrumb":{"@id":"https:\/\/www.infrasoft.com.au\/?p=816#breadcrumb"},"inLanguage":"en-AU","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.infrasoft.com.au\/?p=816"]}]},{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.infrasoft.com.au\/?p=816#primaryimage","url":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams.png","contentUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2022\/05\/TMF-Audit-Trail-Diagrams.png","width":1103,"height":497},{"@type":"BreadcrumbList","@id":"https:\/\/www.infrasoft.com.au\/?p=816#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.infrasoft.com.au\/"},{"@type":"ListItem","position":2,"name":"uLinga for Kafka &#8211; High Performance, Fault-Tolerant Kafka Streaming"}]},{"@type":"WebSite","@id":"https:\/\/www.infrasoft.com.au\/#website","url":"https:\/\/www.infrasoft.com.au\/","name":"Infrasoft","description":"High Availability Software Infrastructure","publisher":{"@id":"https:\/\/www.infrasoft.com.au\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.infrasoft.com.au\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-AU"},{"@type":"Organization","@id":"https:\/\/www.infrasoft.com.au\/#organization","name":"Infrasoft","url":"https:\/\/www.infrasoft.com.au\/","logo":{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/","url":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg","contentUrl":"https:\/\/www.infrasoft.com.au\/wp-content\/uploads\/2021\/10\/cropped-infrasoft_icon.jpg","width":512,"height":512,"caption":"Infrasoft"},"image":{"@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/infrasoft-pty-limited"]},{"@type":"Person","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/ab1cfc963931c33ab30cec69b057ad56","name":"website","image":{"@type":"ImageObject","inLanguage":"en-AU","@id":"https:\/\/www.infrasoft.com.au\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f8ed84048fcdb5afa88338fd16181fdd?s=96&d=mm&r=g","caption":"website"},"sameAs":["https:\/\/www.infrasoft.com.au\/"],"url":"https:\/\/www.infrasoft.com.au\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts\/816"}],"collection":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=816"}],"version-history":[{"count":19,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts\/816\/revisions"}],"predecessor-version":[{"id":844,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=\/wp\/v2\/posts\/816\/revisions\/844"}],"wp:attachment":[{"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=816"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=816"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.infrasoft.com.au\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=816"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}