Data pipeline on cloud
Articles Case Studies Cloud Cloud Computing Confluent Kafka Data Engineering Data Pipeline Data Pipelines Event Streaming Kafka Real time data streaming Real Time Streaming

CASE STUDY: Drug discovery with new data pipelines based on Confluent Cloud


Recursion is a biotechnology company founded in 2013, headquartered in Salt Lake City, Utah. It accelerates drug discovery by combining experimental biology, artificial intelligence, automation, and real-time event streaming. It has built a system that processes over three petabytes of biological image data generated on Recursion’s robotic platform. The company

Data Streaming Use Cases And Applications
Apache Kafka Apache Nifi Articles Data Engineering Data Ingestion Tools Data Modeling Data Pipelines Data Structuring Real time data

Enterprise Data Streaming Architecture


A modern enterprise Data Streaming Architecture will have a lot of communications, these can include application queues, streaming data, control data, data transfers, for backup, syncing, updates, IOT sensor data, system snapshots and other never ending use cases. We would be discussing about some commonly used use-cases in most enterprises.

Big data analytics using Kafka
Apache Kafka Articles Data Pipelines Kafka Architecture Kafka Use Cases Real time data streaming Real Time Streaming

Real-Time Streaming and Data Pipelines with Apache Kafka


Every large corporation and SME have found real-time data analysis quite critical. Many industries such as legal services, financial services and IT operation management are in need of massive real-time data along with historical data. When we are in need of handling high volume data, we should implement the best

How To Build a Scalable Big Data Pipeline
Articles Data Engineering Data Pipeline Data Pipelines

How To Build a Scalable Big Data Pipeline


When you deploy machine learning, big-data analytics, and data science in real-time, you need to remember that model training and analytics tuning occupies only a portion of the work. Around 50% of the effort is dependent upon grooming the data for Machine Learning and Analytics. The rest of the effort

Data Pipeline definition
Articles Data Pipeline Data Pipelines ETL Pipeline Real time data

What is a Data Pipeline? Definition & Examples


Have you ever star-gazed? Let’s imagine that you are counting the number of stars in the sky. Would you be able to count all the stars? You can categorize them for sure. That’s exactly how abundant data is nowadays. When you allow data flow from one location to the next,

Real time data streaming and Analytics
Articles Case Studies Data Pipeline Data Pipelines Real time data Real time data streaming Real Time Streaming

CASE STUDY: Real time analytics & Data management at Charter


Charter Communications, Inc. is a leading broadband connectivity company and cable operator. Through their brand, Spectrum, they offer a full range of state-of-the-art residential and business services including Spectrum Internet®, TV, Mobile, and Voice in 41 states for more than 30 million customers. As customers always require better reliability, competitive

Real time Data Streaming
Apache Kafka Apache Nifi Articles Data Ingestion Tools Data Pipeline Data Pipelines Kafka Architecture Kafka as a Service Kafka Use Cases

Data Ingestion Pipelines & use cases


What is a Data pipeline? A data pipeline is a system where data is transferred in chunks in a serial and systematic manner (Messages, records) between systems. These flows are well defined, audited and might contain sensitive information, which needs to be secured.  These pipelines can be application queues, transfers

Real time Data Streaming
Articles Data Engineering Data Pipeline Data Pipelines Kafka Streams Real Time Streaming

What Is Streaming Data? Guide To Real-time Data And Stream Processing


Data is indeed the new oil, and real-time data processing & real time stream processing is the one that unlocks its potential to drive businesses in this technologically advanced era. Now, businesses need high-speed data pipelines more than before, to streamline their processes and match the customer’s expectations. A report

developers working on the computer
Apache Kafka Articles Data Engineering Data Pipeline Data Pipelines Microservice Architecture Microservices

Implementing MongoDB to Elastic Search 7.X Data Pipeline


In this article, we will see how to implement a data pipeline from an application to Mongo DB database and from there into an Elastic Search keeping the same document ID using Kafka Connect in a Microservice Architecture. In recent days and years, all the microservices architectures are asynchronous in