X-Blog - Data Streaming

What is data streaming

In an increasingly connected and digitized world, the amount of data produced by daily activities is constantly increasing and processing these can be an expensive operation in terms of time and resources. With data streaming technology, it is possible to process them in real time. This allows data to be transmitted to a recipient continuously and constantly while it is generated or collected, instead of waiting for it to be collected and then sent in bulk.
‍
Data streaming is used in many contexts, such as monitoring sensors in real time, collecting financial market data, broadcasting live videos, and sharing data between applications.
‍
In particular, in network orchestration, this technology is used for the transmission of data in real time between different nodes of a network. This can be useful in applications such as telemedicine or remote control of machinery, where the transmission of data in real time is essential for the operation of the application. Still in the context of orchestration, streaming can be used to send data collected by network or IoT sensors to a central server for processing in real time. This way, you can make quick decisions and correct any issues before they become critical. In addition, streaming can also be used for the distribution of updates or multimedia content within a corporate network, allowing for faster and more seamless installation or playback.

What are the advantages of streaming

Real-time processing: data streaming allows you to make decisions in real time based on data, making it possible to analyze and respond immediately to critical events as well as being able to respond more quickly to changes in the context.
Scalability: thanks to the management of large amounts of data in an efficient and scalable way, it is possible to easily manage their flows and increase their processing capacity as the volume of data increases. This allows you to manage data more effectively and efficiently.
Speed: Data streaming reduces the latency between data collection and processing. This means that data processing can take place immediately after collection, without having to wait for all data to be fully collected.

How it works

Data streaming is a continuous process that involves the transmission of data in real time and is based on multiple architectures; the one most suitable for decoupling is a publish-subscribe type, in which data is transmitted in a continuous flow and consumed by a data processing system asynchronously. The technologies used to implement data streaming include Apache Kafka, Amazon Kinesis, Google Cloud Pub/Sub, Microsoft Azure Event Hubs, and many others. The data streaming process generally consists of four phases:

Data collection: This can be collected from a variety of sources, such as sensors, databases, mobile devices, or social media.
Data transmission: these are then transmitted through a network connection to a streaming system. The transmission can take place through a variety of protocols, such as HTTP, MQTT or WebSocket.
Data processing: This may include cleaning data, transforming data into a common format, or processing statistical analysis on the data.
Data visualization: these are displayed so that users can access the information. Data can be visualized through a variety of means, such as dashboards, charts, maps, or reports.

Data streaming technology therefore offers advanced features for flow management, such as real-time data processing, scalability, latency management and integration with other technologies.

Why choose the Xacria solution

The Xacria solution makes it possible to combine the best features offered by streaming and orchestration, creating with XNO a product capable of managing both Network Automation and Network Orchestration flows.

XNO, thanks to its architecture, guarantees complete horizontal and vertical scalability, thus making it highly scalable in any context. This allows, for example, to manage hundreds of thousands of devices without compromising system performance.

Thanks to perfectly integrated streaming patterns, with XNO it is possible to create continuous closed-loop processes to configure and monitor network equipment in real time. By collecting the collected metrics, it is possible to establish network traffic, identify any problems (such as congestion) and intervene to prevent and resolve them.

This process is in line with today's needs of all operators who are moving towards automation to reduce the need for human intervention in key operations.

The collection of this data is also in line with what is indicated in openconfig which defines a protocol for managing equipment, from configuration to the collection of telemetry streaming data.

XNO therefore makes it possible to create complete automation and orchestration systems to obtain a constant streaming flow of communication with the network infrastructure and to take advantage of the following features:

Analysis and collection of telemetry data in real time to assess the state of the network;
Orchestration and automation for equipment provisioning;
Control of the infrastructure through the closed-loop;
Integration with other existing systems.

Data Streaming

What is data streaming

What are the advantages of streaming

How it works

Why choose the Xacria solution

Company