Lightbend, the company behind the JVM language Scala and developer of the Reactive Platform, has announced the official availability of an expansion of its Fast Data Platform: Lightbend Pipelines will cover all aspects from design through provisioning to operation of streaming data pipelines and their integration into microservices Simplify and accelerate applications.

The system follows a modular approach to seamlessly integrate different techniques. Lightbend Pipelines supports a variety of streaming engines, such as Akka Streams or Spark Structured Streaming, so users can also merge components from different data streaming applications, such as extract, transform, load (ETL) processes, or developing machine learning models in a pipeline can. The necessary adjustments Lightbend takes over pipelines – the writing of Boilerplate code can largely be omitted, promises the manufacturer.

Based on pre-integrated Kubernetes environments, the runtime of the system also automates numerous tasks around deployment, the routing of data between the streamlets and data security. Management and monitoring of the streaming data pipelines can be understood via a console with a graphical interface. A first impression of the functionality of Lightbend Pipelines provides the following video.

[su_youtube url=”https://youtu.be/16cujevI83E”]

The Lightbend Platform combines open-source and commercial technologies to design, build, and deploy streaming and fast-data applications as well as reactive microservices. The Fast Data Platform, first introduced in 2017, incorporates features of reactive systems such as application responsiveness, resilience, and elasticity in streaming-based data applications. The platform integrates streaming engines such as Akka Streams, Flink or Kafka Streams as well as the ML frameworks TensorFlow, Kubeflow and Spark ML.