The Linux Foundation has announced a new sub-project: The Presto Foundation is to further the development and dissemination of the originally developed by Facebook Query Engine. An open, neutral governance model for managing development is designed to ensure that Presto grows as well as fits varied applications.
Facebook had launched the SQL Query Engine 2012. It aims at data analysis in the big data environment. Presto enables querying via various data sources such as MySQL, PostgreSQL, MongoDB, Kafka or even Hadoop, S3 and Alluxio. Motivation for the development was that the Facebook developers direct queries via Apache Hive was not performant enough for the data analysis of the social data processed large amounts of data.
Distribute and rule
A special feature of the engine is that Presto processes the data where it is generated or stored instead of first connecting it to a location for queries. This increases the attractiveness especially for use in large clusters. Presto connects two types of servers: Coordinators and Workers. The latter take care of performing the tasks and processing the data, while the former manage the workers by name and take care of processing the commands and planning queries. The workers, in turn, access the actual data sources through connectors.
His first step into the open source world was made by Presto in 2013 under Apache license. As a result, among others, LinkedIn and Netflix participated in the further development. In January of this year, the Presto executives finally founded their own foundation for the query engine.
Launched under the umbrella of the Linux Foundation, the Presto Foundation is therefore a logical next step in the development of the engine. Besides Facebook, Alibaba, Twitter and Uber are among the founding members. According to the company, more than a thousand employees use Presto on Facebook. You’re probably doing millions of queries every day to handle petabytes of data.
Further details on the founding of the Foundation can be found in the announcement from the Linux Foundation. The source code is available on GitHub, and a project page on GitHub.io provides documentation and an overview of the community activities. Recently, Facebook, together with Alibaba, netifi, Pivotal and Lightbend, founded the Reactive Foundation under the umbrella of the Linux Foundation.
Via | Presto Foundation | Linux Foundation | Source Code