AI - Delta Lake

Empowering data-driven organizations with ACID transactions and continuous processing using Delta Lake's open-source, massively collaborative data lake built on Apache Spark.

Logo of Delta Lake
Last Audited At

About Delta Lake

Delta Lake is an open-source, massively collaborative data lake built on Apache Spark and Opensource Deltasquare's computing technology. It aims to bring ACID transactions to apache spark using a delta file format that provides continuous, versioned, and scalable big data processing.

They offer an API for interacting with Delta Lake metadata, ensuring compatibility between different versions of the API and data storage systems. Their roadmap outlines plans for future developments, which are detailed in their GitHub milestones.

To develop with Delta Lake, users can import it as a new project into IntelliJ IDEA following specific instructions provided. The transaction protocol is defined in a dedicated document, allowing for efficient and reliable data processing.

Delta Lake is a part of the Apache Software Foundation and maintains an active community through various communication channels including a public Slack channel, LinkedIn company page, YouTube channel, and Google groups forum. Users can also report issues and contribute to the project as per guidelines provided. The project is licensed under the Apache License 2.0.

Their GitHub repository contains extensive documentation, latest binaries, API documentation, compatibility information, concurrency control mechanisms, and more. For detailed setup instructions and verification procedures, refer to their provided resources.

Was this page helpful?

More companies

Tidyverse

The tidyverse is a harmonious collection of R packages for data manipulation, visualization, and statistical modeling. Easy installation, updates, and a focus on collaboration and inclusivity.

Read more

Sonio

Revolutionizing healthcare through advanced ultrasound solutions powered by AI-driven technology SAS.

Read more

Octolis

Empowering businesses to turn raw data into valuable insights and actions in real-time with Octolis' easy-to-setup and secure data integration platform.

Read more

Tell us about your project

Our Hubs

London, United Kingdom

A global AI hotspot, thrives on innovation, diverse talent, and a dynamic tech ecosystem, offering unparalleled opportunities for AI engineers.

Munich, Germany

A vibrant AI hub, merges cutting-edge technology with rich cultural experiences, creating an inspiring environment for AI engineers.