lakeFS

@lakefs

628 subscribers59 videos1.2M views

lakeFS is an open-source data lake management platform that transforms your object storage into a Git-like repository. lakeFS enables you to manage your #datalake the way you manage your code. Run parallel pipelines for experimentation and #cicd for your data. lakeFS features: - Exabytes scale version control - Git-like operations: branch, commit, merge, revert - Zero copy branching for frictionless experiments - Full reproducibility of data and code - Pre-commit/merge hooks for data CI/CD - Instantly revert changes to data lakeFS works seamlessly with the following data frameworks: - Hive - Apache Spark - Kafka - mlFlow - Apache Zeppelin - Presto - Hadoop - Amazon Athena - Amazon Kinesis - Jupyter - Delta Lake - Apache Iceberg - Apache Hudi - DataBricks - Airflow Deployable in the cloud or on-prem: - Google Cloud - AWS - Azure - Minio - ceph - and any S3 compatible storage… Visit the lakeFS homepage at https://lakefs.io/ and its GitHub repo at https://github.com/treeverse/lakeFS