Running big data analysis and doing data engineering work has always demanded compute, storage, and other orchestration infrastructure setup - most of the time on cloud. Setting all of this up is enough overhead to slow down anyone who just wants to start working with data. When you just want to get started, you want one tool that can run your data stack, that works locally on your machine.
In my past 10 years of big data engineering, I used to always setup python and spark environments and then write sql or pyspark scripts, test locally, and eventually deploy that to cloud environment. This was useful because cloud compute, such as EMR Spark, lacked interactivity and that prevented quick iterations. Sure there are notebook solutions and stuff that offer some relief, but I always wanted a SQL/Py IDE that brings a spark sandbox environment for quick query execution and inspection on smaller data samples, visualization of results, etc. In the recent past, I wanted AI to help with the query writing part of it, so that I don’t have to explain the context over and over again in every prompt. Even though there are point tools and agentic CLIs that one can stitch together, the overall dev setup process had high friction for anyone new.
Today we're releasing Nile Local - a fully local AI Data IDE and local stack. Download it, point it at your data, and get storage, compute, zero-ETL, lineage, versioning, and AI-assisted analytics running entirely on your machine. You can literally run AI assisted data analytics and engineering, while on a flight. Try it for your side projects or on a larger work project to develop in SQL or Pyspark and test your data pipelines locally and interactively, iterate quickly, all before deploying to your cloud infra. You can use the embedded local LLMs or Cloud AI, to ask it questions about your data and get first class data analysis and data engineering done for you. You don’t need expertise in setting up compute, storage environments, LLMs, or orchestration, the tool has it all built in..
All you need is a decent machine with good RAM (16GB supported, 32GB recommended if you want to use local AI). If you just need the non-AI capabilities, you should be ok with a Macbook air or equivalent. I got great results with a Macbook pro with 32GB RAM and a 30B Gemma 4 and Qwen (comes built in). You also have the option to use Cloud LLM if you prefer that.
https://www.youtube.com/watch?v=C6qSFLylryk
The entire data stack runs on your machine. Here's what you get:
Storage + compute
Zero-ETL ingestion
Lineage + versioning
AI-assisted data engineering analytics