SQLFluff

Most of the focus of getting to 2.0.0 was on accuracy and robustness, however much of the focus since has been on performance and speed. This post is a little peek behind the curtain of how we went about that, and what some of the biggest improvements have been.

Today we released SQLFluff 1.0.0, the first stable release of SQLFluff.

This year I spoke at the Coalesce 2021 conference about a topic a bit broader than just SQL linting or data in small businesses.

In June, I had a great chat with the team at the Podcast.__init__ (aka the Python Podcast) about linting, SQL and open source projects.

The dbt project at tails.com has over 600 models and 66k lines of code. With multiple contributors to a project and varying SQL backgrounds, it’s really difficult to maintain consistent readability and comprehension across a codebase like that by hand.

For around a year I’ve been working on SQLFluff, a SQL linter to keep up with the growing place of analytics code as a store of business definitions. This post isn’t about why I wrote it, but it is about how to start making it a bit more appealing.

SQL has been around for a long time, as a language for communicating with databases, as a communication protocol. More recently with the rise of data as a business function, or a domain in it’s own right SQL has also become an invaluable tool for defining the structure of data.