Spark SQL Formatter
Format Apache Spark SQL and Databricks queries with Spark rules for backtick identifiers, strings with backslash escapes, raw r'' strings and ${var} substitutions. CLUSTER BY, DISTRIBUTE BY and SORT BY are kept as clauses, LATERAL VIEW and LEFT SEMI or ANTI JOIN start new lines and SELECT * EXCEPT stays inline. Unclosed strings and parentheses are reported with line and column for data engineers.
Frequently asked questions
Which Spark SQL syntax is understood?
Backtick identifiers, single and double quoted strings with backslash escapes, raw r'' strings and X'' binary literals are recognized. ${var} substitutions, map['key'] and array[0] subscripts and :: casts are kept intact.
Which Spark specific clauses are supported?
CLUSTER BY, DISTRIBUTE BY and SORT BY get their own lines like ORDER BY. LATERAL VIEW, LATERAL VIEW OUTER, LEFT SEMI JOIN and LEFT ANTI JOIN start new lines like joins.
What about SELECT * EXCEPT?
EXCEPT directly after a star stays on the same line. That way SELECT * EXCEPT (col) is not confused with the EXCEPT set operator.
Are ${var} placeholders kept?
Yes. Substitutions such as ${env} or ${hivevar:date} are kept exactly as written, including the colon inside them.
Does it support MERGE INTO?
Yes. WHEN MATCHED and WHEN NOT MATCHED branches of Delta Lake MERGE INTO statements start new lines.
Can I paste SQL from a PySpark string?
Yes, paste the SQL text itself without the surrounding Python quotes. The result can be pasted back into spark.sql() or a notebook cell.
Does it change keyword case?
No. Keywords and names keep the case you wrote. Only whitespace and line breaks change.
Which errors are detected?
Unclosed strings, backtick identifiers, subscripts and block comments, and unbalanced parentheses, each with line and column. Nothing runs on a cluster.
How large can the input be?
Up to 1 MB per request and 128 KB per statement. Notebook exports with many statements are formatted one statement at a time.
Can I choose the indentation, and is my SQL stored?
Use 2 or 4 spaces or tabs. The SQL is formatted in memory only and never saved.