Text2SQL: Natural Language to SQL

Overview

A complete, from-scratch NLP research pipeline that fine-tunes sequence-to-sequence transformer models to translate English questions into SQL queries, built solo for COMP9444 (UNSW's Neural Networks and Deep Learning). Five architectures — T5-small, T5-base, FLAN-T5-base, CodeT5-base, and BART-base — were fine-tuned on the AI4DS/sql_generator_no_cot dataset and compared head-to-head across six metrics: BLEU, ROUGE, Exact Match, Levenshtein distance, token accuracy, and SQL structural similarity. The pipeline covers every stage end-to-end — preprocessing and tokenization, a training loop with mixed precision, gradient accumulation, early stopping and LR scheduling, four decoding strategies at inference (greedy, beam, top-k, top-p), Optuna-driven hyperparameter search, and attention/token-importance explainability visualizations. Serving is handled by a FastAPI REST backend and a Gradio web UI, both Dockerized for lightweight CPU-only deployment. CodeT5-base — the code-aware architecture — gave the best quality/size trade-off and is the model the project is built to deploy.

Key Features

• ▸ Five fine-tuned seq2seq architectures compared head-to-head: T5-small, T5-base, FLAN-T5-base, CodeT5-base, BART-base
• ▸ Full training pipeline: mixed precision, gradient accumulation, early stopping, LR scheduling, checkpointing
• ▸ Six evaluation metrics: BLEU, ROUGE, Exact Match, Levenshtein distance, token accuracy, SQL structural similarity
• ▸ Four decoding strategies at inference: greedy, beam search, top-k, and nucleus (top-p) sampling
• ▸ Optuna-driven hyperparameter tuning plus beam-width, sequence-length, and learning-rate ablation experiments
• ▸ Attention-map and token-importance explainability visualizations
• ▸ FastAPI REST backend and Gradio web UI, Dockerized for CPU-only deployment