Periodic Binary Heads with Two?Stage Training for Efficient Multi?Dataset Text Classification

This project develops a text classification system that integrates multiple classification architectures within a web-based interface. Four benchmark datasets are used: AG News for topic classification with four categories, Papluca for language identification across twenty languages, IMDb for binary sentiment analysis, and DBpedia for fourteen-class topic classification. The implemented models include Spectral Binary Head (SBH) and Wavelet Binary Head (WBH) which apply periodic functions to binarize classification weights while maintaining gradient flow via straight-through estimation. Both models follow a two-stage training strategy where a full-precision warm-up MLP is trained first, followed by the binary head. For comparison, a frozen BERT backbone with multi-layer perceptron heads is applied to Papluca, a fine-tuned BERT model is used for DBpedia, and a bidirectional LSTM with attention mechanism is built for IMDb. Evaluation metrics comprise accuracy, macro-F1, micro-F1, precision, recall, confusion matrices, and ROC-AUC where applicable. Experimental results show WBH achieves 88.83% accuracy on AG News, the enhanced BERT head reaches 97.71% on Papluca, BiLSTM with attention attains 89.80% on IMDb, and fine-tuned BERT obtains 99.17% on DBpedia. A Flask-based web application provides user registration, login, text input, model selection, prediction display, and logout functionality. All trained models are saved as PyTorch state dictionaries alongside vocabulary and configuration files for later loading and single-text prediction. The system demonstrates that periodic binary classification heads offer competitive performance with reduced parameter overhead compared to full-precision equivalents.

Keywords: text classification, periodic binary head, spectral binary head, wavelet binary head, two-stage training, BERT, frozen backbone, fine-tuning, BiLSTM, attention mechanism, AG News, Papluca, IMDb, DBpedia, Flask