Validation-aware weak-supervision framework for suicidal ideation and mental-health benchmarks on Bluesky
New arXiv preprint (arXiv:2609.22696v1) proposes a validation-aware weak-supervision pipeline to build suicidal ideation (SI) and broader mental-health (MH) benchmarks on the decentralized platform Bluesky. The system combines public firehose collection, lexicon filtering, Llama-3-8B-assisted binary annotation, human-adjudicated validation subsets, and transformer-based model benchmarking to produce corpora of 8,346 SI-labeled posts and 9,988 MH-labeled posts; results show model performance depends strongly on task definition and validation protocol, with BERT+LSTM, RoBERTa, and DistilRoBERTa each leading on different SI/MH metrics and human validation revealing SI labels are dominated by false negatives while MH labels are dominated by false positives.