Validation-aware weak-supervision framework for suicidal ideation and mental-health benchmarks on Bluesky

KEY POINTS

1. New arXiv preprint (arXiv:2609.22696v1) proposes a validation-aware weak-supervision pipeline to build suicidal ideation (SI) and broader mental-health (MH) benchmarks on the decentralized platform Bluesky.
2. The system combines public firehose collection, lexicon filtering, Llama-3-8B-assisted binary annotation, human-adjudicated validation subsets, and transformer-based model benchmarking to produce corpora of 8,346 SI-labeled posts and 9,988 MH-labeled posts; results show model performance depends strongly on task definition and validation protocol, with BERT+LSTM, RoBERTa, and DistilRoBERTa each leading on different SI/MH metrics and human validation revealing SI labels are dominated by false negatives while MH labels are dominated by false positives.
3. This work shows decentralized social media can support reproducible mental-health benchmarking but only when label provenance, validation strategy, and system design (including LLM-assisted labeling and human adjudication) are evaluated together.