Fine-tuned Stable Diffusion XL and LLaMA enable controllable Ulos motif generation
arXiv:2609.17987v1 presents a multimodal generative framework that fine-tunes Stable Diffusion XL v1.0 via LoRA and integrates a multimodal LLaMA 1.5-7B to generate culturally faithful Batak Ulos motifs. The system uses four conditioning mechanisms (text, image, representation, semantic map/ControlNet); an ablation across three scenarios found Text+Image+Semantic Map achieved the best FID (270) and CLIP (0.65–0.70) but lowest SSIM (0.65), Text+Image+Representation gave the best overall balance (SSIM 0.84, FID 280), and combining all four worsened FID to 330; qualitative evaluation with nine weavers and 30 public participants showed statistically significant positive acceptance (Wilcoxon p=0.007 and p<0.001), and a web prototype for text- and image-to-image generation was developed.