This research introduces CycleQD, a novel method for training large language models (LLMs) to acquire multiple skills simultaneously. CycleQD leverages the Quality Diversity framework through a cyclic process, alternating which skill is prioritized while others serve as behavioral characteristics. This approach uses model merging and SVD-based mutation to create a composite LLM that surpasses traditional fine-tuning methods. Experiments demonstrate CycleQD's effectiveness on computer science tasks, achieving performance comparable to GPT-3.5-Turbo, and its broader applicability to image segmentation. The method addresses data imbalance and limitations of standard objective functions in LLM training.

https://arxiv.org/ ...