Skip to content

training

Model Merging

Techniques for combining the weights of multiple trained models into a single model that inherits capabilities from all source models. Model merging enables creating new models without additional training, by averaging or interpolating parameters.

In practice

Merging a coding-specialized model with a creative writing model can produce a model good at both tasks.