This study addresses the limitations of deep learning brain age models trained on the UK Biobank (UKBB) dataset, which typically suffer from narrow age ranges and uniform scanner protocols. While fine-tuning these models on external datasets introduces risks of inheriting base-model biases or experiencing catastrophic forgetting, we implement an optimization pipeline to mitigate these issues. By training an ensemble of regularized models and fine-tuning them with an expanded age range across multiple cohorts (using oversampling, Label Distribution Smoothing, and Elastic Weight Consolidation), we successfully expanded the model's demographic reach and scanner compatibility. The fine-tuned model demonstrates excellent generalizability across unseen younger (MR-ART) and elderly (ADNI) cohorts while retaining its original performance on the UKBB baseline with less than 5% performance degradation.