Scaling Up Personalized Aesthetic Assessment via Task Vector Arithmetic
ECCV 2024
Abstract
The task of personalized image aesthetic assessment seeks to tailor aesthetic score prediction models to match individual preferences with just a few user-provided inputs. However, the scalability and generalization capabilities of current approaches are considerably restricted by their reliance on an expensive curated database. To overcome this long-standing scalability challenge, we present a unique approach that leverages readily available databases for general image aesthetic assessment and image quality assessment. Specifically, we view each database as a distinct image score regression task that exhibits varying degrees of personalization potential. By determining optimal combinations of task vectors, known to represent specific traits of each database, we successfully create personalized models for individuals. This approach of integrating multiple models allows us to harness a substantial amount of data. Our extensive experiments demonstrate the effectiveness of our approach in generalizing to previously unseen domains&ndasha challenge previous approaches have struggled to achieve–making it highly applicable to real-world scenarios. Our novel approach significantly advances the field by offering scalable solutions for personalized aesthetic assessment and establishing high standards for future research.
Introduction
Everyone has their own unique aesthetic appetite.
While there are numerous models that assess the aesthetic quality of images,
no model universally reflects the aesthetic preference of all individuals.
Task Vectors in Aesthetics Assessment
We train multiple models using publicly available aesthetic assessment databases.
Task vectors obtained from these models represent the characteristics of each database.
In this way, we effectively use more than 400,000 aesthetic score samples.
Personalization via Task Vector Customization
Combining task vectors produces a new aesthetic score model with unique behaviors.
Given a small set of personal image collections, we optimize the coefficients for combining task vectors.
This allows us to create personalized models by training a minimal number of parameters.
By combining these task vectors, we create personalized models for individuals.
Results
Our approach demonstrates outstanding generalization to unseen domains.
We outperform methods trained on the AADB dataset without pretraining on the AADB dataset.
Our approach does is not trained on the AADB dataset, yet it outperforms existing methods.
Accuracy of personalization increases with the number of available task vectors.
Our scalable approach can also easily integrate additional datasets.
Our approach can easily integrate additional datasets to improve personalization accuracy.
Citation
@article{yun2024scaling,
title={Scaling Up Personalized Aesthetic Assessment via Task Vector Customization},
author={Jooyeol Yun and Jaegul Choo},
year={2024},
eprint={2407.07176},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2407.07176},
}