Publication:

An Empirical Study of Feature Dynamics During Fine-Tuning

 
cris.virtual.department#PLACEHOLDER_PARENT_METADATA_VALUE#
cris.virtual.department#PLACEHOLDER_PARENT_METADATA_VALUE#
cris.virtual.orcid0000-0002-8607-5067
cris.virtual.orcid0000-0003-1546-3852
cris.virtualsource.departmentb4e95a64-316d-496a-94fb-177f312882b8
cris.virtualsource.department07b83c60-4bf4-4bde-8356-0a1c1b0dafa2
cris.virtualsource.orcidb4e95a64-316d-496a-94fb-177f312882b8
cris.virtualsource.orcid07b83c60-4bf4-4bde-8356-0a1c1b0dafa2
dc.contributor.authorBehzadi Khormouji Hamed
dc.contributor.authorDe Roeck, Lena
dc.contributor.authorOramas Mogrovejo, Jose Antonio
dc.date.accessioned2026-11-05T15:24:42Z
dc.date.available2026-11-05T15:24:42Z
dc.date.createdwos2026
dc.date.issued2026
dc.description.abstractFine-tuning is the common practice of adapting a pre-trained model so that the encoded features can be reused for a new target task. Up to now, there have been efforts in the literature to get insights from the fine-tuning process by measuring the internal feature alignment between the source and the target datasets. However, they suffer from three weaknesses. First, their findings are limited to a few datasets, deep models and focused on only one similarity metric which results in discordant observations and doubts regarding reliability. Second, the conducted evaluations are either purely qualitative, which lends itself to subjectivity; or purely quantitative, which suffers from reduced intelligibility. Third, existing analysis focus on the two extremes of the fine-tuning process, i.e. on the model pre and post fine-tuning. In doing so, there is no room for analyzing the dynamics that link these two extremes. Here, we conduct both quantitative and qualitative analyses that aim at shining a light on the feature dynamics during iterative stages of the fine-tuning process. The analysis shows that feature similarity is reduced, even in early stages, between the source model and its fine-tuned counterparts when the target domain is dissimilar. Moreover, it illustrates domain shift across iterations of fine-tuning procedure. We believe the presented methodology could be adopted for the analysis of fine-tuning processes and help pinpoint the reasons why some of these processes are more effective than others. The implementation is available at
dc.description.wosFundingTextThis work is supported by the UAntwerp BOF DOCPRO4-NZ Project (id 41612) "Multimodal Relational Interpretation for Deep Models".
dc.identifier.doi10.1007/978-3-032-25308-8_25
dc.identifier.eissn1865-0937
dc.identifier.isbn978-3-032-25307-1
dc.identifier.issn1865-0929
dc.identifier.urihttps://imec-publications.be/handle/20.500.12860/60753
dc.language.isoeng
dc.provenance.editstepusergreet.vanhoof@imec.be
dc.publisherSPRINGER INTERNATIONAL PUBLISHING AG
dc.relation.ispartofseriesCommunications in Computer and Information Science
dc.source.beginpage363
dc.source.conferenceMachine Learning and Principles and Practice of Knowledge Discovery in Databases International Workshops of ECML PKDD
dc.source.conferencedate2024-09-09
dc.source.conferencelocationVilnius
dc.source.endpage378
dc.source.journalMACHINE LEARNING AND PRINCIPLES AND PRACTICE OF KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2024, PT I
dc.source.numberofpages16
dc.title

An Empirical Study of Feature Dynamics During Fine-Tuning

dc.typeProceedings paper
dspace.entity.typePublication
imec.internal.crawledAt2026-07-14
imec.internal.sourcecrawler
imec.internal.wosCreatedAt2026-09-07
Files
Publication available in collections: