This paper introduces Structuring Whitened Embeddings, a modality- and encoder-agnostic framework designed to optimize encoders for extracting smooth, progression-aware feature representations by aligning real-world samples. Proportional inter-sample relationships are preserved, enabling the capture of subtle and continuous changes. By establishing relationships directly between samples and thereby avoiding the need for data augmentation, the approach is particularly well suited for transformation-sensitive data, such as medical time series, where even minor sample changes can lead to disproportionate shifts in interpretation. Experimental results on early atrial fibrillation prediction and timestamp imputation, modeling both inter- and intra-patient dynamics, demonstrate significant performance improvements using the optimized features. The framework’s augmentation-free design and generalizability across tasks and modalities position it as a practical solution for modeling evolution in complex datasets.
