{"id":1344,"date":"2023-12-30T18:14:48","date_gmt":"2023-12-30T17:14:48","guid":{"rendered":"http:\/\/pages.di.unipi.it\/bacciu\/?page_id=1344"},"modified":"2024-01-02T15:38:23","modified_gmt":"2024-01-02T14:38:23","slug":"journals","status":"publish","type":"page","link":"https:\/\/pages.di.unipi.it\/bacciu\/publications\/journals\/","title":{"rendered":"Journals"},"content":{"rendered":"\n

<\/a><\/form>
59 entries<\/span> «<\/a> ‹<\/a> 1 of 2 ›<\/a> »<\/a> <\/div><\/div>

2024<\/h3>
1.<\/div>

Carta, Antonio; Cossu, Andrea; Lomonaco, Vincenzo; Bacciu, Davide; Weijer, Joost<\/p>

Projected Latent Distillation for Data-Agnostic Consolidation in distributed continual learning<\/a> Journal Article<\/span> <\/p>

In: <\/span>Neurocomputing, <\/span>vol. 598, <\/span>pp. 127935, <\/span>2024<\/span>, ISSN: 0925-2312<\/span>.<\/p>

Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@article{CARTA2024127935,
\r\ntitle = {Projected Latent Distillation for Data-Agnostic Consolidation in distributed continual learning},
\r\nauthor = {Antonio Carta and Andrea Cossu and Vincenzo Lomonaco and Davide Bacciu and Joost Weijer},
\r\nurl = {https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0925231224007069},
\r\ndoi = {https:\/\/doi.org\/10.1016\/j.neucom.2024.127935},
\r\nissn = {0925-2312},
\r\nyear  = {2024},
\r\ndate = {2024-01-01},
\r\nurldate = {2024-01-01},
\r\njournal = {Neurocomputing},
\r\nvolume = {598},
\r\npages = {127935},
\r\nabstract = {In continual learning applications on-the-edge multiple self-centered devices (SCD) learn different local tasks independently, with each SCD only optimizing its own task. Can we achieve (almost) zero-cost collaboration between different devices? We formalize this problem as a Distributed Continual Learning (DCL) scenario, where SCDs greedily adapt to their own local tasks and a separate continual learning (CL) model perform a sparse and asynchronous consolidation step that combines the SCD models sequentially into a single multi-task model without using the original data. Unfortunately, current CL methods are not directly applicable to this scenario. We propose Data-Agnostic Consolidation (DAC), a novel double knowledge distillation method which performs distillation in the latent space via a novel Projected Latent Distillation loss. Experimental results show that DAC enables forward transfer between SCDs and reaches state-of-the-art accuracy on Split CIFAR100, CORe50 and Split TinyImageNet, both in single device and distributed CL scenarios. Somewhat surprisingly, a single out-of-distribution image is sufficient as the only source of data for DAC.},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {article}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
In continual learning applications on-the-edge multiple self-centered devices (SCD) learn different local tasks independently, with each SCD only optimizing its own task. Can we achieve (almost) zero-cost collaboration between different devices? We formalize this problem as a Distributed Continual Learning (DCL) scenario, where SCDs greedily adapt to their own local tasks and a separate continual learning (CL) model perform a sparse and asynchronous consolidation step that combines the SCD models sequentially into a single multi-task model without using the original data. Unfortunately, current CL methods are not directly applicable to this scenario. We propose Data-Agnostic Consolidation (DAC), a novel double knowledge distillation method which performs distillation in the latent space via a novel Projected Latent Distillation loss. Experimental results show that DAC enables forward transfer between SCDs and reaches state-of-the-art accuracy on Split CIFAR100, CORe50 and Split TinyImageNet, both in single device and distributed CL scenarios. Somewhat surprisingly, a single out-of-distribution image is sufficient as the only source of data for DAC.<\/div>
Close<\/a><\/p><\/div>