Publications
2019
Castellana, Daniele; Bacciu, Davide
Bayesian Tensor Factorisation for Bottom-up Hidden Tree Markov Models Conference
Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN 2019I) , IEEE, 2019.
Abstract | Links | BibTeX | Tags: graphical models, hidden tree Markov model, structured data processing, tree structured data; tensor factorization; Bayesian learning
@conference{ijcnn2019,
title = {Bayesian Tensor Factorisation for Bottom-up Hidden Tree Markov Models},
author = {Daniele Castellana and Davide Bacciu},
url = {https://arxiv.org/pdf/1905.13528.pdf},
year = {2019},
date = {2019-07-15},
booktitle = {Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN 2019I) },
publisher = {IEEE},
abstract = {Bottom-Up Hidden Tree Markov Model is a highly expressive model for tree-structured data. Unfortunately, it cannot be used in practice due to the intractable size of its state-transition matrix. We propose a new approximation which lies on the Tucker factorisation of tensors. The probabilistic interpretation of such approximation allows us to define a new probabilistic model for tree-structured data. Hence, we define the new approximated model and we derive its learning algorithm. Then, we empirically assess the effective power of the new model evaluating it on two different tasks. In both cases, our model outperforms the other approximated model known in the literature.},
keywords = {graphical models, hidden tree Markov model, structured data processing, tree structured data; tensor factorization; Bayesian learning},
pubstate = {published},
tppubtype = {conference}
}
Bottom-Up Hidden Tree Markov Model is a highly expressive model for tree-structured data. Unfortunately, it cannot be used in practice due to the intractable size of its state-transition matrix. We propose a new approximation which lies on the Tucker factorisation of tensors. The probabilistic interpretation of such approximation allows us to define a new probabilistic model for tree-structured data. Hence, we define the new approximated model and we derive its learning algorithm. Then, we empirically assess the effective power of the new model evaluating it on two different tasks. In both cases, our model outperforms the other approximated model known in the literature.
Davide, Bacciu; Daniele, Castellana
Bayesian Mixtures of Hidden Tree Markov Models for Structured Data Clustering Journal Article
Neurocomputing, 342 , pp. 49-59, 2019, ISBN: 0925-2312.
Abstract | Links | BibTeX | Tags: graphical models, hidden tree Markov model, structured data processing, tree structured data, unsupervised learning
@article{neucomBayesHTMM,
title = {Bayesian Mixtures of Hidden Tree Markov Models for Structured Data Clustering},
author = {Bacciu Davide and Castellana Daniele},
url = {https://doi.org/10.1016/j.neucom.2018.11.091},
doi = {10.1016/j.neucom.2018.11.091},
isbn = {0925-2312},
year = {2019},
date = {2019-05-21},
journal = {Neurocomputing},
volume = {342},
pages = {49-59},
abstract = {The paper deals with the problem of unsupervised learning with structured data, proposing a mixture model approach to cluster tree samples. First, we discuss how to use the Switching-Parent Hidden Tree Markov Model, a compositional model for learning tree distributions, to define a finite mixture model where the number of components is fixed by a hyperparameter. Then, we show how to relax such an assumption by introducing a Bayesian non-parametric mixture model where the number of necessary hidden tree components is learned from data. Experimental validation on synthetic and real datasets show the benefit of mixture models over simple hidden tree models in clustering applications. Further, we provide a characterization of the behaviour of the two mixture models for different choices of their hyperparameters.},
keywords = {graphical models, hidden tree Markov model, structured data processing, tree structured data, unsupervised learning},
pubstate = {published},
tppubtype = {article}
}
The paper deals with the problem of unsupervised learning with structured data, proposing a mixture model approach to cluster tree samples. First, we discuss how to use the Switching-Parent Hidden Tree Markov Model, a compositional model for learning tree distributions, to define a finite mixture model where the number of components is fixed by a hyperparameter. Then, we show how to relax such an assumption by introducing a Bayesian non-parametric mixture model where the number of necessary hidden tree components is learned from data. Experimental validation on synthetic and real datasets show the benefit of mixture models over simple hidden tree models in clustering applications. Further, we provide a characterization of the behaviour of the two mixture models for different choices of their hyperparameters.
2018
Davide, Bacciu; Daniele, Castellana
Learning Tree Distributions by Hidden Markov Models Workshop
Proceedings of the FLOC 2018 Workshop on Learning and Automata (LearnAut'18), 2018.
Links | BibTeX | Tags: graphical models, hidden tree Markov model, structured data processing, tree structured data
@workshop{learnaut18,
title = {Learning Tree Distributions by Hidden Markov Models},
author = {Bacciu Davide and Castellana Daniele},
editor = {Rémi Eyraud and Jeffrey Heinz and Guillaume Rabusseau and Matteo Sammartino },
url = {https://arxiv.org/abs/1805.12372},
year = {2018},
date = {2018-07-13},
booktitle = {Proceedings of the FLOC 2018 Workshop on Learning and Automata (LearnAut'18)},
keywords = {graphical models, hidden tree Markov model, structured data processing, tree structured data},
pubstate = {published},
tppubtype = {workshop}
}
Davide, Bacciu; Daniele, Castellana
Mixture of Hidden Markov Models as Tree Encoder Conference
Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN'18), i6doc.com, Louvain-la-Neuve, Belgium, 2018, ISBN: 978-287587047-6.
Abstract | BibTeX | Tags: graphical models, hidden tree Markov model, structured data processing, tree structured data, unsupervised learning
@conference{esann2018Tree,
title = {Mixture of Hidden Markov Models as Tree Encoder},
author = {Bacciu Davide and Castellana Daniele},
editor = {Michel Verleysen},
isbn = {978-287587047-6},
year = {2018},
date = {2018-04-26},
booktitle = {Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN'18)},
pages = {543-548},
publisher = {i6doc.com},
address = {Louvain-la-Neuve, Belgium},
abstract = {The paper introduces a new probabilistic tree encoder based on a mixture of Bottom-up Hidden Tree Markov Models. The ability to recognise similar structures in data is experimentally assessed both in clusterization and classification tasks. The results of these preliminary experiments suggest that the model can be successfully used to compress the tree structural and label patterns in a vectorial representation.},
keywords = {graphical models, hidden tree Markov model, structured data processing, tree structured data, unsupervised learning},
pubstate = {published},
tppubtype = {conference}
}
The paper introduces a new probabilistic tree encoder based on a mixture of Bottom-up Hidden Tree Markov Models. The ability to recognise similar structures in data is experimentally assessed both in clusterization and classification tasks. The results of these preliminary experiments suggest that the model can be successfully used to compress the tree structural and label patterns in a vectorial representation.
2015
Davide, Bacciu; J.G., Lisboa Paulo; Alessandro, Sperduti; Thomas, Villmann
Probabilistic Modeling in Machine Learning Incollection
Kacprzyk, Janusz ; Pedrycz, Witold (Ed.): pp. 545–575, Springer Berlin Heidelberg, Berlin, Heidelberg, 2015, ISBN: 978-3-662-43505-2.
Links | BibTeX | Tags: Bayesian networks, generative model, graphical models, hidden Markov models
@incollection{Bacciu2015,
title = {Probabilistic Modeling in Machine Learning},
author = {Bacciu Davide and Lisboa Paulo J.G. and Sperduti Alessandro and Villmann Thomas},
editor = {Kacprzyk, Janusz and Pedrycz, Witold},
url = {http://dx.doi.org/10.1007/978-3-662-43505-2_31},
doi = {10.1007/978-3-662-43505-2_31},
isbn = {978-3-662-43505-2},
year = {2015},
date = {2015-01-01},
pages = {545--575},
publisher = {Springer Berlin Heidelberg},
address = {Berlin, Heidelberg},
keywords = {Bayesian networks, generative model, graphical models, hidden Markov models},
pubstate = {published},
tppubtype = {incollection}
}
2014
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
Integrating bi-directional contexts in a generative kernel for trees Conference
Neural Networks (IJCNN), 2014 International Joint Conference on, IEEE, 2014.
Links | BibTeX | Tags: generative model, graphical models, hidden tree Markov model, kernel methods, structured data processing, tree kernel, tree structured data, tree transductions
@conference{11568_586070,
title = {Integrating bi-directional contexts in a generative kernel for trees},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
doi = {10.1109/IJCNN.2014.6889768},
year = {2014},
date = {2014-01-01},
booktitle = {Neural Networks (IJCNN), 2014 International Joint Conference on},
pages = {4145--4151},
publisher = {IEEE},
keywords = {generative model, graphical models, hidden tree Markov model, kernel methods, structured data processing, tree kernel, tree structured data, tree transductions},
pubstate = {published},
tppubtype = {conference}
}
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
Modeling Bi-directional Tree Contexts by Generative Transductions Conference
Neural Information Processing, 8834 , Springer International Publishing, 2014.
Abstract | Links | BibTeX | Tags: generative model, graphical models, hidden tree Markov model, kernel methods, tree kernel, tree structured data
@conference{11568_665864,
title = {Modeling Bi-directional Tree Contexts by Generative Transductions},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
url = {http://dx.doi.org/10.1007/978-3-319-12637-1_68},
doi = {10.1007/978-3-319-12637-1_68},
year = {2014},
date = {2014-01-01},
booktitle = {Neural Information Processing},
journal = {LECTURE NOTES IN COMPUTER SCIENCE},
volume = {8834},
pages = {543--550},
publisher = {Springer International Publishing},
abstract = {We introduce an approach to integrate bi-directional contexts in a generative tree model by means of structured transductions. We show how this can be efficiently realized as the composition of a top-down and a bottom-up generative model for trees, that are trained independently within a circular encoding-decoding scheme. The resulting input-driven generative model is shown to capture information concerning bi-directional contexts within its state-space. An experimental evaluation using the Jaccard generative kernel for trees is presented, indicating that the approach can achieve state of the art performance on tree classification benchmarks.},
keywords = {generative model, graphical models, hidden tree Markov model, kernel methods, tree kernel, tree structured data},
pubstate = {published},
tppubtype = {conference}
}
We introduce an approach to integrate bi-directional contexts in a generative tree model by means of structured transductions. We show how this can be efficiently realized as the composition of a top-down and a bottom-up generative model for trees, that are trained independently within a circular encoding-decoding scheme. The resulting input-driven generative model is shown to capture information concerning bi-directional contexts within its state-space. An experimental evaluation using the Jaccard generative kernel for trees is presented, indicating that the approach can achieve state of the art performance on tree classification benchmarks.
2013
Davide, Bacciu; A, Etchells Terence; JG, Lisboa Paulo; Joe, Whittaker
Efficient identification of independence networks using mutual information Journal Article
Computational Statistics, 28 (2), pp. 621-646, 2013, ISSN: 0943-4062.
Links | BibTeX | Tags: Bayesian networks, graphical models, mutual information, PC algorithm
@article{bgm2013,
title = {Efficient identification of independence networks using mutual information},
author = {Bacciu Davide and Etchells Terence A and Lisboa Paulo JG and Whittaker Joe},
url = {http://dx.doi.org/10.1007/s00180-012-0320-6},
doi = {10.1007/s00180-012-0320-6},
issn = {0943-4062},
year = {2013},
date = {2013-01-01},
journal = {Computational Statistics},
volume = {28},
number = {2},
pages = {621-646},
publisher = {Springer-Verlag},
keywords = {Bayesian networks, graphical models, mutual information, PC algorithm},
pubstate = {published},
tppubtype = {article}
}
Nicola, Di Mauro; Paolo, Frasconi; Fabrizio, Angiulli; Davide, Bacciu; de Marco, Gemmis; Floriana, Esposito; Nicola, Fanizzi; Stefano, Ferilli; Marco, Gori; A, Lisi Francesca; others,
Italian Machine Learning and Data Mining research: The last years Journal Article
Intelligenza Artificiale, 7 (2), pp. 77–89, 2013.
Links | BibTeX | Tags: graphical models, recurrent neural network, structured data processing
@article{di2013italian,
title = {Italian Machine Learning and Data Mining research: The last years},
author = {Di Mauro Nicola and Frasconi Paolo and Angiulli Fabrizio and Bacciu Davide and de Gemmis Marco and Esposito Floriana and Fanizzi Nicola and Ferilli Stefano and Gori Marco and Lisi Francesca A and others},
url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6353263},
doi = {10.3233/IA-130050},
year = {2013},
date = {2013-01-01},
journal = {Intelligenza Artificiale},
volume = {7},
number = {2},
pages = {77--89},
publisher = {IOS Press},
keywords = {graphical models, recurrent neural network, structured data processing},
pubstate = {published},
tppubtype = {article}
}
2012
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
A Generative Multiset Kernel for Structured Data Conference
Artificial Neural Networks and Machine Learning - ICANN 2012 proceedings, Springer LNCS series, 7552 , Springer-Verlag, BERLIN HEIDELBERG, 2012.
Abstract | Links | BibTeX | Tags: generative model, graphical models, hidden tree Markov model, kernel methods, structured data processing, support vector machine, tree kernel, tree structured data
@conference{11568_156516,
title = {A Generative Multiset Kernel for Structured Data},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
doi = {10.1007/978-3-642-33269-2_8},
year = {2012},
date = {2012-01-01},
booktitle = {Artificial Neural Networks and Machine Learning - ICANN 2012 proceedings, Springer LNCS series},
journal = {LECTURE NOTES IN COMPUTER SCIENCE},
volume = {7552},
pages = {57--64},
publisher = {Springer-Verlag},
address = {BERLIN HEIDELBERG},
abstract = {The paper introduces a novel approach for defining efficient generative kernels for structured-data based on the concept of multisets and Jaccard similarity. The multiset feature-space allows to enhance the adaptive kernel with syntactic information on structure matching. The proposed approach is validated using an input-driven hidden Markov model for trees as generative model, but it is enough general to be straightforwardly applicable to any probabilistic latent variable model. The experimental evaluation shows that the proposed Jaccard kernel has a superior classification performance with respect to the Fisher Kernel, while consistently reducing the computational requirements.},
keywords = {generative model, graphical models, hidden tree Markov model, kernel methods, structured data processing, support vector machine, tree kernel, tree structured data},
pubstate = {published},
tppubtype = {conference}
}
The paper introduces a novel approach for defining efficient generative kernels for structured-data based on the concept of multisets and Jaccard similarity. The multiset feature-space allows to enhance the adaptive kernel with syntactic information on structure matching. The proposed approach is validated using an input-driven hidden Markov model for trees as generative model, but it is enough general to be straightforwardly applicable to any probabilistic latent variable model. The experimental evaluation shows that the proposed Jaccard kernel has a superior classification performance with respect to the Fisher Kernel, while consistently reducing the computational requirements. G, Lisboa Paulo J; H, Jarman Ian; A, Etchells Terence; J, Chambers Simon; Davide, Bacciu; Joe, Whittaker; M, Garibaldi Jon; Sandra, Ortega-Martorell; Alfredo, Vellido; O, Ellis Ian
Discovering Hidden Pathways in Bioinformatics Conference
Lecture Notes in Computer ScienceComputational Intelligence Methods for Bioinformatics and Biostatistics, 7548 , 2012.
Abstract | Links | BibTeX | Tags: Bayesian networks, biomedical data, graphical models, mutual information, PC algorithm
@conference{11568_465481,
title = {Discovering Hidden Pathways in Bioinformatics},
author = {Lisboa Paulo J G and Jarman Ian H and Etchells Terence A and Chambers Simon J and Bacciu Davide and Whittaker Joe and Garibaldi Jon M and Ortega-Martorell Sandra and Vellido Alfredo and Ellis Ian O},
doi = {10.1007/978-3-642-35686-5_5},
year = {2012},
date = {2012-01-01},
booktitle = {Lecture Notes in Computer ScienceComputational Intelligence Methods for Bioinformatics and Biostatistics},
journal = {LECTURE NOTES IN COMPUTER SCIENCE},
volume = {7548},
pages = {49--60},
abstract = {The elucidation of biological networks regulating the metabolic basis of disease is critical for understanding disease progression and in identifying therapeutic targets. In molecular biology, this process often starts by clustering expression profiles which are candidates for disease phenotypes. However, each cluster may comprise several overlapping processes that are active in the cluster. This paper outlines empirical results using methods for blind source separation to map the pathways of biomarkers driving independent, hidden processes that underpin the clusters. The method is applied to a protein expression data set measured in tissue from breast cancer patients (n=1,076)},
keywords = {Bayesian networks, biomedical data, graphical models, mutual information, PC algorithm},
pubstate = {published},
tppubtype = {conference}
}
The elucidation of biological networks regulating the metabolic basis of disease is critical for understanding disease progression and in identifying therapeutic targets. In molecular biology, this process often starts by clustering expression profiles which are candidates for disease phenotypes. However, each cluster may comprise several overlapping processes that are active in the cluster. This paper outlines empirical results using methods for blind source separation to map the pathways of biomarkers driving independent, hidden processes that underpin the clusters. The method is applied to a protein expression data set measured in tissue from breast cancer patients (n=1,076) Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
Input-Output Hidden Markov Models for Trees Conference
ESANN 2012 - The 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning - Proceedings, Ciaco scrl - i6doc.com, 2012.
Abstract | BibTeX | Tags: generative model, graphical models, hidden tree Markov model, structured data processing, tree structured data, tree transductions
@conference{11568_152836,
title = {Input-Output Hidden Markov Models for Trees},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
year = {2012},
date = {2012-01-01},
booktitle = {ESANN 2012 - The 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning - Proceedings},
pages = {25--30},
publisher = {Ciaco scrl - i6doc.com},
abstract = {The paper introduces an input-driven generative model for tree-structured data that extends the bottom-up hidden tree Markov model with non-homogenous transition and emission probabilities. The advantage of introducing an input-driven dynamics in structured-data pro- cessing is experimentally investigated. The results of this preliminary analysis suggest that input-driven models can capture more discrimina- tive structural information than non-input-driven approaches.},
keywords = {generative model, graphical models, hidden tree Markov model, structured data processing, tree structured data, tree transductions},
pubstate = {published},
tppubtype = {conference}
}
The paper introduces an input-driven generative model for tree-structured data that extends the bottom-up hidden tree Markov model with non-homogenous transition and emission probabilities. The advantage of introducing an input-driven dynamics in structured-data pro- cessing is experimentally investigated. The results of this preliminary analysis suggest that input-driven models can capture more discrimina- tive structural information than non-input-driven approaches.
2011
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
Adaptive Tree Kernel by Multinomial Generative Topographic Mapping Conference
Proceedings of the International Joint Conference on Neural Networks, IEEE, Piscataway (NJ), 2011.
Links | BibTeX | Tags: generative model, generative topographic mapping, graphical models, hidden tree Markov model, kernel methods, structured data processing, tree kernel, tree structured data
@conference{11568_145907,
title = {Adaptive Tree Kernel by Multinomial Generative Topographic Mapping},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6033423&contentType=Conference+Publications&refinements%3D4294413850%26sortType%3Dasc_p_Sequence%26filter%3DAND%28p_IS_Number%3A6033131%29},
doi = {10.1109/IJCNN.2011.6033423},
year = {2011},
date = {2011-01-01},
booktitle = {Proceedings of the International Joint Conference on Neural Networks},
pages = {1651--1658},
publisher = {IEEE},
address = {Piscataway (NJ)},
keywords = {generative model, generative topographic mapping, graphical models, hidden tree Markov model, kernel methods, structured data processing, tree kernel, tree structured data},
pubstate = {published},
tppubtype = {conference}
}
2010
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
A Bottom-up Hidden Tree Markov Model Technical Report
Università di Pisa (TR-10-08), 2010.
Links | BibTeX | Tags: generative model, graphical models, hidden tree Markov model, structured data processing, tree structured data
@techreport{11568_254437,
title = {A Bottom-up Hidden Tree Markov Model},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
url = {http://compass2.di.unipi.it/TR/Files/TR-10-08.pdf.gz},
year = {2010},
date = {2010-04-01},
volume = {TR-10-08},
number = {TR-10-08},
pages = {1--22},
institution = {Università di Pisa},
keywords = {generative model, graphical models, hidden tree Markov model, structured data processing, tree structured data},
pubstate = {published},
tppubtype = {techreport}
}
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
Bottom-Up Generative Modeling of Tree-Structured Data Conference
LNCS 6443: Neural Information Processing. Theory and Algorithms. Part I, 6443 , Springer-Verlag, BERLIN HEIDELBERG, 2010.
Links | BibTeX | Tags: generative model, graphical models, hidden tree Markov model, structured data processing, tree structured data
@conference{11568_142187,
title = {Bottom-Up Generative Modeling of Tree-Structured Data},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
doi = {10.1007/978-3-642-17537-4_80},
year = {2010},
date = {2010-01-01},
booktitle = {LNCS 6443: Neural Information Processing. Theory and Algorithms. Part I},
journal = {LECTURE NOTES IN COMPUTER SCIENCE},
volume = {6443},
pages = {660--668},
publisher = {Springer-Verlag},
address = {BERLIN HEIDELBERG},
keywords = {generative model, graphical models, hidden tree Markov model, structured data processing, tree structured data},
pubstate = {published},
tppubtype = {conference}
}
Davide, Bacciu; Alessio, Micheli; Alessandro, Sperduti
Compositional Generative Mapping of Structured Data Conference
Proceedings of the 2010 IEEE InternationalJoint Conference on Neural Networks(IJCNN'10), IEEE, 2010.
Links | BibTeX | Tags: generative topographic mapping, graphical models, hidden tree Markov model, structured data processing, tree structured data, unsupervised learning
@conference{11568_136433,
title = {Compositional Generative Mapping of Structured Data},
author = {Bacciu Davide and Micheli Alessio and Sperduti Alessandro},
doi = {10.1109/IJCNN.2010.5596606},
year = {2010},
date = {2010-01-01},
booktitle = {Proceedings of the 2010 IEEE InternationalJoint Conference on Neural Networks(IJCNN'10)},
pages = {1359--1366},
publisher = {IEEE},
keywords = {generative topographic mapping, graphical models, hidden tree Markov model, structured data processing, tree structured data, unsupervised learning},
pubstate = {published},
tppubtype = {conference}
}
2008
Davide, Bacciu
A Perceptual Learning Model to Discover the Hierarchical Latent Structure of Image Collections PhD Thesis
2008.
Abstract | Links | BibTeX | Tags: biomedical data, clustering, competitive repetition suppression learning, feature selection, generative model, graphical models, image understanding, latent topic model, neural networks, statistics, unsupervised learning
@phdthesis{11568_466874,
title = {A Perceptual Learning Model to Discover the Hierarchical Latent Structure of Image Collections},
author = { Bacciu Davide},
url = {http://e-theses.imtlucca.it/id/eprint/7},
doi = {10.6092/imtlucca/e-theses/7},
year = {2008},
date = {2008-01-01},
publisher = {IMT Lucca},
abstract = {Biology has been an unparalleled source of inspiration for the work of researchers in several scientific and engineering fields including computer vision. The starting point of this thesis is the neurophysiological properties of the human early visual system, in particular, the cortical mechanism that mediates learning by exploiting information about stimuli repetition. Repetition has long been considered a fundamental correlate of skill acquisition andmemory formation in biological aswell as computational learning models. However, recent studies have shown that biological neural networks have differentways of exploiting repetition in forming memory maps. The thesis focuses on a perceptual learning mechanism called repetition suppression, which exploits the temporal distribution of neural activations to drive an efficient neural allocation for a set of stimuli. This explores the neurophysiological hypothesis that repetition suppression serves as an unsupervised perceptual learning mechanism that can drive efficient memory formation by reducing the overall size of stimuli representation while strengthening the responses of the most selective neurons. This interpretation of repetition is different from its traditional role in computational learning models mainly to induce convergence and reach training stability, without using this information to provide focus for the neural representations of the data. The first part of the thesis introduces a novel computational model with repetition suppression, which forms an unsupervised competitive systemtermed CoRe, for Competitive Repetition-suppression learning. The model is applied to generalproblems in the fields of computational intelligence and machine learning. Particular emphasis is placed on validating the model as an effective tool for the unsupervised exploration of bio-medical data. In particular, it is shown that the repetition suppression mechanism efficiently addresses the issues of automatically estimating the number of clusters within the data, as well as filtering noise and irrelevant input components in highly dimensional data, e.g. gene expression levels from DNA Microarrays. The CoRe model produces relevance estimates for the each covariate which is useful, for instance, to discover the best discriminating bio-markers. The description of the model includes a theoretical analysis using Huber’s robust statistics to show that the model is robust to outliers and noise in the data. The convergence properties of themodel also studied. It is shown that, besides its biological underpinning, the CoRe model has useful properties in terms of asymptotic behavior. By exploiting a kernel-based formulation for the CoRe learning error, a theoretically sound motivation is provided for the model’s ability to avoid local minima of its loss function. To do this a necessary and sufficient condition for global error minimization in vector quantization is generalized by extending it to distance metrics in generic Hilbert spaces. This leads to the derivation of a family of kernel-based algorithms that address the local minima issue of unsupervised vector quantization in a principled way. The experimental results show that the algorithm can achieve a consistent performance gain compared with state-of-the-art learning vector quantizers, while retaining a lower computational complexity (linear with respect to the dataset size). Bridging the gap between the low level representation of the visual content and the underlying high-level semantics is a major research issue of current interest. The second part of the thesis focuses on this problem by introducing a hierarchical and multi-resolution approach to visual content understanding. On a spatial level, CoRe learning is used to pool together the local visual patches by organizing them into perceptually meaningful intermediate structures. On the semantical level, it provides an extension of the probabilistic Latent Semantic Analysis (pLSA) model that allows discovery and organization of the visual topics into a hierarchy of aspects. The proposed hierarchical pLSA model is shown to effectively address the unsupervised discovery of relevant visual classes from pictorial collections, at the same time learning to segment the image regions containing the discovered classes. Furthermore, by drawing on a recent pLSA-based image annotation system, the hierarchical pLSA model is extended to process and representmulti-modal collections comprising textual and visual data. The results of the experimental evaluation show that the proposed model learns to attach textual labels (available only at the level of the whole image) to the discovered image regions, while increasing the precision/ recall performance with respect to flat, pLSA annotation model.},
keywords = {biomedical data, clustering, competitive repetition suppression learning, feature selection, generative model, graphical models, image understanding, latent topic model, neural networks, statistics, unsupervised learning},
pubstate = {published},
tppubtype = {phdthesis}
}
Biology has been an unparalleled source of inspiration for the work of researchers in several scientific and engineering fields including computer vision. The starting point of this thesis is the neurophysiological properties of the human early visual system, in particular, the cortical mechanism that mediates learning by exploiting information about stimuli repetition. Repetition has long been considered a fundamental correlate of skill acquisition andmemory formation in biological aswell as computational learning models. However, recent studies have shown that biological neural networks have differentways of exploiting repetition in forming memory maps. The thesis focuses on a perceptual learning mechanism called repetition suppression, which exploits the temporal distribution of neural activations to drive an efficient neural allocation for a set of stimuli. This explores the neurophysiological hypothesis that repetition suppression serves as an unsupervised perceptual learning mechanism that can drive efficient memory formation by reducing the overall size of stimuli representation while strengthening the responses of the most selective neurons. This interpretation of repetition is different from its traditional role in computational learning models mainly to induce convergence and reach training stability, without using this information to provide focus for the neural representations of the data. The first part of the thesis introduces a novel computational model with repetition suppression, which forms an unsupervised competitive systemtermed CoRe, for Competitive Repetition-suppression learning. The model is applied to generalproblems in the fields of computational intelligence and machine learning. Particular emphasis is placed on validating the model as an effective tool for the unsupervised exploration of bio-medical data. In particular, it is shown that the repetition suppression mechanism efficiently addresses the issues of automatically estimating the number of clusters within the data, as well as filtering noise and irrelevant input components in highly dimensional data, e.g. gene expression levels from DNA Microarrays. The CoRe model produces relevance estimates for the each covariate which is useful, for instance, to discover the best discriminating bio-markers. The description of the model includes a theoretical analysis using Huber’s robust statistics to show that the model is robust to outliers and noise in the data. The convergence properties of themodel also studied. It is shown that, besides its biological underpinning, the CoRe model has useful properties in terms of asymptotic behavior. By exploiting a kernel-based formulation for the CoRe learning error, a theoretically sound motivation is provided for the model’s ability to avoid local minima of its loss function. To do this a necessary and sufficient condition for global error minimization in vector quantization is generalized by extending it to distance metrics in generic Hilbert spaces. This leads to the derivation of a family of kernel-based algorithms that address the local minima issue of unsupervised vector quantization in a principled way. The experimental results show that the algorithm can achieve a consistent performance gain compared with state-of-the-art learning vector quantizers, while retaining a lower computational complexity (linear with respect to the dataset size). Bridging the gap between the low level representation of the visual content and the underlying high-level semantics is a major research issue of current interest. The second part of the thesis focuses on this problem by introducing a hierarchical and multi-resolution approach to visual content understanding. On a spatial level, CoRe learning is used to pool together the local visual patches by organizing them into perceptually meaningful intermediate structures. On the semantical level, it provides an extension of the probabilistic Latent Semantic Analysis (pLSA) model that allows discovery and organization of the visual topics into a hierarchy of aspects. The proposed hierarchical pLSA model is shown to effectively address the unsupervised discovery of relevant visual classes from pictorial collections, at the same time learning to segment the image regions containing the discovered classes. Furthermore, by drawing on a recent pLSA-based image annotation system, the hierarchical pLSA model is extended to process and representmulti-modal collections comprising textual and visual data. The results of the experimental evaluation show that the proposed model learns to attach textual labels (available only at the level of the whole image) to the discovered image regions, while increasing the precision/ recall performance with respect to flat, pLSA annotation model.