Caltech Library System

About | Browse | Search | Caltech Student Instructions

Type of Document	Master's Thesis
Author	Angelova, Anelia Nedelcheva
Author's Email Address	anelia AT caltech.edu
URN	etd-05282004-000943
Persistent URL	http://resolver.caltech.edu/CaltechETD:etd-05282004-000943
Title	Data pruning
Degree	Master of Science
Option	Computer Science
Advisory Committee	Advisor Name Title Pietro Perona Committee Member
Keywords	data pruning regularization learning from noisy data outlier detection example elimination
Date of Defense	2004-05-28
Availability	unrestricted
Abstract Could a training example be detrimental to learning? Contrary to the common belief that more training data is needed for better generalization, we show that the learning algorithm might be better off when some training examples are discarded. In other words, the quality of the examples matters. We explore a general approach to identify examples that are troublesome for learning with a given model and exclude them from the training set in order to achieve better generalization. We term this process 'data pruning'. The method is targeted as a pre-learning step in order to obtain better data to train on. The approach consists in creating multiple semi-independent learners from the dataset each of which is influenced differently by individual examples. The multiple learners' opinions about which example is difficult are arbitrated by an inference mechanism. Although, without guarantees of optimality, data pruning is shown to decrease the generalization error in experiments on real-life data. It is not assumed that the data or the noise can be modeled or that additional training examples are available. Data pruning is applied for obtaining visual category data with little supervision. In this setting the object data is contaminated with non-object examples. We show that a mechanism for pruning noisy datasets prior to learning can be very successful especially in the presence of large amount of contamination or when the algorithm is sensitive to noise. Our experiments demonstrate that data pruning can be worth while even if the algorithm has regularization capabilities or mechanisms to cope with noise and has a potential to be a more refined method for regularization or model selection.
Files	Filename       Size       Approximate Download Time (Hours:Minutes:Seconds)     28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access    DataPruning.pdf 1.38 Mb 00:06:23 00:03:17 00:02:52 00:01:26 00:00:07