|
When learning models that are represented in matrix forms, enforcing a low-rank constraint can dramatically improve the memory and run time complexity, while providing a natural regularization of the model. However, naive approaches for minimizing functions over the set of low-rank matrices are either prohibitively time consuming (repeated singular value decomposition of the matrix) or numerically unstable (optimizing a factored representation of the low rank matrix). We build on recent advances in optimization over manifolds, and describe an iterative online learning procedure, consisting of a gradient step, followed by a second-order retraction back to the manifold. While the ideal retraction is hard to compute, and so is the projection operator that approximates it, we describe another second-order retraction that can be computed efficiently, with run time and memory complexity of O((n+m)k) for a rank-k matrix of dimension m x n, given rank one gradients. We use this algorithm, LORETA, to learn a matrix-form similarity measure over pairs of documents represented as high dimensional vectors. LORETA improves the mean average precision over a passive-aggressive approach in a factorized model, and also improves over a full model trained over pre-selected features using the same memory requirements. LORETA also showed consistent improvement over standard methods in a large (1600 classes) multi-label image classification task.
|
Video Length: 0
Date Found: March 28, 2011
Date Produced: March 25, 2011
|
|
VideoLectures |
July 10, 2011
The explosion in growth of the Web of Linked Data has provided, for the first time, a plethora of information in disparate locations, yet bound together by machine-readable, semantically typed relations. Utilisation of the Web of Data has been, until now, restricted to members of the community, ...
|
VideoLectures |
July 10, 2011
Problems cannot be solved with the mentality that has caused them’. Hence, the 2008- crisis cannot be solved with ethics of one-sided and short-term mentality of the industrial and neoliberal economics, which has caused the ‘Bubble Economy’ of several recent decades. Neither the market nor the ...
|
VideoLectures |
July 10, 2011
|
VideoLectures |
July 10, 2011
|
VideoLectures |
July 10, 2011
Social media presents unique challenges for topic classification, including the brevity of posts, the informal nature of conversations, and the frequent reliance on external hyperlinks to give context to a conversation. In this paper we investigate the usefulness of these external hyperlinks ...
|
|
|
|
|
|
Featured Content
Featuring websites that enhance the internet user’s experience.
|