Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | bartholomew-murphy |
View: | 223 times |
Download: | 0 times |
Online Dictionary Learning for Sparse
CodingInternational Conference on Machine Learning, 2009
Julien Mairal, Francis Bach, Jean Ponce and Guillermo Sapiro
Overview
• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion
Introduction
• The linear decomposition of a signal using a few atoms of a learned dictionary has recently led to state-of-the-art results for image processing tasks.• While learning the dictionary has proven to be critical to achieve results,
effectively solving the corresponding optimization problem is a significant computational challenge. (There may include millions of training sets.)
Overview
• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion
Problem Statement
• Classical dictionary learning techniques consider a finite training set of signals and optimize the empirical cost function.
• Note that should be small if D is “good” at represent the signal x.
Problem Statement
• To prevent D from being arbitrarily large (which would lead to arbitrarily small values of α), it is common to constrain columns to have an L-2 norm less than or equal to one.
• The problem of minimizing can be rewritten as a joint optimization problem with respect to dictionary D and coefficient α,
Problem Statement
• A nature approach to solving this problem is to alternate between the two variables, minimizing over one while keeping the other one fixed.• In the case of dictionary learning, classical projected first-order
stochastic gradient descent consists of a sequence of updates of D:
• The dictionary learning method authors present falls into the class of online algorithms based on stochastic approximations, processing one sample at a time, but exploits the specific structure of the problem to efficient solve it.
Overview
• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion
Online Dictionary Learning – Sparse Coding• The sparse coding problem of Eq. (2) with fixed dictionary is an L1-
regularized linear least-squares problem.• The columns of learned dictionaries are in general highly correlated,
so authors use LARS-Lasso algorithm (Osborne et al., 2000; Efron et al., 2004) to provide whole regularization path (i.e. for all possible values of λ).
Online Dictionary Learning – Dictionary Update• Proposed algorithm for updating the dictionary uses block-coordinate
descent with warm restarts, and one of its main advantages is that it is parameter-free and does not require any learning rate tuning.• Since the vectors are sparse, the coefficients of matrix A are in
general concentrated on the diagonal.• Since algorithm uses the value of as warm restart for computing , a
single iteration has empirically been found to be enough.
Overview
• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion
Overview
• Introduction• Problem Statement• Online Dictionary Learning• Experimental Validation• Conclusion
Conclusion
• Authors have introduced in this paper a new stochastic online algorithm for learning dictionaries adapted to sparse coding tasks.• Preliminary experiments demonstrate that it is significantly faster than
batch alternatives on large datasets that may contain millions of training example.
An Efficient Frame-Content Based Intra Frame Rate Control
for HEVCIEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO 7, JULY 2015
Miaohui Wang, King Ngi Ngan, and Hongliang Li
Introduction
• In this letter, authors propose a new gradient based R-λ model for the HEVC intra frame rate control, where the gradient is used to measure the frame-content complexity.• In addition, a novel bit allocation method is developed for CTU rate
control.
Modeling the Relationship Between Rate-Gradient and λ for the HEVC Frame Coding• Due to that different frames have different encoding complexities, the
frame-content complexity measure is incorporated into the proposed method for HEVC intra frame coding.
Bit Allocation – GOP Level Bit Allocation
Original – GOP Level Proposed – GOP Level
Original – Frame Level Proposed – Frame Level
Original – CU Level Proposed – CU Level
Simulation Configuration
1. HM 10.0 : the original HM 10.0 without rate control2. JCT-VC K0103: the original HM 10.0 with the default rate control3. JCT-VC M0257: the original HM 10.0 with the default intra frame rate
control4. Proposed method
Conclusion
• In this letter, a frame-content based rate control method is proposed for the HEVC intra frame coding.• The frame-content complexity is measured by its gradient, which has
been incorporated into an improved R-λ model.• A new bit allocation scheme with content complexity is developed at
the CTU level.