(a) The conventional metric learning based method (b) Our Global Relatedness Decoupled-Distillation (GRDD)
𝜓 ∙ |𝜽1
𝜙 ∙ |𝜽2
Update 𝜽𝟏
Global features
Episodic features
Stage2：Relatedness Decoupled-Distillation (RDD)
Global relatedness
The groups of the
decoupled-relatedness
1
2
𝑁𝑆
...𝑿1S 𝑿1
Q𝑿2Q 𝑿𝑁Q
Q
𝑿1Q
𝑿𝑁QQ
𝑿2Q
𝑿1S
𝑿2S
𝑿𝑁SS
...𝑿2S 𝑿1
Q 𝑿2Q 𝑿𝑁Q
Q
...𝑿𝑁SS
𝑿1Q
𝑿2Q 𝑿𝑁Q
Q
... ...
Query and
support images
Label predictor(e.g. cosine similarity)
Cross-
entropy loss
... 𝜓 ∙ |𝜽1
Random
episodic labels
Update 𝜽𝟏
Episodic features
Query and
support images
Stage1：Learning global category knowledge
Global category knowledge
Figure 1. The brief illustration of our Global Relatedness Decoupled-Distillation method (b) in training the meta-learner𝝍 (·|𝜽1), compared with the conventional metric learning based method (a). Of note, during the relatedness distillation, thewell-trained global learner 𝝓 (·|𝜽2) is frozen.
AbstractDespite the success that metric learning based approacheshave achieved in few-shot learning, recent works reveal theineffectiveness of their episodic training mode. In this paper,we point out two potential reasons for this problem: 1) therandom episodic labels can only provide limited supervisioninformation, while the relatedness information between thequery and support samples is not fully exploited; 2) the meta-learner is usually constrained by the limited contextual in-formation of the local episode. To overcome these problems,we propose a new Global Relatedness Decoupled-Distillation(GRDD) method using the global category knowledge andthe Relatedness Decoupled-Distillation (RDD) strategy. OurGRDD learns new visual concepts quickly by imitating thehabit of humans, i.e. learning from the deep knowledge dis-tilled from the teacher. More specifically, we first train aglobal learner on the entire base subset using category labelsas supervision to leverage the global context information
of the categories. Then, the well-trained global learner isused to simulate the query-support relatedness in globaldependencies. Finally, the distilled global query-support re-latedness is explicitly used to train the meta-learner usingthe RDD strategy, with the goal of making the meta-learnermore discriminative. The RDD strategy aims to decouple thedense query-support relatedness into the groups of sparsedecoupled relatedness. Moreover, only the relatedness ofa single support sample with other query samples is con-sidered in each group. By distilling the sparse decoupledrelatedness group by group, sharper relatedness can be ef-fectively distilled to the meta-learner, thereby facilitatingthe learning of a discriminative meta-learner. We conductextensive experiments on the miniImagenet and CIFAR-FSdatasets, which show the state-of-the-art performance of ourGRDD method.
Woodstock ’18, June 03–05, 2018, Woodstock, NY Zhou et al.
1 IntroductionIn recent years, deep learning has achieved impressive suc-cess in computer vision tasks such as image classification[14, 38], object detection [33, 34] and semantic segmenta-tion [4, 9]. However, it is well known that the deep-learningmodel tends to overfit for scarce training samples and per-forms far from satisfactory. In contrast, humans are still ableto learn new visual concepts quickly in the data-scarce cir-cumstances. This motivates the emergence of research forthe few-shot learning (FSL) problem [7], i.e., having the ma-chine learning system quickly learn new visual conceptsfrom only one or a few labelled training examples.Intuitively, a straightforward solution to the overfitting
problem is augmenting the target training dataset, e.g., bydata synthesis [13, 37] or large-scale weakly labelled or un-labelled datasets [6, 47]. However, the main problem of thedata augmentation based approach is that the augmentationpolicies need to be tailored for different datasets due to do-main gaps [44]. As one of the most widely used methods, themetric learning based method has achieved promising perfor-mance in FSL while maintaining high flexibility. In general,it aims to train a meta-learner for learning the transferablefeature embeddings from the known categories (i.e., the cat-egories of the auxiliary base subset 𝑫𝑏𝑎𝑠𝑒 with adequatetraining data). To bridge the gaps between the training andtesting phases, the episodic meta-training is designed [43].Therefore, for the target FSL task whose sample categoriesare unobserved, the meta-learner first encodes the queryand support samples into the embedding domain. Then, thequery samples are matched with the support sample cate-gories with the highest similarity [43] or the lowest distance[39].Despite the success achieved by metric learning based
methods, recent works [5, 42] show that their episodic train-ing mode is ineffective or even unnecessary. In this paper, wefirst point out two potential reasons for this phenomenon:1) the random episodic labels can only offer limited super-vision information, 2) while the meta-learner is generallyconstrained by the limited intra- and inter-categorical con-text dependencies of the local episode. These issues limit themodel’s capability in producing high-quality transferablefeature embeddings, and thus suppress model performance(the analysis is provided in Table 2 and Figure 3 of Section4.3). To overcome these problems, we propose a new met-ric learning based method, named as Global RelatednessDecoupled-Distillation (GRDD), which mimics the humanhabit of learning new concepts quickly, i.e., learning fromdeep knowledge distilled by the teacher. The differences be-tween our GRDD and the previous typical metric learningbased methods are shown in Figure 1. In the previous metriclearning based methods (e.g. [39, 41, 43]), the meta-learnerlearns from the randomly constructed episodic labels whosesupervision information is limited. In contrast, our GRDD
utilizes the global relatedness between the query and supportsamples to train the meta-learner, which is more informativeand thus makes the learned transferable embeddings morediscriminative.As can be seen in Figure 2, GRDD is designed in a two-
stage training manner as dual-learners are used. In the firsttraining stage, we train the global-learner 𝝓 (·|𝜽2) on the en-tire base subset using category labels as supervision to fullyexploit the global context dependencies of the categories.Then, in the second stage, the well-trained global-learneris used as a teacher to guide the episodic meta-trainingof the meta-learner 𝝍 (·|𝜽1). Specifically, we first use theglobal-learner to simulate the global query-support relat-edness for each episode, via leveraging the learned globalcategory knowledge. Then, the global relatedness informa-tion is explicitly distilled to the meta-learner, which allowsthe meta-learner to know the samples’ relatedness in theglobal context. To facilitate this process, we propose theRelatedness Decoupled-Distillation (RDD) strategy. It decou-ples the dense query-support relatedness into the groupsof sparse decoupled relatedness. In particular, each groupof decoupled relatedness only considers the relatedness ofa single support sample with other query samples. On onehand, the sparser the relatedness is, the easier it can be dis-tilled. On the other hand, decoupled relatedness is sharperin konwledge distillation, which is crucial in learning a dis-criminative meta-learner. To validate our method, extensiveexperiments are conducted on two public FSL datasets, i.e.,miniImagenet [43] and CIFAR-FS [2], which firmly validatesthe effectiveness of our method.
All in all, the contribution of this paper can be summarizedas follows:
• We point out the weaknesses of the current episodictraining mode used in the metric learning based FSLmethods, and propose a newGlobal Relatedness Decoupled-Distillation (GRDD) method to overcome these prob-lems.
• Instead of the random episodic labels, we propose toexplicitly use the distilled global query-support re-latedness to train the meta-learner, which makes thelearned transferable feature embeddings more discrim-inative.
• We introduce the Relatedness Decoupled-Distillation(RDD) strategy to facilitate the relatedness distillation.It decouples the entire query-support relatedness intothe groups of sparse decoupled relatedness to makethe relatedness information sharper and easier to bedistilled.
• On the miniImagenet and CIFAR-FS datasets, our pro-posed GRDD presents the state-of-the-art performancecompared to other counterparts.
Few-shot Learning with Global Relatedness Decoupled-Distillation Woodstock ’18, June 03–05, 2018, Woodstock, NY
2 Related workIn this section, we briefly review the related FSL methodsand introduce the differences between our proposed methodand the most relevant approaches.
Metric learning based method. The metric learningbased methods work in a learning-to-learn paradigm. It aimsto train a meta-learner for learning high-quality transferablefeature embeddings that can be well generalized to solvethe target FSL tasks whose sample categories are unseen.Among the metric learning based methods, MatchNet [43] isa representative work. It develops the episodic meta-trainingto bridge the gaps between the training and testing phasesof FSL, using the random episodic labels as training super-vision. Snell et al. [39] further develop MatchNet by intro-ducing prototype representation so that query samples arecategorized according to their Euclidean distances to theprototypes. Li et al. [20] propose to retrieve the global classrepresentations by using the local features to categorize thequery samples. Moreover, in [19], they introduce the adap-tive margin loss to improve the feature representation of thesamples by further considering the semantic relation of thecategories in Glove [26]. Unlike the above methods that userandom episodic labels as supervision [19, 20, 39, 43], ourGRDD explicitly uses the global query-support relatednessto train the meta-learning with the goal of making the meta-learner more discriminative. Moreover, compared to [19],the distilled sample-wise relatedness is more fine-grainedthan the category-level relation in Glove [26], and thus moreimplicit information can be exploited, leading to accurateclassification, as in Table 2. Additionally, different from theworks [30, 42] simply resorting to the pretraining strategy,our GRDD aims to enhance the performance of the episodictraining mode.
External memory based method. The external mem-ory based methods are inspired by the recent success of Neu-ral Turing Machine [11]. As a representative work, MAML[8] proposes to design the memory module in a key-to-valueparadigm. It first records the useful information of the sup-port set into memory and then reads out the stored infor-mation to categorize the query samples. Ramalho et al. [31]boost the memory module by only memorizing the mostunexpected information, thus suppressing memory redun-dancy. Kaiser et al. [16] design a long-term memory modulesuitable for solving the lifelong learning problems. It shouldbe noted that memory-augmented models generally need tobe fine-tuned on the support set of the target tasks in orderto obtain sufficient useful information of the new categories.In contrast, our GRDD can be used to directly categorize thequery samples without the need for fine-tuning.
Hallucination-based method. The hallucination-basedFSL methods can be divided into two sub-directions, i.e. hal-lucinations of new data [13, 45, 50] and hallucinations ofclassifier weights [10, 27, 29]. Hariharan et al. [13] propose
a non-parametric data hallucination approach that halluci-nates new support features for the novel unseen categoriesusing the inter-category commonality. Wang et al. [45] pro-pose a hallucinator that synthesizes new images with dif-ferent object poses or backgrounds by introducing randomnoise into the original image, while Zhang et al. [50] proposeto hallucinate new data using the guidance of salient objects.In contrast to hallucinating new data, [10, 27, 29] proposeto hallucinate the classifier weights for the novel categoriesaccording to the feature activations of the support samples.
Transductive vs. inductive method. In traditional in-ductive FSL, each query sample is categorized independently.Transductive FSL, on the other hand, aims to categorize allquery samples at once, or to consider the generated episodictasks as a whole, thus leveraging information from boththe support and the query sets. For example, Boudiaf et al.[3] propose to maximize the mutual information betweenthe embedding features and the label prediction. Ziko etal. [51] propose to impose an additional constraint on cat-egory inference, i.e., nearby samples should have the sameconsistent label assignments. In contrast, Liu et al. in [21]propose to propagate labels from labelled instances to un-labelled instances using the manifold structure of the data.As mentioned in [51], the transductive-based methods areusually more accurate than the inductive ones. Neverthe-less, they face an unavoidable drawback, namely that thetransductive model has to be retrained from scratch whennew query samples or new episodic tasks appear. As for ourGRDD, it is inductive and thus can be used to categorizenew query samples or address new tasks directly once thetraining phase is complete. Last but not least, our GRDD canbe easily integrated into the transductive approach, such as‘GRDD-TIM’ in Table 1.
3 MethodIn this section, we first present the preliminaries and thendescribe the proposed method in detail.
3.1 PreliminaryAn FSL task consists of two subsets of data, commonly re-ferred to as the support set 𝑺 =
{(𝑿𝑆
𝑖 , 𝑦𝑆𝑖 )}𝑁𝑆−1𝑖=0 and query
set 𝑸 ={𝑿𝑄
𝑖
}𝑁𝑄−1𝑖=0 . For the default ‘𝐶-way 𝐾-shot’ setting,
the 𝑁𝑆 labelled support samples are prepared by randomlysampling 𝐾 labelled samples from each of the 𝐶 categories(i.e., 𝑁𝑆 = 𝐶 × 𝐾), while the 𝑁𝑄 unlabelled query samplesare also randomly drawn from these 𝐶 categories. Note thatthe instances from the support and query sets are disjoint,i.e., 𝑺 ∩ 𝑸 = ∅. The ultimate goal of FSL is to categorize thequery samples by exploiting the prior knowledge containedin the support set, as in Equation 1:
𝑦𝑖 = arg max�̃�𝑖 ∈{1,..,𝐶 }
𝑃 (𝑦𝑖 |𝑿𝑄
𝑖, 𝑺) . (1)
Woodstock ’18, June 03–05, 2018, Woodstock, NY Zhou et al.
.
Relatedness
decoupling
Relatedness
decoupling
.
The mini-batch image
ϕ ∙ |𝜽2CE
Stage 1: Global Category Knowledge Learning
Update 𝜽𝟏
Global features
Stage 2: Relatedness Decoupled-Distillation
FC ∙ |𝜽3Prediction
...
Episodic features
Category labels
Update 𝜽𝟐
(a) Training on the base subset 𝑫base
......
The episode data
ϕ ∙ |𝜽2
𝜓 ∙ |𝜽1
...
Global features
Query: 𝑿1𝑄~𝑿𝑁𝑄
𝑄
Support: 𝑿1𝑆~𝑿𝑁𝑆
𝑆
KL KL KL...
...𝑿1S 𝑿1
Q𝑿2Q 𝑿𝑁Q
Q
1
Rotation
FC ∙ |𝜽4 CE
Rotation
labels
(b) Testing on the target task
𝑠1
Logistic
Regression
Brid AlpacaTiger
...
𝜓∙|𝜽
1Query: 𝑿1
𝑄~𝑿𝑁𝑄
𝑄 Support: 𝑿1𝑆~𝑿𝑁𝑆
𝑆
...
𝑿1Q
𝑿𝑁QQ
𝑿2Q
𝑿1S
𝑿2S
𝑿𝑁SS
𝑿1Q
𝑿𝑁QQ
𝑿2Q
𝑿1S
𝑿2S
𝑿𝑁SS
...𝑿2S 𝑿1
Q𝑿2Q 𝑿𝑁Q
Q
...𝑿𝑁SS
𝑿1Q
𝑿2Q 𝑿𝑁Q
Q
...𝑿1S 𝑿1
Q𝑿2Q 𝑿𝑁Q
Q
...𝑿2S 𝑿1
Q𝑿2Q 𝑿𝑁Q
Q
...𝑿𝑁SS
𝑿1Q
𝑿2Q 𝑿𝑁Q
Q
2𝑁𝑆
𝑁𝑆21
SUM
Figure 2. The overview for our Global Relatedness Decoupled-Distillation (GRDD) method. In particular, ‘CE’ stands for thecross-entropy loss and ‘KL’ for the KL divergence, while 𝝍 (·|𝜽1) and 𝝓 (·|𝜽2) represent the meta-learner and the global-learner,respectively. Moreover, ‘FC’ indicates the fully connected layer, and ‘SUM’ denotes the summation operation.
In Equation 1, 𝑃 (𝑦𝑖 |𝑿𝑄
𝑖, 𝑺) gives the probability that the
query sample 𝑿𝑄
𝑖is classified as the label 𝑦𝑖 conditioned
on the support set 𝑺 . To handle the FSL task, the metriclearning based approach resorts to an auxiliary base subset𝑫𝑏𝑎𝑠𝑒 . Note that the label spaces of the base subset 𝑫𝑏𝑎𝑠𝑒 andthe target FSL task are disjoint. As with current metric learn-ing based methods, episodic meta-training is commonly usedto bridge the gaps between the training and testing phasesof FSL, using the randomly constructed episodic labels astraining supervision, such as [39, 41, 43]. Obviously, theepisodic training mode has advantages in training a meta-learner with high generalization. Nevertheless, the recentworks (e.g., [42, 46]) reveal its ineffectiveness in training theFSL model. Therefore, this paper highlights two potentialproblems of the current episodic meta-training and proposesa new metric learning based method to alleviate these prob-lems.
3.2 Global Relatedness Decoupled-DistillationWe propose a new metric learning based method, called asGlobal Relatedness Decoupled-Distillation (GRDD), whichaims to imitate the human habit of learning novel concepts,i.e., learning from deep knowledge distilled by the teacher.In our GRDD method, two different learners are used, calledglobal-learner 𝝓 (·|𝜽2) andmeta-learner 𝝍 (·|𝜽1). Accordingly,as shown in Figure 2, GRDD is designed in a two-stage train-ing manner. In the first stage, the global-learner 𝝓 (·|𝜽2) istrained on the entire base subset 𝑫𝑏𝑎𝑠𝑒 , using the categorylabels as training supervision. In this way, category knowl-edge can be exploited in the global contextual dependencies.
In the second training stage, we then use the global query-support relatedness distilled from the global-learner 𝝓 (·|𝜽2)to train the meta-learner 𝝍 (·|𝜽1) based on the episodic train-ing mode. To facilitate the relatedness learning, we proposethe Relatedness Decoupled-Distillation (RDD) strategy inour GRDD, which decouples the dense query-support re-latedness into the groups of sparse decoupled relatedness,making the relatedness sharper and easier to distill. Sections3.2.1 and 3.2.2 present these two training stages in detail.
3.2.1 Global Category Knowledge Learning. To fullyexploit the global context dependencies of the categories,we train the global-learner 𝝓 (·|𝜽2) on the entire base sub-set 𝑫𝑏𝑎𝑠𝑒 at the first training stage, using the category la-bels as supervision. In this process, we use the well-knownmini-batch training strategy for fast model convergence.We also employ the data augmentation strategy [30], whichaugments the input mini-batch images
{𝑿𝑖
}𝑁𝑏−1𝑖=0 by rotat-
ing them to 90◦, 180◦, and 270◦, respectively, and obtains{𝑿𝑅𝑖
}𝑁𝑏−1,270𝑖=0,𝑅=0 . Accordingly, the one-hot rotation labels are
constructed, i.e.,{𝑟𝑅𝑖
}𝑁𝑏−1,270𝑖=0,𝑅=0 . Note that in the following sec-
tions 𝑅 = 0, 90 , 180 or 270 is used when there are no specialstatements. As shown in Figure 2, we first send the aug-mented mini-batch data to the global-learner 𝝓 (·|𝜽2) and usethe global-learner to extract their high-level feature repre-sentations
{𝒉𝑅𝑖
}𝑁𝑏−1,270𝑖=0,𝑅=0 as in Equation 2:
𝒉𝑅𝑖 = 𝝓 (𝑿𝑅𝑖 |𝜽2) (2)
where 𝒉𝑅𝑖 is the features of 𝑿𝑅𝑖 , while 𝜽2 gives the learnable
parameters of the global-learner. Then, a fully connected
Few-shot Learning with Global Relatedness Decoupled-Distillation Woodstock ’18, June 03–05, 2018, Woodstock, NY
layer 𝐹𝐶 (·|𝜽3) is applied to the features 𝒉𝑅𝑖 to predict theircategories, as in Equation 3:
𝒑𝑅𝑖 = 𝐹𝐶 (𝒉𝑅𝑖 |𝜽3). (3)
In Equation 3, 𝒑𝑅𝑖 denotes the one-hot category prediction of
𝒉𝑅𝑖 , while 𝜽3 indicates the learnable parameters of the fullyconnected layer.After that, another fully connected layer 𝐹𝐶 (·|𝜽4) is ap-
plied to 𝒑𝑅𝑖 , aiming to infer the label of rotation angle, as
described in Equation 4:
𝒓𝑅𝑖 = 𝐹𝐶 (𝒑𝑅𝑖 |𝜽4). (4)
Finally, the groundtruth category labels{𝑦𝑖}𝑁𝑏−1𝑖=0 and the
rotation labels{𝑟𝑅𝑖
}𝑁𝑏−1,270𝑖=0,𝑅=0 are used to jointly optimize the
entire network, as shown below:
𝜽 ′2 = 𝜽2 − 𝑙𝑟1 ∗
𝜕(L𝑐 + L𝑎)𝜕𝜽2
(5)
𝜽 ′3 = 𝜽3 − 𝑙𝑟1 ∗
𝜕(L𝑐 + L𝑎)𝜕𝜽3
(6)
𝜽 ′4 = 𝜽4 − 𝑙𝑟1 ∗
𝜕(L𝑐 + L𝑎)𝜕𝜽4
(7)
where
L𝑐 =1
𝑁𝑏 ∗ 4
𝑁𝑏−1∑︁𝑖=0
270∑︁𝑅=0
𝐶𝑏𝑎𝑠𝑒−1∑︁𝑗=0
𝑜𝑛𝑒_ℎ𝑜𝑡 (𝑦𝑖 ) 𝑗 ∗ log(𝒑𝑅𝑖 ) 𝑗 (8)
and
L𝑟 =1
𝑁𝑏 ∗ 4
𝑁𝑏−1∑︁𝑖=0
270∑︁𝑅=0
3∑︁𝑗=0
𝑜𝑛𝑒_ℎ𝑜𝑡 (𝑟𝑅𝑖 ) 𝑗 ∗ log(𝒓𝑅𝑖 ) 𝑗 . (9)
Note that the learning rate 𝑙𝑟1 is initialized as 5𝑒−2 in Equa-tion 5, 6 and 7, and decays in the ‘poly’ manner. 𝑜𝑛𝑒_ℎ𝑜𝑡 (·)indicates the operation for one-hot encoding.
The advantages of learning global category knowledge aretwofold. On one hand, the above training strategy is moreglobal than the episodic training mode, and thus categoryknowledge can be learned in the more global contextual de-pendencies. On the other hand, the global knowledge learnedby the global-learner 𝝓 (·|𝜽2) is more informative than therandom episodic labels, which can be used to better guidethe training of the meta-learner 𝝍 (·|𝜽1). In this way, thepreviously mentioned weaknesses of episodic meta-trainingcan be relieved. In Section 3.2.2, we will elaborate the de-tails of distiling the learned global knowledge to train themeta-learner 𝝍 (·|𝜽1).
3.2.2 RelatednessDecoupled-Distillation. Consideringthat the random episodic labels can only provide limitedsupervision information, we therefore propose to use theglobal-learner 𝝓 (·|𝜽2) for simulating the relatedness betweenthe query and support samples in the global context depen-dencies of the categories, which is then used to explicitlytrain the meta-learner 𝝍 (·|𝜽1). To facility the learning of
relatedness, our GRDD method introduces the RelatednessDecoupled-Distillation (RDD) strategy.
More specifically, for each episodic data{𝑿𝑆𝑖 ,𝑿
𝑄
𝑗
}𝑁𝑆−1,𝑁𝑄−1𝑖=0, 𝑗=0 ,
we first use the global-learner 𝝓 (·|𝜽2), to extract their high-level features
{𝒉𝑆𝑖 ,𝒉
𝑄
𝑗
}𝑁𝑆−1,𝑁𝑄−1𝑖=0, 𝑗=0 . Since the features are ex-
tracted based on the learned global category knowledge,they are referred to as global features in this paper. Based onthese global features, we then extract the global relatednessinformation between the query and support samples (i.e.,𝑹𝑔 ∈ R𝑁𝑆×𝑁𝑞 ) using Equation 10:
𝑹𝑔
𝑖 𝑗=
𝒉𝑆𝑖 ∗ 𝒉𝑄
𝑗
| |𝒉𝑆𝑖| | ∗ | |𝒉𝑄
𝑗| |
(10)
where 𝑹𝑔
𝑖 𝑗denotes the (𝑖, 𝑗) element of 𝑹𝑔 which is the re-
lation between the 𝑖-𝑡ℎ support sample and the 𝑗-𝑡ℎ querysample.Then, the relatedness 𝑹𝑔 is decoupled into the groups of
sparse decoupled relatedness [𝝎𝑔
0 , ...,𝝎𝑔
𝑁𝑆−1] for knowledgedistillation, as in Equation 11:
𝝎𝑔
𝑖=∥
𝑁𝑞−1𝑗=0
𝑒𝑥𝑝 (𝑹𝑖 𝑗/𝑇 )∑𝑗 𝑒𝑥𝑝 (𝑹𝑖 𝑗/𝑇 )
(11)
where ∥ denotes the concatenation operation, while 𝑇 is thetemperature hyperparameter used to smooth the values ofrelatedness for knowledge distillation.
After that, we distill the decoupled relatedness [𝝎𝑔
0 , ...,𝝎𝑔
𝑁𝑆−1]to the meta-learner 𝝍 (·|𝜽1) group by group. For better expla-nation, we use 𝑹𝑒 to represent the query-support relatednesscomputed based on the features extracted from the meta-learner, while the decoupled relatedness computed basedon 𝑹𝑒 is denoted as [𝝎𝑒
0 , ...,𝝎𝑒𝑁𝑆−1]. We first use KL diver-
gence 𝐾𝐿(·, ·) to measure the deviation between each groupof [(𝝎𝑔
0 ,𝝎𝑒0 ), ..., (𝝎
𝑔
𝑁𝑆−1,𝝎𝑒𝑁𝑆−1)], and then the loss of KL de-
viation is summed up as in Equation 12:
L𝑘𝑙 =
𝑁𝑆−1∑︁𝑖=0
𝐾𝐿(𝝎𝑔
𝑖,𝝎𝑒
𝑖 ). (12)
In addition, a regularized term L𝑟𝑡 is used to regularize therelatedness distillation, which constrains that samples withthe same categories have higher relatedness:
L𝑟𝑡 =1𝑁𝑄
𝑁𝑄−1∑︁𝑖=0
𝐶−1∑︁𝑗=0
𝑜𝑛𝑒_ℎ𝑜𝑡 (𝑦𝑄𝑖) 𝑗 ∗ log(𝝈𝑖 ) 𝑗 (13)
where
𝝈𝑖 =
𝑁𝑆−1∑︁𝑗=0
(𝑹𝑒 )𝑇𝑖 𝑗 ∗ 𝑜𝑛𝑒_ℎ𝑜𝑡 (𝑦𝑆𝑗 ). (14)
Finally, the meta-learner parameters are updated via thejoint usage of L𝑘𝑙 and L𝑟𝑡 , as shown in Equation 15:
𝜽 ′1 = 𝜽1 − 𝑙𝑟2 ∗
𝜕(L𝑘𝑙 + 𝛾 ∗ L𝑟𝑡 )𝜕𝜽1
(15)
Woodstock ’18, June 03–05, 2018, Woodstock, NY Zhou et al.
Table 1. The accuracy comparison between our proposed GRDD and the related state-of-the-art approaches on miniImagenetand CIFAR-FS datasets, with 95% confidence interval. It is noteworthy that the methods marked with ‘‡’ are based ontransductive learning, while the remaining methods are inductive. Moreover, ‘Arch.’ denotes the network architecture, while‘n/a’ indicates the unavailable results in original papers.
where the hyperparameter 𝛾 is to control the balance be-tween L𝑘𝑙 and L𝑟𝑡 . Note that during this process, the param-eters of the well-trained global-learner 𝝓 (·|𝜽2) are frozen. Byusing the proposed RDD strategy to train the meta-learner𝝍 (·|𝜽1), our method achieves competitive experimental per-formance compared to other counterparts, which is shownin the next section.
4 ExperimentWith the aim of validating our proposed method, we con-duct extensive experiments on two public FSL datasets, i.e.,miniImagenet [43] and CIFAR-FS [2]. In this section, we first
introduce these datasets and the implementation details ofthe experiments. Then, we compare our GRDD in detail withthe related state-of-the-art approaches.
4.1 Dataset and Implementation DetailsDataset. miniImagenet [43] and CIFAR-FS [2] are the mostcommonly used FSL datasets. In particular, CIFAR-FS is de-rived from the CIFAR-100 [17] dataset, while miniImagenetis derived from the larger ILSVRC-12 [35] dataset. Remark-ably, these two datasets both contain 60000 images with 100different semantic categories. But the image resolutions ofthe datasets are different. Specifically, CIFAR-FS consists of
Few-shot Learning with Global Relatedness Decoupled-Distillation Woodstock ’18, June 03–05, 2018, Woodstock, NY
Table 2. The demonstration for the weaknesses of the cur-rent episodic meta-training and the strength of our relat-edness distillation method in FSL. ‘CL’ or ‘EL’ denotes theexperiments that use category labels or episodic labels astraining supervision. ‘GR’ represents the usage of our globalrelatedness. ‘Arch.’ denotes the network architecture.
Supervision Arch. of 𝝓 ( · |𝜽2) Arch. of 𝝍 ( · |𝜽1) Acc (%)CL 𝑛𝑜 ConvNet-4 64.6 ± 0.9
32 × 32 images, while the images from miniImagenet have aresolution of 84 × 84. Following previous works [30, 46? ],for these two datasets, the 100 categories are divided into 64,16 and 20 for training, validation and testing, respectively.
Implementation Details. All our experiments are builton Pytorch1. Following [2, 39, 46], we respectively use ConvNet-4 [43] and ResNet-12 [14] to implement the meta-learner.Note that the global-learner is implemented by ResNet-12if there is no special declaration. For all experiments, wechoose the Stochastic Gradient Descent (SGD) as the op-timizer, of which the weight decay is empirically set to5𝑒−4. Under the two-stage training manner, different trainingstrategies are applied to the different stages. In particular, forthe first training stage, we adopt the ‘poly’ learning rate, i.e.,𝑙𝑟1 = 𝑙𝑟_𝑖𝑛𝑖𝑡×(1− 𝑖𝑡𝑒𝑟
𝑖𝑡𝑒𝑟_𝑡𝑜𝑡𝑎𝑙 )𝑝𝑜𝑤𝑒𝑟 , where 𝑙𝑟_𝑖𝑛𝑖𝑡 is set to 1𝑒−1
and 𝑝𝑜𝑤𝑒𝑟 is set to 0.9. We also use the well-known mini-batch training strategy for fast model convergence. Note thatfor all datasets, the batch size is set to 64 and the epoch isset to 90. However, in the second training stage, we set asmaller initial learning rate 𝑙𝑟2 and epoch, which are 1𝑒−3
and 15, respectively. Moreover, the learning rate decays by afactor of 0.1 for the last 5 epochs. For the hyperparameters,𝛾 and𝑇 are respectively set to 0.2 and 4, which are validatedin Section 4.3.
4.2 Comparison with the state-of-the-art methodsIn this section, we compare our GRDD with related state-of-the-art approaches summarized in Table 1. Note that fora fair comparison, the methods based on different networkstructures are compared accordingly.
ConvNet-4. In this part, we compare our GRDD with themethods implemented using ConvNet-4. As shown in Table1, our GRDD largely outperforms the compared methods onthe miniImagenet and CIFAR-FS datasets. For example, onthe miniImagenet dataset, GRDD is more accurate than DC
1https://pytorch.org/
(b) CL+EL(a) CL (c) Our GRDD
Figure 3. The t-SNE visualization for the embeddings of the‘CL’, ‘CL+EL’ and our GRDD method based on ResNet-12.
[49] and R2D2 [2] by about 4% and 8%, respectively. On theCIFAR-FS dataset, the accuracy of our GRDD is still higherthan that of R2D2, about 4% higher on the 5-way 1-shot task,while 5% higher on the 5-way 5-shot task.
ResNet-12. In general, higher accuracy can be achievedby using larger models. Thus, by implementing GRDD withResNet-12, GRDD consistently shows better performancethan the ConvNet-4 version. As shown in Table 1, our GRDDis obviously more accurate than most methods, such asSNAIL [22] and VFSL [36]. Moreover, our GRDD is evenbetter compared to the recent works RFS [42], SKD [30] andCBM [46]. For example, on the miniImagenet dataset, theaccuracy of our method is 0.5% and 0.8% higher than thatof SKD on the 1-shot and 5-shot tasks, respectively. On theCIFAR-FS dataset, the accuracy of our GRDD is about 4%and 2% more accurate than RFS, respectively.
Transductive learning. Although our GRDD is proposedas an inductive approach, it can be easily integrated into thetransductive learning approach. For example, we integrateour GRDD with TIM [3], which is called ‘GRDD-TIM’ inTable 1. On the one hand, ‘GRDD-TIM’ achieves a signifi-cant performance gain over the baseline TIM [51]. On theother hand, it also achieves the state-of-the-art performanceamong the transductive counterparts even if our GRDD isimplemented based on a smaller neural network ResNet-12.The above experiments strongly firm the effectiveness
and flexibility of our GRDD. It should be noted that theablation study is conducted in Section 4.3 to further analyzeour proposed method.
4.3 Ablation studyIn this section, we conduct the ablation study for our work.We first analyze the weaknesses of the current episodic train-ing mode. Then, we investigate the impact of each compo-nent in our GRDD. Finally, the settings of two vital haperpa-rameters (i.e.,𝛾 and𝑇 ) are validated. For brevity, we note that‘CL’ denotes the methods pretrained on the Category Labels,while “CL+EL” indicates the methods that further finetrainthe pretrained model using the Episodic Labels. Moreover,
Woodstock ’18, June 03–05, 2018, Woodstock, NY Zhou et al.
63
68
73
78
ConvNet-4 ResNet-12
AC
C (
%)
CIFAR-FSGRDD
w/o RDD
w/o
CL
CL+EL
50
55
60
65
70
ConvNet-4 ResNet-12
AC
C (
%)
miniImagenetGRDD
w/o RDD
w/o
CL
CL+EL
ℒ𝑟𝑡ℒ𝑟𝑡
Figure 4. The influence of each component of GRDD. ‘w/oRDD’ indicates that GRDD is used without the RelatednessDecoupled-Distillation (RDD) strategy and the relatedness isdistilled as a whole instead. ‘w/o L𝑟𝑡 ’ indicates that GRDDis implemented without using the regularized term L𝑟𝑡 .
T=1
T=4
(a) w/o decoupling (b) w/ decoupling
𝑁𝑆 × 𝑁Q matrix
𝑁𝑆 groups
𝑁𝑄
Decoupled relatedness slot
Figure 5. The comparison between relatedness with (b) andwithout (a) relatedness decoupling in knowledge distillation,under different values of temperature 𝑇 . Note that withoutdecoupling, the relatedness is distilled as a whole matrix.While with decoupling, the relatedness is distilled corre-sponding to each decoupled relatedness slot.
88
88.5
89
89.5
72
75
78
81
84
0 0.2 0.4 0.6 0.8 1
Hyperparameter ϒ
CIFAR-FS
5-way 1-shot5-way 5-shot
88
88.5
89
89.5
74
76
78
80
0.5 1 2 4 6 8Hyperparameter T
CIFAR-FS
5-way 1-shot5-way 5-shot
1-s
ho
t A
CC
(%
)
5-s
ho
t A
CC
(%
)
1-s
ho
t A
CC
(%
)
5-s
ho
t A
CC
(%
)
Figure 6. Ablation study for the hyperparameters 𝛾 and 𝑇 .
‘CL+GR’ denotes our GRDD that uses the Global Relatednessextracted from the category labels to train the meta-learner.
Improvement over episodic training. Recent works[42, 46] find that the episodic training mode in FSL is ineffec-tive and unnecessary. Here, we give two potential reasonsfor this phenomenon, which are experimentally analyzedin this part. As shown in Table 2, ‘CL+EL’ does not always
yield a performance gain over the baseline model ‘CL’. Forexample, on ResNet-12, accuracy actually decreases by about2% when episodic labels are further used. This is becausethe episodic labels can only provide limited supervision andthus are unable to boost the quality of feature embeddingseffectively. Instead, the learned global category knowledgemay be destroyed by the local episodic meta-training, whosecontext is very limited. However, when more informativeglobal relatedness is used in meta-training, ‘CL+GR’ achievessignificant improvement in all experiments. In addition, themore accurate the relatedness information is (i.e., extractedby a larger model), the higher the accuracy can be obtained.This proves the effectiveness of our GRDD, while the limitedinformation of the episodic labels is the bottleneck in theepisodic training mode. This conclusion is also consistentwith the visualized analysis in Figure 3, where we can seethat the episodic labels make the embedding space morecompact, but the boundary between different categories be-comes blurred because of the limited guidance of supervisioninformation. However, our relatedness information makesthe embedding space more compact, meanwhile the categoryboundary becomes clearer and more discriminative.
Influence of each component in GRDD. As shown inFigure 4, we first compare our GRDD with two degenerateversions ‘w/o L𝑟𝑡 ’ and ‘w/o RDD’. The results in Figure 4indicate: 1) using the RDD strategy is better than distillingthe relatedness information as a whole matrix; 2) incorporat-ing RDD with the regularized term L𝑟𝑡 is better than usingRDD alone. Moreover, our GRDD also shows consistentlybetter performance than ‘CL’ and ‘CL+EL’. Therefore, theeffectiveness of the two key components of our GRDD can beverified. It is also worth noting that the visualization of therelatedness with and without decoupling in the knowledgedistillation is demonstrated in Figure 5, where the decoupledrelatedness is more discriminative than the relatedness thatis considered as a whole matrix.
Hyperparameter settings. Furthermore, the experimentsin Figure 6 are conducted to validate the settings for two keyhyperparameters in our GRDD, i.e., 𝛾 and 𝑇 . The results inFigure 6 show that the 𝛾 = 0.2 and 𝑇 = 4 setting can yieldbetter performance.
5 ConclusionIn this paper, we show that the bottleneck of the episodictraining mode lies in the limited supervision informationof episodic labels and the scarce category context. To alle-viate these problems, we propose a new Global RelatednessDecoupled-Distillation (GRDD) method that explicitly usesthe more informative global query-support relatedness totrain the meta-learner, making it more discriminative. More-over, the Relatedness Decoupled-Distillation (RDD) strategyis introduced to facilitate this procedure. RDD decouplesthe dense relatedness into the groups of sparse decoupled
Few-shot Learning with Global Relatedness Decoupled-Distillation Woodstock ’18, June 03–05, 2018, Woodstock, NY
relatedness, making the relatedness sharper and easier to bedistilled. Extensive experiments on the miniImagenet andCIFAR-FS datasets validate the effectiveness of our method.In the future, we plan to apply our method in other FSLdomains, such as open-set FSL and domain-shift FSL.
Woodstock ’18, June 03–05, 2018, Woodstock, NY Zhou et al.
References[1] Kelsey Allen, Evan Shelhamer, Hanul Shin, and Joshua Tenenbaum.
2019. InfiniteMixture Prototypes for Few-shot Learning. In Proceedingsof the International Conference on Machine Learning. 232–241.
[2] Luca Bertinetto, Joao F Henriques, Philip Torr, and Andrea Vedaldi.2018. Meta-learning with differentiable closed-form solvers. In Pro-ceedings of the International Conference on Learning Representations.
[3] Malik Boudiaf, Imtiaz Ziko, Jérôme Rony, Jose Dolz, Pablo Piantanida,and Ismail Ben Ayed. 2020. Information Maximization for Few-ShotLearning. Advances in Neural Information Processing Systems 33 (2020).
[4] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Mur-phy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentationwith deep convolutional nets, atrous convolution, and fully connectedcrfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40,4 (2017), 834–848.
[5] Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, andStefano Soatto. 2019. A Baseline for Few-Shot Image Classification. InInternational Conference on Learning Representations.
[6] Matthijs Douze, Arthur Szlam, Bharath Hariharan, and Hervé Jégou.2018. Low-shot learning with large-scale diffusion. In Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition.3349–3358.
[7] Li Fe-Fei et al. 2003. A Bayesian approach to unsupervised one-shotlearning of object categories. In Proceedings Ninth IEEE InternationalConference on Computer Vision. IEEE, 1134–1141.
[8] Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnosticmeta-learning for fast adaptation of deep networks. In Proceedingsof the 34th International Conference on Machine Learning-Volume 70.1126–1135.
[9] Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, andHanqing Lu. 2019. Dual attention network for scene segmentation.In Proceedings of the IEEE Conference on Computer Vision and PatternRecognition. 3146–3154.
[10] Spyros Gidaris and Nikos Komodakis. 2018. Dynamic few-shot visuallearning without forgetting. In Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition. 4367–4375.
[11] Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural turingmachines. arXiv preprint arXiv:1410.5401 (2014).
[12] Fusheng Hao, Fengxiang He, Jun Cheng, Lei Wang, Jianzhong Cao,and Dacheng Tao. 2019. Collect and select: Semantic alignment metriclearning for few-shot learning. In Proceedings of the IEEE InternationalConference on Computer Vision. 8460–8469.
[13] Bharath Hariharan and Ross Girshick. 2017. Low-shot visual recog-nition by shrinking and hallucinating features. In Proceedings of theIEEE International Conference on Computer Vision. 3018–3027.
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deepresidual learning for image recognition. In Proceedings of the IEEEConference on Computer Vision and Pattern Recognition. 770–778.
[15] Muhammad Abdullah Jamal and Guo-Jun Qi. 2019. Task agnostic meta-learning for few-shot learning. In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition. 11719–11727.
[16] Łukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. 2017. Learn-ing to remember rare events. arXiv preprint arXiv:1703.03129 (2017).
[17] Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layersof features from tiny images. (2009).
[18] Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and StefanoSoatto. 2019. Meta-learning with differentiable convex optimization.In Proceedings of the IEEE Conference on Computer Vision and PatternRecognition. 10657–10665.
[19] Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and LiweiWang. 2020. Boosting few-shot learning with adaptive margin loss.In Proceedings of the IEEE Conference on Computer Vision and PatternRecognition. 12576–12584.
[20] Aoxue Li, Tiange Luo, Tao Xiang,WeiranHuang, and LiweiWang. 2019.Few-shot learning with global class representations. In Proceedings ofthe IEEE International Conference on Computer Vision. 9715–9724.
[21] Yanbin Liu, Juho Lee, Minseop Park, Saehoon Kim, Eunho Yang,Sung Ju Hwang, and Yi Yang. 2019. Learning to propagate labels:Transductive propagation network for few-shot learning. In 7th Inter-national Conference on Learning Representations, ICLR 2019. Interna-tional Conference on Learning Representations, ICLR.
[22] Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2018.A Simple Neural Attentive Meta-Learner. In Proceedings of the Interna-tional Conference on Learning Representations.
[23] Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, and AdamTrischler. 2018. Rapid adaptation with conditionally shifted neurons.In International Conference on Machine Learning. PMLR, 3664–3673.
[24] Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. 2018.Tadam: Task dependent adaptive metric for improved few-shot learn-ing. In Advances in Neural Information Processing Systems. 721–731.
[25] Zhimao Peng, Zechao Li, Junge Zhang, Yan Li, Guo-Jun Qi, and JinhuiTang. 2019. Few-shot image recognition with knowledge transfer. InProceedings of the IEEE International Conference on Computer Vision.441–449.
[26] Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.Glove: Global vectors for word representation. In Proceedings of the2014 conference on empirical methods in natural language processing(EMNLP). 1532–1543.
[27] Hang Qi, Matthew Brown, and David G Lowe. 2018. Low-shot learn-ing with imprinted weights. In Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition. 5822–5830.
[28] Limeng Qiao, Yemin Shi, Jia Li, Yaowei Wang, Tiejun Huang, andYonghong Tian. 2019. Transductive episodic-wise adaptive metric forfew-shot learning. In Proceedings of the IEEE International Conferenceon Computer Vision. 3603–3612.
[29] Siyuan Qiao, Chenxi Liu, Wei Shen, and Alan L Yuille. 2018. Few-shot image recognition by predicting parameters from activations. InProceedings of the IEEE Conference on Computer Vision and PatternRecognition. 7229–7238.
[31] Tiago Ramalho and Marta Garnelo. 2018. Adaptive Posterior Learning:few-shot learning with a surprise-based memory module. In Interna-tional Conference on Learning Representations.
[32] Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. 2019. Few-shot learning with embedded class models and shot-free meta training.In Proceedings of the IEEE International Conference on Computer Vision.331–339.
[33] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016.You only look once: Unified, real-time object detection. In Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition.779–788.
[34] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster r-cnn: Towards real-time object detectionwith region proposal networks.IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 6(2016), 1137–1149.
[35] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, SanjeevSatheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla,Michael Bernstein, et al. 2015. Imagenet large scale visual recogni-tion challenge. International journal of computer vision 115, 3 (2015),211–252.
[36] Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Darrell, andZeynep Akata. 2019. Generalized zero-and few-shot learning viaaligned variational autoencoders. In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition. 8247–8255.
Few-shot Learning with Global Relatedness Decoupled-Distillation Woodstock ’18, June 03–05, 2018, Woodstock, NY
[37] Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, MattiasMarder, Abhishek Kumar, Rogerio Feris, Raja Giryes, and Alex MBronstein. 2018. Δ-encoder: an effective sample synthesis method forfew-shot object recognition. In Proceedings of the 32nd InternationalConference on Neural Information Processing Systems. 2850–2860.
[38] Karen Simonyan and Andrew Zisserman. 2014. Very deep convo-lutional networks for large-scale image recognition. arXiv preprintarXiv:1409.1556 (2014).
[39] Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical net-works for few-shot learning. Advances in Neural Information ProcessingSystems 30 (2017), 4077–4087.
[40] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. 2019. Meta-transfer learning for few-shot learning. In Proceedings of the IEEEConference on Computer Vision and Pattern Recognition. 403–412.
[41] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, andTimothy M Hospedales. 2018. Learning to compare: Relation net-work for few-shot learning. In Proceedings of the IEEE Conference onComputer Vision and Pattern Recognition. 1199–1208.
[42] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, andPhillip Isola. 2020. Rethinking Few-Shot Image Classification: a GoodEmbedding Is All You Need? arXiv preprint arXiv:2003.11539 (2020).
[43] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, DaanWierstra, et al.2016. Matching networks for one shot learning. Advances in NeuralInformation Processing Systems 29 (2016), 3630–3638.
[44] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. 2020.Generalizing from a few examples: A survey on few-shot learning.
ACM Computing Surveys (CSUR) 53, 3 (2020), 1–34.[45] Yu-XiongWang, Ross Girshick, Martial Hebert, and Bharath Hariharan.
2018. Low-shot learning from imaginary data. In Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition. 7278–7286.
[46] Zeyuan Wang, Yifan Zhao, Jia Li, and Yonghong Tian. 2020. Coopera-tive Bi-path Metric for Few-shot Learning. In Proceedings of the 28thACM International Conference on Multimedia. 1524–1532.
[47] Yu Wu, Yutian Lin, Xuanyi Dong, Yan Yan, Wanli Ouyang, and YiYang. 2018. Exploit the unknown gradually: One-shot video-basedperson re-identification by stepwise learning. In Proceedings of the IEEEConference on Computer Vision and Pattern Recognition. 5177–5186.
[48] Ziyang Wu, Yuwei Li, Lihua Guo, and Kui Jia. 2019. Parn: Position-aware relation networks for few-shot learning. In Proceedings of theIEEE International Conference on Computer Vision. 6659–6667.
[49] Shuo Yang, Lu Liu, and Min Xu. 2021. Free Lunch for Few-shot Learn-ing: Distribution Calibration. arXiv preprint arXiv:2101.06395 (2021).
[50] Hongguang Zhang, Jing Zhang, and Piotr Koniusz. 2019. Few-shotlearning via saliency-guided hallucination of samples. In Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition.2770–2779.
[51] Imtiaz Ziko, Jose Dolz, Eric Granger, and Ismail Ben Ayed. 2020. Lapla-cian regularized few-shot learning. In International Conference on Ma-chine Learning. PMLR, 11660–11670.
![Multi-Task Transfer Methods to Improve One-Shot Learning ...mori/research/papers/yan-bmvc15.pdf · Zhou et al. [35] consider the distances between task models as the relatedness measure,](https://static.documents.pub/doc/80x56/5f0bc1c67e708231d432109f/multi-task-transfer-methods-to-improve-one-shot-learning-moriresearchpapersyan-.jpg)
