A Survey on Contrastive Self-Supervised Learning(对比式自监督学习研究)-----pretext tasks、Downstream task解释

• 摘自MDPI：A Survey on Contrastive Self-Supervised Learning

摘要部分：

Self-supervised learning(自监督学习) has gained popularity because of its ability to avoid the cost of annotating(给…做注释) large-scale datasets. It is capable of adopting self-defined pseudolabels(伪标签) as supervision and use the learned representations(模型表示) for several downstream tasks. Specifically, contrastive learning has recently become a dominant component (重要的部分)in self-supervised learning for computer vision, natural language processing (NLP), and other domains(领域). It aims at embedding(把…嵌入) augmented(增强的) versions of the same sample close to each other while trying to push away embeddings from different samples. This paper provides an extensive review of self-supervised methods that follow the contrastive approach. The work explains commonly used pretext tasks(借口任务、代理任务) in a contrastive learning setup, followed by different architectures that have been proposed(提议) so far. Next, we present(可作提出) a performance comparison of different methods for multiple downstream tasks such as image classification, object detection, and action recognition. Finally, we conclude with the limitations of the current methods and the need for further techniques and future directions to make meaningful progress.

Keywords: contrastive learning; self-supervised learning; discriminative(有区别的) learning; image/video classification; object detection; unsupervised learning; transfer learning

翻译：自监督学习因为它可以避免给大规模数据做标注的成本而获得普及。它有能力采用自定义的伪标签做监督并使用学习好的模型表示几个下游任务。特别的，最近对比学习在自监督学习像计算机视觉、自然语言处理和其它领域已经是重要的组成部分。它旨在将相同样本的增强版本彼此靠近的嵌入，然后进一步推进不同样本的嵌入。这篇文章对遵循对比方法的自监督方法进行了广泛的回顾。这项工作解释了在对比学习程序中通常使用的代理任务，然后解释了到目前为止已经提出的不同的构架。接下来我们提出了几个下游任务的不同方法的性能比较，比如图像识别、物体检测、行为识别。最后，我们总结的当前方法的局限性和对未来技术和目标的需求，以取得有意义的进步。

关键词：对比学习、自监督学习、判别式学习、图像/视频分类、物体检测、无监督学习、迁移学习

名词理解：

pretext tasks：常被翻译为代理任务、借口任务。可以理解为是一种为达到特定训练任务而设计的间接任务。比如在训练神经网络时，我们需要自己设置一些参数的值，那么传统上我们会毫无目标的的随机进行设置，这样我们往往需要对参数进行大量的调整。但是我们现在拿出一部分数据先进行训练，得到一组参数的值，然后用这组参数的值作为初始值，那么在接下来的训练中往往大大减少了调整参数的工作量。这种有助于模型更好的执行目标任务的任务就称为pretext tasks。

Downstream task：常被翻译成下游任务，其就是利用预训练的模型在当前数据集的效果