The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers

S. H. Son; Jegwang Ryu; Namhoon Lee; Jaeho Lee

doi:10.1007/978-3-031-72855-6_22

ScienceGate Book Chapters

BOOK-CHAPTER

The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers

S. H. Son Jegwang Ryu Namhoon Lee Jaeho Lee

Year: 2024 Lecture notes in computer science Pages: 379-396 Publisher: Springer Science+Business Media

DOI: 10.1007/978-3-031-72855-6_22

Get Full-Text PDF Get Analytical Report

Keywords:

Computer science Distillation Transformer Artificial intelligence Masking (illustration) Pattern recognition (psychology) Computer vision Electrical engineering Chromatography Voltage Chemistry Engineering

Metrics

Cited By

3.80

FWCI (Field Weighted Citation Impact)

Refs

0.91

Citation Normalized Percentile

Is in top 1%

Is in top 10%

Citation History

Topics

Industrial Vision Systems and Defect Detection

Physical Sciences → Engineering → Industrial and Manufacturing Engineering

Visual Attention and Saliency Detection

Physical Sciences → Computer Science → Computer Vision and Pattern Recognition

Currency Recognition and Detection

Physical Sciences → Computer Science → Computer Vision and Pattern Recognition

The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers

Metrics

Citation History

Topics

Related Documents

DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers

Local masking meets progressive freezing: crafting efficient vision transformers for self-supervised learning

When Self-supervised Transformers Meet Knowledge Distillation: Efficient Chest X-Ray Classification

Cumulative Spatial Knowledge Distillation for Vision Transformers

Supervised Masked Knowledge Distillation for Few-Shot Transformers