ScienceGate Book Chapters
JOURNAL ARTICLE

Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation

Ta-Chung ChiTing-Han FanAlexander I. Rudnicky

Year: 2024 Pages: 132-148
DOI: 10.18653/v1/2024.findings-naacl.10
   Get Full-Text PDF    Get Analytical Report
Keywords:
Extrapolation Transformer Computer science Artificial intelligence Electrical engineering Mathematics Voltage Engineering Mathematical analysis

Metrics

0
Cited By
0.00
FWCI (Field Weighted Citation Impact)
0
Refs
0.10
Citation Normalized Percentile
Is in top 1%
Is in top 10%

Topics

Neural Networks and Applications
Physical Sciences →  Computer Science →  Artificial Intelligence
Industrial Vision Systems and Defect Detection
Physical Sciences →  Engineering →  Industrial and Manufacturing Engineering

Related Documents

JOURNAL ARTICLE

Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings

Ta-Chung ChiTing-Han FanLiwei ChenAlexander I. RudnickyPeter J. Ramadge

Year: 2023 Pages: 1183-1193
BOOK-CHAPTER

Graph Attention Networks with Positional Embeddings

Liheng MaReihaneh RabbanyAdriana Romero-Soriano

Lecture notes in computer science Year: 2021 Pages: 514-527
JOURNAL ARTICLE

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation

Yihang GaoMinbin HuangXin JiangJingyao LiYu LiZhenguo LiMichael K. NgXiaozhe RenHan ShiJing XiongChuanyang Zheng

Year: 2024 Pages: 26659-26700
BOOK-CHAPTER

IPE Transformer for Depth Completion with Input-Aware Positional Embeddings

Bocen LiGuozhen LiHaiting WangLijun WangZhenfei GongXiaohua ZhangHuchuan Lu

Lecture notes in computer science Year: 2021 Pages: 263-275
JOURNAL ARTICLE

Attention-based BiLSTM with positional embeddings for fake review detection

Jindong ChenTian ZhangZhihua YanZhichao ZhengWen ZhangJian Zhang

Journal:   Journal Of Big Data Year: 2025 Vol: 12 (1)
© 2026 ScienceGate Book Chapters — All rights reserved.