DAPE V2: Process Attention Score as Feature Map for Length Extrapolation

Chuanyang Zheng; Yihang Gao; Han Shi; Jing Xiong; Jiankai Sun; Jingyao Li; Minbin Huang; Xiaozhe Ren; Michael K. Ng; Xin Jiang; Zhenguo Li; Yu Li

doi:10.18653/v1/2025.acl-long.522

JOURNAL ARTICLE

DAPE V2: Process Attention Score as Feature Map for Length Extrapolation

Chuanyang Zheng Yihang Gao Han Shi Jing Xiong Jiankai Sun Jingyao Li Minbin Huang Xiaozhe Ren Michael K. Ng Xin Jiang Zhenguo Li Yu Li

Year: 2025 Pages: 10628-10666

DOI: 10.18653/v1/2025.acl-long.522

Get Full-Text PDF Get Analytical Report

Keywords:

Extrapolation Feature (linguistics) Computer science Process (computing) Artificial intelligence Pattern recognition (psychology) Statistics Mathematics Programming language

Metrics

Cited By

0.00

FWCI (Field Weighted Citation Impact)

Refs

0.36

Citation Normalized Percentile

Is in top 1%

Is in top 10%

Topics

Manufacturing Process and Optimization

Physical Sciences → Engineering → Industrial and Manufacturing Engineering

Fault Detection and Control Systems

Physical Sciences → Engineering → Control and Systems Engineering

Advanced Statistical Process Monitoring

Social Sciences → Decision Sciences → Statistics, Probability and Uncertainty

DAPE V2: Process Attention Score as Feature Map for Length Extrapolation

Metrics

Topics

Related Documents

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation

Semantic Segmentation with Attention-Modulated Feature Fusion in HRNET V2

Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation

Extending Context Window of Attention Based Knowledge Tracing Models via Length Extrapolation

A Training-Free Length Extrapolation Approach for LLMs: Greedy Attention Logit Interpolation