Newest papers

Efficient Edge-aware Attention Network for Graph Generation

Vincent Jung, Alba Carballo-Castro, Yiming Qin, Lonneke van der Plas, Pascal Frossard B · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28618 · cited by 0 works (OpenAlex) · open-access PDF

German

Graph-generative Modelle verwenden normalerweise grafspezifische Transformer-Architekturen, um Node- und Edge-Funktionen gemeinsam zu aktualisieren. Diese Architekturen profitieren jedoch nicht direkt von den jüngsten Hardware-bewussten Aufmerksamkeits-Implementierungen und beschränken ihre Skalierbarkeit. Die Anpassung ist jedoch nicht trivial: Kanten- und Grafenebene-Funktionen müssen in Paar-Aufmerksamkeits-Scores injiziert werden, Grafikmasken müssen respektiert werden, und dynamische Kanten-Updates bleiben eine wichtige Quelle der quadratischen Speicherverwendung. Wir schlagen eine Kanten-bewusste Aufmerksamkeits-Lage für die Grafikgenerierung vor, die strukturelle Informationen durch Kopf-Score-Modulation

Machine translation (Meta M2M100); the English original follows.

Abstract

Graph generative models commonly use graph-specific Transformer architectures to jointly update node and edge features. However, these architectures do not directly benefit from recent hardware-aware attention implementations, limiting their scalability. The adaptation is, however, non-trivial: edge and graph-level features must be injected into pairwise attention scores, graph masks must be respected, and dynamic edge updates remain a major source of quadratic memory usage. We propose an edge-aware attention layer for graph generation that incorporates structural information through head-wise score modulation, while updating edge representations with a lightweight residual mechanism that retains attention-score dependent edge updates. This layer can be effectively implemented using PyTorch's FlexAttention. We achieve similar generation quality to our baseline while reducing peak GPU memory by around 65% and substantially improving training and sampling throughput. These results suggest that the proposed architecture is a practical step toward scalable graph foundation models.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Scalability, Graph, Theoretical computer science, Limiting, Residual

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange