Jakob Uszkoreit acted as the originator and catalyst by proposing the replacement of recurrence with self-attention, but he was not the sole inventor of the full Transformer system.

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?, Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?, Attention is All you Need agree on this point.

The Transformer was not composed solely of attention mechanisms, as non-attention components and optimizer schedules were critical to the system's empirical success.

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?, Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?, Attention is All you Need agree on this point.

The public record of the paper contains multiple mutually inconsistent English–French translation BLEU figures.

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?, Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?, Attention is All you Need agree on this point.

Reimplementation of the Transformer demonstrated that the learning-rate schedule was an unusually consequential and sensitive implementation detail.

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?, Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?, Attention is All you Need agree on this point.

The maps diverge on whether the Transformer's decisive breakthrough was trading sample efficiency for TPU cluster parallelizability or establishing feed-forward sublayers as associative factual storage.

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?: The Transformer architecture traded algorithmic sample efficiency for hardware-level training parallelizability on TPU clusters. Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?: Feed-forward sublayers function as associative key-value memories that store the substantive knowledge in Transformer models.

Harita aileleri

Attention is All you Need

1 investigation · 3 maps · 342 claims · Last updated 22 Eki 2026

LinkedIn X Facebook WhatsApp E-posta

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise? (Markdown)Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage? (Markdown)Attention is All you Need (Markdown)

Overview

Attention is All you Need: 1 herkese açık inceleme ve 342 iddia.

Harita genel bakışları

Sırala

View

2
Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
submitted 22 Eki 2026 by @nineDan 203
122 iddia
+2
Attention is All you Need kaynağından çatallandı
OverviewThis map questions the paper's rhetorical focus on attention by proposing that feed-forward networks function as key-value associative memories that store model knowledge. It situates this structural hypothesis within the broader authorship history, revisions, and mechanisms of the Transformer.
1
Attention is All you Need
submitted 22 Eki 2026 by @nineDan 203
101 iddia
+2
OverviewThis map provides a foundational overview of the 2017 paper, tracing its authorial lineage across Google Brain and Research, the release of the open-source software ecosystem, and its core thesis against sequential recurrence. It also tracks the paper's multi-year bibliographic revisions and metric inconsistencies.
3
Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
submitted 22 Eki 2026 by @nineDan 203
119 iddia
+2
Attention is All you Need kaynağından çatallandı
OverviewThis map investigates whether the Transformer's ascent was driven by hardware-level TPU parallelizability and economics rather than algorithmic superiority. It balances technical trade-offs between sample efficiency and compute throughput against the historical record of project leadership and implementation details.

Last generated 22 Eki 2026 17:10 by a member

Daha fazla bilgi

22 Eki 2026 17:10 karşılaştırıldı

Haritalar

Haritayı aç

Haritalar karşılaştırılıyor…

Themes across the family

A broad read of every map in this family: the themes that run through them and how each map treats them. Structured scans give a more targeted side-by-side.

Attribution and Architectural Provenance
3 haritadan 3 tanesi
The family documents the inception of the Transformer as a collaborative synthesis where Jakob Uszkoreit provided the initial catalytic hypothesis of replacing recurrence with self-attention, complemented by specialized architectural contributions across Google Brain and Research.
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
Identifies Uszkoreit as the conceptual initiator and catalyst while emphasizing that design and mechanisms were distributed among coauthors such as Vaswani, Polosukhin, and Shazeer.
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
Acknowledges Uszkoreit's foundational role while highlighting Shazeer's invention of scaled dot-product attention.
M3Attention is All you Need
Traces the close cross-building collaboration between Google Brain and Google Research teams that produced the early model implementations.
Architectural Components Beyond Attention
3 haritadan 3 tanesi
All maps qualify the rhetorical claim that attention is all that is needed, showing that feed-forward sublayers, learning rate schedules, and optimizer engineering were critical to model viability.
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
Stresses that the finished Transformer was not composed solely of attention and relied on load-bearing optimizer schedules.
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
Contends that feed-forward sublayers play a crucial role as associative key-value memory storage rather than being incidental to attention.
M3Attention is All you Need
Points out that independent replication required extracting subtle learning-rate schedules and hyperparameters not described by pure attention.
Hardware Parallelization and Recurrence Bottlenecks
3 haritadan 2 tanesi
The research maps examine whether overcoming the sequential computational constraints of recurrent networks was fundamentally a hardware-throughput trade-off.
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
Probes whether the architecture sacrificed algorithmic sample efficiency specifically to capitalize on parallel throughput across TPU clusters.
M3Attention is All you Need
Focuses on the paper's core empirical proposition that sequential recurrence prevented intra-example training parallelization.
Publication Inconsistencies and Reporting Revisions
3 haritadan 3 tanesi
The family details persistent reporting discrepancies in benchmark translation metrics across successive arXiv versions and conference proceedings.
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
Documents multiple conflicting English–French BLEU scores and missing implementation details in the original paper text.
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
Notes the evolution of the paper across seven arXiv versions and the delayed introduction of headline BLEU scores.
M3Attention is All you Need
Highlights direct internal contradictions between abstract and body BLEU figures within the official NeurIPS proceedings.

Jakob Uszkoreit acted as the originator and catalyst by proposing the replacement of recurrence with self-attention, but he was not the sole inventor of the full Transformer system.
3 haritadan 3 tanesi
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0013 Uszkoreit role unclear
The best-supported interpretation of Uszkoreit's role is originator and initial project catalyst, not sole architect or inventor.
TartışmalıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0013 Uszkoreit role unclear
TartışmalıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0028 Uszkoreit role unclear
TartışmalıYüksek önem düzeyiChatGPT
The Transformer was not composed solely of attention mechanisms, as non-attention components and optimizer schedules were critical to the system's empirical success.
3 haritadan 3 tanesi
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0080 Transformer not all attention
The finished Transformer model did not literally replace every RNN operation with attention and was not composed solely of attention.
TartışmalıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0083 Transformer not all attention
TartışmalıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0033 Transformer not all attention
TartışmalıYüksek önem düzeyiChatGPT
The public record of the paper contains multiple mutually inconsistent English–French translation BLEU figures.
3 haritadan 3 tanesi
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0052 Inconsistent BLEU figures
The public record of the "Attention is All you Need" paper contains at least four mutually inconsistent English–French BLEU figures.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0055 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0090 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
Reimplementation of the Transformer demonstrated that the learning-rate schedule was an unusually consequential and sensitive implementation detail.
3 haritadan 3 tanesi
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0091 Reviewer reimplemented system
An independent NeurIPS reviewer attempted to reimplement the system from "Attention is All you Need" and found the learning-rate schedule to be unusually consequential.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0094 Reviewer reimplemented system
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0097 Reviewer reimplemented system
Kaynak bağlantılıYüksek önem düzeyiChatGPT

Build on the map here to find more

The maps diverge on whether the Transformer's decisive breakthrough was trading sample efficiency for TPU cluster parallelizability or establishing feed-forward sublayers as associative factual storage.
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0022 Hardware Determinism over Algorithmic EleganceYapay zeka ile araştır
The Transformer architecture traded algorithmic sample efficiency for hardware-level training parallelizability on TPU clusters.
ÇıkarımlıYüksek önem düzeyiModel kaynağı kullanılamıyor
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0022 The 'Attention' Misnomer and MLP DominanceYapay zeka ile araştır
Feed-forward sublayers function as associative key-value memories that store the substantive knowledge in Transformer models.
ÇıkarımlıYüksek önem düzeyiModel kaynağı kullanılamıyor

Build on the map here to find more

Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise? uniquely highlights the extensive GPU compute time and memory limitations encountered during independent empirical exploration.
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0027 Transformer paper compute requirements
The empirical exploration experiments for the Popel and Bojar (2018) Transformer training tips paper required roughly 4 years of cumulative GPU compute time.
Kaynak bağlantılıDüşük önem düzeyiClaude
Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage? uniquely connects feed-forward sublayers to classical associative memory models formulated in the 1970s.
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0039 Linear associative memory foundation
ROME's key-value memory formulation relies on classical linear associative memory frameworks originally established by Anderson (1972) and Kohonen (1972).
ÇıkarımlıDüşük önem düzeyiClaude
Attention is All you Need uniquely highlights the contemporaneous June 2017 open-sourcing of Google's Tensor2tensor framework.
M3Attention is All you Need
C0043 Tensor2tensor open-sourced June 2017
Tensor2tensor was open-sourced by Google, announced in June 2017, the same month as the "Attention Is All You Need" paper.
ÇıkarımlıDüşük önem düzeyiKimi

The reported English–French translation performance conflicts across publications, claiming 41.8 BLEU in the current arXiv version but 41.0 BLEU in Table 2 of the NeurIPS proceedings PDF.
M3Attention is All you Need
C0089 BLEU score reported
The current arXiv abstract and Table 2 for "Attention is All you Need" report an English–French BLEU score of 41.8.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0094 BLEU score report
Table 2 of the NeurIPS proceedings PDF for "Attention is All you Need" reports an English–French BLEU score of 41.0.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
The NeurIPS proceedings text conflicts internally by reporting an English–German score of 27.5 BLEU in the abstract while reporting 28.4 BLEU in the body text.
M3Attention is All you Need
C0094 Model size and BLEU score
The NeurIPS proceedings abstract for "Attention is All you Need" describes a 165-million-parameter model scoring 27.5 English–German BLEU.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0073 BLEU discrepancy noted
The proceedings PDF body of "Attention is All you Need" uses 28.4 English–German BLEU, creating an abstract/body discrepancy.
Kaynak bağlantılıYüksek önem düzeyiChatGPT

https://arxiv.org/abs/1706.03762
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0004 BLEU score reported
The current arXiv abstract and Table 2 for "Attention is All you Need" report an English–French BLEU score of 41.8.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0052 Inconsistent BLEU figures
The public record of the "Attention is All you Need" paper contains at least four mutually inconsistent English–French BLEU figures.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0055 ArXiv versions listed
The arXiv record for "Attention is All you Need" includes versions v6 on July 24, 2023, and v7 on August 2, 2023.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0073 BLEU discrepancy noted
The proceedings PDF body of "Attention is All you Need" uses 28.4 English–German BLEU, creating an abstract/body discrepancy.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0075 Model size and BLEU score
The NeurIPS proceedings abstract for "Attention is All you Need" describes a 165-million-parameter model scoring 27.5 English–German BLEU.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0089 Vaswani submitted paper v1
Ashish Vaswani was identified as the submitter for v1 of "Attention is All you Need" on arXiv, while the current record identifies Llion Jones.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0094 BLEU score report
Table 2 of the NeurIPS proceedings PDF for "Attention is All you Need" reports an English–French BLEU score of 41.0.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0105 Attention paper version 7
The latest official version of "Attention is All you Need" on arXiv is v7, not v5.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0004 BLEU score reported
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0055 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0058 ArXiv versions listed
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0076 BLEU discrepancy noted
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0078 Model size and BLEU score
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0092 Vaswani submitted paper v1
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0097 BLEU score report
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0108 Attention paper version 7
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0089 BLEU score reported
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0090 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0091 ArXiv versions listed
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0093 BLEU discrepancy noted
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0094 Model size and BLEU score
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0096 Vaswani submitted paper v1
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0098 BLEU score report
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0099 Attention paper version 7
Kaynak bağlantılıYüksek önem düzeyiChatGPT
https://papers.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Reviews.html
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0052 Inconsistent BLEU figures
The public record of the "Attention is All you Need" paper contains at least four mutually inconsistent English–French BLEU figures.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0073 BLEU discrepancy noted
The proceedings PDF body of "Attention is All you Need" uses 28.4 English–German BLEU, creating an abstract/body discrepancy.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0075 Model size and BLEU score
The NeurIPS proceedings abstract for "Attention is All you Need" describes a 165-million-parameter model scoring 27.5 English–German BLEU.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0084 NeurIPS attention paper
The NeurIPS proceedings abstract for "Attention is All you Need" reports an English–French BLEU score of 41.1.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0091 Reviewer reimplemented system
An independent NeurIPS reviewer attempted to reimplement the system from "Attention is All you Need" and found the learning-rate schedule to be unusually consequential.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0094 BLEU score report
Table 2 of the NeurIPS proceedings PDF for "Attention is All you Need" reports an English–French BLEU score of 41.0.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0111 Paper replication issues
A reviewer of "Attention is All you Need" found that important replication details, such as numerous hyperparameters, had to be extracted from the code rather than the submitted paper.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0055 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0076 BLEU discrepancy noted
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0078 Model size and BLEU score
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0087 NeurIPS attention paper
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0094 Reviewer reimplemented system
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0097 BLEU score report
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0114 Paper replication issues
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0090 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0093 BLEU discrepancy noted
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0094 Model size and BLEU score
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0095 NeurIPS attention paper
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0097 Reviewer reimplemented system
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0098 BLEU score report
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0101 Paper replication issues
Kaynak bağlantılıYüksek önem düzeyiChatGPT
https://arxiv.org/pdf/1706.03762
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0003 Kaiser affiliated with Google
Łukasz Kaiser was listed as being affiliated with “Google Brain” in arXiv:1706.03762, with the email lukaszkaiser@google.com.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0061 Noam Shazeer affiliation
Noam Shazeer was listed as being affiliated with “Google Brain” in arXiv:1706.03762, with the email noam@google.com.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0066 Ashish Vaswani affiliation
Ashish Vaswani was listed as being affiliated with “Google Brain” in arXiv:1706.03762, with the email avaswani@google.com.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0085 Polosukhin affiliation clarified
Illia Polosukhin's affiliation in arXiv:1706.03762 was clarified by an explicit footnote stating “‡ Work performed while at Google Research.”
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0110 Illia Polosukhin departure
Illia Polosukhin's departure from Google occurred before the June 12, 2017, arXiv v1 posting of the "Attention Is All You Need" paper.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0027 Transformer paper compute requirements
The empirical exploration experiments for the Popel and Bojar (2018) Transformer training tips paper required roughly 4 years of cumulative GPU compute time.
Kaynak bağlantılıDüşük önem düzeyiClaude
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0003 Kaiser affiliated with Google
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0064 Noam Shazeer affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0069 Ashish Vaswani affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0088 Polosukhin affiliation clarified
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0113 Illia Polosukhin departure
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0077 Kaiser affiliated with Google
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0081 Noam Shazeer affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0082 Ashish Vaswani affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0085 Polosukhin affiliation clarified
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0087 Illia Polosukhin departure
Kaynak bağlantılıOrta önem düzeyiGrok
https://arxiv.org/html/1706.03762
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0003 Kaiser affiliated with Google
Łukasz Kaiser was listed as being affiliated with “Google Brain” in arXiv:1706.03762, with the email lukaszkaiser@google.com.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0061 Noam Shazeer affiliation
Noam Shazeer was listed as being affiliated with “Google Brain” in arXiv:1706.03762, with the email noam@google.com.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0066 Ashish Vaswani affiliation
Ashish Vaswani was listed as being affiliated with “Google Brain” in arXiv:1706.03762, with the email avaswani@google.com.
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0085 Polosukhin affiliation clarified
Illia Polosukhin's affiliation in arXiv:1706.03762 was clarified by an explicit footnote stating “‡ Work performed while at Google Research.”
Kaynak bağlantılıOrta önem düzeyiGrok
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0110 Illia Polosukhin departure
Illia Polosukhin's departure from Google occurred before the June 12, 2017, arXiv v1 posting of the "Attention Is All You Need" paper.
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0003 Kaiser affiliated with Google
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0064 Noam Shazeer affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0069 Ashish Vaswani affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0088 Polosukhin affiliation clarified
Kaynak bağlantılıOrta önem düzeyiGrok
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0113 Illia Polosukhin departure
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0077 Kaiser affiliated with Google
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0081 Noam Shazeer affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0082 Ashish Vaswani affiliation
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0085 Polosukhin affiliation clarified
Kaynak bağlantılıOrta önem düzeyiGrok
M3Attention is All you Need
C0087 Illia Polosukhin departure
Kaynak bağlantılıOrta önem düzeyiGrok
https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0052 Inconsistent BLEU figures
The public record of the "Attention is All you Need" paper contains at least four mutually inconsistent English–French BLEU figures.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0111 Paper replication issues
A reviewer of "Attention is All you Need" found that important replication details, such as numerous hyperparameters, had to be extracted from the code rather than the submitted paper.
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0055 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0114 Paper replication issues
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0090 Inconsistent BLEU figures
Kaynak bağlantılıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0101 Paper replication issues
Kaynak bağlantılıYüksek önem düzeyiChatGPT
https://arxiv.org/abs/1606.01933
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0043 Proposal attributed to Uszkoreit
The explicit sentence attributing the proposal to Uszkoreit does not appear in arXiv versions v1–v4 (June 12, 19, 20, 30, 2017) but appears in v5 (December 6, 2017) with detailed contribution statements.
TartışmalıYüksek önem düzeyiChatGPT
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0046 Proposal attributed to Uszkoreit
TartışmalıYüksek önem düzeyiChatGPT
M3Attention is All you Need
C0030 Proposal attributed to Uszkoreit
TartışmalıYüksek önem düzeyiChatGPT

How should credit attribution for proposing the replacement of RNNs with self-attention be formally interpreted given evolving paper revisions?
M1Did TPU Hardware Economics, Not Algorithmic Superiority, Drive the Transformer's Rise?
C0005 Uszkoreit proposed self-attentionYapay zeka ile araştır
Jakob Uszkoreit is credited with proposing replacing RNNs with self-attention in the "Attention Is All You Need" paper.
ÇıkarımlıDüşük önem düzeyiClaude
M2Did the Attention Mechanism Disguise the True Role of Feed-Forward Storage?
C0005 Uszkoreit proposed self-attentionYapay zeka ile araştır
ÇıkarımlıDüşük önem düzeyiClaude
M3Attention is All you Need
C0005 Uszkoreit proposed self-attentionYapay zeka ile araştır
ÇıkarımlıDüşük önem düzeyiClaude

Discussion

Yorumda bir haritayı etiketlemek için @m yazın veya @kullanıcıadı ile birinden bahsedin.

Henüz yorum yok.

Attention is All you Need

Overview

Harita genel bakışları

Haritalar

Themes across the family

Görüş birliği4

Görüş ayrılığı1

Farklılaşan vurgu noktaları3

Kanıt çelişkisi2

Claims support6

Ortak açık sorular1

Ortak alıntılar8

Discussion