Skip to content

Train transformers on synthetic graph search problems to measure their scaling behavior and to do mechanistic analysis.

Notifications You must be signed in to change notification settings

asaparov/learning_to_search

Folders and files

NameName
Last commit message
Last commit date

Latest commit

67e53d9 · Jan 29, 2025
Oct 22, 2024
Nov 27, 2024
Sep 13, 2024
Oct 22, 2024
Oct 22, 2024
Jan 29, 2025
Nov 1, 2023
Nov 30, 2024
Dec 28, 2023
Nov 28, 2024
Nov 7, 2024
Nov 19, 2024
Nov 14, 2024
Nov 1, 2023
Nov 30, 2024
Jun 2, 2024
Oct 20, 2024
Dec 15, 2023
Oct 22, 2024
Oct 22, 2024
Oct 22, 2024
Nov 4, 2024
Nov 27, 2024
Nov 30, 2024
Nov 27, 2024
Jan 22, 2024
Mar 26, 2024

Repository files navigation

This repo contains code to perform the experiments and analysis in our paper: Transformers Struggle to Learn to Search

If you use this code in your work, please cite:

@inproceedings{
  TransformersStruggleToSearch,
  title={Transformers Struggle to Learn to Search},
  author={Abulhair Saparov and Srushti Pawar and Shreyas Pimpalgaonkar and Nitish Joshi and Richard Yuanzhe Pang and Vishakh Padmakumar and Seyed Mehran Kazemi and Najoung Kim and He He},
  booktitle={The Thirteenth International Conference on Learning Representations},
  year={2025},
  url={https://openreview.net/forum?id=qFVVBzXxR2V}
}

About

Train transformers on synthetic graph search problems to measure their scaling behavior and to do mechanistic analysis.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published