Recent searches

No recent searches

Search options

Only available when logged in.
sigmoid.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A social space for people researching, working with, or just interested in AI!

Administered by:

Server stats:

586
active users

sigmoid.social: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.1

David Pfau

OK, now that I've moved over to sigmoid.social, time for my first...uh...tootprint? Mastoscript? Manudon? Screw it - we wrote a paper and I want to share it with you.

Very pleased to be able to share this one: is attention all you need to solve the Schrödinger equation? arxiv.org/abs/2211.13672

arXiv.orgA Self-Attention Ansatz for Ab-initio Quantum ChemistryWe present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural networks like the FermiNet and PauliNet have been used to significantly improve the accuracy of these first-principle calculations, but they lack an attention-like mechanism for gating interactions between electrons. Here we show that the Psiformer can be used as a drop-in replacement for these other neural networks, often dramatically improving the accuracy of the calculations. On larger molecules especially, the ground state energy can be improved by dozens of kcal/mol, a qualitative leap over previous methods. This demonstrates that self-attention networks can learn complex quantum mechanical correlations between electrons, and are a promising route to reaching unprecedented accuracy in chemical calculations on larger systems.
#paperthread
··Web
8boosts·25favorites
David Pfau

For the last several years, numerous groups have shown that neural networks can make calculations in quantum chemistry much more accurate - FermiNet, PauliNet, etc. We wrote a review article about it here: arxiv.org/abs/2208.12590

*
David Pfau

Most work since then has only made small tweaks to these basic neural network ansatzes. Instead, we tried to reinvent neural network ansatzes from the ground up. The result is a model we call the Psiformer: basically, a Transformer encoder designed for quantum chemistry.

David Pfau

One problem with the FermiNet was that it seemed to decrease in accuracy as system size increased. We find that the Psiformer is uniformly more accurate than the FermiNet on all systems we investigated.

David Pfau

Most impressively, the bigger the system size, the bigger the improvement with the Psiformer relative to the FermiNet. On the largest system we looked at, the benzene dimer (84 electrons!) the Psiformer with VMC is more accurate than the FermiNet with *diffusion* Monte Carlo!

David Pfau

I really never thought I’d be an “attention is all you need” guy, but at least in this case, it seems like neural network ansatzes using self-attention are a clear improvement over prior models, and present a path to unprecedented accuracy in quantum chemical calculations.

David Pfau

This was work led by Ingrid von Glehn and in collaboration with
James Spencer. For those at , I’ll be speaking about this and other topics on deep learning and quantum chemistry at the workshop on Saturday! ml4physicalsciences.github.io/

TrendingLive feeds
About