Learning better with Dale’s Law: A Spectral Perspective - #NeurIPS2023 contribution by Li et al. (2023). It discusses how to train brainlike #RNNs with separate inhibitory and excitatory units with similar performance as standard RNNs:
#rnns
0 posts0 participants0 posts today
1997 with the advent of Long Short-Term Memory recurrent #neuralnetworks marks the subsequent step in our brief history of )large) #languagemodels from last week's #ise2023 lecture. Introduced by Sepp Hochreiter and Jürgen Schmidhuber #LSTM #RNNs enabled efficient processing of sequences of data.
Slides: https://drive.google.com/file/d/1atNvMYNkeKDwXP3olHXzloa09S5pzjXb/view?usp=drive_link
#nlp #llm #llms #ai #artificialintelligence #lecture @fizise
Simplifying and Understanding State Space Models with Diagonal Linear RNNs
'Minimal Width for Universal Property of Deep RNN', by Chang hoon Song, Geonho Hwang, Jun ho Lee, Myungjoo Kang.
Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learning
Matthew Kyle Schlegel, Volodymyr Tkachuk, Adam M White, Martha White
Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learning
OpenReviewInvestigating Action Encodings in Recurrent Neural Networks in...Building and maintaining state to learn policies and value functions is critical for deploying reinforcement learning (RL) agents in the real world. Recurrent neural networks (RNNs) have become a...