Belated advertising for our NeurIPS 2022 paper. ReLU-based policies partition the input space into piecewise linear regions. How many regions are there? (Ans: related to # neurons; only weakly related to depth). Does the region density increase in areas that the policy learns to visit? (yes, moderately). See here for other lessons learned: https://www.cs.ubc.ca/~van/papers/2022-NeurIPS-understanding/index.html
#RL
