The ironic thing about these memes is that you actually can’t have nontrivial forking logic in neural network modules because it can’t be traced by jit compilers.