I find it surprising that you actually have performance problems. We used to run lemmy.ml on a 2 cpu vps for 10€ a month, without any problems. Recently we upgraded to 4 cpu due to some kind of ddos, but now resource usage is quite low (load average under 1). Maybe you can share some screenshots of htop or similar (via pm if you want).
Its true that lemmy-ui (nodejs) is not very efficient, but in practice it seems barely noticable. And nginx is very fast, written in C.
The easiest way to get started would be if you replace the templates and assets, and leave api calls and routing to lemmyBB. That way you could create something usable within a few days. Templates use handlebars syntax.