Way too much choice. Do you think uninformed users make the right choices? Do you think all people you interact with are capable of gathering information or even willing to think about what app to install, what server to choose etc.?

If you try Signal.org or WhatsApp.com or Telegram.org, you see that the streamlined walled garden is way easier for people to sign up to. Just count the clicks from Matrix.org until you can chat with someone compared to one of the aforementioned apps.