Maybe? Its after the tower of Hera and fighting the wizard but I think you can be pulled into the dark world then without the orb.

That’s probably the last NPC you can get to before you need dark world objects to explore.