I’m assuming that the lady in the picture that is facing him, doing the “shh” motion, is his lover, and they aren’t allowed to be public with their love. He may have stopped by to see if she could sneak away, and did not know they would all be out on the porch, spending time together. That’s my interpretation, at least.