Private mail is a place where you can see invitations from guild leaders and your sent requests. To view it, press the envelope icon in the Guild section.
All the invitations sent to you by guild leaders are shown on the Incoming tab. You can review the information about these guilds and accept or reject the invitations. You can't have more than 10 unanswered invitations. Once you reach this cap, other guilds won't be able to invite you.
All the requests you've sent to guilds will be on the Outgoing tab. You can cancel any request if you wish.