1. Blog gets fresh content.
Search engines love that. If you ever heard anything useful about authority on the web (and the web is all about the authority, content only comes second), you already know blogs with frequent updates rank higher than blogs with similar authority but without fresh content. The owner of the blog should be always happy when somebody offers free original content although I wouldn't recommend to accept every offer.
Sometimes a blogger just can't refresh his/her blog for some reason (life happens, you know) and few guest posts can help to keep the positions and steady traffic. If content is good (and possibly related with other content on the blog), owner should use it to interlink guest posts with already existing posts.
2. A guest post is a promotion for guest blogger.
It brings him/her fresh eyes from already established blog in (hopefully) related niche, what of course means new visitors for his/her blog. Even if the number of new visitors is not significant, a guest blogger gets backlinks and they bring authority. Authority on the web always translates in higher ranking positions.
3. Guest blogging is essentially an interaction between real people.
This can (not always, but sometimes) result in more cooperation on- and off-line. We all know interaction between real human beings is something search engines (check first point about fresh content and check second point about authority) should eventually reward.