“Non-junk” is in the eye of the beholder. YOU may know that they are “non-junk” but a rules-based email filtering program may not.
The ordinary Internet user has no idea how much junk mail has to be screened out by your email service provider in order for you to see more good email than junk. Trust me, if your email address has existed more than a few weeks, you would not want to see the sewer that is an unfiltered mail feed.
The service providers each maintain their own filtering rules, and there are slight differences, due in large part to the fact that they see different profiles of bad mail.
I act as my own email service provider (i.e. I run a mailserver on the Linux box in my home office), and I have outsourced my mail filtering to a Danish friend in Switzerland, who does this as his business, catering to customers in Switzerland and Germany. (And I get it for free, because I helped him with beta testing when he started up.) The spam that I get is different from what his German customers get, so my feedback is helpful to him.
One of the layers of his filtering is this: For every email received, he looks up the combination of “From:” address, “To:” address and the IP address from which the connection comes in. If this combination is new, he rejects the mail with a return code that says “We can’t talk to you right now, come back later”, and then marks that he has seen this combination before. So when the sending “Post Office” tries again (usually 15–30 minutes later), it goes through. We had a correspondent in Brazil, whose email was never getting through. We had him send it to a different mailbox and then sent the evidence to my filtering provider, who checked their logs. It turned out, that the sending post office was a cluster of over 200 machines, and it might take days before the retries landed on a machine that had been used before. So he had to program an exception for this provider’s IP address range.
Why does he do this particular screening layer? Because it gets rid of over 90% of the mail, which comes from hijacked PC’s belonging to home internet users on cable modem connections which get a spam message and a list of 10,000 email addresses to deliver it to. They will not retry, because that is too much work, not worth it – just try the next address in your list.
I highly recommend SpamChek for this kind of service.