Personal Website of Yakov Shafranovich

Weird Email Problems and rDNS

For the past few days I have been troubleshooting a very strange problem at one of my clients. They have a small Windos network of about 7 computers connected to the Internet through a single DSL line. Recently, they begun to experience slowdowns when sending email through Outlook - when email is sent, it takes a minute or so until it actually gets to the server. Needless to say a very annoying thing.

While at the office, I started troubleshooting with the first computer that had the issue. I tried telneting to port 25 to several mail servers. To my suprise, only two mail servers - the ones they used had the problem, while others such as my own were fine. Their two mail servers - smtp.cavtel.net and smtp.concentric.net would just sit there with a blinking cursor for 30 to 45 seconds until the 220 SMTP banner would come up and then they worked just fine.

I basically worked my way back from that computer to their Internet connection:
1. Removed the antivirus on that one computer, no change.
2. Change firewall settings, no change.
3. Tried other computers in the office, same problem.
4. Brought my own computer, same problem.
5. Upgraded firmware on their router, no change.
6. Disconnected the router and connected directly to the DSL modem with my own computer, no change.

At this point I was pretty sure that the issue wasn’t on their end. However, what made the problem more weird is that only some mail servers were affected but those operated by different companies, AND it wasn’t a flat out failure but rather a delay of 30 seconds.

The next step was to call their ISP - Cavalier Telecom. After getting assurances that there was no blockages throttling on their site, the technician I spoke to went to get helps from higher ups. For the next 45 minutes he was putting me own, asking basic questions like “Can you ping our mail server?” and going back to consulting with other technicians. To their credit, they actually knew what they were talking about - something that cannot be said about many other tech support calls nowadays.

While I was on hold with the ISP, I was googling for different problems. By accident I ran across a mention how AOL rejects any SMTP sessions originating from an IP without an reverse DNS entry (PTR). On a hunch, I went to check if my client’s static IP had an rDNS entry - and guess what - they DID NOT! So the next time the technician got on the phone, I asked him to add one in just in case. Half a hour later, mail become blazing fast.

To summarize, what I believe happened is that the two mail servers in questions were running similar software (Postfix I think) and were trying to resolve the rDNS of the connecting IP when starting the SMTP session. For some reason, the rDNS timeout was set pretty high, and the session basically sat there doing nothing until the lookup failed completely. A very unusual and interesting problem.

Posted in Spam and Email | No Comments »

RSS Auto-Discovery via Email

Andy’s blog entry and Meng’s whitepaper got me thinking about RSS discovery via email. What if some of the emails sent today such as lists, sites like Politech, etc. included a header pointing to an RSS version of the same content? What if email clients like Thunderbird which already have RSS support, would automatically parse those heades and let the user subscribe to the emails? Then, the next logical step would be something along the lines of Meng’s system where an empty email pointing to an RSS feed can be sent to the email receiver, automatically parsed by his reader and retreived.

After the mention of the “Link” header in Meng’s paper and some digging around, I ran across Mozilla’s Pre-fetching FAQ which then pointed me to RFC 2068, section 19.6.2.4. In there, a “link” HTTP header is defined which is semantically the same as the “link” HTML tag currently used for autodiscovery. So what if transplant this header into email as follows:

Link: <http://www.shaftek.org/blog/atom.xml>;
 rel=alternate;
 title="RSS 1.0";
 type="application/rss+xml"

It should be trivial to write a Thunderbird extension to parse this and present the user with an option to auto-subscribe. Now we just need someone who wants to write it…

Posted in Programming | No Comments »

RSS vs. Mailing Lists

Recently I have been observing myself switching slowly from mailing lists to RSS feeds for some of the many lists I am on. So far, I have established a principle that anything that I need to get via email can come via RSS instead if it is something that I do not need to reply to. The most recent “victim” of this was the CERT security stuff which I just switched to RSS.

It is interesting how segregating non-repliable email traffic into RSS from the regular email traffic is making my inbox less and less full every day. And of course, I like it!

Posted in Spam and Email, Technology | No Comments »

New Survey of MTAs Needed?

Following up on my earlier post from two weeks ago about MTA market share, it seems that no one really has any accurate numbers for that. Did time come for a new survey, perhaps using a distributed network of scanners similar to Grub?

Posted in Technology | 1 Comment »

Finding MTA Market Share

One interesting piece of information that I am currently trying to hunt down is the current market share for Mail Transfer Agents (MTAs) on the Internet. This includes only programs that receive and send email between domains through the Internet (as opposed to private networks), and should include open source products as well which would normally be excluded from a commercial survey (since they are free). For an example, the Radicati group publishes a market share analysis for email messaging products BUT excludes almost all open source products.

Anyway, so far I had managed to find the following surveys, mainly done via analyzing SMTP banner headers directly:

1. DJB’s SMTP surveys (1997 - 2001).
2. Falko Timme’s two surveys (March and April of 2004).
3. Uwe Obse’s surveys (1999 - 2000).
4. Small SMTP survey (2002 - 2005).
5. Credentia.cc Surveys (2002 - 2003).
6. TTY1.NET (2003 - 2004).

Unfortunatly, the small number of servers surveyed, the research methodology and age of data means that these numbers are basically useless. For example, #1, #3, and #5 are too old. For 2004 there is only #2, #4 and #6. Of these, both TTY1.net and Linuks surveys have only 50,000 hosts which were picked under uncertain criteria. The Falko Timme survey includes the largest number (over 400,000) but the results are very divergent from the others. Considering that back in 2000 DJB estimated the number of mail servers to be 4,000,000 this number is the closest real data. However, Falko does not include the methodology that was used to pick out the hosts.

Perhaps, it is time to have a new survey based on sound statistical design with inclusion of large numbers of hosts. If you have any additional links or data, please email them to me to blog /at/ shaftek [dot] org.

UPDATE: Markus Stumpf pointed to another 2003 survey of 53,000 hosts done by Charles Cazabon.

Posted in Spam and Email | Comments Off