Trust metrics in email delivery processes

[rob_au]

Following on from my comments about thr employ of reputation or trust metrics in the decision process for delivering mail in the thread "Boycott this kind of system !" (http://www.emaildiscussions.com/...threadid=27026), I was interested to come across the following press release from Mail Cruncher - http://www.emediawire.com/releases/2004/9/emw161107.htm

Mail Cruncher, a new anti-spam subscription service, relies on a "business trust rating" database called the Outbound Index. Email is sorted based on facts such as domain age, relationships between server and domain, and sender stability.

Could this mechanism (the employ of reputation and trust metrics) be the new way by which spam is filtered?

[fmnewbee]

Here is how they says it themselves at

http://www.mailcruncher.com/

"Save Time

Mail Cruncher reveals sender addresses, not just the names spammers try to fool you with.

Spammers play on the fact that everyone knows someone named "Cynthia" - and will open the mail to see how she is doing.

But Mail Cruncher reveals the return address user is x9zrhayqewtt2 and the sending domain is @dailyexamplebargains4you.biz, lined up with 5 others just like it.

Now you don't have to open the email to know it's not your old colleague Cynthia!

Stay Focused

It's your choice - after you have used Mail Cruncher for a few weeks - if you prefer to automatically reject the suspect messages or spend a minute a day scanning the Mail Cruncher list.

Either way, your inbox will remain a cleaner place, letting you focus on the business communications of your choice that will bring your company increased revenue. "

here is another good citation from your link

"The Mail Cruncher email sorting service uses Outbound Index ratings exclusively to sort email.

Messages with a high rating go immediately to the subscriber's inbox.

Messages with a low rating are held back. Once a day, Mail Cruncher subscribers are sent an email with a sorted, color-coded list of suspicious emails that can be scanned in seconds.

"My inbox stays cleaner, and I don't lose mail I want to get," says Fran Microulis, a Web site administrator for American Biophysics Corporation.

"Once a day I get a single email with the Mail Cruncher list.

Even if there are one hundred messages on the list, I can look it over in less than a minute, deleting everything I don't want with one click.

I used to spend at least a half hour a day trying to figure out what was spam in my inbox and getting rid of it.""

[fmnewbee]

Would it be ok to use it outside of their patent for personal usage.

Is it possible to set up a php or asp or pl or any other such language script to do the same and not using it commercially this could help us with spam much better than the tech used now?

Or am I too noob to not get it.

Why aren't You fellow members of EMD not head over heals in love with it?

Trew is head over heals in love with it as a concept.

[OutboundIndex]

As the creator of the Outbound Index and Mail Cruncher, I would encourage a discussion of the ethical and intellectual property issues with fellow members of this forum.

I created Mail Cruncher as a demonstration of the usefulness of the Outbound Index, and because we needed an effective way for our own users to deal with spam without losing wanted mail.

Mail Cruncher is just a user interface for compressing the time spent per day separating wanted / unwanted into one time block, as short as possible, with minimal clicking. The design goal assumes that

- no ISP or anti-spam method can know what the recipient wants, thus give the recipient the opportunity to recover items s/he wants, once a day so that not too much time passes from an important message arriving.

Mail Cruncher is independent of what anti-spam method is used to tag, flag or sort the mail to a quarantine or bulk folder. It's just intended as a useful user interface to the bulk folder. As such, the useful elements we have contributed to quarantine management are likely to be incorporated into existing and future quarantine systems.

I do envision Mail Cruncher in PHP and/or hopefully the most efficient platform. I prototyped in Cold Fusion because that is my native language, but I always envisioned migration to other languages by other programmers.

Possibly the discussion here is really about the Outbound Index business trust ratings rather than the Mail Cruncher as a user interface. I'm the creator of the Outbound Index as well, or the universe working through me, is the creator, along with at least thousands of programmers who came before me, supplying the tools that I have used to create this.

I believe that one of our (people here on earth's) greatest challenges is to figure out how to learn to work together in the creation of and bringing to market of ideas that will help the world move forward positively.

I don't agree with people who are horrified by profit or business - someone like me, a single mother with no inheiritance, cannot produce useful practical products and services while focusing on basic survival like how do I feed and clothe my son or get healthcare for myself.

While I myself went through the process to get a patent for HARDWARE in 1990-5, I do not believe that ideas of all nature, especially those built on the work of thousands, should always depend on litigation for business potential or defense. Litigation also advantages the bigco with endless $ for legal team and prevents the entrepreneur with useful ideas for the world from effectively competing - or even being able to bring their product/service to market - even in the software / internet world.

What does it mean for the good-of-the-world, when investors are reluctant to fund the commercialization of a practical and useful tool, because others may "steal" it or litigate it out of existence? I think it means that the ideas that would help the world, surface, and then are frozen and prevented from doing their good work.

This is a puzzle we as earth inhabitants may need to work through. I don't say that I have the answer, but I am open to the answer.

- April Lorenzen

[robert@fm]

Welcome to the forums, April!

If you PM Edwin (the Admin of these forums), you can get your "Representative of" field set.

[fmnewbee]

Me not being a native speaker might have come down badly so apology if any misunderstanding. I surely support that you have your rightful share.

what I liked about your system is that it make it much easier to wade through all the spam coming in.

the best would be to find out a way to not send out any. To stop it at the smtp servers.

but any software that is effective to sort out the most likely to be personal emails is most welcomed.

Whish more people here in EMD would contribute in this thread. Me being a noob is not of much help.

Trew

[OutboundIndex]

No offense taken, Trew. I think your questions are valid as to whether Mail Cruncher or the Outbound Index "ideas" are unencumbered, or available for use personally or commercially etc.

My initial thought was to use Mail Cruncher as a demonstration, and that others would use the style or concept to improve the lives of many. I hoped along the way that they'd say "hey that April Lorenzen, pretty smart, let's see what else she is offering" and might use the Outbound Index ratings as the tagging method input to their Mail Crunchers, and/or contribute to our efforts in various ways.

I find it odd for people to swear never to pay for any software or anti-spam service - maybe they should swear never to pay more than it is worth to them. At this point the Outbound Index service is expected to cost 4 cents per user per month, for people who don't need hand-holding and consulting services. (That's a query service of the business trust ratings - not the "Mail Cruncher" service.)

Anyway, Trew, it is nice to meet you in this forum! Please do not apologize for noobness. I'm imagining that noobs have been the inspiration for a lot of positive change in the world

- April Lorenzen

[rob_au]

Originally posted by OutboundIndex
Mail Cruncher is independent of what anti-spam method is used to tag, flag or sort the mail to a quarantine or bulk folder. It's just intended as a useful user interface to the bulk folder. As such, the useful elements we have contributed to quarantine management are likely to be incorporated into existing and future quarantine systems.

I do envision Mail Cruncher in PHP and/or hopefully the most efficient platform. I prototyped in Cold Fusion because that is my native language, but I always envisioned migration to other languages by other programmers.

Whilst I see a place in metric based sorting systems such as that described for Mail Cruncher, my interest lies primarily with the employ of trust metrics, such as those employed in the sort of mail, in the decision process for mail delivery (which I believe the Outbound Index refers to). This is primarily because my own interest in anti-spam systems is based upon the consent within the mail transaction, rather than the content of the mail message itself - Indeed, I see the employ of challenge-response in the Bluebottle mail environment as a mechanism for measure of trustworthiness of the mail sender.

[rob_au]

Originally posted by OutboundIndex
I find it odd for people to swear never to pay for any software or anti-spam service - maybe they should swear never to pay more than it is worth to them. At this point the Outbound Index service is expected to cost 4 cents per user per month, for people who don't need hand-holding and consulting services. (That's a query service of the business trust ratings - not the "Mail Cruncher" service.)

There are two issues which I see with this approach - The first is the mechanism of query for mail providers seeking to employ these trust metrics in their delivery decision process, and is the second is the likelihood of fragmentation of the service field providing trust metrics for email and business correspondence. (April, I do recognise that each of these are relatively large issues which may be better addressed by email correspondence between ourselves - If this is a preferred medium for correspondence of these issues, please feel free to contact me via email, rob.casey@bluebottle.com)

[OutboundIndex]

Please explain further if I have misunderstood your points...

The query mechanism offered by the Outbound Index is the SIQ protocol which passes both an IP and an RHS domain part in the query:

http://wiki.OutboundIndex.net/ProtocolDiscussion

The SIQ protocol is not proprietary to the Outbound Index; it is in the public domain, submitted to the IETF as an Internet Draft, and under consideration as the preferred protocol by at least two other reputation companies not related to us. We've been using the SIQ protocol in it's present version for the past 6 months or so, and a very similar incarnation for over a year.

It has functioned as well for queries between France and the US with a volume of 50K to 100K queries per day, as it does on the local LAN.

We have 2 sendmail milters working in production on numerous machines and an MS Exchange plug in under development. These are designed to work with any server conforming to the SIQ protocol spec.

Regarding fragmentation of trust services - I am having to guess or fill in the blanks here - do you mean: If there is more than one trust service, people will have to check many services, they may disagree, how will we know who to trust anyway?

Kind of the situation we have now with a variety of pay-to-whitelist services?

Thank you,

- April Lorenzen

[rob_au]

Originally posted by OutboundIndex
It has functioned as well for queries between France and the US with a volume of 50K to 100K queries per day, as it does on the local LAN.

I've had a quick read through the Server Index Query draft and whilst it appears to be fairly complete in it's coverage, I am concerned about it's ability to scale if pre-DATA queries are to be performed for every unique mail server connection, particularly if this were to be employed over the HTTP transport. In terms of query volumes, the volume stated above would likely be that generated daily by Bluebottle alone (at our current user levels) were we to employ SIQ in our pre-DATA processing. This concern may however be bourne more out of my relatively superficial understanding of the SIQ protocol at this point in time than failings in the protocol itself.

Regarding fragmentation of trust services - I am having to guess or fill in the blanks here - do you mean: If there is more than one trust service, people will have to check many services, they may disagree, how will we know who to trust anyway?

Kind of the situation we have now with a variety of pay-to-whitelist services?

This is the exact scenario that concerns me - Are we not simply replacing one modality of trust metric with another? What advantage does this change in trust metric offer if there is no critical mass motivating the transition?

(Please note that my questions and responses are not intended to be critical of the solution which you propose - Rather on the contrary, I am most interested in how you see that trust metrics such as the Outbound Index can be leveraged to provide an improved, more "intelligent" decision table for mail processing)

[OutboundIndex]

Scaling of SIQueries: The SIQ clients cache answers. So in practice there is not a query for each pre-DATA connection.

These are UDP queries - same packet as DNS queries. Most inbound servers do at least one and usually multiple DNS queries for each pre-DATA connection.

We haven't used the HTTP fallback protocol - as the UDP packet size as been entirely sufficient. I think it was good design thoroughness on Anthony Howe's part to put in a fallback protocol given the 512 byte limit on UDP packets.

Trust Metrics: The SSL certificate market is surviving so far, with more than one trusted authority. For fun let's try going to other extremes:

If absolute power corrupts absolutely, how do we know that a single worldwide trust authority will remain trust worthy after they have the world sewed up?

Is a single, corruptible (human) authority inherently more trustable than a web of trust authorities where an abberrant manipulation COULD be revealed by the others? Consider GOSSIP

http://www.seedwiki.com/page.cfm?doc=GOSSiP&wikiid=5874

Or, I decide which store to trust to buy groceries from:

- will the food have bacteria and make me sick?

- will the roof fall in on me while I shop?

- will they steal the money out of my bank account if I use my debit card?

I choose the store I will trust with my life based on a variety of observable characteristics and historical data.

Is my life safer if there is only one grocery store chain?

I also wonder about reproducible results. Spam samples can't really be proven / tested by someone wanting to check if they are real. I can say that I give a high probabilitiy to them being real, due to belief in the repuation of the person who posted it, or that a variety of DIVERSE UNRELATED people posted the same one. I think reproducible results that anyone could spot check and verify are nice. Transparency if at all possible.

Right now there are at least 50 blacklists. People choose which one to use and which one they trust or like the style of.

Rob, I'm not saying all the above examples apply or are correct - I'm just opening up the topic for exploration. I am interested to hear your input.

Thanks,

- April Lorenzen