On 4/5/2012 6:58 PM, Darren New wrote:
> On 4/1/2012 8:43, Warp wrote:
>> (it broke the 50% mark of all email traversing
>> the internet long time ago)
>
> It broke the 90% mark a long time ago.
>
>  > twharts any kind of comprehensive automatic
>> traffic analysis of email (or at least makes it impractical and
>> expensive).
>
> Classifying spam isn't difficult. It's just that ISPs don't want to
> carry 10x as much email data as they need to, and it's hard to track
> down the source.
>
They could go a long way by changing the protocol so you can't "fake" 
the source, and the tracking, with respect to how it got there, is kept, 
and correctly reported, so that, even if you changed the supposed start 
point, somehow, it would be more obvious that the source, as it 
traversed the network, wasn't the source being reported. Half the time 
email systems consider this information "inconvenient" and actually make 
it hard, or impossible, to even look at, never mind actually tell you 
that there is a discrepancy of any kind.

If the thing comes from a proxy, its obviously not from where ever it 
was sent from in reality. Might need some rules on whether its legal for 
the proxy itself to misrepresent itself as a) not in the chain, or b) a 
different source. But, once it leaves the proxy, there is still, in 
principle, a way to trace back the address, to the server it claims to 
come from, thereby finding that there is no way in hell the trace in the 
email's own path could match with the claimed source (but, that would 
require an automatic traceroute, and even doing that, from some 
machines, won't work in cases like Windows, where generating the packets 
needed in anything other than the control paths is **not allowed**, as a 
possible detected exploit, and where your ISP, modem, or something else, 
is denying those control commands).

But, yeah, its hardly "impossible" to at least figure out where the hell 
it comes from, and probably easier to use something like that, to ferret 
out new "bad" messages, than all the stupid assed, "Lets look at the 
content, then panic when legit mail contains X formating, and Y list of 
keywords!!!" Hotmail has flagged legit stuff on me, for example, once a 
week, at times, as "possibly dangerous", yet, at almost as much of a 
regular basis, it has failed to flag idiots trying to cell me viagra... 
And, while they suggest to leave the bad emails in there, to better 
handle new bad ones, if you have a good one end up in the trap, you can 
miss it in "page after page" of invalid ones, simply because having one 
good email, on the 50th page, or 800 actual spam messages... really 
isn't a viable solution. Its almost better, if you have fairly low 
volume, to turn the damn spam trap off, and just delete them yourself.

Post a reply to this message

> Classifying spam isn't difficult.

I've yet to see a single system which can do this reliably. Maybe one 
exists, but I haven't seen it.

> It's just that ISPs don't want to
> carry 10x as much email data as they need to, and it's hard to track
> down the source.

This.

Post a reply to this message

> They could go a long way by changing the protocol so you can't "fake"
> the source, and the tracking

Yeah, but that means a breaking change to the mail protocol.

Will. Not. Happen.

Heck, they invented a way to check that the source mail server is 
authorised to send mail, and large ISPs manage to screw that up.

(When receiving mail, you're supposed to look up the domain of the 
source server and check for an authorisation record in the DNS. 
Apparently some doofus thought it would be a good idea to also check the 
domain OF THE EMAIL ADDRESS - despite the RFC explicitly saying that you 
must not do this...)

Post a reply to this message

On 6-4-2012 6:52, Patrick Elliott wrote:
> On 4/5/2012 6:58 PM, Darren New wrote:
>> On 4/1/2012 8:43, Warp wrote:
>>> (it broke the 50% mark of all email traversing
>>> the internet long time ago)
>>
>> It broke the 90% mark a long time ago.
>>
>> > twharts any kind of comprehensive automatic
>>> traffic analysis of email (or at least makes it impractical and
>>> expensive).
>>
>> Classifying spam isn't difficult. It's just that ISPs don't want to
>> carry 10x as much email data as they need to, and it's hard to track
>> down the source.
>>
> They could go a long way by changing the protocol so you can't "fake"
> the source, and the tracking, with respect to how it got there, is kept,
> and correctly reported, so that, even if you changed the supposed start
> point, somehow, it would be more obvious that the source, as it
> traversed the network, wasn't the source being reported. Half the time
> email systems consider this information "inconvenient" and actually make
> it hard, or impossible, to even look at, never mind actually tell you
> that there is a discrepancy of any kind.

Often I receive mail that was not sent by the person that is in the 
from: line. Many people also get mail that claims to be sent by me. I 
even get myself mail sent by me often from places that I might wish to 
visit, but haven't done so yet.
What I never fully understood is if this is legal or not. I know it is 
easy to do, and hard to track down, but I would expect it to be illegal 
anyway. Anyone here knows?


-- 
tip: do not run in an unknown place when it is too dark to see the 
floor, unless you prefer to not use uppercase.

Post a reply to this message

> Often I receive mail that was not sent by the person that is in the
> from: line. Many people also get mail that claims to be sent by me. I
> even get myself mail sent by me often from places that I might wish to
> visit, but haven't done so yet.
> What I never fully understood is if this is legal or not. I know it is
> easy to do, and hard to track down, but I would expect it to be illegal
> anyway. Anyone here knows?

Sending an email is like sending a postcard; you write on one side who 
it's from, and on the other side who it's to. Most people write who it's 
/really/ from, but there's absolutely nothing to stop you pretending to 
be anybody you fancy. (Whether the recipient will believe you is another 
matter...) People seem to think because it's on a computer it must 
somehow be "secure", but it isn't.

Is it illegal? Well, is it illegal to send a postcard claiming to be 
from somebody it isn't?

Clearly trying to deceive somebody for financial gain is fraud, which is 
illegal no matter which way you try to do it. But is pretending to be 
somebody else illegal in itself? I don't know. (And I'd guess it varies 
by country anyway.)

Post a reply to this message

Le 2012-04-06 00:52, Patrick Elliott a écrit :
> On 4/5/2012 6:58 PM, Darren New wrote:
>> On 4/1/2012 8:43, Warp wrote:
>>> (it broke the 50% mark of all email traversing
>>> the internet long time ago)
>>
>> It broke the 90% mark a long time ago.
>>
>> > twharts any kind of comprehensive automatic
>>> traffic analysis of email (or at least makes it impractical and
>>> expensive).
>>
>> Classifying spam isn't difficult. It's just that ISPs don't want to
>> carry 10x as much email data as they need to, and it's hard to track
>> down the source.
>>
> They could go a long way by changing the protocol so you can't "fake"
> the source,

There are many webhosting services that also offer e-mail with their 
package.  In those cases, the source would always appear fake since the 
source would be "mailrelay.webhostingcompany.com" instead of 
"mail.francoispetgroomingservices.biz"

> and the tracking, with respect to how it got there, is kept,

It is.  Look at the "Received:" lines of the header.

> and correctly reported, so that, even if you changed the supposed start
> point, somehow, it would be more obvious that the source, as it
> traversed the network, wasn't the source being reported.

Internal RFC-1918 addressing and dicrepancies between internal DNS vs. 
public DNS names make this impossible.

As an outsider, how can you tell if fred.remoteoffice.mycompany 
(10.2.5.14) and pebbles.datacenter.mycompany (10.254.13.56) are valid 
sources without knowing the internal e-mail architecture of the company?

> Half the time
> email systems consider this information "inconvenient" and actually make
> it hard, or impossible, to even look at, never mind actually tell you
> that there is a discrepancy of any kind.
>
> If the thing comes from a proxy, its obviously not from where ever it
> was sent from in reality.

There's no such thing as a proxy in e-mail parlance.  Only mail relays. 
  And because most companies and ISPs try to limit the path that e-mails 
take to known and trusted sources, you can't get rid of them.

> Might need some rules on whether its legal for
> the proxy itself to misrepresent itself as a) not in the chain, or b) a
> different source.

How should a machine with an internal DNS name of 
pebbles.datacenter.mycompany and an IP address of 10.254.13.56 which 
gets natted by the outside firewall to 209.209.209.209 (and which 
resolves to mx.mycompany.com) represent itself?

> But, once it leaves the proxy, there is still, in
> principle, a way to trace back the address, to the server it claims to
> come from,

Not if the server is behind a firewall (which is should be), or if is 
used RFC-1918 IP addressing (which it should).

> thereby finding that there is no way in hell the trace in the
> email's own path could match with the claimed source (but, that would
> require an automatic traceroute, and even doing that, from some
> machines, won't work in cases like Windows, where generating the packets
> needed in anything other than the control paths is **not allowed**, as a
> possible detected exploit, and where your ISP, modem, or something else,
> is denying those control commands).
>

There are various tricks used by mail relays to try and assert the true 
identity of a mail-relay that contacts them, such as doing DNS lookups 
and reveser lookups to make sure they match the SMTP "HELO" command, 
verifying that the machine is a valid MX record for the domain it claims 
to represent, etc...  But as stated above, these can sometimes prevent 
valid e-mails from small businesses that don't have their own e-mail 
infrastructure from being delivered.

> But, yeah, its hardly "impossible" to at least figure out where the hell
> it comes from, and probably easier to use something like that, to ferret
> out new "bad" messages, than all the stupid assed, "Lets look at the
> content, then panic when legit mail contains X formating, and Y list of
> keywords!!!" Hotmail has flagged legit stuff on me, for example, once a
> week, at times, as "possibly dangerous", yet, at almost as much of a
> regular basis, it has failed to flag idiots trying to cell me viagra...
> And, while they suggest to leave the bad emails in there, to better
> handle new bad ones, if you have a good one end up in the trap, you can
> miss it in "page after page" of invalid ones, simply because having one
> good email, on the 50th page, or 800 actual spam messages... really
> isn't a viable solution. Its almost better, if you have fairly low
> volume, to turn the damn spam trap off, and just delete them yourself.

While it would potentially cut down on the phishing e-mails, even if you 
did manage to make sure that the source was real, there's no way to 
programatically determine if an e-mail that says "get viagra at 80% off" 
that comes from online.farmacia.cr is something you're interested in or not.

-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

On 4/5/2012 21:52, Patrick Elliott wrote:
> They could go a long way by changing the protocol so you can't "fake" the
> source, and the tracking, with respect to how it got there, is kept, and
> correctly reported,

It is, assuming that you don't have a corrupted routing node. The basic 
problem is getting everyone to switch to a brand new email protocol all at 
once, and getting everyone to support your tracking proposal.

The received-by header isn't something you can completely forge.

> so that, even if you changed the supposed start point,
> somehow, it would be more obvious that the source, as it traversed the
> network, wasn't the source being reported. Half the time email systems
> consider this information "inconvenient" and actually make it hard, or
> impossible, to even look at, never mind actually tell you that there is a
> discrepancy of any kind.

Nah. The received-by headers are always carried along in the email. They're 
just not that useful because they weren't secure from the beginning, so if 
you reject all mail from insecure mail exchanges, you'll cut people off.

> If the thing comes from a proxy, its obviously not from where ever it was
> sent from in reality. Might need some rules on whether its legal for the
> proxy itself to misrepresent itself as a) not in the chain, or b) a
> different source. But, once it leaves the proxy, there is still, in
> principle, a way to trace back the address, to the server it claims to come
> from, thereby finding that there is no way in hell the trace in the email's
> own path could match with the claimed source (but, that would require an
> automatic traceroute, and even doing that, from some machines, won't work in
> cases like Windows, where generating the packets needed in anything other
> than the control paths is **not allowed**, as a possible detected exploit,
> and where your ISP, modem, or something else, is denying those control
> commands).

I don't think you understand how internet email routing works.

> But, yeah, its hardly "impossible" to at least figure out where the hell it
> comes from,

It really is, if you want to do it reliably without breaking all email 
systems currently deployed.

> and probably easier to use something like that, to ferret out
> new "bad" messages,

Sure. You first. Just reject all email that doesn't come from your new 
protocol, and see how that works out for you.

> Its almost better, if you
> have fairly low volume, to turn the damn spam trap off, and just delete them
> yourself.

Annnnnd... you just answered your own question. The system has to be as 
reliable as the delivery is in the first place.

-- 
Darren New, San Diego CA, USA (PST)
   "Oh no! We're out of code juice!"
   "Don't panic. There's beans and filters
    in the cabinet."

Post a reply to this message

On 4/6/2012 2:11 AM, Orchid Win7 v1 wrote:
>> They could go a long way by changing the protocol so you can't "fake"
>> the source, and the tracking
>
> Yeah, but that means a breaking change to the mail protocol.
>
> Will. Not. Happen.
>
> Heck, they invented a way to check that the source mail server is
> authorised to send mail, and large ISPs manage to screw that up.
>
> (When receiving mail, you're supposed to look up the domain of the
> source server and check for an authorisation record in the DNS.
> Apparently some doofus thought it would be a good idea to also check the
> domain OF THE EMAIL ADDRESS - despite the RFC explicitly saying that you
> must not do this...)
Uh.. Why would that make any sense, I mean other than the possibility 
that some moron would use a domain that didn't have a record at all, and 
thus generate, "Yep, absolutely a complete fake!" Still, if you do check 
both, and the two being checked do not come back with the same 
information, that is also a sign there is something wrong, its just.. I 
really doubt, from your comment, that they did that, instead of 
something much stupider. lol

Post a reply to this message

On 4/7/2012 5:36 AM, Francois Labreque wrote:
> Le 2012-04-06 00:52, Patrick Elliott a écrit :
>> On 4/5/2012 6:58 PM, Darren New wrote:
>>> On 4/1/2012 8:43, Warp wrote:
>>>> (it broke the 50% mark of all email traversing
>>>> the internet long time ago)
>>>
>>> It broke the 90% mark a long time ago.
>>>
>>> > twharts any kind of comprehensive automatic
>>>> traffic analysis of email (or at least makes it impractical and
>>>> expensive).
>>>
>>> Classifying spam isn't difficult. It's just that ISPs don't want to
>>> carry 10x as much email data as they need to, and it's hard to track
>>> down the source.
>>>
>> They could go a long way by changing the protocol so you can't "fake"
>> the source,
>
> There are many webhosting services that also offer e-mail with their
> package. In those cases, the source would always appear fake since the
> source would be "mailrelay.webhostingcompany.com" instead of
> "mail.francoispetgroomingservices.biz"
>
>> and the tracking, with respect to how it got there, is kept,
>
> It is. Look at the "Received:" lines of the header.
>
>> and correctly reported, so that, even if you changed the supposed start
>> point, somehow, it would be more obvious that the source, as it
>> traversed the network, wasn't the source being reported.
>
> Internal RFC-1918 addressing and dicrepancies between internal DNS vs.
> public DNS names make this impossible.
>
> As an outsider, how can you tell if fred.remoteoffice.mycompany
> (10.2.5.14) and pebbles.datacenter.mycompany (10.254.13.56) are valid
> sources without knowing the internal e-mail architecture of the company?
>
Hmm. Yeah, there is that. In theory, this isn't as big an issue with 
IPv6, since, in principle, every single device, not just IP connection, 
could have its own unique identifier. You could even use MAC for that, 
only.. some people got the idea that you should be allowed to screw with 
those too, so again... Sigh..

In any case, I am not talking about a "complete" solution, just one that 
is marginally less stupid than the ones in existence. Sure, a few "small 
businesses" might look bad, but what would you rather get, 500 different 
emails from 10.254.23.1, each one with a different domain name, or the 
ability to mask out ones that go through 2 other "external" IPs, which 
you can surmise makes it a probable fraud, and only have to look at 1-2 
emails that come from similar locations.

For the most part, unless something goes **very** wrong in a network, or 
a major change happens to its morphology, its not just the endpoint that 
can be used to figure where it came from.

Instead of even an attempt at a smart solution, what we get is clients 
that hide the routing information, and let the scammers add 
"http://www.wellsfargo.com/accounts" to the "mouse over" for all the 
damn links, so that you either a) copy and paste that (it doesn't copy 
the real address under it), and end up at the legit point, of you click 
the link, and end up at "wells.fargo.scam.robyoublind.ru". In other 
words, the ***EXACT OPPOSITE*** of better security, and threat 
identification.

Post a reply to this message

On 4/7/2012 2:56 PM, Darren New wrote:
>> If the thing comes from a proxy, its obviously not from where ever it was
>> sent from in reality. Might need some rules on whether its legal for the
>> proxy itself to misrepresent itself as a) not in the chain, or b) a
>> different source. But, once it leaves the proxy, there is still, in
>> principle, a way to trace back the address, to the server it claims to
>> come
>> from, thereby finding that there is no way in hell the trace in the
>> email's
>> own path could match with the claimed source (but, that would require an
>> automatic traceroute, and even doing that, from some machines, won't
>> work in
>> cases like Windows, where generating the packets needed in anything other
>> than the control paths is **not allowed**, as a possible detected
>> exploit,
>> and where your ISP, modem, or something else, is denying those control
>> commands).
>
> I don't think you understand how internet email routing works.
>
In principle, it works like any other protocol, but, in principle, the 
message grows as it goes through each node, since it tracks where its 
been. Its also possible to route it specifically, but that is *way* over 
most people's heads.

>> But, yeah, its hardly "impossible" to at least figure out where the
>> hell it
>> comes from,
>
> It really is, if you want to do it reliably without breaking all email
> systems currently deployed.
>

That is what exceptions are for. You might still have to check the trap, 
but it would be a "slightly" smarter trap. Right now, the trap tries to 
rely on blacklist data, and keyword identification, using programs that 
are, fundamentally, quite stupid (as in not even using halfway decent 
AI, which might have some limited capacity to guess whether the word 
viagra is someone trying to sell it to you, or your friend, telling you 
about needing to go to the hospital, because they took too much of it. 
All the program knows is "viagra", and if a few other words are there, 
its flagged, hence the moronic fact that those slip through, while 
Hotmail has **multiple** times actually flagged legit emails Origin, 
about things going on with Star Wars: KOTOR.

 From a human perspective, that the later seems to be a possibly threat, 
while the former changes less than 1-2 words, between emails, but always 
makes it through the filters, is just... WTF?

Post a reply to this message