Many people use their hosts file for resources blocking purposes, especially against ads or known malicious sites.

Since your hosts file takes precedence over your DNS in domain name resolution, you can redirect undesired domain to invalid IP addresses, saving both bandwidth and CPU because resolved IPs are cached.

Unluckily, most information sources about this useful technique, including the Wikipedia article above, instruct the reader to use 127.0.0.1 (the local loopback IP) as the dead-end destination, rather than a truly invalid address such as 255.255.255.0. This is not very smart, especially if you installed a web server on the loopback interface (like many web developers do), because you're spamming it with dummy requests whenever you browse an ad-laden web site.

Furthermore, I'm currently receiving several reports about ABE warnings popping up everywhere. If you read my post about ABE yesterday, you know that it ships with a built in "SYSTEM" ruleset containing just one rule which alone implements the whole LocalRodeo functionality:

# Prevent Internet sites from requesting LAN resources. Site LOCAL Accept from LOCAL Deny

Such a rule blocks any HTTP request for resources placed in your local network, including localhost (127.0.0.1) and any other LAN IP, unless it is originated from your local network as well. This protects your internal servers and devices (e.g. routers and firewalls exposing web interfaces) against CSRF and XSS attacks performed from the internet.

As a side effect, though, if you're redirecting arbitrary hosts to 127.0.0.1, you'll get bombed by a storm of ABE warnings whenever those sites are linked from external web sites. The solution is simple: just open your host file and replace

127.0.0.1

with

255.255.255.0

everywhere it's used to block something, but being careful to keep

127.0.0.1

on the

localhost

entry and other really local domains, if any.

Update:

NoScript 1.9.5.5 beta automatically suppresses notifications for the commonest case covered here (HTTP requests for a domain name resolving to 127.0.0.1 on the default port), and also introduces an option to disable all ABE notifications.

33 Responses to “ABE Warnings Everywhere OMG!”

  1. #1 GµårÐïåñ says:

    Excellent points Giorgio and thanks for taking the time to share it with the public. I wish that the folks at Spybot would take that to heart and stop loading people's host files with the 127.0.0.1 loop back and use the more appropriate destination. At least this way the user knows what's the best practice for the ones they insert themselves.

  2. #2 PX says:

    Dumb request/question -- what about just turning off the warning? Or flashing something in the bottom icon of NoScript?

    Some anti-spyware/etc... programs automatically write the rules in the HOSTS file... so this might be a better solution for those of us under those circumstances.

    Great work, by the way!

  3. #3 News ABE Warnings Everywhere OMG! | Web 2.0 Designer says:

    [...] See the original post: ABE Warnings Everywhere OMG! [...]

  4. #4 candy says:

    I use hosts file with NoScript and ABE, but I don't receive any warning. Why?

  5. #5 Giorgio says:

    @PX:
    An option to turn off the warning will be given in next release, too.
    Also I'm toying with the idea of checking if a web server actually exists on 127.0.0.1 first time you hit it, and if it's not found skipping the warning all together.

    @candy:
    Maybe you're not using 127.0.0.1 as your redirect destination, or you're using Adblock Plus to block the same hosts.

  6. #6 Tom T. says:

    I'll call this to the attention of the very nice people at http://www.mvps.org/winhelp2002/hosts.htm, who supply a free subscription service of Hosts files, updated about once a month, but directing to localhost.

    Question: Is there harm in directing to 5.x.x.x, non-existent locally, rather than 255.etc? (As a minor side benefit, with about 15,000 entries in Hosts, the 5.x keeps the file about 90k smaller.) TIA.

  7. #7 Tom T. says:

    P. S. Most people don't realize how easy the change is. If you're not comfortable inside your system, don't do this, but all that is involved in Win XP is: Open Windows > system32 > drivers > etc and you will see the Hosts file. Open it with Wordpad. (You may have to choose Wordpad from a list of options with which to open.) Click Edit > Replace. In the Find/Replace window, enter 127.0.0.1 in "Find", and in "Replace", enter the desired replacement, like the 255.255.255.0 suggested by Giorgio. Then click "Replace all". If you have 15,000 entries as I do, it will take some seconds. When you get the "finished" message, LOOK FOR LOCALHOST AND CHANGE IT BACK TO 127.0.0.1 !!!! (It is the first entry in mine, which makes that easy.) Save the file, close it, and close all folders. No reboot necessary.

    CAUTION: Back up the Hosts file first, perhaps by placing a copy on your desktop. DO NOT DO THIS IF NOT COMFORTABLE EDITING SYSTEM FILES.

    Offered in the hope that it might be of use, BUT WITHOUT WARRANTIES, EXPRESS OR IMPLIED.
    *** USE AT YOUR OWN RISK ***

  8. #8 Neil Rashbrook says:

    I have to say that's the first time I've seen 255.255.255.0 suggested as a dummy IP address. Is there any particular reason why it might be better than, say, 0.0.0.0?

  9. #9 PX says:

    @Giorgio

    Thanks [alot!]!
    And again, great work :)

    Maybe a better option would be a checkbox to ignore 127.0.0.1 (rather than checking to see if it exists as a webserver)

  10. #10 Ocie says:

    Here is what WhoIsByIPAddress (http://tools.whois.net/whoisbyip/) returns for 255.255.255.0:

    OrgName: Internet Assigned Numbers Authority
    OrgID: IANA
    Address: 4676 Admiralty Way, Suite 330
    City: Marina del Rey
    StateProv: CA
    PostalCode: 90292-6695
    Country: US

    NetRange: 240.0.0.0 - 255.255.255.255
    CIDR: 240.0.0.0/4
    NetName: RESERVED-240
    NetHandle: NET-240-0-0-0-0
    Parent:
    NetType: IANA Special Use
    Comment: Please see RFC 3330 for additional information.
    RegDate:
    Updated: 2002-10-14

    OrgAbuseHandle: IANA-IP-ARIN
    OrgAbuseName: Internet Corporation for Assigned Names and Number
    OrgAbusePhone: +1-310-301-5820
    OrgAbuseEmail: abuse@iana.org

    OrgTechHandle: IANA-IP-ARIN
    OrgTechName: Internet Corporation for Assigned Names and Number
    OrgTechPhone: +1-310-301-5820
    OrgTechEmail: abuse@iana.org

    # ARIN WHOIS database, last updated 2009-07-01 19:10

  11. #11 Ocie says:

    Here is what WhoIsByIPAddress returns for 0.0.0.0:

    OrgName: Internet Assigned Numbers Authority
    OrgID: IANA
    Address: 4676 Admiralty Way, Suite 330
    City: Marina del Rey
    StateProv: CA
    PostalCode: 90292-6695
    Country: US

    NetRange: 0.0.0.0 - 0.255.255.255
    CIDR: 0.0.0.0/8
    NetName: RESERVED-1
    NetHandle: NET-0-0-0-0-1
    Parent:
    NetType: IANA Special Use
    Comment: Please see RFC 3330 for additional information.
    RegDate:
    Updated: 2002-10-14

    OrgAbuseHandle: IANA-IP-ARIN
    OrgAbuseName: Internet Corporation for Assigned Names and Number
    OrgAbusePhone: +1-310-301-5820
    OrgAbuseEmail: abuse@iana.org

    OrgTechHandle: IANA-IP-ARIN
    OrgTechName: Internet Corporation for Assigned Names and Number
    OrgTechPhone: +1-310-301-5820
    OrgTechEmail: abuse@iana.org

    # ARIN WHOIS database, last updated 2009-07-01 19:10
    ----------

    I guess we should all read RFC 3330!

  12. #12 Nils R Grotnes says:

    Back when I used a hosts file for blocking, the main reason to use 127.0.0.1 was that pointing to an ip-address without a webserver led to waiting on annoying timeouts.

  13. #13 Ocie says:

    http://www.rfc-editor.org/rfc/rfc3330.txt

    It has seven pages, but only the first three or four contain the essential data. :-)

  14. #14 Giorgio says:

    @Nils R Grotnes:
    Using an invalid address is even faster, since the browser aborts the request even before hitting the network.

    @Ocie:
    Any address ending with 0 is invalid as a TCP endpoint, but I suggested using 255.255.255.0 rather than 127.0.0.0 or 0.0.0.0 because local IP tests in currently shipping stable ABE match addresses starting with 127 and 0 even if they end with 0. This has been corrected, hence with next release you can use any x.y.z.0 invalid address if you wish to.

  15. #15 Peter Graney says:

    I'd like to agree with response N02.

    I do not have the expertise to go through and edit my hosts file. Just give me the option to turn the notifications off!!!

    Thanks Peter

  16. #16 Giorgio says:

    @Peter Graney:
    You've got it in latest development build 1.9.5.5.

  17. #17 Rob says:

    Thanks for the info, Giorgio!

    FYI, I updated the Wikipedia article. See what you think.

  18. #18 Cd-MaN says:

    A quick note: I've done some speed tests and using 0.0.0.0 (or other invalid addresses) instead of 127.0.0.1 has the added benefit of speed. I found that the former was ~100 times faster than the later (of course we are talking about very small time intervals anyways, but still, it is an other benefit). See here:

    http://hype-free.blogspot.com/2009/07/speedy-hosts-blocklists.html
    http://hype-free.blogspot.com/2009/07/more-benchmarking-in-127001-vs-0000.html

  19. #19 Ocie Hudson says:

    Regarding #14: "Any address ending with 0 is invalid as a TCP endpoint, ..."

    That is interesting, because running NETSTAT -a -b -v -n from a command prompt on my computer often reports a long list of "connections". Usually three or four that are reported as TCP have 0.0.0.0:0 as one endpoint, often as the "foreign" IP address -- but maybe simultaneously have the "local address" as 0.0.0.0:ddd where "ddd" is a nonzero port number.

  20. #20 Tom T. says:

    I received the following strong objection from the providers of the Hosts service that I use. Pertinent email follows:

    Tom T. wrote: (2 July 09)

    Giorgio Maone, the developer of the NoScript safety extension for Firefox web browser, has introduced a new feature, Application Boundary Enforcer (ABE), which is causing some conflicts with Hosts redirecting to localhost. Giorgio recommends redirecting to 255.255.255.0. .... please consider his advice and whether it would be desirable to adjust your excellent service in this manner. A flood of alerts is being triggered by the present situation for those who use both tools.
    ****************

    Mike Burgess replied: (3 July 09)

    re: "Giorgio recommends redirecting to 255.255.255.0"

    Obviously he did not put much thought into this ... there are
    only two known "industry standards" being "127.0.0.1" or "0.0.0.0"

    The reason I say "not much thought" is things have to be compatible
    with each other ... all antivirus/antispyware program recognize "127.0.0.1"

    I'm afraid he is going to run into even more problems/issues with these
    programs as well as the many authors of HOSTS files.

    I have no intention of altering "127.0.0.1" just because of some poorly
    researched extension ...

    re: "Do you see any problem with this"

    Try "0.0.0.0" instead and see what happens ...
    (link provided by Mike in which he recommends 0.0.0.0 for those running a server-TT):
    (FAQ) Can I use the HOSTS file if I'm running a "Server"?
    http://www.mvps.org/winhelp2002/hostsfaq.htm#Personal

    Mike Burgess
    Microsoft MVP - Consumer Security
    "There's no place like 127.0.0.1"
    http://www.mvps.org/winhelp2002/hosts.htm
    ******************

    In response to another mail from Tom T.,
    Mike Burgess replied: (4 July 09)

    It is ridiculous to think that every program in the world should
    make changes just because of this poorly thought out extension.

    Think about it ... what about all the HOSTS manager programs
    that are coded to add/remove entries using "127.0.0.1" should they
    now all redesign their programs? (you must be kidding!) ...

    These are just two examples ... there are many many more ...
    HostsMan - http://abelhadigital.com/
    HostsXpert - http://www.funkytoad.com/

    What about all the Antivirus/Antispyware programs that can edit the
    HOSTS file ... is "255.255.255.0" compatible with all these? You
    *must* be sure before any author makes these type changes ...

    Advising "joe average" to make changes (using Replace) is a very
    dangerous method that will no doubt lead to problems ... while it may
    be easy for you or I to do this ... it is NOT for "joe average" ... so what
    are they to do?

    re: "NoScript has been downloaded by over 50 million users"
    It doesn't matter how many users if you are using an extension coded
    that is not compatible or has an option that "joe average" can use/understand ...

    Is Microsoft wrong about using "127.0.0.1" ??
    http://www.google.com/search?hl=en&safe=off&num=20&q=%22microsoft%22+%2B+%22127.0.0.1%22&aq=f&oq=&aqi=g10
    --
    The free utility Spybot - Search & Destroy, for example, includes a feature called "Immunize" that populates the hosts file with thousands of URLs of such websites redirected to 127.0.0.1 to block them.
    http://en.wikipedia.org/wiki/Hosts_file
    --
    So are you going to get SpyBot S&D to recode their program?

    Perhaps you could resolve the end-user problem by having the extension
    offer to edit the existing HOSTS file and "Replace" the needed address.

    Mike Burgess
    Microsoft MVP - Consumer Security
    "There's no place like 127.0.0.1"
    http://www.mvps.org/winhelp2002/hosts.htm
    ******************
    ******************
    @Giorgio, what is your response to Mike's comments?

    (For the record, I did not advise "Joe Average" to edit Hosts. My posts above contain the standard disclaimers - experienced users only, back up, etc.)

  21. #21 Giorgio says:

    @Tom T.:

    Mike Burgess replied: (3 July 09):

    Obviously he did not put much thought into this … there are
    only two known "industry standards" being "127.0.0.1" or "0.0.0.0"

    Who uses 127.0.0.1 for adblocking did not put enough thought into his business.
    There's no "industry standard" about hosts file used as a poor man adblocker, which is a hack at best.

    There are IETF's Networking Group RFCs (which are not standards either, but are the specification which the internet is built on and are much more authoritative than Microsoft itself, let alone any MVP): according RFC3330, networking, which say "127.0.0.1" is the address of the loopback interface, where you can actually bind a web server, while "0.0.0.0" is an alias for "all the network interfaces on this hosts", and it's used almost exclusively as a placeholder for server binding purposes. These designations don't hint (or are even compatible) with any adblocking purpose.

    That said, any IPv4 "address" whose rightmost byte is 0 is practically invalid as a destination, therefore I suggested 255.255.255.0 initially because the "0.0.0.0" network is a local network anyway and caught by ABE, but at this moment (in current NoScript beta) "0.0.0.0" should be equally good since invalid addresses like this are properly recognized.

    The reason I say "not much thought" is things have to be compatible
    with each other … all antivirus/antispyware program recognize "127.0.0.1"

    Eat shit! One billion flies can't be wrong.

    Think about it … what about all the HOSTS manager programs
    that are coded to add/remove entries using "127.0.0.1" should they
    now all redesign their programs? (you must be kidding!) …

    They should, not because NoScript, but because it has been a poor design choice.

    Is Microsoft wrong about using "127.0.0.1" ??

    God forbid, Microsoft is always right!

    Perhaps you could resolve the end-user problem by having the extension
    offer to edit the existing HOSTS file and "Replace" the needed address.

    I already "resolved" (worked around, actually) by omitting notifications when an external request is directed to 0.0.0.0 or to 127.0.0.1 on a standard port.

    This does not mean using 127.0.0.1 as an adblocking destination is less stupid, though.

  22. #22 Lewis says:

    I have a domain on static servers. I have home.example.com as a CNAME pointing to a DYNDNS name. This means that home.example.com routes correctly to my home machine, no matter what IP it has. on my home machine, so that home.example.com resolves, I have in my /etc/hosts

    127.0.0.1 localhost home.example.com

    This works fine, and has always worked. Trouble is, today I was accessing a local page on my own machine and was making some changes. The first five or six went through fine, then all of a sudden I started getting these abe warnings.

    I looked at editing the SYSTEM.abe config file, but there was zero info on what it expected, and precious little in the sample file to go on. And no, there is no reason to run DNS on my local machine. I don't need, nor do I want, DNS on this machine.

    For now I've simply disabled abe.

    As for using 127.0.0.1 for adblocking there is a very good reason, and that is the time it takes to load a page. If I redirect 101com.com to 127.0.0.1 then when a site tries to load an ad from there, it simply doesn't find the picture it needs and doesn't load. If I put 0.0.0.0 there, it tries to load from 0.0.0.0. 127.0.0.1 is the one address that I can guarantee is 1) reachable 2) has a webserver 3) has no ads 4) will not take more than a few milliseconds to resolve/respond.

  23. #23 Tom T. says:

    @ Giorgio:

    Thank you for your analysis and reply to Mr. Burgess. I'm not sure whether his mind is open to any further discussion, so I will ponder whether to forward it to him, but I certainly appreciate having the information for my own use.

    For the record, although his Hosts service might have started originally as an ad-blocker, I use it more as a malware-blocker. I already have ad-blocking sw, Adblock (original), with which I am very happy (although I understand that the majority prefers Plus). But his Hosts started adding sites that are found to be known installers of drive-by malware, even though they are not advertisers. Consider porn.example.com. Surely the site displays ads, but they can be blocked if desired. But it is known that many porn sites are malware installers (so are some non-porn sites, of course), while others might be properly-behaved and seek their revenue from advertising, subscriptions, etc. His Hosts file would prevent the browser from connecting to any site, porn.com or Brittanica.com, that had been proven to be a malware-installer. Whether it incidentally blocks any ads along the way is not important, at least to me. But it's good to have a community devoted to identifying malsites and providing a blocker that requires no user action, no installation of an application, etc. The saving in page-loading time and bandwidth is a fringe benefit.

    With that in mind, may I repeat my original question? What about redirecting Hosts to 5.xxx, which does not exist on my LAN? I used to use Hamachi, which uses the 5 domain, but if so, I would just choose a different 5xxx address. It seems this would be safe (no server there, no web page, nothing) and fast (internal address; browser would abort quickly).

    http://network-tools.com/default.asp?prog=lookup&host=5.8.9.7
    IP address: 5.8.9.7
    No host name is associated with this IP address or no reverse lookup is configured.
    Error:Host not found
    5.8.9.7 is from Other(XX) in region Unclassified

    So is there any problem, or advantage or disadvantage, to using this address for Hosts? Thanks.

    ******************
    Feat req: Preview button and window, for poor typists like myself. :-)

  24. #24 Giorgio says:

    @Lewis:
    In your specific case (web site on localhost meant to be referenced from outside), you just need an exception at the beginning of the SYSTEM rule like this:

    Site http://home.example.com
    Allow
    

    Regarding 127.0.0.1 VS 0.0.0.0, it works the opposite of what you believe. You may want to check this comment by Cd-Man reporting a ~100 times speed improvement of using an invalid *.0 address over 127.0.0.1.

    While I can't swear about Cd-Man's numbers, I can swear about Firefox internals: invalid addresses like 0.0.0.0 or 255.255.255.0 don't generate any network traffic, and their rejection is immediate: therefore they're a far better candidate for adblocking.

    @Tom T.:
    I could not recommend using a valid unassigned external IP, not only for the reason above (unneeded network activity, even if just DNS resolution like in your case) but because tomorrow it might get assigned.
    Just use an invalid address.

  25. #25 Tom T. says:

    Thanks, Giorgio. Will use *.0 as advised.

    Speed information:

    Pinging 127.0.0.1 with 32 bytes of data:

    Reply from 127.0.0.1: bytes=32 time

  26. #26 Tom T. says:

    Not sure what happened to the table of ping times I was attempting to post, - I see only a couple of lines of it now -- but no matter. The bottom line is *.0 is the way to go.

  27. #27 SteveSkinz says:

    Thanks, Giorgio.
    For everything includingall very good instant page links.
    I would also like to say a big thankyou to all poster's also, in every post i read i found a direct link or some usefull information.
    I have had my PC some 4 yesrs now & have alway's used free/freeware/shareware products/software coz i'm a low income family of 4 and i wont pay for anything unless i know it is the best, i used to buy Norton Security suite's but they suck IMO especially for price I even think AVG free is better lol.
    I'm currently using Firefox 3.7 and have notied alit of scripting erors when i run a reg cleaner, probabbly the by a storm of ABE warnings linked from external web sites.

    The solution is simple: So say's you, I didnt really understand?
    I know my IP config is:-
    localhost 127.168.0.1 home.example.com (This sets up my Router)
    Localhost2 127.168.0.3 (Thisis for my Daughter's WiFi Lap lop Settings)
    Subnetmask 25.255.255.0 (i think)
    Loop I think i 192.168.101 (Is that right)?

    I dontreally understand IP Config's especially blocking sripts which is what i want and need to be able to do (This PC is XP the Lap Top is VISTA).

    If you can explain in dummy instrutions, thanks!

    Have a great morning/day/afternon and night- Slep Well ;-) .

  28. #28 Giorgio says:

    @SteveSkinz:
    Your net configuration has probably nothing to do with the errors you get.
    Please feel free to join our web tech forum if you want to discuss this issue.

  29. #29 j says:

    thank you for creating this

  30. #30 j says:

    this is a great site as i have searche alll over to find a buffet of script coders

  31. #31 Sorrow says:

    So, amidst all this discussion, the bottom line is:

    Use 0.0.0.0 as your target IP address instead of 127.0.0.1, except in the case of the first line, which should be your redirect address, AKA computer.

    Correct? No compatibility issues whatsoever?

  32. #32 Giorgio says:

    @Sorrow:
    Correct, and no compatibility issues.

  33. #33 Kevin Chadwick says:

    127.0.0.0/8 and 0.0.0.0/8 should likely be blocked on any incoming interface especially wireless. I believe 255.255.255.0 would be a much better choice in general than 127.0.0.1. The only problem may be if a spyware program incorrectly blocks a clean site that you want to visit and then corrects this mistake by removing that entry. I would expect the domain to be searched for ignoring the ip and so even that case could easily be made irrelevent. If you have edited this file in the first place and use noscript, it would be likely that that person can backup or reset the host file or even delete it, likely without huge consequences (maybe a badly made game server host etc). On a large network with an unusual setup the consequences of using 127.0.0.1 maybe far greater.

    As for speed it may be that a firewall is dropping connections to 255.255.255.0 and not 127.0.0.1, therefore it may be slower to respond or timeout those connections but if you do not want to connect to that address, then why should it matter.

    Trust a microsoft employee to think existing practice should outrank best practice, lol. Especially when the changes to fix these spyware programs would be tiny and may already be unnecessary and or uneventful.

    "I have no intention of altering "127.0.0.1" just because of some poorly
    researched extension …"

    Should have been
    "I have no intension of recommending companies follow the rfcs which provide an industry standard of best practice methods just because of some well researched extension and I also recommend people who understand the risks of scripting don't use the windows gui or activate windows (scripted) or accept adobe licensing agreements as these actions would be prevented by our own windows security scripting policies when enabled and it should be noted that activation and license acceptance actually gains us something and the user nothing but potential headaches via security problems and unidentifiably poor software".

Bad Behavior has blocked 2709 access attempts in the last 7 days.