Sooner or later you, dear NoScript user, may face this puzzle: you already allowed every single script source looking “legitimate” on a certain page, but the damn thing stubbornly refuses to work as it should. Then, in a moment of enlightenment, you dig inside your Untrusted menu, and there you find google-analytics.com. You put it there long ago because you don’t like to be tracked, but now you cross your fingers and temporarily allow it… et voilà, the page starts behaving!

Now, you may ask why the hell a web site requires Google Analytics scripts to be enabled for providing its basic features, and you might be right: no reason for that. On the other hand, a growing number of web sites leverage Google Analytics for more than just tracking page views and navigation: they also try to collect finer grained usage data about some specific features of theirs. Therefore they call functions or reference objects from http://www.google-analytics.com/ga.js from their own “1st party” specific scripts, e.g. when you click a certain button: if google-analytics.com is blocked by NoScript (or by AdBlock Plus, or by your hosts file, for the matter), the 1st party code referencing it will obviously fail and the button will stay dead. You can be hinted about Google Analytics being the culprit by opening Tools|Error Console and watching for errors like “urchinTracker is not defined” or “_gat is not defined”.

So far, all you could do about that was allowing google-analytics.com. But latest NoScript (1.8.9.7 or above) implements a new feature, called Surrogate Scripts, which works around this problem out of the box and is customizable enough to cope with similar 3rd party script issues in the future. How does it work? Very simple: whenever an external script is blocked, NoScript checks if its URL matches a certain pattern, and if it does an alternate user-provided surrogate script gets executed instead, in the context of the loading page. There you can define surrogates for any required object or function.

There’s no UI for this feature (yet?), but its intended audience is likely geeky enough not no need one.
You can specify as many URL/surrogate mappings as you want, by creating a couple of about:config preference entries under the noscript.surrogate root.
The built-in Google Analytics mapping can be regarded as a reference:

  • noscript.surrogate.ga.sources: *.google-analytics.com
  • noscript.surrogate.ga.replacement: var _0=function(){};with(window)urchinTracker=_0,_gat={_getTracker:function(){return {__noSuchMethod__:_0}}}
    var _0=function(){};with(window)urchinTracker=_0,pageTracker={_setDomainName:_0,_trackPageview:_0,_initData:_0},_gat={_getTracker:function(){return window.pageTracker}}*

If you want to exempt some pages from this replacement (e.g. because they already provide a graceful fallback for missing external scripts), you can add an URL pattern to the noscript.surrogate.ga.exceptions preference, e.g. *.mozilla.org *.mozilla.com. Script Surrogates can be disabled globally by setting noscript.surrogate.enabled to false.

Happy hacking :)

*Update

Jesse Andrew told me about some Google Analytics API not covered by the original surrogate, so rather than trying to find out every possible tracker method, present and future, I decided to catch them all by exploiting a nifty Mozilla JavaScript feature, i.e. __noSuchMethod__. Please get 1.8.9.8 (latest development build) if you want to this more reliable approach right now.

Update 2

Since I’ve been asked by concerned non-geeks (especially those who can’t read JavaScript code) what exactly the Google Analytics surrogate does, here’s a plain English description: the Google Analytics surrogate script does NOTHING. It’s a dummy “catch all” replacement for the most common Google Analytics functions: it makes the calling pages happy, helping not to break sites, but doesn’t send nor receive anything to/from Google.

Update 3

NoScript now supports a second kind of script surrogates, called “Page-level surrogates”: if the sources patterns start with the “@” character, the replacement scripts are executed before the matched page starts to be parsed and before any other script can run. This allows preemptive patching of the loading page, similarly to GreaseMonkey but in a more pervasive way, since GreaseMonkey’s scripts can run only when the DOM is already parsed (i.e. after page scripts already run).

53 Responses to “Surrogate Scripts vs Google Analytics”

  1. #1 Jesse Andrews says:

    Perhaps add _trackEvent to the included GA surrogate script - event tracking is out of beta and recommended to be used where people would use page tracking before.

    http://code.google.com/apis/analytics/docs/eventTrackerGuide.html

  2. #2 Giorgio says:

    @Jesse:
    Thanks. Should it be a pageTracker method?
    More in general, is there a concise documentation about that?

  3. #3 Jesse Andrews says:

    @Giorgio,

    yep, _trackEvent is a method that the pageTracker object provides.

  4. #4 Giorgio says:

    @Jesse:
    Eh, I looked at the API page and there were far too many methods to be covered. So I decided to go the smart way ;)

  5. #5 Dave says:

    This is brilliant. Seriously. If you weren’t doing this, I don’t know who would. Thank you so much. (I’ve donated, but under another name — this is my pseudonym). Now, how about something else I’ve always wondered about in the "synthetic" realm: cookies. Wouldn’t it be great to always allow doubleclick.com (and other trackers), but instead returning the cookie they’ve stored on your machine, return a randomized cookie whenever a certain tracking site requests one.
    Thank you again for a great add-on.
    Dave

  6. #6 Giorgio says:

    @Dave:
    Thanks, you’re too good.
    I don’t want to fiddle with cookies for non-security purposes, because there are already excellent extensions for cookie management.
    I, for instance, use CS Lite. Did you try that?

  7. #7 MacOtaku says:

    Would it be possible to avoid mapping over any hypothetical global object other than those from the blocked and replaced code and still accomplish the same? Unfortunately, some sites do use rather silly/meaningless variable names, such as _. Would this surrogate script still work without raising any security issues if wrapped in an anonymous function call, so _ would be a local variable, discarded after function call finished? In JS, when a global object is assigned from a local object, it becomes a copy of (and not a reference to) the local, doesn’t it?

    (function(){var _0=function(){};with(window)urchinTracker=_0,_gat={_getTracker:function(){return {__noSuchMethod__:_0}}}})()

    BTW, this is exactly what I was asking for a while back in http://userscripts.org/topics/2293 — but GM doesn’t provide a safe method (that I know of) to run script in the context of the page, before the page’s own scripts run.

  8. #8 Giorgio says:

    @MacOtaku:
    it’s already wrapped inside an anonymous function behind the curtains, if your concern was about “_0″ polluting the global namespace.
    In facts, whatever you put in a “noscript.surrogate.xyz.replacement” preference gets wrapped inside a block like this:

    try { (function() { ...replacement code here... })() } catch(e) {}
    

    Therefore, if you prefix your vars with “var” or “let”, you’re not going to pollute anything.

  9. #9 Dave says:

    @Giorgio #6:
    Thanks, I have tried CS Lite and a few others, but I kept running into the same problem with functionality as experienced with calls to scripts, namely the "Oh, it appears you have cookies disabled; you need to enable them to use this site" problem. I completely understand your reluctance to broaden NoScript’s reach, and thank you again for your constant effort in this area. I know Leo LaPorte and Steve Gibson think highly of you as well, and share that frequently on their podcasts.
    Dave

  10. #10 dude says:

    Or just get over yourself and remove NoScript entirely.

  11. #11 Matthew Flaschen says:

    "Or just get over yourself and remove NoScript entirely."

    Why would I do that, when it does exactly what I want it to do?

  12. #12 Non-Geek says:

    Giorgio, I think you might underestimate the number of non-geek (non-code-reading) users who use NS and want to take full advantage of its features. Before I disable surrogates entirely, could you please explain in plain, dummy English *what* the GA surrogate script does? What information does it send to Google, and what, if any, information comes back to the web site?

    @Dave: Yes, but if you remember, Gibson almost gave up on NS because he found the notifications annoying. Members of his newsgroup had to tell him that he could customize or disable notifications, after which he became a big fan.

    Which proves my point: if a guy with Gibson’s level of knowledge can’t figure it out, the rest of us are clueless. A plain-English explanation of major changes, along with the tech explanation, would be really cool.

  13. #13 Giorgio says:

    @Non-Geek:
    the “geeky” part of this feature is about writing or customizing surrogate scripts, therefore there’s not much I can add to simplify it if you can’t code.
    The good news is that the Google Analytics “fix” works out of the box for anyone, geeks and non-geeks: you don’t need to customize anything nor even know it’s there.

    could you please explain in plain, dummy English *what* the GA surrogate script does? What information does it send to Google, and what, if any, information comes back to the web site?

    It does NOTHING. It’s a dummy “catch all” replacement for the most common Google Analytics functions: it makes the calling pages happy, helping not to break sites, but doesn’t send nor receive anything to/from Google.

  14. #14 kba says:

    ah, now this is really clever. I love the idea, thanks for this feature!

  15. #15 Nan M says:

    Giorgio writes:
    "The good news is that the Google Analytics “fix” works out of the box for anyone, geeks and non-geeks: you don’t need to customize anything nor even know it’s there."

    That, for this plain ordinary user, is the genius of Giorgio’s approach; right from the outset, NoScript has this huge underpinning of **anticipatory** effort to make NoScript transparent for us mums and dads. 99 percent perspiration and one percent inspiration!

    Superb work Prof!

    from your Western Australian fans.

  16. #16 Tomalak says:

    @Giorgio: There still seems to be something not right with either my configuration or the "surrogate scripts"-extension itself.

    Firebig still gives me "_gat is not defined" on a Google Analytics enabled page, and I can’t see why.

    google-analytics.com is in my untrusted list, and I’m using NoScript version 1.8.9.7. What would I need to check to find out what’s wrong?

  17. #17 Giorgio says:

    @Tomalak:
    That might be a known issue.
    Could you check if it persists in 1.8.9.8, and if it does tell me the page where it happens?
    Thanks.

  18. #18 Tomalak says:

    The development version behaves the same. The first page load results in a flurry of JS errors about things not being defined, a subsequent load with (CTRL-F5) results in "_gat is not defined" only.

    Affected page is stackoverflow.com, the statement that triggers the error is: "var pageTracker = _gat._getTracker("UA-5620270-1");".

    Looking at the replacement dummy you define, this definitely should not be happening. Thanks for looking into this! :-)

  19. #19 Giorgio says:

    @Tomalak:
    Weird, it’s working just fine for me (no error at all, just the usual CSS warnings).
    Could you try NoScript Options|Reset and, if the problem persists, Standard Diagnostic?

  20. #20 Tomalak says:

    Just made a blank profile and installed nothing but FireBug and NoScript 1.8.9.7. Sure enough the error disappeared. Restored my whitelist, still works.

    So I guess one of my (nearly) 1000 active extensions interferes. ;-) No. 1 candidate could be GreaseMonkey, will have a closer look. If I find something that indicates an interoperability issue I will post it here.

  21. #21 Tomalak says:

    Hmmm… this looks like an interoperability issue with AdBlock Plus.

    Both "http://easylist.adblockplus.org/abp-tracking-filter.txt" and "http://maltekraus.de/Firefox/adblock_site-specific-elemhiding.txt" trigger the error independently. When I disable them, the page loads smoothly.

    Both seem to contain a filter that interferes here, but I can’t find it because I have no idea where to look. Maybe you have more luck?

  22. #22 Giorgio says:

    @Tomalak:
    The culprit being AdBlock Plus comes to no surprise.
    Recent NoScript versions move their content policy on the bottom of the call list, in order to allow AdBlock Plus to operate first.
    This is done on purpose and by popular demand, in order to make the two extensions to better interoperate by allowing items already blocked by NoScript to be definitely zapped away by AdBlock plus, and vice-versa prevent NoScript placeholders to be shown for objects already filtered by NoScript.
    As a side effect, if AdBlock Plus is blocking Google Analytics (or any other script), NoScript’s surrogates won’t run.
    Good news is that finding the ABP filter to disable is fairly easy: just right-click the ABP icon and select “Blockable items”, then watch for the red lines until you find google-analytics.com.

  23. #23 Bob says:

    You underestimate the non-geeks. I need instructions for dummies to do something to the about:config file. Very clear and very simple. Thanks.

  24. #24 Non-Geek says:

    Giorgio, thanks much for your answer and the update to your post. As #15, Nan M says, NS is popular precisely because it’s good to go OOB — Mom and Pop can use it. Hope you’ll continue to add a dummy-friendly line to high-tech explanations where needed.

    And I have always used the original AdBlock despite all the fans of AB+, *precisely because* AB minds its own business and just takes care of its one task, not interfering with NS or anything else. As you once said, Giogio, "Do one thing and do it well."
    NS +AdBlock = no problems.

  25. #25 Non-Geek says:

    Edit: "Giorgio". I can spell, but I can’t type :-)

  26. #26 Scott Trotter says:

    Thank you very much for doing this. You rock!

  27. #27 Jakub Gedeon says:

    Hmm, here is an idea… would it be possible to automatically generate surrogate scripts? I don’t know how feasible it would be, but you could simply select "Fake Allow" from the menu, and it would read through the script and create a dummy, do-nothing surrogate.

  28. #28 Domenico says:

    […]Giorgio Maone ha integrato una nuova interessante funzione nel suo NoScript: i surrogate scripts. Cosa sono e a cosa servono? Per capirlo, provo a fare un esempio.[…]

  29. #29 Art says:

    Bravissimo Giorgio!

    How did you know this was just the kind of feature I wanted?

    NoScript once again proves that, even on its own, it would be a good-enough reason to switch to FireFox.
    Many Thanks

  30. #30 botted says:

    Giorgio, thanks for the surrogate scripts feature and for applying it to “Google Analytics.” There is another tracker also with market share more than sufficient to form the basis for full profiles on any user of the internet, Omniture’s SiteCatalyst. http://www.slideshare.net/shamel67/emetrics-industry-insight-washington-dc-081020-presentation

    It’s 1st party script and the URI path often includes “/s_code.js” but not always ( http://www.ford.com/js/s.js ). ABP Tracking Filter for Adblock Plus no longer blocks it at all because doing so reportedly causes trouble when signing onto certain websites.

    If you could figure out a way to stop SiteCatalyst, standard and non-standard URI paths, without the use of a message content filter that wildly slows the loading of every webpage, the millions who use ABP Tracking Filter and NoScript because they don’t want to provide such tracking data would appreciate it surely. (Even stopping SiteCatalyst on its two most popular URI paths would be a help.)

  31. #31 hakre says:

    Nice! Thanks! I strongly dislike this epidemic analytics plague and I as well strongly dislike those webdevs that are so dumb to not test their sites w/o javascript or certain scripts disabled. Now I do not need to care about these negative feelings any longer!

  32. #32 James says:

    Thank you for adding this. This is an excellent feature. I *hate* it when designers force me to pretty much select "temporarily allow all" just to view a webpage. That is a horrible design choice - and not just because it inconveniences. Many blind people who use screen readers are essentially locked out from using those websites.

    Even burgerking.com is unusable without Javascript enabled (unless you use Google to bypass the Javascript nonsense).

  33. #33 Shane says:

    This is a great feature, one thing that I found is you can add additional sites to the Google Analytics
    e.g.
    *.google-analytics.com;*.someothersite.com;*.etc.com

  34. #34 Firefox Add-On NoScript Updated to Version 1.9.0.6 | Infosecurity.US says:

    […] Enhanced page-level script surrogates. […]

  35. #35 Noone says:

    SO do I have to add the parameters in about:config? Like noscript.surrogates.enabled, etc…?

  36. #36 Giorgio says:

    @Noone:
    Yes, you have to.

  37. #37 Uli says:

    It’s noscript.surrogate.enabled, not noscript.surrogates.enabled ;) In my 1.9.08 it is automatically enabled.

  38. #38 Giorgio says:

    @Uli:
    Fixed, thanks.

  39. #39 Gym says:

    Thanks for this info. At least there is a solution for individuals, but I wonder if anyone has done anything about this at a corporate level. We started blocking Google Analytics on our 40,000 user network a couple of years ago, after a serious GA-related problem we experienced over a week or two. One day, lots of users started getting redirected to one particular e-commerce site (rapidracking.com) when they tried to access many normal everyday sites. Pretty soon the WWW became unusable, except for a few websites that always worked fine. We were also performing a fairly effective DoS on rapidracking.com. Clearing caches on clients and on various proxy servers in the chain got us working for a while, and we verified that there was no DNS poisoning going on. Eventually we discovered that the problem always started re-appearing just after the proxy logs showed a standard get request to www.google-analytics.com. So we tried blocking requests to the google-analytics.com domain at the firewall, and hey presto! Problem disappeared. We were never able to ascertain what exactly we were pulling down from GA that caused the redirection. Maybe there was a pool of compromised GA servers, maybe very selective cache poisoning, maybe our ISP lying to us about no other customers having problems. But it explained why some sites (non-GA ones) worked fine throughout. We’ve never removed that block, and that may explain why we get the occasional site that just won’t work through the firewall the way others do.

  40. #40 orngjce223 says:

    A similar fix might be in order for yieldmanager.net

    Just saying

  41. #41 Giorgio says:

    @orngjce:
    Done in NoScript 1.9.0.92.

  42. #42 Noone says:

    Wow! Very interesting stuff Gym. It looks like this surrogate function might be a wise idea to have. I’m thinking some maleware can utilize a tracker URL - possibly their own- to compromise FF. That "Nosuch method" is ingenious. LOL

  43. #43 JoeP says:

    To explain surrogates:

    The surrogates idea is akin to stubbing out a function, or commenting out lines of code. In coding, if one has a function one knows is not working for some reason (say for example it is scrambling some global variable unexpectedly, or some other unforseen side effect), one can write a short, "surrogate" function that does nothing more than the expected things, such as always returning 0 or true or whatever else is a "good" return for callers. Similarly, if problematic code is identified somewhere in the middle of a function, one can put comment characters before or around such misbehaving lines, thus making those lines not execute.

    Similarly, NS provides a facility to substitute Java/ECMAscript for scripts one finds objectionable for whatever reasons. Such replacements can do the bare minimum the main page expects, and bypass unwanted behaviors.

    To say that GA (or any other) surrogate scripts do nothing is sort of a misnomer; it DOES do something, in that it makes our Web experiences better! Thank you all who worked on this.

  44. #44 JP says:

    In case you’re looking for "new challenges", the following site goes into an endless update loop if I don’t allow GA:

    http://traffic.511.org/traffic_map.asp

    I let them know what was causing the issue, but they don’t seem to be inclined to fix it.

    thanks…

  45. #45 jj says:

    great tool!
    im always happy to read the changelogs^^

    thx from your swiss fans

  46. #46 khopesh says:

    It looks like NoScript (1.9.9.14) has this code by default, plus a little extra. However, it doesn’t work for me. I presume that’s because AdBlock Plus is nabbing it first?

  47. #47 Giorgio says:

    @khopesh:
    Likely. Just whitelist it in ABP.

  48. #48 Matt says:

    For people who find CS Lite under powered and want something that works more like No Script, you might try Cookie Monster. I’ve found that Cookie Monster better handles the issues that caused trouble for me with CS Lite. YMMV. I vaguely recall fooling with the options to get it to work the way that I wanted (e.g. with second level domain names on).

  49. #49 hackademix.net » NoScript against Pop-unders says:

    […] some time now NoScript has been providing a page-level script surrogate to kill imagefap.com’s pop-unders. Actually, since most recent NoScript versions execute […]

  50. #50 Nick says:

    Good grief, third try is the charm I hope:

    I have a specific issue I think NoScript surrogates can help with but I’m having a hard time getting it to work. If you have any ideas I’d love to hear them.

    There is a page which does a check to see if it has been opened in a new window. It does this in a page-level <script> element located in the <head> of the page. Inside this script element it does a check and then runs

    document.write(’<META HTTP-EQUIV="refresh" CONTENT="0; URL=notallowed.html">’);

    I’ve gotten the page-level surrogate script to run, but haven’t been able to figure out a good way to cancel this out. I’ve tried removing the element, but when the surrogate runs the script element doesn’t seem to even exist in DOM yet. I’ve tried overriding the document.write function so it doesn’t do anything, but that doesn’t seem to carry over into the page scripts.

    Since the browser begins the redirect as soon as the <script> element is parsed, so the only way I can think to accomplish this is with NoScript’s surrogates (since they happen early enough) but I’m wondering if maybe they happen *too* early?

    Any tips would be great!

  51. #51 Giorgio says:

    @Nick:
    This one works for me:

    document.write = function(s){ if (!/<meta/i.test(s)) document.__proto__.write.apply(this, arguments) };
    
  52. #52 Nick says:

    Sorry for my slow response, but that seems to work perfectly. Thanks for the help and the fast reply!

  53. #53 hackademix.net » Google Analytics Opt-Out Snake Oil says:

    […] in fact, almost one year and half ago, this very issue prompted the development of NoScript’s Script Surrogates feature, which prevents the breakage by “emulating” the blocked script with dummy […]

Bad Behavior has blocked 7330 access attempts in the last 7 days.