Sooner or later you, dear NoScript user, may face this puzzle: you already allowed every single script source looking "legitimate" on a certain page, but the damn thing stubbornly refuses to work as it should. Then, in a moment of enlightenment, you dig inside your Untrusted menu, and there you find google-analytics.com. You put it there long ago because you don't like to be tracked, but now you cross your fingers and temporarily allow it... et voilà , the page starts behaving!
Now, you may ask why the hell a web site requires Google Analytics scripts to be enabled for providing its basic features, and you might be right: no reason for that. On the other hand, a growing number of web sites leverage Google Analytics for more than just tracking page views and navigation: they also try to collect finer grained usage data about some specific features of theirs. Therefore they call functions or reference objects from http://www.google-analytics.com/ga.js from their own "1st party" specific scripts, e.g. when you click a certain button: if google-analytics.com is blocked by NoScript (or by AdBlock Plus, or by your hosts file, for the matter), the 1st party code referencing it will obviously fail and the button will stay dead. You can be hinted about Google Analytics being the culprit by opening Tools|Error Console and watching for errors like "urchinTracker is not defined" or "_gat is not definedâ€.
So far, all you could do about that was allowing google-analytics.com. But latest NoScript (1.8.9.7 or above) implements a new feature, called Surrogate Scripts, which works around this problem out of the box and is customizable enough to cope with similar 3rd party script issues in the future. How does it work? Very simple: whenever an external script is blocked, NoScript checks if its URL matches a certain pattern, and if it does an alternate user-provided surrogate script gets executed instead, in the context of the loading page. There you can define surrogates for any required object or function.
There's no UI for this feature (yet?), but its intended audience is likely geeky enough not no need one.
You can specify as many URL/surrogate mappings as you want, by creating a couple of about:config preference entries under the noscript.surrogate root.
The built-in Google Analytics mapping can be regarded as a reference:
- noscript.surrogate.ga.sources: *.google-analytics.com
- noscript.surrogate.ga.replacement: var _0=function(){};with(window)urchinTracker=_0,_gat={_getTracker:function(){return {__noSuchMethod__:_0}}}
var _0=function(){};with(window)urchinTracker=_0,pageTracker={_setDomainName:_0,_trackPageview:_0,_initData:_0},_gat={_getTracker:function(){return window.pageTracker}}*
If you want to exempt some pages from this replacement (e.g. because they already provide a graceful fallback for missing external scripts), you can add an URL pattern to the noscript.surrogate.ga.exceptions preference, e.g. *.mozilla.org *.mozilla.com. Script Surrogates can be disabled globally by setting noscript.surrogate.enabled to false.
Happy hacking :)
*Update
Jesse Andrew told me about some Google Analytics API not covered by the original surrogate, so rather than trying to find out every possible tracker method, present and future, I decided to catch them all by exploiting a nifty Mozilla JavaScript feature, i.e. __noSuchMethod__. Please get 1.8.9.8 (latest development build) if you want to this more reliable approach right now.
Update 2
Since I've been asked by concerned non-geeks (especially those who can't read JavaScript code) what exactly the Google Analytics surrogate does, here's a plain English description: the Google Analytics surrogate script does NOTHING. It’s a dummy “catch all†replacement for the most common Google Analytics functions: it makes the calling pages happy, helping not to break sites, but doesn’t send nor receive anything to/from Google.
Update 3
NoScript now supports a second kind of script surrogates, called "Page-level surrogates": if the sources patterns start with the "@" character, the replacement scripts are executed before the matched page starts to be parsed and before any other script can run. This allows preemptive patching of the loading page, similarly to GreaseMonkey but in a more pervasive way, since GreaseMonkey's scripts can run only when the DOM is already parsed (i.e. after page scripts already run).
January 25th, 2009 at 8:54 pm
Perhaps add _trackEvent to the included GA surrogate script - event tracking is out of beta and recommended to be used where people would use page tracking before.
http://code.google.com/apis/analytics/docs/eventTrackerGuide.html
January 25th, 2009 at 9:01 pm
@Jesse:
Thanks. Should it be a pageTracker method?
More in general, is there a concise documentation about that?
January 25th, 2009 at 9:26 pm
@Giorgio,
yep, _trackEvent is a method that the pageTracker object provides.
January 25th, 2009 at 10:03 pm
@Jesse:
Eh, I looked at the API page and there were far too many methods to be covered. So I decided to go the smart way ;)
January 25th, 2009 at 10:38 pm
This is brilliant. Seriously. If you weren't doing this, I don't know who would. Thank you so much. (I've donated, but under another name -- this is my pseudonym). Now, how about something else I've always wondered about in the "synthetic" realm: cookies. Wouldn't it be great to always allow doubleclick.com (and other trackers), but instead returning the cookie they've stored on your machine, return a randomized cookie whenever a certain tracking site requests one.
Thank you again for a great add-on.
Dave
January 25th, 2009 at 11:00 pm
@Dave:
Thanks, you're too good.
I don't want to fiddle with cookies for non-security purposes, because there are already excellent extensions for cookie management.
I, for instance, use CS Lite. Did you try that?
January 26th, 2009 at 12:40 am
Would it be possible to avoid mapping over any hypothetical global object other than those from the blocked and replaced code and still accomplish the same? Unfortunately, some sites do use rather silly/meaningless variable names, such as _. Would this surrogate script still work without raising any security issues if wrapped in an anonymous function call, so _ would be a local variable, discarded after function call finished? In JS, when a global object is assigned from a local object, it becomes a copy of (and not a reference to) the local, doesn't it?
(function(){var _0=function(){};with(window)urchinTracker=_0,_gat={_getTracker:function(){return {__noSuchMethod__:_0}}}})()
BTW, this is exactly what I was asking for a while back in http://userscripts.org/topics/2293 -- but GM doesn't provide a safe method (that I know of) to run script in the context of the page, before the page's own scripts run.
January 26th, 2009 at 12:49 am
@MacOtaku:
it's already wrapped inside an anonymous function behind the curtains, if your concern was about "_0" polluting the global namespace.
In facts, whatever you put in a "noscript.surrogate.xyz.replacement" preference gets wrapped inside a block like this:
try { (function() { ...replacement code here... })() } catch(e) {}Therefore, if you prefix your vars with "var" or "let", you're not going to pollute anything.
January 26th, 2009 at 1:37 am
@Giorgio #6:
Thanks, I have tried CS Lite and a few others, but I kept running into the same problem with functionality as experienced with calls to scripts, namely the "Oh, it appears you have cookies disabled; you need to enable them to use this site" problem. I completely understand your reluctance to broaden NoScript's reach, and thank you again for your constant effort in this area. I know Leo LaPorte and Steve Gibson think highly of you as well, and share that frequently on their podcasts.
Dave
January 26th, 2009 at 1:38 am
Or just get over yourself and remove NoScript entirely.
January 26th, 2009 at 5:20 am
"Or just get over yourself and remove NoScript entirely."
Why would I do that, when it does exactly what I want it to do?
January 26th, 2009 at 9:37 am
Giorgio, I think you might underestimate the number of non-geek (non-code-reading) users who use NS and want to take full advantage of its features. Before I disable surrogates entirely, could you please explain in plain, dummy English *what* the GA surrogate script does? What information does it send to Google, and what, if any, information comes back to the web site?
@Dave: Yes, but if you remember, Gibson almost gave up on NS because he found the notifications annoying. Members of his newsgroup had to tell him that he could customize or disable notifications, after which he became a big fan.
Which proves my point: if a guy with Gibson's level of knowledge can't figure it out, the rest of us are clueless. A plain-English explanation of major changes, along with the tech explanation, would be really cool.
January 26th, 2009 at 10:12 am
@Non-Geek:
the "geeky" part of this feature is about writing or customizing surrogate scripts, therefore there's not much I can add to simplify it if you can't code.
The good news is that the Google Analytics "fix" works out of the box for anyone, geeks and non-geeks: you don't need to customize anything nor even know it's there.
It does NOTHING. It's a dummy "catch all" replacement for the most common Google Analytics functions: it makes the calling pages happy, helping not to break sites, but doesn’t send nor receive anything to/from Google.
January 26th, 2009 at 2:50 pm
ah, now this is really clever. I love the idea, thanks for this feature!
January 26th, 2009 at 4:22 pm
Giorgio writes:
"The good news is that the Google Analytics “fix” works out of the box for anyone, geeks and non-geeks: you don’t need to customize anything nor even know it’s there."
That, for this plain ordinary user, is the genius of Giorgio's approach; right from the outset, NoScript has this huge underpinning of **anticipatory** effort to make NoScript transparent for us mums and dads. 99 percent perspiration and one percent inspiration!
Superb work Prof!
from your Western Australian fans.
January 26th, 2009 at 8:35 pm
@Giorgio: There still seems to be something not right with either my configuration or the "surrogate scripts"-extension itself.
Firebig still gives me "_gat is not defined" on a Google Analytics enabled page, and I can't see why.
google-analytics.com is in my untrusted list, and I'm using NoScript version 1.8.9.7. What would I need to check to find out what's wrong?
January 26th, 2009 at 8:58 pm
@Tomalak:
That might be a known issue.
Could you check if it persists in 1.8.9.8, and if it does tell me the page where it happens?
Thanks.
January 26th, 2009 at 9:10 pm
The development version behaves the same. The first page load results in a flurry of JS errors about things not being defined, a subsequent load with (CTRL-F5) results in "_gat is not defined" only.
Affected page is stackoverflow.com, the statement that triggers the error is: "var pageTracker = _gat._getTracker("UA-5620270-1");".
Looking at the replacement dummy you define, this definitely should not be happening. Thanks for looking into this! :-)
January 26th, 2009 at 10:14 pm
@Tomalak:
Weird, it's working just fine for me (no error at all, just the usual CSS warnings).
Could you try NoScript Options|Reset and, if the problem persists, Standard Diagnostic?
January 26th, 2009 at 11:02 pm
Just made a blank profile and installed nothing but FireBug and NoScript 1.8.9.7. Sure enough the error disappeared. Restored my whitelist, still works.
So I guess one of my (nearly) 1000 active extensions interferes. ;-) No. 1 candidate could be GreaseMonkey, will have a closer look. If I find something that indicates an interoperability issue I will post it here.
January 26th, 2009 at 11:17 pm
Hmmm... this looks like an interoperability issue with AdBlock Plus.
Both "http://easylist.adblockplus.org/abp-tracking-filter.txt" and "http://maltekraus.de/Firefox/adblock_site-specific-elemhiding.txt" trigger the error independently. When I disable them, the page loads smoothly.
Both seem to contain a filter that interferes here, but I can't find it because I have no idea where to look. Maybe you have more luck?
January 27th, 2009 at 12:00 am
@Tomalak:
The culprit being AdBlock Plus comes to no surprise.
Recent NoScript versions move their content policy on the bottom of the call list, in order to allow AdBlock Plus to operate first.
This is done on purpose and by popular demand, in order to make the two extensions to better interoperate by allowing items already blocked by NoScript to be definitely zapped away by AdBlock plus, and vice-versa prevent NoScript placeholders to be shown for objects already filtered by NoScript.
As a side effect, if AdBlock Plus is blocking Google Analytics (or any other script), NoScript's surrogates won't run.
Good news is that finding the ABP filter to disable is fairly easy: just right-click the ABP icon and select "Blockable items", then watch for the red lines until you find google-analytics.com.
January 27th, 2009 at 3:39 am
You underestimate the non-geeks. I need instructions for dummies to do something to the about:config file. Very clear and very simple. Thanks.
January 27th, 2009 at 3:41 am
Giorgio, thanks much for your answer and the update to your post. As #15, Nan M says, NS is popular precisely because it's good to go OOB -- Mom and Pop can use it. Hope you'll continue to add a dummy-friendly line to high-tech explanations where needed.
And I have always used the original AdBlock despite all the fans of AB+, *precisely because* AB minds its own business and just takes care of its one task, not interfering with NS or anything else. As you once said, Giogio, "Do one thing and do it well."
NS +AdBlock = no problems.
January 27th, 2009 at 3:44 am
Edit: "Giorgio". I can spell, but I can't type :-)
January 27th, 2009 at 5:25 am
Thank you very much for doing this. You rock!
January 27th, 2009 at 6:19 pm
Hmm, here is an idea... would it be possible to automatically generate surrogate scripts? I don't know how feasible it would be, but you could simply select "Fake Allow" from the menu, and it would read through the script and create a dummy, do-nothing surrogate.
January 28th, 2009 at 2:05 pm
[...]Giorgio Maone ha integrato una nuova interessante funzione nel suo NoScript: i surrogate scripts. Cosa sono e a cosa servono? Per capirlo, provo a fare un esempio.[...]
January 28th, 2009 at 2:48 pm
Bravissimo Giorgio!
How did you know this was just the kind of feature I wanted?
NoScript once again proves that, even on its own, it would be a good-enough reason to switch to FireFox.
Many Thanks
January 28th, 2009 at 11:20 pm
Giorgio, thanks for the surrogate scripts feature and for applying it to “Google Analytics.” There is another tracker also with market share more than sufficient to form the basis for full profiles on any user of the internet, Omniture’s SiteCatalyst. http://www.slideshare.net/shamel67/emetrics-industry-insight-washington-dc-081020-presentation
It’s 1st party script and the URI path often includes “/s_code.js” but not always ( http://www.ford.com/js/s.js ). ABP Tracking Filter for Adblock Plus no longer blocks it at all because doing so reportedly causes trouble when signing onto certain websites.
If you could figure out a way to stop SiteCatalyst, standard and non-standard URI paths, without the use of a message content filter that wildly slows the loading of every webpage, the millions who use ABP Tracking Filter and NoScript because they don’t want to provide such tracking data would appreciate it surely. (Even stopping SiteCatalyst on its two most popular URI paths would be a help.)
January 31st, 2009 at 12:10 am
Nice! Thanks! I strongly dislike this epidemic analytics plague and I as well strongly dislike those webdevs that are so dumb to not test their sites w/o javascript or certain scripts disabled. Now I do not need to care about these negative feelings any longer!
February 8th, 2009 at 1:31 am
Thank you for adding this. This is an excellent feature. I *hate* it when designers force me to pretty much select "temporarily allow all" just to view a webpage. That is a horrible design choice - and not just because it inconveniences. Many blind people who use screen readers are essentially locked out from using those websites.
Even burgerking.com is unusable without Javascript enabled (unless you use Google to bypass the Javascript nonsense).
February 9th, 2009 at 9:01 pm
This is a great feature, one thing that I found is you can add additional sites to the Google Analytics
e.g.
*.google-analytics.com;*.someothersite.com;*.etc.com
February 23rd, 2009 at 9:05 pm
[...] Enhanced page-level script surrogates. [...]
February 27th, 2009 at 1:30 pm
SO do I have to add the parameters in about:config? Like noscript.surrogates.enabled, etc...?
February 27th, 2009 at 1:50 pm
@Noone:
Yes, you have to.
March 5th, 2009 at 3:19 pm
It's noscript.surrogate.enabled, not noscript.surrogates.enabled ;) In my 1.9.08 it is automatically enabled.
March 5th, 2009 at 11:06 pm
@Uli:
Fixed, thanks.
March 6th, 2009 at 11:44 pm
Thanks for this info. At least there is a solution for individuals, but I wonder if anyone has done anything about this at a corporate level. We started blocking Google Analytics on our 40,000 user network a couple of years ago, after a serious GA-related problem we experienced over a week or two. One day, lots of users started getting redirected to one particular e-commerce site (rapidracking.com) when they tried to access many normal everyday sites. Pretty soon the WWW became unusable, except for a few websites that always worked fine. We were also performing a fairly effective DoS on rapidracking.com. Clearing caches on clients and on various proxy servers in the chain got us working for a while, and we verified that there was no DNS poisoning going on. Eventually we discovered that the problem always started re-appearing just after the proxy logs showed a standard get request to www.google-analytics.com. So we tried blocking requests to the google-analytics.com domain at the firewall, and hey presto! Problem disappeared. We were never able to ascertain what exactly we were pulling down from GA that caused the redirection. Maybe there was a pool of compromised GA servers, maybe very selective cache poisoning, maybe our ISP lying to us about no other customers having problems. But it explained why some sites (non-GA ones) worked fine throughout. We've never removed that block, and that may explain why we get the occasional site that just won't work through the firewall the way others do.
March 8th, 2009 at 8:49 am
A similar fix might be in order for yieldmanager.net
Just saying
March 10th, 2009 at 1:53 am
@orngjce:
Done in NoScript 1.9.0.92.
March 11th, 2009 at 6:36 pm
Wow! Very interesting stuff Gym. It looks like this surrogate function might be a wise idea to have. I'm thinking some maleware can utilize a tracker URL - possibly their own- to compromise FF. That "Nosuch method" is ingenious. LOL
October 28th, 2009 at 6:13 pm
To explain surrogates:
The surrogates idea is akin to stubbing out a function, or commenting out lines of code. In coding, if one has a function one knows is not working for some reason (say for example it is scrambling some global variable unexpectedly, or some other unforseen side effect), one can write a short, "surrogate" function that does nothing more than the expected things, such as always returning 0 or true or whatever else is a "good" return for callers. Similarly, if problematic code is identified somewhere in the middle of a function, one can put comment characters before or around such misbehaving lines, thus making those lines not execute.
Similarly, NS provides a facility to substitute Java/ECMAscript for scripts one finds objectionable for whatever reasons. Such replacements can do the bare minimum the main page expects, and bypass unwanted behaviors.
To say that GA (or any other) surrogate scripts do nothing is sort of a misnomer; it DOES do something, in that it makes our Web experiences better! Thank you all who worked on this.
November 18th, 2009 at 5:32 am
In case you're looking for "new challenges", the following site goes into an endless update loop if I don't allow GA:
http://traffic.511.org/traffic_map.asp
I let them know what was causing the issue, but they don't seem to be inclined to fix it.
thanks...
November 18th, 2009 at 11:36 am
great tool!
im always happy to read the changelogs^^
thx from your swiss fans
December 2nd, 2009 at 12:06 am
It looks like NoScript (1.9.9.14) has this code by default, plus a little extra. However, it doesn't work for me. I presume that's because AdBlock Plus is nabbing it first?
December 2nd, 2009 at 1:13 am
@khopesh:
Likely. Just whitelist it in ABP.
December 16th, 2009 at 5:29 pm
For people who find CS Lite under powered and want something that works more like No Script, you might try Cookie Monster. I've found that Cookie Monster better handles the issues that caused trouble for me with CS Lite. YMMV. I vaguely recall fooling with the options to get it to work the way that I wanted (e.g. with second level domain names on).
January 6th, 2010 at 11:36 pm
[...] some time now NoScript has been providing a page-level script surrogate to kill imagefap.com’s pop-unders. Actually, since most recent NoScript versions execute [...]
January 15th, 2010 at 5:02 am
Good grief, third try is the charm I hope:
I have a specific issue I think NoScript surrogates can help with but I'm having a hard time getting it to work. If you have any ideas I'd love to hear them.
There is a page which does a check to see if it has been opened in a new window. It does this in a page-level <script> element located in the <head> of the page. Inside this script element it does a check and then runs
document.write('<META HTTP-EQUIV="refresh" CONTENT="0; URL=notallowed.html">');
I've gotten the page-level surrogate script to run, but haven't been able to figure out a good way to cancel this out. I've tried removing the element, but when the surrogate runs the script element doesn't seem to even exist in DOM yet. I've tried overriding the document.write function so it doesn't do anything, but that doesn't seem to carry over into the page scripts.
Since the browser begins the redirect as soon as the <script> element is parsed, so the only way I can think to accomplish this is with NoScript's surrogates (since they happen early enough) but I'm wondering if maybe they happen *too* early?
Any tips would be great!
January 15th, 2010 at 7:38 pm
@Nick:
This one works for me:
document.write = function(s){ if (!/<meta/i.test(s)) document.__proto__.write.apply(this, arguments) };January 19th, 2010 at 12:33 am
Sorry for my slow response, but that seems to work perfectly. Thanks for the help and the fast reply!
May 26th, 2010 at 11:09 pm
[...] in fact, almost one year and half ago, this very issue prompted the development of NoScript’s Script Surrogates feature, which prevents the breakage by “emulating” the blocked script with dummy [...]