Matter & Energy

Copy CSS styles without additional HTTP requests

michael — Fri, 10 Jun 2011 21:18:33 +0000

I’m working on an editor product for Plone that has a realtime preview in an iframe. In order to style the preview, I needed the same stylesheets loaded in that iframe as were in the parent window, but iframes don’t directly provide a way to do that. Enter Javascript.

A straightforward approach (and the first one I tried) would be to copy the stylesheet elements from the parent window to the iframe. To do that, I put this code in the of the child document:

window.onload = function() {
  // Get a ref to the parent document's stylesheets.
  var parentdoc_ss = window.parent.document.styleSheets;
  var head = document.head;
  for (var i=0; i
The problem with that approach is that it can cause an additional HTTP request for each stylesheet. In practice, the original stylesheets may already be cached in the browser, on the server, or both. But even so, let’s see if we can copy the styles without making any HTTP requests.
window.onload = function() {
  // Get a ref to the parent document's stylesheets.
  var parentdoc_ss = window.parent.document.styleSheets;
  // Grab a stylesheet from the current document.
  if (document.styleSheets.length<1) {
    // If one doesn't exist, create it.
    document.head.appendChild(document.createElement("style"));
  }
  var ss = document.styleSheets[document.styleSheets.length-1];
  var rule_index = 0; // We'll need this to help us add the rules in order.
  for (var i=0; i
This whole time I’ve been referencing iframes, but it really should work for any documents that share the same origin. While I only tested on Firefox 4, I wouldn’t be surprised if it worked on WebKit, too. It would definitely need tweaking for Internet Explorer.
While it’s easy enough to manipulate individual styles with Javascript,  even with JQuery, I had to do a fair amount of research and  trial-and-error before I had this nailed down on Firefox. I haven’t found  any other documentation for copying styles without reloading the stylesheets themselves, so I hope someone finds this useful, at least for theory. If you have any suggestions or find a better approach, please leave a comment.

Plone with a clean development environment on Leopard

michael — Sun, 05 Jun 2011 18:29:22 +0000

I still have a G5 PowerMac. It runs Mac OS X 10.5 Leopard. I know it’s old, but since it’s a PowerPC I can’t upgrade the MacOS. It’s still very relevant for doing Plone development, as Apple has a long history of installing old libraries anyway, so managing that stuff is just part of life with Apple. Here’s how I set it up to do development for Plone in a clean way:

When you install Plone using the installers with default options you get a virtual environment. Changes you make to your buildout in there generally don’t affect your operating system. When you want to develop a Plone product or do core development, however, you’ll be working from a Subversion working copy, not a normally-installed Plone setup. The development environment instructions gets you set up with some nice tools, but there are a couple of additional steps you can do to keep your development environment nice and tidy.

Things you’ll need to have a sane development situation on OS X:

XCode Tools – Version 3.1.4 seems to be the last PPC-compatible version. It’s there, but you’ll need to register and dig for it. XCode 3.1.4 includes…
Subversion – version 1.4.4, [update: Plone now uses git] which seems to be just new enough to work. If you are reading this later on and 1.4.4 doesn’t work anymore, install a new version using…
Homebrew – or another package manager of some kind. I’ve used MacPorts, Fink, Gentoo Prefix, and now I’m on the PowerPC branch of Homebrew. I got this idea from David Glick. However, I almost gave up when complications arose from removing MacPorts. I’m sure you can adapt these instructions to whatever package manager you want. I like Homebrew because it doesn’t duplicate what the OS provides, such as…
Python – but Plone doesn’t work with Python 2.5, so you’ll need to install Python manually, because Homebrew doesn’t duplicate what the OS provides. You’ll want Python 2.6 for Plone 4 and Python 2.4 for Plone 3. For 2.4 you’ll want to compile it with a special MACOSX_DEPLOYMENT_TARGET=10.5 flag. Whatever Python you have, you’ll want…
PIP – You can get by with easy_install, but despite OS X having surprisingly good segregation for site-packages, who wants to live without uninstall? Just easy_install pip, and never use easy_install again. Then use pip to install…
virtualenv – Creates silos in which to put Plone setups so they don’t contaminate your system and your system doesn’t contaminate your Plone. You can pip install this with the system Python and point it to a specific Python version later, when you use it.

How to set up a virtualized Plone development environment:

Set up a virtual environment.

$ virtualenv -v -p /usr/local/bin/python2.6 --no-site-packages --distribute ./py26env

Check out the Plone development buildout.

$ svn co  https://svn.plone.org/svn/plone/buildouts/plone-coredev/branches/4.1/ ./plone41devel

[Update: Plone now uses git.]

Change directories into your working copy.

$ cd ./plone41devel

Invoke bootstrap.py with the virtualenv’s python.

$ ../py26env/bin/python bootstrap.py && bin/buildout

Set up a local buildout configuration

The buildout configuration files for Plone are in Subversion. If you are a core developer and make a change you run the risk of accidentally committing that change to the core. A file named ‘local.cfg’ should be in svn:ignore, which means you can create it and edit it to your heart’s content without worrying about it getting caught up in your feverish fixing. It needs to hook back into the real buildout.cfg though, so it should look something like this:

[buildout]
extends =
	buildout.cfg

Then you just run bin/buildout -c local.cfg and after that you can use bin/develop rb.

Fixing PIL and lxml for OS X

Plone 4.1 needs PIL and lxml to be truly happy, but OS X makes it particularly challenging to install these. Fortunately, others have solved this problem, so I’m just going to give you my local.cfg that takes care of it. PIL is easy: Alex Clark’s Pillow does all the work for us. The real trick is the redefinition of the [instance] part with an arbitrary dependency on [lxml]. That’s necessary to force the lxml part to be run before anything else, which is the only way it will work.

[buildout]
extends =
	buildout.cfg
eggs +=
	Pillow
parts +=
	lxml

[lxml]
recipe = z3c.recipe.staticlxml
egg = lxml
build-libxslt = true
build-libxml2 = true
static-build = true
libxml2-url = ftp://xmlsoft.org/libxslt/LATEST_LIBXML2
libxslt-url = ftp://xmlsoft.org/libxslt/LATEST_LIBXSLT

[instance]
recipe = plone.recipe.zope2instance
user = admin:admin
http-address = 8080
eggs = ${buildout:eggs}
environment-vars =
	zope_i18n_compile_mo_files true
	_dummy ${lxml:egg}

Finally, if you want to check out a package, you can modify [sources] in local.cfg. Just add a sources part:

[buildout]
extends =
…
[sources]
my.product = svn svn://uri

and run bin/develop co my.product && bin/develop rb to check out and install most eggifiable repositories.

Plone 4.1 with Apache and mod_wsgi (sorta)

michael — Tue, 31 May 2011 02:03:05 +0000

Update 2011-06-01: Still not working perfectly, but I did manage to clean up a couple steps by using more of mod_wsgi’s bells and whistles:

Using a .pth file and the WSGIPythonPath directive you don’t need to manually load all the eggs into sys.path
Using Daemon Mode per Graham Dumpleton’s comment. To be honest I haven’t noticed any difference caused by this change…

Issues:

The site is still slow (despite using Daemon Mode)

I get signal errors. This is a known problem when trying to serve signal-dependent python stuff via mod_wsgi, but I’m not sure it causes any real problems.

[Wed Jun 01 17:59:40 2011] [warn] mod_wsgi (pid=12739): Callback registration for signal 10 ignored.
[Wed Jun 01 17:59:40 2011] [warn]   File "/var/www/Plone-ZEO-4.1rc2/zeocluster/zope2.wsgi", line 3, in
[Wed Jun 01 17:59:40 2011] [warn]     application = make_wsgi_app(None, '/var/www/Plone-ZEO-4.1rc2/zeocluster/parts/client1/etc/zope.conf')
[Wed Jun 01 17:59:40 2011] [warn]   File "/var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/Zope2-2.13.7-py2.6.egg/Zope2/Startup/run.py", line 68, in make_wsgi_app
[Wed Jun 01 17:59:40 2011] [warn]     starter.prepare()
[Wed Jun 01 17:59:40 2011] [warn]   File "/var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/Zope2-2.13.7-py2.6.egg/Zope2/Startup/__init__.py", line 90, in prepare
[Wed Jun 01 17:59:40 2011] [warn]     self.registerSignals()
[Wed Jun 01 17:59:40 2011] [warn]   File "/var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/Zope2-2.13.7-py2.6.egg/Zope2/Startup/__init__.py", line 340, in registerSignals
[Wed Jun 01 17:59:40 2011] [warn]     self.cfg.trace])
[Wed Jun 01 17:59:40 2011] [warn]   File "/var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/Zope2-2.13.7-py2.6.egg/Signals/Signals.py", line 115, in registerZopeSignals
[Wed Jun 01 17:59:40 2011] [warn]     SignalHandler.registerHandler(SIGUSR1, showStacks)
[Wed Jun 01 17:59:40 2011] [warn]   File "/var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/Zope2-2.13.7-py2.6.egg/Signals/SignalHandler.py", line 37, in registerHandler
[Wed Jun 01 17:59:40 2011] [warn]     signal.signal(signum, self.signalHandler)

In many cases I can’t POST to the site. I can log in as long as it’s not HTTP Basic, but I can’t edit pages or make site-setup configuration changes. Not sure why.

Notes:

For the following instructions I did every step as the user apache, so Plone runs as that user, as does zeo. This may not be the absolute best practice, but it made things a bit simpler.
You can substitute instance in probably every case I used client1.

Install Apache and mod_wsgi. Make sure to specify Python 2.6 for mod_wsgi.
Grab a Plone 4.1 release candidate and install it as the user apache. Use the same Python 2.6 as you did for mod_wsgi.

Create a path configuration file:

$ ( # Do this in a subshell so we don't contaminate the IFS
>   # variable in our normal shell.
> cd /var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs
> eggs=( *.egg )
> IFS=$'\n'
> echo "${eggs[*]}" > mod_wsgi.pth
> )

In your buildout directory create an empty file called ‘zope2.wsgi.in’ as a collective.recipe.template template.~~copy the bin/client1 file into zope2.wsgi.in (because you need all the egg paths)~~

Put these two lines in it:

from Zope2.Startup.run import make_wsgi_app
application = make_wsgi_app(None, '${zope-conf}')

Add a section called wsgi to your buildout.cfg file:

[buildout]
…
parts =
        …
        wsgi
        …
…
[wsgi]
recipe = collective.recipe.template
input = zope2.wsgi.in
output = zope2.wsgi
zope-conf = ${client1:location}/etc/zope.conf

In your apache config:

…
WSGIPythonPath /var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/
WSGIDaemonProcess neon processes=1 threads=1 python-path=/var/www/Plone-ZEO-4.1rc2/buildout-cache/eggs/
WSGIProcessGroup neon
WSGIScriptAlias / /var/www/Plone-ZEO-4.1rc2/zeocluster/zope2.wsgi


        Order allow,deny
        Allow from all

…

Run buildout and start the zeoserver.
I had some problems with HTTP Basic Authentication through WSGI, so I avoided it by starting the client without WSGI for the purpose of logging in to the ZMI to create the Plone site. bin/client1 fg
Log in to http://localhost:8080 and create the Plone site.
Once I had actually created the Plone site, I killed the client1 instance and removed the section from zope.conf
Start apache, and observe the logs as you navigate around your Plone site on port 80.

As I said, my site is slow and a lot of errors appear in the logs, but it’s functional, which is better than I’ve seen it so far. More work on this later.

The Future of the Internet is in the Client and the Cloud

michael — Sat, 28 May 2011 20:59:45 +0000

Commentary on: http://www.zdnet.com/blog/btl/the-future-of-the-internet-its-in-the-app/49512?tag=nl.e539

The Forrester CEO cited in this article strikes me as remarkably naïve
(or maybe badly misquoted). I’m not downplaying the importance of
apps, but 90% of the popular apps on the market are popular because
they exchange data with a web server. “Leaves cloud computing in the
dust”? I don’t think so. The app model depends on cloud computing.

Now, there’s been some interesting development in the area of merging
the concept of apps with websites. Both Google and Mozilla are working
on it from opposite ends: Google with its ChromeOS efforts (oddly,
ChromeOS is diametrically opposite of the space occupied by Android,
but I think Google’s doing that strategically and on purpose) and
Mozilla with its Open Web Applications project.

I’m not sure what approach Microsoft will really take. They’ve made
their position on HTML 5 clear, and that’s a good thing, but as for
apps they seem to just be covering their bases. (They’ve made it
possible to install Android apps on the Windows Mobile platform, for
example.)

I am interested in the app model, definitely, but the Internet isn’t
going to change as drastically as Colony seems to think. The article
does rightly point out that the Internet will use more of the power of
client machines than it has in the past, but that has less to do with
apps than it does with the growth of HTML 5, powerful Javascript
engines like V8 and frameworks like Sencha and JQuery. (Those tend to
be hidden behind app development anyway.)

So I’m following the app discussion carefully, but really focusing my
efforts on the cross-platform frameworks like Mozilla’s OWA, because I
think that’s where the biggest bang for my buck will end up.

How to build a simple shared zeoclient-only setup for Plone 3

michael — Sun, 08 May 2011 15:15:12 +0000

Note: The following is probably very adaptable to Plone 4. In fact, I suspect it’s even easier in Plone 4. But I’m working on Plone 3 right now, so there you go.

This is the second in a two-part guide on how to separate the Plone database server from the client instances. Even with load-balancing proxies, convenient high-availability in Plone always remains just out of reach. As long as the zeoserver and its clients are on the same machine, you have to consider both server and clients in your change control processes. Separation is simplification. I’ve given other reasons to care in my previous post, but as an update, here is the approximate storage size of a single client-only site with a lot of eggs:

$ du -hs *
53M	Python-2.4
73M	Zope-2.10.11-final-py2.4
149M	testcase

Getting started with the clients

Decide how you will structure your filesystem:
- Shared Python 2.4: /opt/plone/Python-2.4
- Shared Zope 2.10: /opt/plone/Zope-2.10.11-final-py2.4
- Buildout for zeo: /opt/plone/testcase
Create the user that the clients will run as. Create the parents of the above paths and give that user ownership. Become that user for the rest of these instructions.
```
useradd plone
mkdir /opt/plone && chown plone /opt/plone
su - plone || sudo -u plone bash
```
Download and unpack the Universal Installer (UI) from Plone.org. Use the UI to install a shared Zope and Python, but nuke the rest of what it installs for now.
```
./install.sh --target=/opt/plone zeo && rm -rf /opt/plone/{buildout-cache,zeocluster}
```

Use the UI to install an actual zeoserver using that shared Python and Zope:

./install.sh --target=/opt/plone/testcase --with-python=/opt/plone/Python-2.4/bin/python --with-zope=/opt/plone/Zope-2.10.11-final-py2.4 zeo

Go to the zeocluster directory and clean out the buildout stuff that is only needed for Plone clients. You really only need bin/buildout, buildout.cfg. If you’re using bash you can do this quickly with extglob:
```
cd /opt/plone/testcase/zeocluster && shopt -s extglob
rm -rf ../buildout-cache/ bin/!(buildout) !(bin|buildout.cfg|versions.cfg) .installed.cfg
```

Create a backup of the buildout file and begin working on it. You can delete almost everything in there, just keep the zope2, zopepy and client parts and fix any broken references. Here’s what mine looks like:

[buildout]
extends = versions.cfg
versions = versions
extensions = buildout.dumppickedversions
zeo-address = 10.0.0.87:8100
client1-address = 8080
client2-address = 8081
eggs = Plone
zcml =
develop =
debug-mode = off
parts =
        zope2
        client1
        client2
        zopepy

[zope2]
recipe = plone.recipe.zope2install
fake-zope-eggs = true
additional-fake-eggs =
        ZConfig
        pytz
location = /opt/plone/Zope-2.10.11-final-py2.4

[client1]
recipe = plone.recipe.zope2instance
zeo-client = true
zeo-address = ${buildout:zeo-address}
effective-user = plone
user = admin:abc
http-address = ${buildout:client1-address}
debug-mode = ${buildout:debug-mode}
verbose-security = ${buildout:debug-mode}
deprecation-warnings = ${buildout:debug-mode}
eggs = ${buildout:eggs}
zcml = ${buildout:zcml}
environment-vars = PYTHON_EGG_CACHE ${buildout:directory}/var/.python-eggs
zope2-location = ${zope2:location}

[client2]
< = client1
http-address = ${buildout:client2-address}

[zopepy]
recipe = zc.recipe.egg
eggs = ${buildout:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy

[versions]
Cheetah = 2.0.1
Paste = 1.7.2
PasteScript = 1.7.3
ZopeSkel = 2.11.1
collective.recipe.backup = 1.1
plone.recipe.command = 1.0
plone.recipe.distros = 1.5
plone.recipe.osxcontroller = 0.3
plone.recipe.precompiler = 0.3
plone.recipe.unifiedinstaller = 0.9
collective.recipe.zope2cluster = 1.0
PasteDeploy = 1.3.3
zc.recipe.egg = 1.2.2

Once you’re satisfied with your buildout configuration, run buildout, fix permissions and try to start the instance. (Note: buildout always crashes the first time I do this because of a setuptools incompatibility. Just run it again if it fails.):
```
bin/buildout || bin/buildout && bin/client1 start && bin/client2 start
```
That’s it! Now to create a new client setup, just repeat steps 5-8, updating the paths and port numbers (zeo-address, clientN-address) to suit.

That’s really all there is to it.

How to build a simple shared zeoserver-only setup for Plone 3

michael — Sun, 08 May 2011 03:39:25 +0000

Note: The following is probably very adaptable to Plone 4. In fact, I suspect it’s even easier in Plone 4. But I’m working on Plone 3 right now, so there you go.

When you first install Plone, you get everything together: server, client, whatever, it just works. I’m sure not everybody knows, or cares, that Zope can be run as a zeoserver distinct from the zeoclient. But there are some reasons to care:

Simplicity: Plone can seem complex, but zeoserver is dead simple. Separating it from the clients is the easiest way to see this.
Flexibility: Need to reboot or migrate a server without site downtime? You can’t, unless you’re using load-balancing proxies. But with zeoserver on a separate machine from the clients, the clients can withstand short outages of the zeoserver without apparent downtime, even if you don’t have a load balancer! (YMMV depending on demand, zeoserver’s reboot time and the clients’ caches.) Moving the clients gets easier, too.
Backups: I don’t need to backup all of Plone. The only important part is the Data.fs (and in Plone 4, the blobs, but that’s for another day). Now my backup scenario can effectively ignore the client servers, and just back up what’s on the server with the ZODBs on it.
Scalability: I’ll let these results speak for themselves:
Before
```
$ du -hs --exclude '*.log' plone
526M     site1
407M     site2
676M     site3
```
After:
```
$ du -hs *
53M	Python-2.4
73M	Zope-2.10.11-final-py2.4
11M	site1
11M	site2
11M	site3
```
Well, that’s not quite fair – the client code isn’t listed there, but it’s no longer an issue for this server.

Hopefully that’s enough to get you interested.

Getting started

Decide how you will structure your filesystem:
- Where the Data.fs will be stored: /srv/plone/testcase/Data.fs
- Shared Python 2.4: /opt/plone/Python-2.4
- Shared Zope 2.10: /opt/plone/Zope-2.10.11-final-py2.4
- Buildout for zeo: /opt/plone/testcase
Create the user that the zeoservers will run as. Create the parents of the above paths and give that user ownership. Become that user for the rest of these instructions.
```
useradd zeo
mkdir /srv/plone && chown zeo /srv/plone
mkdir /opt/plone && chown zeo /opt/plone
su - zeo || sudo -u zeo bash
```
Download and unpack the Universal Installer (UI) from Plone.org. Use the UI to install a shared Zope and Python, but nuke the rest of what it installs for now.
```
./install.sh --target=/opt/plone zeo && rm -rf /opt/plone/{buildout-cache,zeocluster}
```

Use the UI to install an actual zeoserver using that shared Python and Zope:

./install.sh --target=/opt/plone/testcase --with-python=/opt/plone/Python-2.4/bin/python --with-zope=/opt/plone/Zope-2.10.11-final-py2.4 zeo

Go to the zeocluster directory and clean out the buildout stuff that is only needed for Plone clients. You really only need bin/buildout, buildout.cfg. If you’re using bash you can do this quickly with extglob:
```
cd /opt/plone/testcase/zeocluster && shopt -s extglob
rm -rf ../buildout-cache/ bin/!(buildout) !(bin|buildout.cfg|versions.cfg) .installed.cfg
```

Create a backup of the buildout file and begin working on it. You can delete almost everything in there, just keep the zope2 and zeoserver parts and fix any broken references. Here’s what mine looks like:

[buildout]
extends = versions.cfg
versions = versions
extensions = buildout.dumppickedversions
dump-picked-versions-file = picked-versions.cfg
zeo-address = 8100
file-storage = /srv/plone/testcase/Data.fs
parts =
	zope2
	zeoserver

[zope2]
recipe = plone.recipe.zope2install
url = ${versions:zope2-url}
fake-zope-eggs = true
additional-fake-eggs =
	ZConfig
	pytz

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = ${buildout:zeo-address}
effective-user = zeo
file-storage = ${buildout:file-storage}

Once you’re satisfied with your buildout configuration, run buildout, fix permissions and try to start the instance. (Note: buildout always crashes the first time I do this because of a setuptools incompatibility. Just run it again if it fails.):
```
bin/buildout || bin/buildout && chown -R zeo:zeo . && bin/zeoserver start
```
That’s it! Now to create a new zeoserver, just repeat steps 5-8, updating the paths and port number (zeo-address) to suit.

That’s really all there is to it. Tomorrow I’ll post how to put together the client side.

How to force HTTPS on Facebook and Twitter

michael — Wed, 10 Nov 2010 23:43:52 +0000

HTTPS is a way of providing security and privacy for your communication with web sites. You should expect it from sites that deal with your money, such as your bank, or Amazon.com, but you might not realize how important it is for sites like Facebook or Twitter. Think of it this way: When you protect your financial information, you’re protecting yourself, but when you protect your Facebook account, you’re protecting your friends. Using https keeps your accounts and reputation secure.

Many sites, Facebook included, have support for https, but immediately redirect you back to the insecure site, or forget to record your login session in a secure way. A Firefox Add-on called “NoScript” provides a solution to both these problems. Here’s how to make it happen:

Install the Add-on.
Restart Firefox
In Firefox, go to Tools –> Add-ons –> NoScript –> NoScript Preferences –> Advanced tab
In the “Behavior” tab under “Force the following sites to use secure (HTTPS) connections:” add the sites you want to secure. If you want to secure every subdomain of a domain, start with a dot. My entry looks like:
```
.facebook.com
.twitter.com
```
In the “Cookies” tab check “Enable Automatic Secure Cookies Management”.
Under “Force encryption for all the cookies set over HTTPS by the following sites:” add the same sites again. This will keep your login secure even over open wireless.

Now, in the future, your logins to those sites will be secure. NoScript will also do other things, such as block JavaScript and Flash by default for sites it doesn’t know about. If you want Javascript on for a site, you can just whitelist that site. NoScript will make it clear when it’s blocking JavaScript. You’ll see it in action the first time you visit Facebook and Twitter.

Let’s try it:

Go to http://facebook.com. First, notice that the URL changes to https://www.facebook.com. Now, you’ll also see NoScript letting you know that it blocked facebook.com. Just select “Allow all this page” from NoScript’s options.

Now you know enough to surf safely and protect your accounts on sites like facebook.com and twitter.com. For more information, read the NoScript FAQ.

How to drop 32-bit support from AMD64 Gentoo

michael — Fri, 08 Oct 2010 01:09:27 +0000

In the wake of the recent kernel exploits I determined I didn’t really need 32-bit support on most of my 64-bit machines. The vulnerabilities in question depend upon 32-bit support, so dropping that support resolves the issue without me having to patch the kernel or really change what software I’m running at all. Here’s how I did it.

I should point out that you cannot simply go back to having 32-bit support after you do this. This is a one-way change!

Step 1: Make sure you have a 64-bit capable bootloader:

You can use lilo or grub2. Lilo is documented in the Gentoo handbook, and I haven’t tried Grub2.

Step 2: Switch to non-multilib profile:

WEB-SVN ~ # eselect profile list Available profile symlink targets: [1] default/linux/amd64/10.0 * [2] default/linux/amd64/10.0/desktop [3] default/linux/amd64/10.0/desktop/gnome [4] default/linux/amd64/10.0/desktop/kde [5] default/linux/amd64/10.0/developer [6] default/linux/amd64/10.0/no-multilib [7] default/linux/amd64/10.0/server [8] hardened/linux/amd64/10.0 [9] hardened/linux/amd64/10.0/no-multilib [10] selinux/2007.0/amd64 [11] selinux/2007.0/amd64/hardened [12] selinux/v2refpolicy/amd64 [13] selinux/v2refpolicy/amd64/desktop [14] selinux/v2refpolicy/amd64/developer [15] selinux/v2refpolicy/amd64/hardened [16] selinux/v2refpolicy/amd64/server WEB-SVN ~ # eselect profile set 6 && eselect profile show Current make.profile symlink: default/linux/amd64/10.0/no-multilib

Step 3: Emerge packages whose useflags have changed:

WEB-SVN ~ # emerge -1 sandbox glibc gcc ...much noise... WEB-SVN ~ # rm /etc/env.d/04multilib && > env-update && > . /etc/profile && > fix_libtool_files.sh "$(gcc -dumpversion)" ...well, fix_libtool_files.sh didn't seem to change anything, but it was a good idea anyway... WEB-SVN ~ # cat /etc/ld.so.conf # to check if there are 32-bit libs left # ld.so.conf autogenerated by env-update; make all changes to # contents of /etc/env.d directory /usr/local/lib include ld.so.conf.d/*.conf /usr/x86_64-pc-linux-gnu/lib /usr/lib/gcc/x86_64-pc-linux-gnu/4.4.3 /usr/lib64/postgresql-8.4/lib64
Looks good!

Remove 32-bit support in the kernel

WEB-SVN ~ # cd /usr/src/linux && make menuconfig ... Executable file formats / Emulations ---> [ ] IA32 Emulation ... WEB-SVN ~ # make && > mount -o remount,rw /boot && > make install modules_install && > module-rebuild -X rebuild && > shutdown -r now

That’s it!

Doubleclick.wtf

michael — Wed, 01 Sep 2010 22:49:19 +0000

Steve and I were asked to implement doubleclick.net for some site by “oh you just drop some code in the page and it works great OK?”

No. We never do this, because we’re actually responsible for the crap that gets served from our servers, and there’s already enough clean-up we have to do.

So let’s take a look at this code (with identifying marks removed to protect the funky).

Here’s the original code:

Well that’s special. It generates an iframe so it can load whatever content it wants from doubleclick.net’s servers. That makes me slightly nervous and annoyed, but what’s worse, the code is invalid XHTML strict, so I’m going to have to rewrite it to be valid. Might as well rewrite the whole thing, since the Javascript is pretty stinky, too. (At least they took the trouble to write a noscript version)

var axel = Math.random() + "";
var a = axel * 10000000000000;

What does it do? Well, at first glance it looks like it tries to create a very long string. But actually no, in Javascript, "12345.67" * 1 == Number(12345.67). So this can be rewritten to make sense, be more efficient, and be one line: var a = 10000000000000 * Math.random();

Next, we can build the attributes in a way that makes this whole block of code more reusable:

var url_src = 1234567;
var url_type = "feline123";
var url_cat = "tabby012";
var url_ord = 1;
// and just for completeness
var url_num = a;
var data = "http://fls.doubleclick.net/activityi" +
  ";src=" + url_src +
  ";type=" + url_type +
  ";cat=" + url_cat +
  ";ord=" + url_ord +
  ";num=" + url_num + "?";

Then we’ve got the invalid iframe element. The object tag can be used in most cases in place of the iframe tag, so let’s use that. We build the element into the DOM:

var o = document.createElement("object");
o.data = data;
o.width = 1;
o.height = 1;
// Ignore that "frameborder" attribute because
// it's neither valid nor valuable.

…and since we were asked to insert this code “as close as possible to the opening tag,” insert it before the first child of the body element:

var b = document.body;
b.insertBefore(o, b.firstChild);

Putting it all together:

// Remember me? I got renamed!
var url_num = 10000000000000 * Math.random();
var url_src = 1234567;
var url_type = "feline123";
var url_cat = "tabby012";
var url_ord = 1;
var data = "http://fls.doubleclick.net/activityi" +
  ";src=" + url_src +
  ";type=" + url_type +
  ";cat=" + url_cat +
  ";ord=" + url_ord +
  ";num=" + url_num + "?";

var o = document.createElement("object");
o.data = data;
o.width = 1;
o.height = 1;

var b = document.body;
b.insertBefore(o, b.firstChild);

When I ran this code in Firebug, it produced the following DOM node on my page:

So I figured I would grab a copy of that URL using wget and see what it looked like. It looks like this:

So… Wait, what? The only differences between the URL in that and the URL generated for the is fls has become ad and activityi has become activity. So why didn’t we just load that in the first place? Only Doubleclick knows for sure, but loading the iframe and then the image does tell them a little bit more about browser capabilities, because it makes two different requests to their servers from your browser. Clever, but very irritating. On the other hand, maybe they’re just using the .

Using find in Subversion working copies

michael — Tue, 31 Aug 2010 21:18:07 +0000

The POSIX find command is extremely convenient and powerful for searching out files and features of the filesystem. In combination with its -exec action it can make changes to many files based on nuanced characteristics. I can’t live without it.

However, it’s inconvenient to use find in Subversion working copies because of all the .svn folders. Find searches within all of them, and since they contain duplicates of the files in the working copy itself, things get cumbersome fast.

You can work around this by pruning .svn:

find . -name .svn -prune -o -print

but find’s -prune flag is pretty intricate and befuddles even the smartest administrators and developers from time-to-time, especially if you don’t understand the default action, -print.

To make this easier, I’ve written a shell function that automatically prunes “.svn” and anything found in the svn property “svn:ignore” in the target path. Feel free to use this if you find it convenient.

svnfind() {
	# find things in an svn working copy
	# excluding .svn dirs and anything
	# in the target directory's svn:ignores
	local ignores=()
	local IFS=$'\n'
	local path

	# GNU no-path compatibility
	case "$1" in
		-*) path=".";;
		*) path="$1"; shift;;
	esac

	set -f # turn off globbing temporarily
	local _ignores=( $(svn pg svn:ignore "$path") )
	for i in "${_ignores[@]}"; do
		ignores+=( -o -name "$i" )
	done
	set +f

	# If find contains no "actions" other than -prune,
	# append the default action of -print
	local default="-print"
	for arg; do
		case "$arg" in
			-delete|-exec|-execdir|-fls|-fprint|-fprint0|-ls|-ok|-okdir|-print|-print0|-printf|-quit)
				unset default
				break;;
		esac
	done
	# $default must be unquoted here
	find "$path" \( -name .svn "${ignores[@]}" \) -prune -o "$@" $default
}