Mapping specific requests to a single ZEO client instance

As a follow up to Apache Round Robin for ZeoClients:

we needed to send session specific things to a single instance.

An example of why this is necessary is for collective.captcha. If the connections are being round robin-ed you get audio and image files from different instances. That is BAD. It doesn't work.

First make sure varnish doesn't cache the captcha images and audio or your captcha protected form page.
In varnish.vcl add:

# Do NOT cache captcha image, audio, request
  if (   req.url ~ "/@@captcha"
      || req.url ~ "/contact-info"
     ) {
        pipe;
       }

Next we need to map a server to do our captcha stuff.
In zopeservers.map add the line:

CAPTCHA localhost:6970

I will leave it to you to think of how to creatively balance requests to the other instances. Perhaps remove teh one that does some captcha and other special work from ALL, or maybe add the other instances more than once... OK I didn't leave much, but there are probably smarter ways to do it.

Finally we need to add a rule for mapping captcha requests to the correct instance.
To the appropriate apache.conf (http.conf) add:

RewriteRule ^/(@@captcha/(audio|image)|contact-info)   \
   http://${zopeservers:CAPTCHA}/VirtualHostBase/http/%{SERVER_NAME}:80/msrd/VirtualHostRoot/$1 [L,P]

Don't forget to restart apache and varnish!!

Thanks to Chris Shenton for suggesting the solution.

Apache Round Robin for ZeoClients

There is a common approach to running Plone/Zope behind some service that distributes requests across numerous ZeoClients. Here's how I have been doing it using Apache (2.2.x).

                                  ________Zeoclient(6968)
                                 | _______ZeoClient(6969)
client-->varnish(80)-->apache(81)|<_______ZeoClient(6970)
                                 |________ZeoClient(6971)

Everybody knows how to setup varnish, right? So I am only going to show the Apache bits here.

First you need a rewrite map.

#zopeservers.map
ALL     localhost:6968|localhost:6969|localhost:6970|localhost:6917

Of course there can be more or less hosts in the map and they don't have to be all localhost either.

Next you need to enter the right stuff in the apache configuration. As here in a VirtualHost.

<VirtualHost *:81>
...
RewriteMap  zopeservers rnd:/usr/local/apache2/conf/zopeservers.map
RewriteRule ^/(.*) \
    http://${zopeservers:ALL}/VirtualHostBase/http/%{SERVER_NAME}:80/ploneroot/VirtualHostRoot/$1 [L,P]
...
</VirtualHost>

How does Emiril say... BAM!

Not so hard at all. A couple of notes:

1. I don't think apache knows how to detect if a client is down or broken. So it will serve up all the errors it gets.

2. Varnish doesn't cache https, so it needs to be handled wihtout varnish...

Serving up a ZODB on demand from a repozo backup

Since I said I would on the mailing list/forum.

Here it is:

import os
import shutil
import tarfile
import tempfile

def application(environ, start_response):
  status = '200 OK'

  tempdir  = tempfile.mkdtemp()
  datafile = '%s/Data.fs' % tempdir
  tarball = '%s.tar.bz2' % os.path.join(tempdir,
                                        os.path.basename(datafile))

  ## If I were smarter or had more time
  ## I would import from repozo or at
  ## least use subprocess
  os.system("/usr/local/zope/2.9.7/bin/repozo.py -v -z -R -r \
         /usr/local/zope/sites/2.9.7/msrd/zeo/var/backup/ -o %s" % datafile)

  out = tarfile.TarFile.open(tarball, 'w:bz2')
  os.chdir(tempdir)
  out.add('Data.fs')
  out.close()


  response_headers = [('Content-type', 'application/octet-stream'),
                      ('Content-Length', str(len(open(tarball).read()))),
                      ('Content-Disposition',
                       'inline; filename="Data.fs.tar.bz2"')]

  start_response(status, response_headers)
  try:
      return [open(tarball, 'rb').read()]
  finally:
      shutil.rmtree(tempdir)

I did it really quick. But it solved my problem very well. It gets a copy of the Data.fs from the last repozo backup. So all developers don't need shell access to the server...

A couple things I could see doing that would make this cooler are:

• Not using os.system
• implement a method to do HEAD requests with the time of the last repozo delta so I don't compress and transfer this beast if there are no changes...
  

If you have mod_wsgi installed and working you just need to add something like:

WSGIScriptAlias /Data.fs.tar.bz2 /usr/local/apps/getDataArchived.py

to the appropriate configuration file.

ipython in zope3 zopectl debug

The solution in my earlier post doesn't fly in zope3. To get it to work in zope3 I modified:
~/instances/z3/bin/debugzope

if __name__ == '__main__':    db = startup()
     del startup
     from zope.app.debug import Debugger
     debugger = app = Debugger.fromDatabase(db)
     root = app.root()
     del db
     del Debugger
     import IPython
     IPython.Shell.IPShell(user_ns={'root': root,
                                     'app': app,                         
                                    'top' : app.root() 
                                     }).mainloop(sys_exit=1)

I haven't figured out much about how the debugger differs in Zope3. But it gives me top as app.root().

In [1]: print top.items()
<OOBTreeItems object at 0x355c598>

In [2]: app.root()['foo']
Out[2]: <zope.app.folder.folder.Folder object at 0x36dfef0>

In [3]: top
Out[3]: <zope.app.folder.folder.Folder object at 0x36a27f0>

In [4]: top['foo']
Out[4]: <zope.app.folder.folder.Folder object at 0x36df830>

In [5]: print [x for x in top.items()]
[(u'foo', <zope.app.folder.folder.Folder object at 0x36df830>)]

ipython in zopectl debug

A long time ago (6mos? a year) I got tired of manually typing support for history and tab completion into the zope debugger. So I just hooked the debug command up to read my .pythonrc file by importing user in the do_debug method of zopectl.py.

def do_debug(self, arg):
   cmdline = self.get_startup_cmd(self.options.python + ' -i',
   "import Zope2; app=Zope2.app(); import user;")

Today I tried getting ipython working as described in the plone docs. I also tried vanrees notes

and a couple others. I could not get it going though. So I decided to return to the hack above and extend it to load ipython.

def do_debug(self, arg):
    cmdline = self.get_startup_cmd(self.options.python + ' -i',
    "import Zope2; app=Zope2.app(); import user;     
     ns={'__name__':'msrd','app':app}; import IPython;   
     IPython.Shell.IPShell(user_ns=ns).mainloop(sys_exit=1);")

Poof, now debug comes up with ipython. Yay... What does that mean you may ask?

In [2]: ??app.msrd
Type: ImplicitAcquirerWrapper
Base Class:
String Form:
Namespace: Interactive
Length: 1
Docstring [source file open failed]:
Make PloneSite subclass CMFSite and add some methods.
This will be useful for adding more things later on.

I know this is probably the wrong way to do this, but nyah nyah. It is working now. I do welcome feedback...

As a note: I also have import user in my args. This picks up stuff from your file defined in PYTHONSTARTUP. But then if you already use PYTHONSTARTUP you probably already know that.

Plone site broken? Fix ir through the zodb

Today someone changed the ip on a dev box running an instance that had the ip as part of the url in CacheFu. So of course it won't load a page anymore. What to do? reinstall? nah. Let 's think for a second. Plone is a nice interface to Zope/CMF. Zope/CMF is a lovely interface to ZODB. ZODB is a persistence engine for Python objects. CacheFu's settings are persisted as atributes of a cache object in the ZODB. Let's just fix it directly....

/usr/local/{instance}/bin/zopectl debug
Starting debugger (the name "app" is bound to the top-level Zope object)
...{ 8< snip } ...
>>> import readline, rlcompleter, transaction
>>> readline.parse_and_bind("tab: complete")
# now we have tab completion
# bind the plone root to s
>>> s = app.itcd
# bind our (s)ites cache settings to cc
>>> cc = s.portal_cache_settings
>>> cc.getDomains()
('http://1.2.3.4:80',) ## <<== look there is our old value!!
# yup it needs to be this new value
>>> cc.setDomains("http://4.3.2.1:8080")
# and persisted. Otherwise it will be aborted at exit
>>> transaction.commit()

That was easy, no?