as part of an ongoing project to get zope3 running well on google app engine, i worked on loading python code from zipped egg files. I started by following guido’s hint in the corresponding appengine issue. As google appengine doesn’t contain the zipimport builtin extension, i relied on using the python svn sandbox code that implements imports in pure python, and uses the zipfile module on top of that to provide a zip import facility. I checked out the import in py code, and added the following to get it working in google app engine dev server.
import os.path, imp, types from zipimport_ import zipimport imp.new_module = types.ModuleType imp.PY_SOURCE = 1 imp.PY_COMPILED = 2 def load_zipegg( egg_path ): egg_file = os.path.basename( egg_path ) egg_name = egg_file.split('-',1) head = egg_name.split('.') importer = zipimport.zipimporter( egg_path ) # we load the head, the importer loads any contained modules importer.load_module( head )
unfortunately while it works ok on the dev server, it doesn’t work on appengine😦, due to a non implementation of marshall.dumps. i’ve appended a comment to that effect on the issue.
in a nutshell you hand it hand it the path to an egg, and it loads the egg code. Its not 100% perfect, it has some issues with minor namespace’d eggs (stomps on __path__, extraneous sys.modules entry created in subpackages). Hopefully its useful to those wanting to build larger applications or utilize frameworks other than the builtin django.
It does add considerably to the startup time for an app, which went from .3-.5s to .9-1s. after this initial load that particular app server instance has a cached sys.modules to work with, and startup time is neglible.
The porting of zope3 to appengine, still needs some support for zip contained resources, such as configuration, presentation templates, and browser resources. Even just zipping the packages without those, gets a simple zope3 application down to ~350 files.
Currently i’m just zipping individual directory eggs, with some filtering for pyc, so, text, and test files. Load time would likely go considerably faster if addons were packaged in a single egg.