0002422: make emi modules reloadable

Please see now our new Bugtracker system: JIRA

View Issue Details [ Jump to Notes ]

[ Issue History ] [ Print ]

Project

Category

View Status

Date Submitted

Last Update

0002422

Endian Firewall

Other Services

public

2009-11-18 20:45

2010-09-24 16:31

Reporter

peter-endian

Assigned To

peter-endian

Priority

normal

Severity

feature

Reproducibility

have not tried

Status

assigned

Resolution

open

Platform

OS Version

Product Version

Target Version

2.5

Fixed in Version

Summary

0002422: make emi modules reloadable

Description

with commands.emi.reload all modules will be reloaded, since the cherrypy instance will be re-instantiated completely
That's ok in some cases, but not ok when you want to do something like this:

emicommand commands.emi.reload
emicommand commands.migration.run --pkg=qos

which will be done in %post hook for the efw-qos package for example.

Now, we could also remove all cherrypy references to an emi controller, remove the modules from modules registry, re-import and re-init the emi module. That would be the ideal, because after an upgrade only that proper module will be reloaded.

This can be done by the following code examples:
import sys
import cherrypy
delattr(cherrypy.root.manage, "qos")
for i in filter(lambda x: x.startswith("endian.qos"), sys.modules.keys()):
    del sys.modules[i]
import endian.qos.web
endian.hotspot.web.init(cherrypy.root.manage)

So far so good..
Not every EMI module is reloadable so far. hotspot for example tries to re-register the model and qos tries to re-register datasource filetypes.
So we need to check that every emi module is reloadable.

That means we need to do:
- registries should replace same items and not raise an error (datasource)
- module reload probably need to close also the corresponding database hub,
  since reload will reopen it and thus register already registered classes.
- registering modules need to become more intelligent and store for every child
  which have been connected its parent-node module string in a searchable
  structure. not every controller is connected to manage, so we can't
  remove it if we don't find it.
- removing controller need also to remove command and event registrations and
  subscriptions. They will however be overwritten, but if a module has an error
  during reload or will be removed, those registrations will not be
  garbage-collected
- removing modules should be generic, in order to be able to do that also on
  %postun hook, when an emi module passes by.
- installing modules could also become dynamic in order to be able to call
  from emicommand, so installations of new modules do not require an emi restart

hope that's all.

Tags

No tags attached.

Attached Files

Relationships

Notes
(0003489) peter-endian (administrator) 2009-12-01 21:40	EMI could store persistently of the version of every module of which that module has been loaded last time. with this information EMI can understand during module loading (reload/restart) which module needs to restart its services and restart or enqueue them to be scheduled for restart. this way we don't need to hook those restarts anymore within rpm %post hooks, since every EMI module hooks reload of itself.

(0003860) peter-endian (administrator) 2010-02-25 10:50	after talk with andrea we concluded that it is better to implement lazy loading of the EMI modules and completely destroy all turbogears objects whenever EMI get a SIGHUP. This then can be done on every module update. So, next access to the module will reload it then This is way easier to implement and causes few hassles for sure, since that's how cherrypy implements auto-reload of python modules. When it is working ok for cherrypy, it should also for EMI.

Issue History
Date Modified	Username	Field	Change
2009-11-18 20:45	peter-endian	New Issue
2009-11-18 20:45	peter-endian	Assigned To	=> peter-endian
2009-12-01 21:40	peter-endian	Note Added: 0003489
2010-02-25 10:50	peter-endian	Note Added: 0003860
2010-03-08 19:56	peter-endian	Status	new => confirmed
2010-09-24 16:31	peter-endian	Status	confirmed => assigned
2010-09-24 16:31	peter-endian	Target Version	2.4 => 2.5