A Proposal for Packaging Domain Specific Extensions to Radia

I am a long time user of different CMS systems, mostly the myriad of
popular PHP content management systems. The main problem I find with
these systems is their lack of domain specificity. We all know now that
publishing on the web is not a one size fits all problem and as such it
doesn’t have a one size fits all solution. I wanted to talk to the group
a little about the ideas that I think have caused many other CMS’s to be
problematic and why I feel that radiant has a chance to break through
these difficulties, along with how some of the problems might be solved.

The Problem
Several popular solutions to the domain problem have emerged, the
main ones being ‘components’, ‘modules’, or ‘mambots’ (scary). The
problem with components is a one of segmentation and complexity. Adding
components increases the complexity of your system and still doesn’t
turn your system into one that is tailored to your domain. In most cases
it creates many segments of your CMS in which to work in that are
separate from the main interface and separate from one another.

Why Radiant?
My vision for Radiant, even though I’m not a member of the core team
would be one of a ‘base’ or ‘core’ CMS that Radiant is today. Radiant
would be a relatively neutral system from which developers could easily
extend to solve problems within their specific domain. My ideal
situation would be to simply be able to type something like
‘script/radiant_plugin install radiant_blog’ and automatically get
comments, trackbacks, tagging and other blog functionality automatically
integrated into the database and admin interface. Make a note that I
don’t want to drop modules on top of the CMS, I want to fundamentally
change its operation to suit the problem domain.

What has to be covered?
I will assume that the installation of Radiant is clean and has not
been installed over anything else. My proposal is for a plugin system
that works easily and seamlessly alongside of the rails plugin system.
The goals of my proposals are the following:

  1. The plugins should be compatible with the rails plugin system
  2. The plugins should not be dependent on any plugins not controlled by
    the radiant core team
  3. All plugins should be non-destructive and should not heavily modify
    the workings of the core CMS
  4. All plugins should be easily removed and all changes should be
    reversible (this is particularly important to migrations)
  5. The burden of extending the functionality of the system should lie on
    the developer of the plugin which means as little modification as
    possible should be done to the current radiant system.

There are 4 main considerations when extending Radiant in a way that
allows things to be easily packaged. They are as follows:

  1. Modifying controllers and models
  2. Adding behaviors and radius tags
  3. Adding table to the database
  4. Modifying the administration views

Proposed Solutions

  1. Modifying controllers and models
    This is what rails plugins were meant to do. Since our plugins are
    rails plugins we get this for free. There is one consideration. In order
    for a plugin to not break compatibility as radiant gets upgraded the
    preferred method of modifying controller and model logic is to use
    before and after hooks so that if Radiant’s core functionality is
    changed in the future the plugin will not break that functionality.

  2. Adding behaviors and radius tags
    If you use the normal rails plugin structure then adding tags and
    behaviors to radiant is easy.

  3. Migrating the database
    Migrating the database is slight more difficult as rails doesn’t
    have any built in methods for migrating out of plugin directories. This
    is where I want to suggest another file in the main plugin directory
    that sits alongside init.rb called radiant.rb. Radiant.rb is a file that
    contains initialization code that is specific to the plugin being a
    radiant plugin. Radiant.rb does not replace init.rb. This file will
    refer to the directory with the database migrations along with any
    future improvements to the plugin system that might come along.
    A separate rake task will be needed to run the plugin migrations but
    can easily be included in lib/tasks. This task will look for all plugins
    with a radiant.rb file and attempt to migrate them into the current
    environment’s database. The one rule that should be applied to radiant
    plugin migrations is that forward migrations must be non-destructive to
    the ‘core’ CMS structure. In the event that two different plugins’
    migrations clash the migration process itself will cause an error to be
    reported to the user. There is the possibility that these migrations
    don’t increment the version number of the database and simply work as an
    atomic database change that the normal migrations are unaware of. This
    needs to be investigated a bit further, but this can be solved.
    There should be a separate script command that gets the plugin like
    a normal rails plugin, and then runs the migrations if the user chooses
    to do so. There should also be a script for reverting the migrations and
    deleting the plugin. These are relatively trivial to code.

  4. Modifying the administration views
    The problem of modifying the admin interface really can be a show
    stopper. I have thought of a few solutions and I will present the one
    solution that would get most of the way there and is easiest to
    implement into the current system.

    The solution is to rely on helpers to place callbacks during the
    rendering of the of the administrative views. These filters would be
    before each item in a list, form, or set of tabs and before the ending
    of the list, form or set of tabs. The goal is to break down the
    administrative interface into smaller pieces that can be independently
    added onto by either changing the data before a list item or form
    control gets rendered or by rendering new list items or form controls.

    How coarse or fine should these helper callbacks be? Well my idea
    for instance on the ‘Edit Page’ action is to place a call to a blank
    helper method called before_title_input just before the title input box
    and before_body_input just before the body input. This would allow
    plugin developers to easily extend the existing forms and lists by
    assigning before filters to the form or list element that is about to be
    rendered. Tabs could be achieved through a bit of coercing of the same
    methodology only with helpers that make callbacks inside of javascript
    blocks (this might take some more investigation)

    I’m of the opinion that simplicity should trump flexibility in most
    of these cases and that the callbacks should be kept to a minimum to
    allow modification of the default behavior. What might an implementation
    of this system look like? Lets try and add a trackback URL to our New
    Page and Edit Page action.

The plugin helper might look like

module Admin::PageHelper
before_filter :trackback_input, :only => [:end_new_form,
:end_edit_form]

def trackback_input(*args)
trackback_input = ‘<textarea name=“trackbacks” … >’
end
end

The plugin controller might look like

class Admin::PageController < ApplicationController
after_filter :do_trackback, :only => [:new, :edit]

def do_trackback
  if request.post?
    if @page.errors.blank?
    # code for making a trackback requests using 

params[:trackbacks]…
end
end
end
end

What might we have to do to the radiant system to have this
functionality?

The radiant helpers might look like

module ApplicationHelper

holds the list of filter functions in some structure (possibly a

nested hash?)
attr_accessor :filters

def before_filter(filtered_method, options = {})
# add a new lambda with appropriate options to the filters list
end

def method_missing(*args)
# check if it starts with ‘filters_for’ and then run filters for the
proper helper
end
end

module Admin::PageHelper
def end_new_form(*args)
filters_for_end_new_form(args)
end

def end_edit_form(*args)
filters_for_end_edit_form(args)
end

… More Callbacks Go Here …
end

The radiant view might look like

<%= start_form_tag %>

… default form stuff here …

<%= end_new_form %>

<%= end_form_tag %>

The potential to have a long list of callbacks is definitely there

but I still think it is one of the easiest way to implement flexibility
into the administration system without hacking deeper into rails to
alter how views are rendered. I’d say there would be about 7 or 8
callbacks per helper. It might end up not being manageable but if there
is a standard naming convention it shouldn’t be bad at all. This would
allow new callbacks to be added easily without breaking old
functionality. It also allows more than one plugin to register into a
callback chain so that more than one domain specific plugin can be
present at one time, leading to a cross-domain specific CMS.

That's all I have so far, I'm working on building out these features

off of the latest trunk and I’ll have a patch and a working copy with
some examples so that people can take the idea for a spin. I hope to
have a full implementation of the radiant_blog plugin soon using these
conventions.

Josh F.

  1. Migrating the database
    Migrating the database is slight more difficult as rails doesn’t
    have any built in methods for migrating out of plugin
    directories. This is where I want to suggest another file in the main
    plugin directory
    that sits alongside init.rb called radiant.rb. Radiant.rb is
    a file that contains initialization code that is specific to the
    plugin being a
    radiant plugin.

My thought on doing this would be for plugins to have their own
migrations directory, a “plugin_schema” table to keep track of the
schema version of each plugin and a “rake migrate_plugin” script to run
the migrations. It seems so obvious to me, and yet I couldn’t find
anything about something similar, which leads me to assume that it’s
harder than I think.

Dan.


Radiant mailing list
[email protected]
http://lists.radiantcms.org/mailman/listinfo/radiant

to prevent mixins with the Rails core one could also extent
the code in
routes.rb to automagically fetch routes that are defined in plugins.

The Bounty Source svn browser (https://bssvnbrowser.bountysource.com/)
registers itself at a route. It does this fairly badly, but it does it.
I haven’t really looked deeply at the implementation of what they do (I
think they’re aliasing methods in the core routing class) but they add a
controller complete with views to your app. Maybe it’s a good place to
look for ideas.

Dan.

Not quite. There is no direct support for controllers or views in Rails
plugins.

Plugins will still work for controllers since they are just ruby

code that gets run after the /app directory is loaded. What I suggest to
not smash up radiant when new versions come out is to only use before
and after filters to process new form elements or create new instance
variables before the page is rendered.

Relatively easy to do, though you need to update the routes from your
plugin.

I hadn't thought about adding new routes. The only simple method for

adding new routes I can think of is to call a special map.before_routes
and map.after_routes method inside of the mapping block that will let
plugin developers put high priority and low priority routes into the
routes.rb file. I still stress though that the idea behind plugins
should be one of creating before and after hooks for key methods, not of
mixing back into the system by modifying default pieces of rails.

As far as installing and modifying plugins go I think I'm now more

on the side of using the already existing plugin system that rails has
in place and then just using a special rake task to do the plugin
migrations up and down. This is the easiest way to ensure that Radiant
does not become a beast to maintain and support as rails and radiant
both grow, which is the #1 priority in my opinion.

How about just extending the default rake migration task to also include
the migrations of all Radiant plugins?

I don't want to do that since every time a new version of rails

comes out with modifications to the rake tasks Radiant would have to
upgrade it’s ‘special’ tasks file so that it would have all of the
capabilities of the newer version of rails. This would lead to things
like Radiant being on 1.0.0 and Rails being on 1.1.3 since no one has
had the time to fix the pieces that are changed. That is why it is
important to simply leave the Rails system completely intact and
unchanged. It would be easy enough to put tasks into the lib/tasks
directory (which gets automatically searched by rake) and just do ‘rake
radiant:migrate’ or something like that. Then new tasks could be easily
added to the ‘radiant’ task namespace.

Migrations are going to take more work than just that. I think there

is going to need to be a special table where radiant can remember which
plugin migrations it has run by plugin name and file name so that
running the rake task never attempts to re-migrate. This is also the
only way to not have them be versioned.

I'm not entirely sure that callbacks are the right solution to the

view problem but I know that they are quick and easy to implement and
getting something done is better than nothing. I know a lot of people
are trying to extend radiant right now so this is a pretty hot topic and
I just wanted to put in my proposal…:slight_smile:

I will be working on these things very soon and I will look at your
patch…:slight_smile:

Josh F.

Actually yeah I’m working on this right now. I am using a bit of code
from the engines plugin. I am modifying this:

module ::ActionView
class Base
private
def full_template_path(template_path, extension)

    # If the template exists in the normal application directory,
    # return that path
    default_template = "#{@base_path}/#{template_path}.#{extension}"
    return default_template if File.exist?(default_template)

    # Otherwise, check in the engines to see if the template can be

found there.
# Load this in order so that more recently started Engines will
take priority.
Engines.each(:precidence_order) do |engine|
site_specific_path = File.join(engine.root, ‘app’, ‘views’,
template_path.to_s + ‘.’ + extension.to_s)
return site_specific_path if File.exist?(site_specific_path)
end

    # If it cannot be found anywhere, return the default path, where 

the
# user should have put it.
return “#{@base_path}/#{template_path}.#{extension}”
end
end
end

I am packaging this into a plugin called ‘radiant_plugins’ that will
also include all of the helper callbacks and stuff so that everyone can
drop this into their radiant installation and build plugins with it. Yes
I know even the plugin system is a plugin, kinda crazy.

Josh F.

On 29-Jun-2006 09:20 +1000, Daniel S. was heard to say:

The Bounty Source svn browser (https://bssvnbrowser.bountysource.com/)
registers itself at a route. It does this fairly badly, but it does it.
I haven’t really looked deeply at the implementation of what they do (I
think they’re aliasing methods in the core routing class) but they add a
controller complete with views to your app. Maybe it’s a good place to
look for ideas.

They uses a similarly bad method as I use in my patch, though less
flexible. They also have the same problem I am trying to solve right now
which is that as soon as you change the template_root of a controller,
it
also tries to find layouts in this new root. This is not always desired
as
in Radiant’s case, where I’d like to use Radiant’s administration layout
but
provide my own views. Any ideas on how to solve that cleanly?

Oliver

Oh yeah I forgot to mention if in your plugin directory require a file
that has normal routing stuff in it you can add and probably override
routes pretty easily. Take a look at the radiant routes file for more
examples of how John prefers to do his routes…:slight_smile:

ActionController::Routing::Routes.draw do |map|
map.with_options(:controller => ‘admin/page’) do |page|
page.comments ‘admin/pages/comments/:id’, :action => ‘comments’
end
end

Josh F.

That’ll teach me to say things before I get them running…:wink:

Josh

On 28-Jun-2006 20:01 -0400, Josh F. was heard to say:

ActionController::Routing::Routes.draw do |map|
map.with_options(:controller => ‘admin/page’) do |page|
page.comments ‘admin/pages/comments/:id’, :action => ‘comments’
end
end

If you look at the implementation of the draw method you see that it
actually overwrites the routes that have been specified before. Hence,
after
the plugin routes have been created they get overwritten by the
application
routes since they are loaded after the plugins. Also, the plugin routes
should have higher priority than the application routes, since one
should
assume they only “improve” the application and the last route in the
application route matches all request URL that have not been matched
yet,
so appending routes doesn’t work either.

Oliver