Mike November

One thing I've regularly found missing from SilverStripe is the ability to paste a screenshot directly into the CMS. Of course, that's not yet possible using JS or Flash directly, but Java does give you the option to access the clipboard, and using an applet, I've hooked this capability into the CMS.

It's not as fluent as clicking the wysiwyg and hitting Ctrl+V (though I'm sure someone could script those actions...), but clicking ctrl+shift+V when the "Insert Image" toolbar is open will paste to the applet, then you need to manually click the 'Upload' button. This will select the image for you in the list on the right, so you then can just click "Insert Image".

The other nice thing is being able to hook into the Pixlr editor. If you have the 'and Edit' box checked, then the pixlr editor will be launched after clicking upload, allowing you to modify the image immediately.

You can get it from Github. It's currently self signed, so you'll get a java nag warning, and I've found that it performs terribly on OpenJDK on Ubuntu (though no problems with the Sun JRE). So let me know if you have any specific issues.

I just found the link to some slides that Andrew Short prepared with a bit of input from myself that he gave at the Sydney PHP User Group meetup a few months ago. If I can make it, I'll probably roll them out at the Melbourne User Group soon...

Slides on slideshare

Not much going on at the moment outside of some client development work. A couple of projects that may throw up some interesting modules, but that will have to wait for a few weeks to see what's produced.

If you're in Sydney, there's a PHP User Group meetup that's showcasing SilverStripe. Andrew Short will be there to give a developer's insight to the platform, and Shane Weddell will be there to give a more business spin to the whole show.

Also, SilverStripe Australia is having a meetup to celebrate the end of year. There'll be a few things spoken about, but it will mostly be a case of a few beers and unwinding (at least, that's my plan!). I'll have a couple of module things to talk about, and we've got a partner giving an overview of migrating a site to SilverStripe from MySource Matrix. Come along!

After much tweaking and stalling, the QueuedJobs module has been released (and submitted to SilverStripe.org, should be up soon). What does it provide?

The Queued Jobs module provides a framework for SilverStripe developers to define long running processes that should be run as background tasks. This asynchronous processing allows users to continue using the system while long running tasks proceed when time permits. It also lets developers set these processes to be executed in the future.

In essence, the goal is to not leave users with a seemingly 'hanging' connection that may eventually time out when they trigger an action that might take a while to process. Not many actions in SilverStripe do this, but your site might have particular need for it. Some areas where we're using it

• Generating google sitemap.xml files for sites with hundreds and thousands of pages.
• Publishing large subtrees of pages
• Re-indexing content in systems that use external search engines such as Solr.

If you want to test it out, download it from GitHub or SilverStripe.org. After extracting it to the queuedjobs folder, and running dev/build, you'll need to make sure you have a cronjob setup to run the main processor (preferably as the webserver user).

*/1 * * * * php /path/to/silverstripe/sapphire/cli-script.php dev/tasks/ProcessJobQueueTask */15 * * * * php /path/to/silverstripe/sapphire/cli-script.php dev/tasks/ProcessJobQueueTask queue=2

See the wiki page for more info.

I recommend you test out the GenerateGoogleSitemapJob to get a feel for what's going on under the covers (and it actually has a functional benefit!). To create the initial instance, go to http://path.to.silverstripe/dev/tasks/CreateDummyJob?name=GenerateGoogleSitemapJob which will create it (it will recreate itself as it processes. To make things easier, I'll step through the code so you get an idea of what's important when doing your own jobs

    public function __construct() {
        $this->pagesToProcess = DB::query('SELECT ID FROM "SiteTree_Live" WHERE "ShowInSearch"=1')->column();

        $this->currentStep = 0;

        $this->totalSteps = count($this->pagesToProcess);
    }

When constructing the job, I get a list of all the Live pages on the site (these are the only ones that are going to be indexed by google) that are set to show in the search. We're only interested in the ID of these pages though, not the actual objects. This is because we're going to store the full list of IDs of the pages we need to process - the $this->pagesToProcess variable here gets serialized and stored in the database between processing events, enabling us to stop and start processing at any time.

    public function getJobType() {
        if ($this->totalSteps > 100) {
            return QueuedJob::LARGE;
        }

        return QueuedJob::QUEUED;
    }

Here we're arbitrarily making the judgement that > 100 pages to generate an XML file for is enough for the job to be classified as 'large'. There's no real processing difference for this at the moment; the main reason for doing so is to not clog up one of the queues with a job that will take several minutes to execute.

    public function getSignature() {
        return md5(get_class($this));
    }

To prevent multiple instances of the same job being added to a queue, each job defines a signature. The base AbstractQueuedJob defines a default that should be good enough for 95% of jobs, but in some cases you want to ensure that a job is the only one of its kind, regardless of parameters.

    public function  setup() {
        parent::setup();
        increase_time_limit_to();
        
        $tmpfile = tempnam(getTempFolder(), 'sitemap');
        if (file_exists($tmpfile)) {
            $this->tempFile = $tmpfile;
        }
    }

The setup() method is called just before a job starts for the first time. In this case, we're wanting to make sure that a temporary file (that we're going to build the sitemap.xml file into first) exists for us to work with.

    public function prepareForRestart() {
        parent::prepareForRestart();
        // if the file we've been building is missing, lets fix it up
        if (!$this->tempFile || !file_exists($this->tempFile)) {
            $tmpfile = tempnam(getTempFolder(), 'sitemap');
            if (file_exists($tmpfile)) {
                $this->tempFile = $tmpfile;
            }
            $this->currentStep = 0;
            $this->pagesToProcess = DB::query('SELECT ID FROM SiteTree_Live WHERE ShowInSearch=1')->column();
        }
    }

The prepareForRestart() method is executed whenever the job has been paused then restarted. It could have been restarted by a user manually pausing, or an error that caused it to stop. Either way, it gives us a chance to check the state of the job, and if necessary restart it. We could just as easily flag the job as complete here and not continue, but in this case we're making sure our temporary file still exists, and if it doesn't, creating a new one from scratch.

    public function process() {
        $remainingChildren = $this->pagesToProcess;

        // if there's no more, we're done!
        if (!count($remainingChildren)) {
            $this->completeJob();
            $this->isComplete = true;
            return;
        }

        // lets process our first item - note that we take it off the list of things left to do
        $ID = array_shift($remainingChildren);

        // do some processing work that adds content to $tmpfile
        // ... snip ...

        // and now we store the new list of remaining children
        $this->pagesToProcess = $remainingChildren;
        $this->currentStep++;

        if (!count($remainingChildren)) {
            $this->completeJob();
            $this->isComplete = true;
            return;
        }
    }

The process() method is where all the actual work for this job happens, but it still needs to do a minimum of things to keep the container happy and in sync with things. First, it retrieves the list of pages still to be processed, and checks to see if there's anything left, if not marking the job complete. Next, it does the actual work with the next item in the list, then updates $this->pagesToProcess to make sure that next run through is onto the next item. It updates how many steps have been processed, then does another check to see whether the job has completed.

    protected function completeJob() {

        // ... snip ... 

        if (file_exists($this->tempFile)) {
            unlink($this->tempFile);
        }

        $nextgeneration = new GenerateGoogleSitemapJob();
        singleton('QueuedJobService')->queueJob($nextgeneration, date('Y-m-d H:i:s', time() + self::$regenerate_time));
    }

Finally, our completeJob() method actually copies our temp file to the right location, then cleans up the old file. Lastly, this job creates a NEW job and adds it to the queue to be processed at a date in the future; in this case it executes once every day.

Okay, that was a whole lot of words, but hopefully gives an idea of what's involved in writing a queued job, or more to the point, what you don't have to worry about. The framework around this manages everything to do with error handling and reporting, including automatically pausing and restarting jobs and notifying on broken jobs. It manages the persistence of job state so that jobs can be picked up after they've been paused and still continue on. It also manages the scheduling of jobs in the future, so you can use the module almost as a cron replacement.

I've been toying with a simple dependency injector for the last few days, and have started to think about ways in which it (or another dependency injector) could benefit SilverStripe. Some ideas I've been toying with

• Put in authentication / authorization as part of the pre request dispatch.
• Refactor authentication methods to be injected services linked in
• Refactor persistence and data loading away from DataObject class directly
• Using non* SQL based datastores
• Refactor to use PDO or something with proper prepared statements
• PermissionService * for sites that don't need complex permissions, have something that returns true, for those that do configure to use something else
• VersionService
• WorkflowService
• SearchService
• Aop filters for some things
  • Auditing method calls
  • Access restriction on method calls (force developers to think before calling * >write())
  • Would almost work as a replacement to $this* >extend('augmentXXX') type functionality

Along the lines of the first point, I put some code together to see how it works

// injector configuration
Injector::inst(array(
    array(
        'class' => 'RequestProcessor',
        'properties' => array(
            'preFilters' => array(
                '#$AuthenticationFilter',
                '#$AuthorisationFilter',
            )
        ),
    ),
    'AuthenticationFilter',
    'AuthorisationFilter',
    array(
        'class' => 'AuthenticationService',
        'properties' => array(
            'authenticators' => array(
                '#$DbAuthenticationProvider',
            )
        ),
    ),
    'DbAuthenticationProvider',
);

And then adding a hook in before the Director dispatches the request

Injector::inst()->get('RequestProcessor')->preFilter($req, $session); // $session required to get around a SS quirk

The 'grunt' work comes in the AuthneticationFilter

class AuthenticationFilter implements RequestFilter {
    /**
     * Automatically injected based on convention
     */
    public $authenticationService;
    
    public function preRequest(SS_HTTPRequest $request, Session $session) {
        // lets try authenticating
        if (isset($_REQUEST['auth']) || isset($_REQUEST['action_dologin'])) {
            $email = $request->postVar('Email');
            $pass = $request->postVar('Password');

            $member = $this->authenticationService->authenticate($email, $pass);
            if ($member) {
                $member->logIn($request->postVar('Remember'));
                
                // dirty hack for now...
                $session->inst_set('loggedInAs', $member->ID);
            }

            // because we have the request here, we can do a bunch of redirects or
            // whatever we like - but this is just a poc for now so we won't bother with
            // anything else. 
        }
    }

    public function postRequest(SS_HTTPRequest $request, SS_HTTPResponse $response) {

    }
}

So the actual 'authenticate' work is done via an authentication service, which in turn has registered within it a bunch of AuthenticationProvider implementors - in this case I only have one (DbAuthenticationProvider) which simply calls MemberAuthenticator::authenticate(). I could quite easily add on any number of other providers (LDAP, OpenID, etc etc) in a chain, and let the user login via the first that succeeds. Makes for an elegant way to have a single login form.

The next step is to add some logic to the AuthorisationFilter to prevent requests to certain URLs continuing if the user is not authorized (I'm looking at you /dev requests). This stops unauthorised requests very early in the request cycle, and help centralise the logic for this for a single maintainable area for developers to refer to. But that's for another day...