Posts Tagged ‘python’

Adding content to your Buildbot’s Home page

Monday, April 9th, 2012

Those of you who use the excellent Buildbot Continuous Integration system probably spend a fair amount of time looking at it’s Waterfall display. Which shows the status of your builders, and probably looks something like this.

One of the problems with an OOTB Buildbot install however, is that while the Waterfall is pretty awesome (when it’s green), the Home Page tends to be a little… let’s say dry. Which is a shame for what should be valuable URL real-estate.

All of the Web pages served by Buildbot are customizable however, although (at the time of writing) there seems to be little documentation on how to customize the home page in an interesting way – say by passing through extra context variables that can be used in the template.

Redressing that is the purpose of this post!



Basic Customisation

Buildbot will look for a directory called “templates” adjacent to your master.cfg for any custom templates you want to define. These are then rendered using the Jinja2 templating engine.

The template for the homepage is called root.html. You may like to use the current Buildbot default root.html template as a basis to work from – you can find the source here.

If you simply add

<div>
Hello Beautiful World
</div>

before the final closing endblock tag, well you can probably guess what shows up towards the bottom of your Buildbot’s home page…



Adding Variables

In order to pass extra variables through to the template, we’ll have to override a couple classes.

The frist thing we’ll want to do is provide a class that will render the homepage. This is normally buildbot.status.web.root.RootPage. We want an instance variable that will hold our extra variables, so we subclass and override __init__

from buildbot.status.web import root
 
class ContextWebRoot(root.RootPage):
 
    def __init__(self, context, *args, **kwargs):
        self.context = context
        root.RootPage.__init__(self, *args, **kwargs)

The rendering actually gets done by the content() method of our page class, which has a signature that helpfully includes the request and the template context as a dictionary. This means that we can simply update the dictionary and continue as we otherwise would…

    def content(self, request, cxt):
        """
        Let's update the context with our extra vars
        """
        cxt.update(self.context)
        return root.RootPage.content(self, request, cxt)

Which is all well and good, but now we need to have this class be the one that gets called when we visit the “/” url.

Now, in order to do this, we’ll want to subclass the buildbot.status.web.baseweb.WebStatus class that provides er… the Web status functionality. You can do that by adding the following to your master.cfg

from buildbot.status.web import baseweb
 
class ContextWebStatus(baseweb.WebStatus):
 
    def __init__(self, context={}, *args, **kwargs):
        self.context = context
        baseweb.WebStatus.__init__(self, *args, **kwargs)
 
    def setupSite(self):
        """
        Use the existing setup machinery, then replace the page that
        the "/" url points at.
        """
        baseweb.WebStatus.setupSite(self)
        our_root = ContextWebRoot(projects=self.projects)
        self.site.resource.delEntity("")
        self.site.resource.putChild("", our_root)

Now all that remains is to set the custom WebStatus class in the master.cfg.

tpl_context= {
    'foo': 'bar',
    'goo': 'car',
    'hoo': 'dar'
}
 
BuildmasterConfig['status'] = [ContextWebStatus(
    context=tpl_context,
    http_port=80,
    allowForce=True
)]

You can now access the context variables from the template with e.g.

{{ foo }} <br/>
{{ goo }} <br/>
{{ hoo }} <br/>



Putting it all together

So, the status section of your master.cfg should now look something like this:

from buildbot.status.web import baseweb, root
 
class ContextWebRoot(root.RootPage):
 
    def __init__(self, context, *args, **kwargs):
        self.context = context
        root.RootPage.__init__(self, *args, **kwargs)
 
    def content(self, request, cxt):
        """
        Let's update the context with our extra vars
        """
        cxt.update(self.context)
        return root.RootPage.content(self, request, cxt)
 
class ContextWebStatus(baseweb.WebStatus):
 
    def __init__(self, context={}, *args, **kwargs):
        self.context = context
        baseweb.WebStatus.__init__(self, *args, **kwargs)
 
    def setupSite(self):
        """
        Use the existing setup machinery, then replace the page that
        the "/" url points at.
        """
        baseweb.WebStatus.setupSite(self)
        our_root = ContextWebRoot(projects=self.projects)
        self.site.resource.delEntity("")
        self.site.resource.putChild("", our_root)
 
tpl_context= {
    'foo': 'bar',
    'goo': 'car',
    'hoo': 'dar'
}
 
BuildmasterConfig['status'] = [ContextWebStatus(
    context=tpl_context,
    http_port=80,
    allowForce=True
)]

Mercurial Hook to ensure issue references in Trac/Redmine

Saturday, June 4th, 2011

As much as I love the version control integration of the bug trackers I use (Redmine and Trac) I find myself forgetting to include the issue reference more often than I’d like to admit.

Add to this the merry dance you have to go through to ammend commit messages in Mercurial and things get even worse. (Oh how I’d love for hg to implement something similar to git’s $ git commit –amend )

Hence, a Pre-transaction-commit hook for hg that will ask me if I’m sure I want to commit without an issue number. If want to commit anyway, it’s just two extra keystrokes, and saves a whole lot of rollback/apply nonsense.

#!/usr/bin/env python
import os
import re
import subprocess
import sys
 
cmd = subprocess.Popen(['hg', 'log', '-vr', os.environ['HG_NODE']],
                       stdout=subprocess.PIPE).communicate()[0]
msg = cmd.split('description:')[1]
issue_regexes = [
    # Trac
    r'#\d+',
    # Redmine
    r'fixes #\d+',
    r'refs #\d+',
    ]
if not filter(lambda x: re.search(x, msg), issue_regexes):
    print "No issue ref or fix... message is:"
    print msg
    sys.stdout.write("Continue? [y/n] ")
    resp = raw_input().lower()
    if resp == 'n':
        sys.exit(9)

Then add the following to your project’s .hg/hgrc:

[hooks]
pretxncommit = path/to/your/pretxncommit.py

Saving you endless embarrassment:

davidmiller@pascal:~/src/buggy_repo$ hg commit -m "A context-less void"
No issue ref or fix... message is:
 
A context-less void
 
 
 
Continue? [y/n] n
transaction abort!
rollback completed
abort: pretxncommit hook exited with status 9

Love regards etc

Announcing Pony Mode – a Django editing mode for Emacs

Saturday, May 28th, 2011

I’m pleased to announce the first beta ‘release’ of Pony Mode, a minor mode for working on Django projects in Emacs.

This mode provides integration with the django management commands within emacs, as well as test integration, a minor-mode with syntax highlighting for editing templates, will determine whether you are using Fabric or Buildout, as well as much more!

This mode is under active development, so please file any bugs at the Github page, and feel free to provide any feedback/feature requests etc.

Current features include:

  • Run dev server in an emacs buffer [C-c C-p r]
  • * Checks to see if runserver_plus is available
  • * If not uses in-built runserver
  • Jump to current project in browser (start server if required) [C-c C-p b]
  • Run test case at point in buffer [C-c C-p t]
  • Run tests for current app in buffer [C-c C-p t]
  • Run Syncdb on current project
  • Management commands for current project in interactive buffer
  • South integration – run south convert, schemamigration, migrate
  • Run django shell in buffer [C-c C-p s]
  • * Checks for shell_plus
  • * If not defaults to shell
  • Fabric integration [C-c C-p f]
  • Startapp and dumpdata on current project within emacs
  • Database integration with Emacs sql-mode interactive buffer [C-c C-c d
  • Django Template minor mode with syntax highlighting for django
  • template tags

  • Snippet collection for django
  • generate tags table for project
  • run manage commands in interactive buffer
  • Buildout integration
  • Generate TAGS table for project to enable quick navigation
  • Jump to template at point or from editing view [C-c C-p g t]

Grab it while it’s hot!

coverage command for separate test runner in django

Friday, February 18th, 2011

I’m a long-time fan of django-test-coverage. It takes Ned Batchelor‘s coverage.py and wraps it in a django test runner for you, which is great.

Particularly with large test suites though, the speed increase is too large to go unnoticed (In my projects normally in the region of 150% YMMV).

And your tests can never be too fast right?

One solution to this is to only use the coverage test runner in a seperate management command. That way you always have access to the stats, but your ‘regular’ test run doesn’t take the performance hit.

Not only that, but you can also run the coverage stats on your continuous integration server once every few hours for a consistently recent picture of how your coverage is looking.

The code itself is nothing dead simple, just subclasses the django BaseCommand class and then fetches the right test runner:

"""
Run our tests with coverage turned on
"""
import sys
 
from django.core.management.base import BaseCommand
 
class Command(BaseCommand):
    """
    We totally want to get coverage details, but that's so slow!
    """
    option_list = BaseCommand.option_list
    help = "Run our tests with coverage turned on"
    args = "[appname ...]"
 
    requires_model_validation = False
 
    def handle(self, *tests, **options):
        """
        Actually do the test run
 
        Arguments:
        - `*tests`: test labels
        - `**options`: passed opts
        """
        verbosity = int(options.get('verbosity', 1))
        interactive = options.get('interactive', True)
        mod = __import__("django-test-coverage.runner")
        failures = mod.runner.run_tests(tests, verbosity=verbosity,
                                        interactive=interactive)
        if failures:
            sys.exit(bool(failures))

This then needs to live somewhere in an a management dir of one of your installed apps…

yourproject/yourapp/management/commands/coverage.py

…and you’ll need to

$ easy_install django-test-coverage

So that the test runner is there in your environment.

Grab the raw file here.

Alternatively you could patch django itself. Until they merge those changes though, I’ll be sticking with the seperate command.