Intro to some Python dependencies of OpenStack

Well, it’s 2014, so I might as well get my one blog post a year out of the way :).

I started working on OpenStack around the middle of last year. Since that time, I’ve been exposed to quite a lot of things that I was previously unfamiliar with. In addition to all of the OpenStack projects, there are a fair amount of tools and libraries used by those projects that were new (or relatively new) to me. I thought I’d briefly mention a few of those libraries here in a post. As can be expected, they are part of the Python ecosystem since OpenStack is written primarily in Python.

None of these libraries are specific to OpenStack, they just seem to be widely used across the various code bases.

For more in depth documentation, check out the links for each library:

tox - https://pypi.python.org/pypi/tox

At it’s core, tox manages a set of python virtualenv’s and automates running a set of commands in each. It’s meant to allow you to run your test suite, syntax checkers, test installations, etc, against different isolated environments of Python. tox uses a tox.ini file for configuration.

testrepository - https://pypi.python.org/pypi/testrepository

testrepository provides quite a bit of tooling around running Python tests. Notably, it provides parallel test execution to help speed up test runs. A fair amount of OpenStack specifics have been captured at: https://wiki.openstack.org/wiki/Testr. In particular, the last item on that page about debugging tests with pdb is very helpful and a question I’ve seen come up a few times.

testtoolshttps://pypi.python.org/pypi/testtools

testtools adds some functionality on top of the existing unit testing framework. In particular it adds some additional assert methods for ease of use.

stevedorehttps://pypi.python.org/pypi/stevedore

So, I admit when I first saw this I thought, “Who is Steve Dore and why did he name a module after himself?”. Yes, I get the meaning now.

stevedore is a library that provides a framework for dynamically loading plugins or extensions for Python applications. It uses setuptools entrypoints as the implementation. Using plugins to extend base functionality is a fairly repetitive pattern in Python (and all of software). I know I’ve written a few in the past. So, it’s nice to see a module that aims to provide that common functionality in a reuseable fashion.

cliff - https://pypi.python.org/pypi/cliff

cliff stands for Command Line Interface Formulation Framework. It’s a framework for CLI tools. Have you ever written boiler plate CLI code that implements command managers, subcommands, handlers, formatters, outputters, etc?  If so, consider taking a look at cliff. It’s also highly pluggable to allow for different extensions that add functionality (using stevedore of course).

It has some neat built-ins as well, such as an interactive shell implementation, and the ability to generate a bash completion script for your CLI application.

Of note, the unified OpenStack client is being written using cliff: https://wiki.openstack.org/wiki/OpenStackClient

alembic - https://pypi.python.org/pypi/alembic

alembic is a database schema migration tool. I haven’t looked too closely at it, but it seemed to have a fair amount of momentum in the discussions at the last OpenStack summit in Hong Kong.

pecan - https://pypi.python.org/pypi/pecan

pecan is another entry in the world of Python web frameworks.  It seems particularly well suited to REST based API’s.  It uses the object-dispatch routing paradigm to map HTTP methods to corresponding methods on your controller classes.

I believe most existing OpenStack API’s are moving to the pecan framework, while many of the newer projects such as Ceilometer, Ironic and Tuskar are all aready based on pecan.

wsme - https://pypi.python.org/pypi/wsme

WSME provides typing for REST based API’s. It aims to be framework agnostic for easy integration. It handles serializing HTTP responses and deserializing HTTP requests based on the types that you’ve set in your Controller classes (via decorators).

 

Tags: ,

Posted in Programming


Leave a Reply