Python UK business days with pandas

Here’s how to calculate UK business days (well at least for England & Wales) using pandas ‘s holiday calendar.

First you’ll need this calendar for UK holidays:

from import (
    AbstractHolidayCalendar, DateOffset, EasterMonday,
    GoodFriday, Holiday, MO,
    next_monday, next_monday_or_tuesday)
class EnglandAndWalesHolidayCalendar(AbstractHolidayCalendar):
    rules = [
        Holiday('New Years Day', month=1, day=1, observance=next_monday),
        Holiday('Early May bank holiday',
                month=5, day=1, offset=DateOffset(weekday=MO(1))),
        Holiday('Spring bank holiday',
                month=5, day=31, offset=DateOffset(weekday=MO(-1))),
        Holiday('Summer bank holiday',
                month=8, day=31, offset=DateOffset(weekday=MO(-1))),
        Holiday('Christmas Day', month=12, day=25, observance=next_monday),
        Holiday('Boxing Day',
                month=12, day=26, observance=next_monday_or_tuesday)

It was tested with the dates from so should be fine to use, but please let me know if you find anything wrong with it.

Now you can do stuff like:

from datetime import date
from pandas.tseries.offsets import CDay
business = CDay(calendar=EnglandAndWalesHolidayCalendar())
>>>, 3, 2)
>>> five_business_days_later = + 5 * business
>>> five_business_days_later
Timestamp('2016-03-09 00:00:00')
>>>, 3, 9)
>>> - business
>>> date(2016, 12, 25) + business
Timestamp('2016-12-28 00:00:00')

You can also just retrieve the UK holidays for a specific year as a list of datetime objects using e.g.:

>>> holidays = EnglandAndWalesHolidayCalendar().holidays(
    start=date(2016, 1, 1),
    end=date(2016, 12, 31))
>>> holidays.tolist()
[Timestamp('2016-01-01 00:00:00'), Timestamp('2016-03-25 00:00:00'), Timestamp('2016-03-28 00:00:00'), Timestamp('2016-05-02 00:00:00'), Timestamp('2016-05-30 00:00:00'), Timestamp('2016-08-29 00:00:00'), Timestamp('2016-12-26 00:00:00'), Timestamp('2016-12-27 00:00:00')
>>> holidays.to_pydatetime()
array([datetime.datetime(2016, 1, 1, 0, 0),
       datetime.datetime(2016, 3, 25, 0, 0),
       datetime.datetime(2016, 3, 28, 0, 0),
       datetime.datetime(2016, 5, 2, 0, 0),
       datetime.datetime(2016, 5, 30, 0, 0),
       datetime.datetime(2016, 8, 29, 0, 0),
       datetime.datetime(2016, 12, 26, 0, 0),
       datetime.datetime(2016, 12, 27, 0, 0)], dtype=object)
>>> h.to_native_types()
['2016-01-01', '2016-03-25', '2016-03-28', '2016-05-02', '2016-05-30', '2016-08-29', '2016-12-26', '2016-12-27']

Pandas has all sorts of funny stuff you can do with series and time series in particular. Also check pandas’s docs about Custom Business Days especially the warning about possible timezone issues.

If you don’t need the power of pandas or you just don’t like it (maybe because it pulls in a bazillion dependencies and includes a gazillion modules), workalendar looks pretty good.


The origins of the class Meta idiom in python

So I keep finding this class Meta idiom in python APIs lately. Found it in factory-boy and WTForms and I suspected they both got it from Django, but I googled and couldn’t find any explanations of the reason for it or where it came from or why they’re all it class Meta. So here it is!

TL;DR What it is

The inner Meta class has absolutely no relation to python’s metaclasses. The name is just a coincidence of history (as you can read below).

There’s nothing magical about this syntax at all, here’s an example from Django’s documentation:

class Ox(models.Model):
    horn_length = models.IntegerField()
    class Meta:
        ordering = ["horn_length"]
        verbose_name_plural = "oxen"

Having an inner Meta class makes it easier for both the users and the ORM to tell what is a field on the model and what is just other information (or metadata if you like) about the model. The ORM can simply do your_model.pop('Meta') to retrieve the information it needs. You can also do this in any library you implement just as factory-boy and WTForms have done.

Some early Django history

Now for the longer story. I did some software archaeology, which should totally be a thing!, and discovered the first commit which mentions class Meta (actually class META1) in Django: commit 25264c86. There is a Release Notes wiki page which includes that change.

From there we can see how Django models were declared before the introduction of the internal class Meta. A Django Model class had a few special attributes. The db_table attribute held the SQL table name. A fields attribute was a tuple (!) of instances of field types (e.g. CharField, IntegerField, ForeignKey). These mapped to SQL table columns. One other interesting attribute was admin which was mostly used to describe how that model would behave in django’s admin interface. Now all these classes were defined in the django.core.meta package i.e. meta.Model, meta.CharField, meta.ForeignKey, meta.Admin. That’s so meta! (and probably where the name came from in the end)

In a django-developers mailing list thread from July 2005 titled Cleaner approach to ORM fields description user deelan suggests bringing some of SQLObject ‘s ideas to Django’s ORM. This seems to be the first seed of the idea of having an inner class in Django to store part of a model’s attributes:

it’s desiderable to avoid name clashes between fields, so it would be
good to have a way to wrap fields into a private namespace.
deelan, Cleaner approach to ORM fields description

At the end of the thread, django ticket 122 is created which seems to contain the first mention of a separate internal Meta class.

What started off as a backwards-compatible change, soon turned backwards-incompatible and was the first really big community-driven improvement to Django as Adrian Holovaty will later describe it in the release announcement which included the change.

The first patch on ticket 122 by Matthew Marshall started by suggesting that fields should be able to be defined directly on the model class, as class attributes (Build models using fieldname=FieldClass) rather than in the fields list. So:

class Poll(meta.Model):
    question = meta.CharField(maxlength=200)
    pub_date = meta.DateTimeField('date published')

rather than:

class Poll(meta.Model):
    fields = (
        pub_date = meta.DateTimeField('date published'),

But also that there should be two ways of defining a ForeignKey:

ForeignKey = Poll, {'edit_inline':True, 'num_in_admin':3}
#the attribute name is irrelevant here:
anything = ForeignKey(Poll, edit_inline=True, num_in_admin=3)

In his first comment, mmarshall introduces the inner class Meta to hold anything that’s not a field: the table name (strangely renamed to module_name) and the admin options. The fields would be class attributes.

The decision over what goes in an inner class and what goes in the outer class seems to be left to the user. An optional class Field inner class would be supported so the fields would live there and the metadata would live as class attributes (this seemed to offer the advantage of being backwards-compatible with the admin class attribute while allowing tables to have a column that’s also named admin.

There are some other ideas thrown around and the syntax for ForeignKey is also discussed. At one point, Adrian Holovaty (adrian) intervenes to say (about the original class Meta/class Field suggestion):

It’s too flexible, to the point of confusion. Making it possible to do either class Meta or class Field or plain class attributes just smacks of “there’s more than one way to do it.” There should be one, clear, obvious way to do it. If we decide to change model syntax, let’s have class Meta for non-field info, and all fields are just attributes of the class.
— Adrian Holovaty, django ticket 122, comment 9

The thread goes on from there. There are some detractors to the idea (citing performance and conformance to other python APIs), there are discussions about implementation details and talking again about the ForeignKey syntax.

Then, in a dramatic turn of events, Adrian Holovaty closes the ticket as wontfix!:

Jacob [Kaplan-Moss] and I have talked this over at length, and we’ve decided the model syntax shouldn’t change. Using a fieldname=FieldClass syntax would require too much “magic” behind the scenes for minimal benefit.
— Adrian Holovaty, django ticket 122, comment 33

It’s interesting, because IMHO this was a huge differentiator in terms of making django’s models API more human and was also what other frameworks like Rails and SQLObject were doing at the time.

An IRC discussion is then referenced in the ticket.2 From that discussion, it seems that adrian’s reasons for closing were mostly concerns about the ForeignKey syntax and making a backwards-incompatible change to the model. rmunn does a great job of moderating the discussion, clarifying the situation and everyone’s opinions while strongly pushing for the new syntax.

The trac ticket is reopened as a consequence and it looks like smooth-sailing from the on. Some days later the new syntax is merged and the ticket is once again closed, this time with Resolution set to fixed.

Adrian will later announce the change in a django-developers mailing list post. Here are some interesting fragments from that mailing list post:

I apologize for the backwards-incompatibility, but this is still unofficial software. ;-) Once we reach 1.0 — which is much closer now that the model syntax is changed — we’ll be very dedicated to backwards-compatibility.

I can’t think of any other backwards-incompatible changes that we’re planning before 1.0 (knock on wood). If this isn’t the last one, though, it’s at least the last major one.
— Adrian Holovaty, IMPORTANT: Django model syntax is changing

Things didn’t go as planned. In May 2006, came commit f69cf70e which was exactly another let’s-change-everything-in-one-huge-branch commit which was released as part of Django 0.95. As part of this API change, class META was renamed to class Meta (because it’s easier on the eyes). You can find the details on RemovingTheMagic wiki page. It’s funny how in ticket 122 all the comments use the Meta capitalization, except for the last person (who I guess submitted the patch) who uses META. There was some discussion, both in the ticket and on IRC, about it and a few people had concerns that users of Django would actually want to have a field called Meta in their models and the inner class name would clash with that.

That’s it. Almost…

Anyway, so that’s the end of the story of how Django got its class Meta. Now what if I told you that all of this had already happened more than one year before in the SQLObject project? Remember that first post to django-developers which said Django models should hold some of its attributes in a separate inner class like SQLObject already does?

In April 2004, Ian Bicking (creator of SQLObject) sent an email to the sqlobject-discuss mailing list:

There’s a bunch of metadata right now that is being stored in various instance variables, all ad hoc like, and with no introspective interfaces. I’d like to consolidate these into a single object/class that is separated from the SQLObject class. This way I don’t have to worry about name clashes, and I don’t feel like every added little interface will be polluting people’s classes. (Though most of the public methods that are there now will remain methods of the SQLObject subclasses, just like they are now) So I’m looking for feedback on how that should work.
— Ian Bicking, Metadata container

His code example:

class Contact(SQLObject):
     class sqlmeta(SQLObject.sqlmeta):
         table = 'contact_table'
         cacheInstances = False
     name = StringCol()

SQLObject’s community did not seem nearly as animated as Django’s. There were a couple of emails on the sqlobject-discuss mailing list from Ian Bicking which included the proposal and asked for feedback. I suspect some discussion happened through some other channels, but this community was neither as big nor as good at docummenting its functioning as Django. (And sourceforge’s interface to the mailing list archives and cvs logs does not make this easy to navigate).

A year later, Ian Bicking takes part in the django-developers mailinglist discussion where he makes some small syntax suggestions, but it does not seem that he made any other contributions to the design of this part of the Django models API.


As far as I could tell, Ian Bicking is the originator of the idea of storing metadata in a metadata container inner class. Although it was the Django project which settled on the class Meta name and popularised it outside of its own community.

Anyway, that’s the end of the story. To me, it shows just how awesome open source and the open internet can be. The fact that I was able to find all of this 11 years later, complete with the original source code, commit logs and all the discussion around the implementation on the issue tracker, mailing lists and IRC logs is just amazing community work and puts a tear in my eye.

Hope you’ve enjoyed the ride!

1 because in 2005, people were less soft-spoken

2 It’s very fun, you should read it. At some point someone’s cat catches a blue jay. And I think they meant it literally.


Run py.test test case inside any python class

I wanted a way to run the current test I was editing when I know the name of the method, but I’m too lazy to scroll up and see which class it’s defined in and then have to type that out as well. So I found a way for py.test to run any test in any class in a file by just giving it the name of that test or part of its name.

Instead of typing:

$ py.test path/to/my/test/

I can just type:

$ py.test path/to/my/test/ -k test_my_thing

Or even:

$ py.test path/to/my/test/ -k test_my

Or even:

$ py.test path/to/my/test/ -k "thing or stuff"

Woo py.test!

FWIW, nosetests can also do this with the -m option which supports regexp. But AFAIK it does not support the human-friendly " or " interface like in the last example above.


AST literal_eval

Safely evaluate an expression node or a Unicode or Latin-1 encoded string containing a Python literal or container display. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.

This can be used for safely evaluating strings containing Python values from untrusted sources without the need to parse the values oneself. It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing.

—From python’s ast library

I used to discredit everything that meant converting a string to an arbitrary data structure. This is a nice third option which seems like it would be useful in the majority of cases.


Change the system timezone for the python interpreter

Wrapping my head around timezones is hard. And testing the implications of working with different timezones is especially difficult since I now live in GMT (which is mostly the same as UTC).

I found a way to change the timezone, that is used by most of python’s stdlib, by changing the TZ environment variable:

$ python -c 'import time; print(time.tzname)'
('GMT', 'BST')
$ TZ='Europe/Stockholm' python -c 'import time; print(time.tzname)'
('CET', 'CEST')

The XY Problem

So there’s a website devoted to The XY Problem:

The XY problem is asking about your attempted solution rather than
your actual problem. This leads to enormous amounts of wasted time and
energy, both on the part of people asking for help, and on the part of
those providing help.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.