Category Archives: python

Monitoring celery – your distributed task queue

Hey. I recently noticed I needed some kind of celery monitoring. Here’s a summary of what I learned while searching for a solution

TL;DR: Sign up at, add checks and integrations then add a single celery task that runs every minute. Configure the Period and have it drop an email or a Slack message when the check is not invoked. Also, this post is not sponsored in any way.

Here’s a situation. I’m running a web app that offloads work to celery tasks. This web app uses the task engine to run code that potentially uses a lot of time, like language detection, label scanning, calculating similarity, indexing into a search engine, adding synonyms, adding emoji names, etc after receiving content via an API call. Then, after some time, you start to notice that the celery tasks are no longer running while the processes are running. Simple process monitoring would not have caught this as the processes were still running, but tasks were not being picked up.

Monitoring your website is easier: just invoke an endpoint and test the results. Tools like are really good at that. Celery tasks are different as they’re running in the background and typically not directly available via the internet.

Here’s what I do, in a simple list:

    • Sign-up at and add a check via Add Check. Make sure you configure the expected period and grace period and add the integrations you need.
    • Copy the newly created URL and store it somewhere in the projects
    • Add a task that invokes the endpoint for your service. An example implementation is below
    • Use a celerybeat schedule that runs the task at an interval.
    • Wait.

Now here’s some example code which you should modify to your needs. It’s in use for some time now, but I wouldn’t say it’s battle tested.

from celery.utils.log import get_task_logger
from django.apps import apps
from requests import post, get, ConnectTimeout, status_codes
from brownpapersession.celery import app as celery_app, CeleryMailingTask
log = get_task_logger(__name__)
@celery_app.task(name='integration.send_healthcheck', ignore_result=True, base=CeleryMailingTask)
def send_healthcheck(**kwargs):
  config = apps.get_app_config('integration')
  if not config.HEALTHCHECKS_URI:
    log.warning('service=healthchecks, reason=configuration')
    response = get(
        'User-Agent': '',
  except ConnectTimeout as e:
    log.warning('service=healthchecks, uri={uri}, detail={detail}, reason=timeout'.format(
    raise AssertionError(e)
  if == response.status_code:
    log.debug('service=healthchecks, uri={uri}, http.status={status}'.format(
    log.warning('service=healthchecks, uri={uri}, http.status={status}'.format(

Now what’s the beauty in this? Well, this actually notifies you when something is wrong and it will not bother you when everything is running properly. Integrating is lightweight and easy. On top of that the healthchecks website shows you when the last trigger was received. Take these integrations one step further and add a webhook which resolves the issue or displays a message on your website.

Wondering what CeleryMailingTask is? It will be the subject of a next post.

Really happy to take suggestions or improvements via the comments.

weasyprint and glyphicons font on Windows 7: couldn’t load font “Glyphicons Halflings Not-Rotated 21px”

I recently developed tooling using python 2.7. It runs on Windows 7 (using portable python) and Linux, will do something and produce both HTML and PDF output. I used Twitter Bootstrap to format the HTML page and I created a PDF using that HTML page. Should be easy, as weasyprint is pretty mature and actually works on Windows, but pango was emitting couldn't load font messages. Simply installing the font was impossible, as it required administrator access to the operating system Continue reading weasyprint and glyphicons font on Windows 7: couldn’t load font “Glyphicons Halflings Not-Rotated 21px”

First steps on building PDF files from a django app

I’ve been building a web application that operates on information extracted from a storage-engine used by GnuCash. GnuCash 2.6 comes with python-bindings, allowing a developer to rapidly build applications working on data maintained by the application Continue reading First steps on building PDF files from a django app

Parsing json with HTML style comments using python

I recently needed to add version control information into a file which had data stored in json format. As it turns out, I’m not the only one who learned about the issues surrounding json and comments. I wanted an HTML style comment at the end of the file. The solution was real simple, when using pyparsing. Continue reading Parsing json with HTML style comments using python

Making python xmlrpclib.ServerProxy log method names and parameters revisited

In my previous post I described a procedure to log XMLRPC client calls. The procedure will do the job, but it does not suffice when you’re connecting to multiple XMLRPC servers from a single process and you want to log calls sent to one specific server. So, here’s an improvement. Continue reading Making python xmlrpclib.ServerProxy log method names and parameters revisited

Making python xmlrpclib client log method calls

I’ve updated the steps described in this post, please see the revisited post.

I’m developing python scripting which uses XMLRPC. Every now and then I find myself in a situation where I wish I could just get the calls written to my terminal, in the order I called them in, including parameters – especially when I need support. This is what I came up with in my quest to find a decent way to make the XMLRPC API write each call before it’s actually invoked Continue reading Making python xmlrpclib client log method calls