Category Archives: FYI

Monitoring celery – your distributed task queue

Hey. I recently noticed I needed some kind of celery monitoring. Here’s a summary of what I learned while searching for a solution

TL;DR: Sign up at, add checks and integrations then add a single celery task that runs every minute. Configure the Period and have it drop an email or a Slack message when the check is not invoked. Also, this post is not sponsored in any way.

Here’s a situation. I’m running a web app that offloads work to celery tasks. This web app uses the task engine to run code that potentially uses a lot of time, like language detection, label scanning, calculating similarity, indexing into a search engine, adding synonyms, adding emoji names, etc after receiving content via an API call. Then, after some time, you start to notice that the celery tasks are no longer running while the processes are running. Simple process monitoring would not have caught this as the processes were still running, but tasks were not being picked up.

Monitoring your website is easier: just invoke an endpoint and test the results. Tools like are really good at that. Celery tasks are different as they’re running in the background and typically not directly available via the internet.

Here’s what I do, in a simple list:

    • Sign-up at and add a check via Add Check. Make sure you configure the expected period and grace period and add the integrations you need.
    • Copy the newly created URL and store it somewhere in the projects
    • Add a task that invokes the endpoint for your service. An example implementation is below
    • Use a celerybeat schedule that runs the task at an interval.
    • Wait.

Now here’s some example code which you should modify to your needs. It’s in use for some time now, but I wouldn’t say it’s battle tested.

from celery.utils.log import get_task_logger
from django.apps import apps
from requests import post, get, ConnectTimeout, status_codes
from brownpapersession.celery import app as celery_app, CeleryMailingTask
log = get_task_logger(__name__)
@celery_app.task(name='integration.send_healthcheck', ignore_result=True, base=CeleryMailingTask)
def send_healthcheck(**kwargs):
  config = apps.get_app_config('integration')
  if not config.HEALTHCHECKS_URI:
    log.warning('service=healthchecks, reason=configuration')
    response = get(
        'User-Agent': '',
  except ConnectTimeout as e:
    log.warning('service=healthchecks, uri={uri}, detail={detail}, reason=timeout'.format(
    raise AssertionError(e)
  if == response.status_code:
    log.debug('service=healthchecks, uri={uri}, http.status={status}'.format(
    log.warning('service=healthchecks, uri={uri}, http.status={status}'.format(

Now what’s the beauty in this? Well, this actually notifies you when something is wrong and it will not bother you when everything is running properly. Integrating is lightweight and easy. On top of that the healthchecks website shows you when the last trigger was received. Take these integrations one step further and add a webhook which resolves the issue or displays a message on your website.

Wondering what CeleryMailingTask is? It will be the subject of a next post.

Really happy to take suggestions or improvements via the comments.

First steps on building PDF files from a django app

I’ve been building a web application that operates on information extracted from a storage-engine used by GnuCash. GnuCash 2.6 comes with python-bindings, allowing a developer to rapidly build applications working on data maintained by the application Continue reading First steps on building PDF files from a django app

Broadsoft XSI event consumer on .NET

I’ve been developing a Windows application that consumes Broadsoft XSI events. Broadsoft has a “comprehensive range of VoIP Applications in a Single Platform” and XSI is a way to consume data produced by their platform, allowing integration of a modern switchboard service with other types of applications. Continue reading Broadsoft XSI event consumer on .NET

Scripted changing of XML files made easy

Changing the content of a configuration file on an Unix system, like Linux, can be easy. That task quickly becomes a whole different ballgame when you’re dealing with XML files, because this format imposes a couple of restrictions on the XML element values, for instance. You’ll probably be able to get the job done when using traditional tools like sed and awk, but I would advise against it. Continue reading Scripted changing of XML files made easy

Data structure of the XING User Profile resource

XING aims to be a platform for business professionals, offering a platform through which supply meets demand. They seem to be quite good at it. Take a moment to review the services they’re offering and judge for yourself. I think of them mainly as an European competitor to LinkedIn® and they have an API too, which is in beta testing right now. Just request a beta account and you may get an invitation. Continue reading Data structure of the XING User Profile resource

Automagically minimize javascript in Eclipse on Windows

The WordPress tool I’m developing uses jQuery scripting. I’ve seen people including a minimized version of the scripting and I think that’s a good approach. I’m using Eclipse on Windows on my development system and it happened more than once that I forgot to include the minimized version of the scripting. I found a solution in my search for a way to do this automatically. Continue reading Automagically minimize javascript in Eclipse on Windows

Using the wp settings API with a non-admin user

WordPress comes with the settings API, an API that simplifies development of configuration pages. It’s a complete API, with regard to function and security. This post will take a look at security, more specific, at access control. Continue reading Using the wp settings API with a non-admin user

Personalize your LinkedIn Public Profile url

I’ve been seeing a lot of LinkedIn the last few weeks. The reason for that will be in a post in the near future. If you have a LinkedIn profile, you’ve probably noticed the “Public Profile” url on it. This url links directly to your profile. Every profile has one and the default link is hard to remember: it consists of your name and additional data. You can change that link yourself and it’s real easy. The steps are below. Continue reading Personalize your LinkedIn Public Profile url