Native Django alerting package

Here's another idea for a django package I would like to see (which is code for I may build this one day...) or someone will tell me it already exists.

The other week I was debugging a issue with a client and had nailed the ad-hoc query to pull out some rows that needed updating. I then proceeded to fix them and all was well with the world. But then a thought occurred to me that I would like to get notified if this query was ever non-zero again. In fact I check the query again today and it was non-zero, so I still have a bug to track down.

On to the package idea, I would like to easily store a query (ideally as a Django ORM query) and a threshold and/or comparison. Then these queries would get checked regularly and would notify something when the threshold was reached or exceeded.

I do understand that the description is fairly vague and it might not actually be a package and probably more a service, especially given some of the likely infrastructure requirements But it would have a nice simplicity as a package that could be integrated into an existing project.

What do you think? Does a package exist like this? or am I just looking for a monitoring service?