findig.tools.counter — Hit counters for apps and resources

The findig.tools.counter module defines the Counter tool, which can be used as a hit counter for your application. Counters can count hits to a particular resource, or globally within the application.

class findig.tools.counter.Counter(app=None, duration=-1, storage=None)

A Counter counter keeps track of hits (requests) made on an application and its resources.

Parameters:

• app (findig.App, or a subclass like findig.json.App.) – The findig application whose requests the counter will track.
• duration (datetime.timedelta or int representing seconds.) – If given, the counter will only track hits that occurred less than this duration before the current time. Otherwise, all hits are tracked.
• storage – A subclass of AbstractLog that should be used to store hits. By default, the counter will use a thread-safe, in-memory storage class.

attach_to(app)

Attach the counter to a findig application.

Note

This is called automatically for any app that is passed to the counter’s constructor.

By attaching the counter to a findig application, the counter is enabled to wrap count hits to the application and fire callbacks.

Parameters:app (findig.App, or a subclass like findig.json.App.) – The findig application whose requests the counter will track.

partition(name, fgroup)

Create a partition that is tracked by the counter.

A partition can be thought of as a set of mutually exclusive groups that hits fall into, such that each hit can only belong to one group in any single partition. For example, if we partition a counter by the IP address of the requesting clients, each possible client address can be thought of as one group, since it’s only possible for any given hit to come from just one of those addresses.

For every partition, a grouping function must be supplied to help the counter determine which group a hit belongs to. The grouping function takes a request as its parameter, and returns a hashable result that identifies the group. For example, if we partition by IP address, our grouping function can either return the IP address’s string representation or 32-bit (for IPv4) integer value.

By setting up partitions, we can query a counter for the number of hits belonging to a particular group in any of our partitions. For example, if we wanted to count the number GET requests, we could partition the counter on the request method (here our groups would be GET, PUT, POST, etc) and query the counter for the number of hits in the GET group in our request method partition:

counter = Counter(app)

# Create a partition named 'method', which partitions our
# hits by the request method (in uppercase).
counter.partition('method', lambda request: request.method.upper())

# Now we can query the counter for hits belonging to the 'GET' 
# group in our 'method' partition
hits = counter.hits()
number_of_gets = hits.count(method='GET')

Parameters:

• name – The name for our partition.
• fgroup – The grouping function for the partition. It must] be a callable that takes a request and returns a hashable value that identifies the group that the request falls into.

This method can be used as a decorator factory:

@counter.partition('ip')
def getip(request):
    return request.remote_addr

A counter may define more than one partition.

every(n, callback, after=None, until=None, resource=None)

Call a callback every n hits.

Parameters:

• resource – If given, the callback will be called on every n hits to the resource.
• after – If given, the callback won’t be called until after this number of hits; it will be called on the (after+1)th hit and every nth hit thereafter.
• until – If given, the callback won’t be called after this number of hits; it will be called up to and including this number of hits.

If partitions have been set up (see partition()), additional keyword arguments can be given as {partition_name}={group}. In this case, the hits are filtered down to those that match the partition before issuing callbacks. For example, we can run some code on every 100th GET request after the first 1000 like this:

counter.partition('method', lambda r: r.method.upper())

@counter.every(100, after=1000, method='GET')
def on_one_hundred_gets(method):
    pass

Furthermore, if we wanted to issue a callback on every 100th request of any specific method, we can do this:

@counter.every(100, method=counter.any)
def on_one_hundred(method):
    pass

The above code is different from simply every(100, callback) in that every(100, callback) will call the callback on every 100th request received, while the example will call the callback of every 100th request of a particular method (every 100th GET, every 100th PUT, every 100th POST etc).

Whenever partition specs are used to register callbacks, then the callback must take a named argument matching the partition name, which will contain the partition group for the request that triggered the callback.

at(n, callback, resource=None)

Call a callback on the nth hit.

Parameters:resource – If given, the callback will be called on every n hits to the resource.

Like every(), this function can be called with partition specifications.

This function is equivalent to every(1, after=n-1, until=n)

after_every(n, callback, after=None, until=None, resource=None)

Call a callback after every n hits.

This method works exactly like every() except that callbacks registered with every() are called before the request is handled (and therefore can throw errors that interupt the request) while callbacks registered with this function are run after a request has been handled.

after(n, callback, resource=None)

Call a callback after the nth hit.

This method works exactly like at() except that callbacks registered with at() are called before the request is handled (and therefore can throw errors that interupt the request) while callbacks registered with this function are run after a request has been handled.

hits(resource=None)

Get the hits that have been recorded by the counter.

The result can be used to query the number of total hits to the application or resource, as well as the number of hits belonging to specific partition groups:

# Get the total number of hits
counter.hits().count()

# Get the number of hits belonging to a partition group
counter.hits().count(method='GET')

The result is also an iterable of (datetime.datetime, partition_mapping) objects.

Parameters:resource – If given, only hits for this resource will be retrieved.

class findig.tools.counter.AbstractLog(duration, resource)

Abstract base for a storage class for hit records.

This module provides a thread-safe, in-memory concrete implementation that is used by default.

__init__(duration, resource)

Initialize the abstract log

All implementations must support this signature for their constructor.

Parameters:

• duration (datetime.timedelta or int representing seconds.) – The length of time for which the log should store records. Or if -1 is given, the log should store all records indefinitely.
• resource – The resource for which the log will store records.

__iter__()

Iter the stored hits.

Each item iterated must be a 2-tuple in the form (datetime.datetime, partitions).

count(**partition_spec)

Return the number of hits stored.

If no keyword arguments are given, then the total number of hits stored should be returned. Otherwise, keyword arguments must be in the form {partition_name}={group}. See Counter.partition().

track(partitions)

Store a hit record

Parameters:partitions – A mapping from partition names to the group that the hit matches for the partition. See Counter.partition().

Counter example

Counters can be used to implement more complex tools. For example, a simple rate-limiter can be implemented using the counter API:

from findig.json import App
from findig.tools.counter import Counter
from werkzeug.exceptions import TooManyRequests

app = App()

# Using the counter's duration argument, we can set up a
# rate-limiter to only consider requests in the last hour.
counter = counter(app, duration=3600)

LIMIT = 1000

@counter.partition('ip')
def get_ip(request):
    return request.remote_addr

@counter.every(1, after=1000, ip=counter.any)
def after_thousandth(ip):
    raise TooManyRequests("Limit exceeded for: {}".format(ip))