NASD Programmer's Documentation
Freelists

Introduction

Many applications and subsystems within the NASD tree have a need to dynamically allocate and deallocate fixed-size structures. The preferred mechanism for doing so is the freelist mechanism. This set of interfaces provides support for maintaining pools of fixed-size chunks of memory, which may require explicit initialization and deinitialization to use.

If a programmer wishes to maintain caches of allocated but unused memory, the freelist mechanism should be used. There are several reasons for this. One is that using a consistent set of interfaces to do so helps others to read code they are unfamiliar with and identify what it does. Another is that the freelist mechanism is capable of collecting and reporting statistics on how each pool of memory was used, allowing better tuning of the system. A third is that by using a common, unified mechanism for managing allocated but currently unused chunks of memory, the NASD system is capable of reclaiming chunks of memory which are currently unused. This is especially useful in low-memory environments.

Using freelists

To use freelists, be sure the memory module is properly initialized, and include nasd/nasd_freelist.h. Freelists have type nasd_freelist_t. The freelist interface is implemented as a set of macros for efficiency. To minimize overhead and debugging complexity, the freelist mechanism does not maintain additional data for individual allocations. Instead, it requires that users of the freelist interface provide for it typing and dereferencing information for the items in the freelist. This means that many of the freelist macros take as arguments the cast of the item type maintained in the list. The name of the structure or union element within this cast type is a pointer to the item type itself. For example, if one were maintaining a list of nasd_foo_t, that could mean that:

Given
typedef struct nasd_foo_s nasd_foo_t;
struct nasd_foo_s {
  /* actual data for nasd_foo_t here */
  nasd_foo_t  *another_foo;
};

The cast for items in the freelist is (nasd_foo_t *). The pointer to the item type, henceforth referred to as the next pointer, is another_foo. To minimize overhead, the freelist mechanism allows users to set the next pointer arbitrarily whenever items are not in the freelist. This is handy for items which are maintained in lists when they are allocated - the list pointer can then be reused as the next pointer.

Sometimes, it is desirable to maintain freelists of items which do not naturally contain fields which are correctly-formed next pointers. If there is a void pointer in the item type, it is acceptable to use this in place of the next pointer. The preferred mechanism for dealing with this is to create a union. For instance, to maintain a freelist of arrays of eight kilobytes of memory:

typedef union nasd_foo_u nasd_foo_t;
union nasd_foo_u {
  char         data[8192];
  nasd_foo_t  *next;
};
In this example, next is a valid next pointer for the freelist mechanism, and the data field of nasd_foo_t is an eight kilobyte array.

Basic freelists

The most basic kind of freelist is one which maintains chunks of data whose contents may be arbitrary, but require no special initialization or deinitialization. To use such a freelist, first declare a pointer to type nasd_freelist_t. Create the empty freelist by calling NASD_FREELIST_CREATE(). This takes four arguments. The first is the freelist pointer. The second is the maximum number of these items which should ever reside in the freelist at a particular time (extras will be returned to the system immediately). The third is how many additional items to allocate whenever the freelist is empty and an allocation is desired. The final argument is the size of the item. If the freelist pointer is NULL after evaluating NASD_FREELIST_CREATE(), the list itself could not be created.

It is often desirable at this point to make this newly-created freelist be non-empty, so that when the code begins executing, initial trips through not-yet-executed codepaths do not incur tremendous allocation costs. Do this by calling NASD_FREELIST_PRIME(), which takes four arguments. The first argument is the freelist pointer. The second is the number of items to create and add to the list. The third is the next pointer, and the fourth is the item cast.

To retrieve an item from the freelist, call NASD_FREELIST_GET(). This takes four arguments. The first is the freelist pointer. The second is a pointer to be assigned with the address of the object retrieved from the freelist. The third is the next pointer, and the fourth is the item cast.

To return an item to a freelist, call NASD_FREELIST_FREE(). This takes three arguments. The first is the freelist pointer. The second is the address of the item to return. The third is the next pointer.

When a freelist is no longer needed, it (along with its current contents) may be deallocated with NASD_FREELIST_DESTROY(). This macro takes three arguments. The first is the freelist pointer. The second is the next pointer. The third is the item cast.

Example: Let's say we have a type nasd_foo_t, for which we wish to maintain a freelist. We might have something like:


typedef struct nasd_foo_s nasd_foo_t;
struct nasd_foo_s {
  /* actual data for nasd_foo_t here */
  nasd_foo_t  *another_foo;
};

nasd_freelist_t *nasd_foo_freelist; #define NASD_MAX_FREE_FOO 1024 /* Maximum number of foos * to have in the freelist * at a time */ #define NASD_FOO_INC 64 /* How many foos to add to * the freelist at a time * when we run out */ #define NASD_FOO_INITIAL 32 /* How many foos to create * at start of day */

nasd_status_t nasd_init_foo_freelist() { NASD_FREELIST_CREATE(nasd_foo_freelist, NASD_MAX_FREE_FOO, NASD_FOO_INC, sizeof(nasd_foo_t)); if (nasd_foo_freelist == NULL) { return(NASD_NO_MEM); } NASD_FREELIST_PRIME(nasd_foo_freelist, NASD_FOO_INITIAL,next, (nasd_foo_t *)); return(NASD_SUCCESS); }

nasd_status_t nasd_get_foo(   nasd_foo_t  **foo_p) { NASD_FREELIST_GET(nasd_foo_freelist,*foo_p,next,(nasd_foo_t *)); if (*foo_p == NULL) return(NASD_NO_MEM); return(NASD_SUCCESS); }

void nasd_free_foo(   nasd_foo_t  *foo) { NASD_FREELIST_FREE(nasd_foo_freelist,foo,next); }

void nasd_shutdown_foo_freelist() { NASD_FREELIST_DESTROY(nasd_foo_freelist,next,(nasd_foo_t *)); }


Freelists with initialized items

Sometimes, it is desirable to have the items in a freelist maintain state across allocate and free operations. For instance, each item might contain a mutex. Rather than initialize and destroy a mutex each time an item is allocated from or freed to the freelist, it is more desirable to initialize a mutex each time an item is created for the freelist, and deinitialize it whenever the item is returned to the system. To that end, the freelist mechanism provides variants on the above interfaces: NASD_FREELIST_PRIME_INIT(), NASD_FREELIST_GET_INIT(), NASD_FREELIST_FREE_CLEAN(), and NASD_FREELIST_DESTROY_CLEAN().

NASD_FREELIST_PRIME_INIT() and NASD_FREELIST_GET_INIT() are very similar to NASD_FREELIST_PRIME() and NASD_FREELIST_GET(), respectively. Each takes an additional argument, however, which is an initialization function. This function should take a pointer to the item type as its sole argument, and return nasd_status_t. If the initialization is successful, it should return NASD_SUCCESS. Otherwise, it should return a meaningful error code. Likewise, NASD_FREELIST_FREE_CLEAN(), and NASD_FREELIST_DESTROY_CLEAN() take an additional argument which is a function returning void that takes a pointer to the item type as its sole argument. This function is responsible for reversing the action of the init function. For example, if we added a mutex and a condition variable to nasd_foo_t in our earlier example, we would get:


typedef struct nasd_foo_s nasd_foo_t;
struct nasd_foo_s {
  NASD_DECLARE_MUTEX(mutex)
  NASD_DECLARE_COND(cond)
  /* other data for nasd_foo_t here */
  nasd_foo_t  *another_foo;
};

nasd_freelist_t *nasd_foo_freelist; #define NASD_MAX_FREE_FOO 1024 /* Maximum number of foos * to have in the freelist * at a time */ #define NASD_FOO_INC 64 /* How many foos to add to * the freelist at a time * when we run out */ #define NASD_FOO_INITIAL 32 /* How many foos to create * at start of day */

static nasd_status_t init_foo( nasd_foo_t *foo) { nasd_status_t rc; rc = nasd_mutex_init(&foo->lock); if (rc) return(rc); rc = nasd_cond_init(&foo->cond); if (rc) { nasd_mutex_destroy(&foo->lock); return(rc); } return(NASD_SUCCESS); } static void clean_foo( nasd_foo_t *foo) { nasd_status_t rc; rc = nasd_mutex_destroy(&foo->lock); if (rc) { printf("WARNING: got 0x%x (%s) destroying foo lock\n", rc, nasd_error_string(rc)); } rc = nasd_cond_destroy(&foo->cond); if (rc) { printf("WARNING: got 0x%x (%s) destroying foo cond\n", rc, nasd_error_string(rc)); } } nasd_status_t nasd_init_foo_freelist() { NASD_FREELIST_CREATE(nasd_foo_freelist, NASD_MAX_FREE_FOO, NASD_FOO_INC, sizeof(nasd_foo_t)); if (nasd_foo_freelist == NULL) { return(NASD_NO_MEM); } NASD_FREELIST_PRIME_INIT(nasd_foo_freelist, NASD_FOO_INITIAL, next, (nasd_foo_t *), init_foo); return(NASD_SUCCESS); }

nasd_status_t nasd_get_foo(   nasd_foo_t  **foo_p) { NASD_FREELIST_GET_INIT(nasd_foo_freelist, *foo_p, next, (nasd_foo_t *), init_foo); if (*foo_p == NULL) return(NASD_NO_MEM); return(NASD_SUCCESS); }

void nasd_free_foo(   nasd_foo_t  *foo) { NASD_FREELIST_FREE_CLEAN(nasd_foo_freelist, foo, next, clean_foo); }

void nasd_shutdown_foo_freelist() { NASD_FREELIST_DESTROY(nasd_foo_freelist, next, (nasd_foo_t *), clean_foo); }

Now every nasd_foo_t resulting from a call to nasd_get_foo() contains validly-initialized mutex and condition variables.

Advanced topics

Sometimes item initialization and cleanup functions might desire additional out-of-band data. For this reason, the _INIT and _CLEAN macros also have _INIT_ARG and _CLEAN_ARG variants. These variants take an additional argument after the init or clean function which is passed as a second argument to the init or clean functions themselves. Because the freelist interface is entirely macroized, these arguments may have any type.

Freelists protect against accesses by multiple threads by using internal mutexes. These mutexes may be accessed directly by operationg on NASD_FREELIST_MUTEX_OF(freelist_pointer). To lock this mutex, use NASD_FREELIST_DO_LOCK(freelist_pointer). To unlock it, use NASD_FREELIST_DO_UNLOCK(freelist_pointer). The header file nasd_freelist.h provides variants of many of the freelist interfaces which do not take or release locks themselves. If you use this, you are responsible for correctly synchronizing access to the freelist. This has the opportunity for providing greater efficiency when batching operations, or when performing operations already protected by other locks.

If NASD_FREELIST_STATS is defined nonzero in nasd_options.h, when each freelist is destroyed, statistics about operations performed on it are printed, including the number of times items were allocated and freed from the list, how many times the list ran empty and how many more items had to be allocated, the largest number of unused items that was ever in the list, and the largest number of items that was ever allocated at a time, among others.
<--- ---> ^<br>|<br>|
Memory Time NASD Programmer's Documentation