5.2 Usage of Multi-threaded FFTW

Here, it is assumed that the reader is already familiar with the usage of the uniprocessor FFTW routines, described elsewhere in this manual. We only describe what one has to change in order to use the multi-threaded routines.

First, programs using the parallel complex transforms should be linked with -lfftw3_threads -lfftw3 -lm on Unix, or -lfftw3_omp -lfftw3 -lm if you compiled with OpenMP. You will also need to link with whatever library is responsible for threads on your system (e.g. -lpthread on GNU/Linux) or include whatever compiler flag enables OpenMP (e.g. -fopenmp with gcc).

Second, before calling any FFTW routines, you should call the function:

int fftw_init_threads(void);

This function, which need only be called once, performs any one-time initialization required to use threads on your system. It returns zero if there was some error (which should not happen under normal circumstances) and a non-zero value otherwise.

Third, before creating a plan that you want to parallelize, you should call:

void fftw_plan_with_nthreads(int nthreads);

The nthreads argument indicates the number of threads you want FFTW to use (or actually, the maximum number). All plans subsequently created with any planner routine will use that many threads. You can call fftw_plan_with_nthreads, create some plans, call fftw_plan_with_nthreads again with a different argument, and create some more plans for a new number of threads. Plans already created before a call to fftw_plan_with_nthreads are unaffected. If you pass an nthreads argument of 1 (the default), threads are disabled for subsequent plans.

You can determine the current number of threads that the planner can use by calling:

int fftw_planner_nthreads(void);

With OpenMP, to configure FFTW to use all of the currently running OpenMP threads (set by omp_set_num_threads(nthreads) or by the OMP_NUM_THREADS environment variable), you can do: fftw_plan_with_nthreads(omp_get_max_threads()). (The ‘omp_’ OpenMP functions are declared via #include <omp.h>.)

Given a plan, you then execute it as usual with fftw_execute(plan), and the execution will use the number of threads specified when the plan was created. When done, you destroy it as usual with fftw_destroy_plan. As described in Thread safety, plan execution is thread-safe, but plan creation and destruction are not: you should create/destroy plans only from a single thread, but can safely execute multiple plans in parallel.

There is one additional routine: if you want to get rid of all memory and other resources allocated internally by FFTW, you can call:

void fftw_cleanup_threads(void);

which is much like the fftw_cleanup() function except that it also gets rid of threads-related data. You must not execute any previously created plans after calling this function.

We should also mention one other restriction: if you save wisdom from a program using the multi-threaded FFTW, that wisdom cannot be used by a program using only the single-threaded FFTW (i.e. not calling fftw_init_threads). See Words of Wisdom-Saving Plans.

Finally, FFTW provides a optional callback interface that allows you to replace its parallel threading backend at runtime:

void fftw_threads_set_callback(
    void (*parallel_loop)(void *(*work)(void *), char *jobdata, size_t elsize, int njobs, void *data),
    void *data);

This routine (which is not threadsafe and should generally be called before creating any FFTW plans) allows you to provide a function parallel_loop that executes parallel work for FFTW: it should call the function work(jobdata + elsize*i) for i from 0 to njobs-1, possibly in parallel. (The ‘data‘ pointer supplied to fftw_threads_set_callback is passed through to your parallel_loop function.) For example, if you link to an FFTW threads library built to use POSIX threads, but you want it to use OpenMP instead (because you are using OpenMP elsewhere in your program and want to avoid competing threads), you can call fftw_threads_set_callback with the callback function:

void parallel_loop(void *(*work)(char *), char *jobdata, size_t elsize, int njobs, void *data)
{
#pragma omp parallel for
    for (int i = 0; i < njobs; ++i)
        work(jobdata + elsize * i);
}

The same mechanism could be used in order to make FFTW use a threading backend implemented via Intel TBB, Apple GCD, or Cilk, for example.