C++: function pointers with runtime context

cpp

In this post, I’ll consider techniques that allow adding runtime context to C/C++ function pointers.

• Background
  • Functions as variables
  • Capturing runtime context for a function
• Problem
• Solution
  • Approach 1: change the API
  • Approach 2: just-in-time compilation
  • Approach 3: capture runtime context as a template argument
    • Mapping runtime values into compile-time values
    • Static registry for runtime contexts
    • “Greeting callback” example
    • Binary size implications
• TL;DR

Background

Functions as variables

When you code, sometimes you need a variable to reference a function. Maybe, you need to pass a callback into an asynchronous function to be called upon completion. Or, maybe, you need to abstract certain logic behind a callable interface in a lightweight way without introducing any new interfaces or types into the system. How can this be done in C++?

In Modern C++, there are two ways available out-of-box: function pointers and std::function.

Function pointers are the old C way, so it has the benefits of being small, fast and backward-compatible with C libraries.

std::function was added in C++11, so it has a newer, more ergonomic API and semantics (ex.: argument binding). These features come at a cost of storing extra memory and having additional indirections for function calls.

The following demo shows how both ways can be used:

using GreetFnPtr = void(*)();          // Function pointer type.
using GreetFn = std::function<void()>; // `std::function` type.

void GreetAlice() {
  std::cout << "Hello, Alice!" << std::endl;  
}

void GreetBob() {
  std::cout << "Hello, Bob!" << std::endl;  
}

void DemoFunctionPointers() {
  GreetFnPtr alice_greet = &GreetAlice;
  GreetFnPtr bob_greet = &GreetBob;
  GreetFnPtr random_greet = (rand() & 16384) ? alice_greet : bob_greet;
  
  alice_greet();  // Prints "Hello, Alice!"
  bob_greet();    // Prints "Hello, Bob!"
  random_greet(); // Prints either "Hello, Alice!" or "Hello, Bob!"
}

void DemoStdFunctions() {
  GreetFnPtr alice_greet = GreetAlice;
  GreetFnPtr bob_greet = GreetBob;
  GreetFnPtr random_greet = (rand() & 16384) ? alice_greet : bob_greet;
  
  alice_greet();  // Prints "Hello, Alice!"
  bob_greet();    // Prints "Hello, Bob!"
  random_greet(); // Prints either "Hello, Alice!" or "Hello, Bob!"
}

Capturing runtime context for a function

There is one major difference between std::function and function pointers that I want to scrutinize in this post: the ability to capture runtime context and store it alongside a function upon its creation. This great feature not only leads to better code reusability - sometimes, this is the only way one can succeed in solving a technical task.

std::function was designed with context captures in mind. Assuming one’s codebase is written in Modern C++, std::function should be leveraged instead of function pointers most of the time to get an easy access to this feature (among many others).

Let’s see how the previous demo can be re-written to leverage context captures via std::function:

using GreetFn = std::function<void()>;

void GreetByName(const std::string& name) {
  std::cout << "Hello, " << name << "!" << std::endl;  
}

void DemoStdFunctions() {  
  std::string alice_name = "Alice";
  std::string bob_name = "Bob";

  // Capture context using `std::bind`.
  GreetFn alice_greet = std::bind(GreetByName, alice_name);
  GreetFn bob_greet = std::bind(GreetByName, bob_name);
  GreetFn random_greet = (rand() & 16384) ? alice_greet : bob_greet;
  
  alice_greet();  // Prints "Hello, Alice!"
  bob_greet();    // Prints "Hello, Bob!"
  random_greet(); // Prints either "Hello, Alice!" or "Hello, Bob!"
  
  // Capture context using lambdas.
  alice_greet = [alice_name] { GreetByName(alice_name); };
  bob_greet = [bob_name] { GreetByName(bob_name); };
  random_greet = (rand() & 16384) ? alice_greet : bob_greet;
  
  alice_greet();  // Prints "Hello, Alice!"
  bob_greet();    // Prints "Hello, Bob!"
  random_greet(); // Prints either "Hello, Alice!" or "Hello, Bob!"
}

As one can see, we can now have a single generic GreetByName function instead of per-name functions (ex.: GreetAlice, GreetBob) and then bind runtime data to it later.

Problem

Out-of-box, function pointers don’t come with any support for capturing runtime context. However, if your C++ code works with a C library then you may find yourself using function pointers quite heavily. How can you pass some runtime context alongside a function pointer?

Solution

In this section, we’ll consider 3 approaches for adding runtime context to C/C++ function pointers. Each one has pros and cons, so ultimately there’s a valid use-case for each of the approaches.

Approach 1: change the API

One common technique is to change the C library API and make it carry runtime context alongside a function. The C library will also become responsible for passing the runtime context each time the function is called:

using GreetFnPtrWithContext = void(*)(const char*);

void GreetByName(const char* name) { 
  std::cout << "Hello, " << name << "!" << std::endl;  
}

void DemoFunctionPointers() {
  std::string alice_name = "Alice";
  std::string bob_name = "Bob";

  GreetFnPtrWithContext alice_greet = &GreetByName;
  // Not: `alice_name` must outlive `alice_greet_context`.
  const char* alice_greet_context = alice_name.c_str();
  
  GreetFnPtrWithContext bob_greet = &GreetByName;
  // Not: `bob_name` must outlive `bob_greet_context`.
  const char* bob_greet_context = bob_name.c_str();
  
  GreetFnPtrWithContext random_greet = nullptr;
  const char* random_greet_context = nullptr;
  if (rand() & 16384) {
    random_greet = alice_greet;
    random_greet_context = alice_greet_context;
  } else {
    random_greet = bob_greet;
    random_greet_context = bob_greet_context;
  }
  
  alice_greet(alice_greet_context);   // Prints "Hello, Alice!"
  bob_greet(bob_greet_context);       // Prints "Hello, Bob!"
  random_greet(random_greet_context); // Prints either "Hello, Alice!" or
                                      //               "Hello, Bob!"
}

Besides the potential memory lifetime issues (which come up a lot when doing C/C++ interop anyways), this approach is totally workable! If there’s a way to change the C library API, one should prefer this route. Sometimes, however, there is no flexibility to make such a change. What should one try next?

Approach 2: just-in-time compilation

On some platforms, you can leverage runtime (just-in-time, or JIT) compilation techniques to generate a platform-specific assembly code for a simple function that:

1. Keeps the function pointer and the context as local variables;
2. Calls the function pointer with the context as an argument;
3. Returns the result.

This option is not universally accessible as some major platforms don’t allow writing executable memory (ex. iOS). It’s quite an advanced programming technique too! Unless you’re an expert in the field, you best bet is to let LLVM handle JIT compilation. Otherwise, you’d have to implement and maintain different assembly code generators for different chip architectures.

This approach will not be covered in detail here as it’s a topic of its own and deserves a dedicated post.

Approach 3: capture runtime context as a template argument

This approach is based on C++ templates. In order to make this work, we’ll have to narrow down the scope a bit:

1. C++14 compiler must be used (or newer);
2. Only a small number of different integer values can be captured at runtime (~100-10000);
3. Runtime contexts must be mapped against a small integer key in a static registry.

A particular use-case might or might not be aligned with this new set of constraints. I came up with this technique when establishing a mapping between native functions in C++ and functions of a scripting language. As I only had hundreds offunctions to deal with, this approach worked out for me beautifully. This way, I know for sure there is at least one use-case that it serves well :) In case you have a use-case that can live with these constraints, congrats and please keep reading!

Mapping runtime values into compile-time values

First, we’ll need to have some functionality that can map a runtime value into a template, compile-time value.

Given that all C++ templates should be known at compile-time, this is when we’ll need to make a decision about a range of runtime values that we want to support. In the following example, we will use size_t as the type and will allow users to specify the maximum supported runtime value by setting the std::size_t kMaxSwitchIdx template argument.

#include <cstddef>
#include <utility>

namespace lib {
namespace internal {

template <template <std::size_t> typename F, std::size_t... Indices>
auto GetCompileSwitchCallbackImpl(std::index_sequence<Indices...>,
                                  std::size_t switch_idx) {
  decltype(&F<0>::CallbackFn) result = nullptr;
  std::initializer_list<std::size_t>(
      {(switch_idx == Indices ? (result = &F<Indices>::CallbackFn),
        std::size_t()         : std::size_t())...});

  return result;
}

}  // namespace internal

// Provides a compile-time functionality that emulates the C/C++ Switch-Case
// operator.
//
// If `switch_idx <= kMaxSwitchIdx`, this function returns a pointers to the 
// `F<kIndex>::CallbackFn` static method where `kIndex` equals to `switch_idx`.
//
// If `switch_idx > kMaxSwitchIdx`, this function returns `nullptr`.
//
// Requirements:
//
// `F` must be a template class/struct with one `std::size_t` template argument.
// `F` must have a static method called `CallbackFn`. The signature of the
// method must be the same for all instantiations.
template <std::size_t kMaxSwitchIdx, template <std::size_t> typename F>
auto GetCompileSwitchCallback(std::size_t switch_idx) {
  return internal::GetCompileSwitchCallbackImpl<F>(
      std::make_index_sequence<kMaxSwitchIdx + 1>(), switch_idx);
}

}  // namespace lib

Although GetCompileSwitchCallback is not quite enough to make the greeting example work, it’s a very useful utility that can be shared across many use-cases.

Static registry for runtime contexts

To proceed, we’ll need to have a static registry that maps between size_t and std::string. For the sake of simplicity, we won’t bother about thread safety or garbage collection here:

#include <algorithm>
#include <string>
#include <vector>

std::vector<std::string>& GetStrCollection() {
  static std::vector<std::string> str_collection;
  return str_collection;
}

// Returns the index of `str` in `GetStrCollection()`. If not present, adds
// `str` into the collection and generates a new index.
std::size_t FindOrInsertStr(const std::string& str) {
  auto& str_collection = GetStrCollection();

  auto it = std::find(str_collection.begin(), str_collection.end(), str);
  if (it == str_collection.end()) {
    str_collection.push_back(str);
    return str_collection.size() - 1;
  } else {
    return it - str_collection.begin();
  }
}

“Greeting callback” example

Finally, we can assemble all the pieces together to provide an implementation for our “greeting callback” example from earlier sections:

using GreetFnPtr = void (*)();

template <std::size_t kIndex>
struct GreetFnPtrHelper {
  static void CallbackFn() {
    std::cout << "Hello, " << GetStrCollection()[kIndex] << "!" << std::endl;
  }
};

GreetFnPtr CreateGreetFnPtrByName(const std::string& name) {
  static constexpr std::size_t kMaxNumUniqueNames = 100;
  return lib::GetCompileSwitchCallback<kMaxNumUniqueNames, GreetFnPtrHelper>(
      FindOrInsertStr(name));
}

void DemoFunctionPointers() {
  std::string alice_name = "Alice";
  std::string bob_name = "Bob";

  GreetFnPtr alice_greet = CreateGreetFnPtrByName(alice_name);
  GreetFnPtr bob_greet = CreateGreetFnPtrByName(bob_name);
  GreetFnPtr random_greet = (rand() & 16384) ? alice_greet : bob_greet;
  
  alice_greet();  // Prints "Hello, Alice!"
  bob_greet();    // Prints "Hello, Bob!"
  random_greet(); // Prints either "Hello, Alice!" or "Hello, Bob!"
}

You can find a complete code for this example here as a Github Gist.

Binary size implications

One thing worth mentionting are implications on the resulting binary size. As one can imagine, fully specializing a C++ template N times can lead to quite a bloated binary.

Below, one can find the binary sizes for the main_string.cc example from the Github Gist compiled for various values of N. The example is compiled with Clang 13.0.0 on a M1 MacBook Pro (arm64 arch) with the -std=c++14 -O2 flags:

N	Binary Size (in bytes)
10	42,616
100	92,232
1000	524,744
10000	5,004,152

As one can see, the binary size can get quite large as N grows. For this reason, this approach is quite niche and shouldn’t be pursuied unless there no flexibility to change the C library API (approach #1). However, sometimes there is no opportunity to do so, so it’s good to be aware of the whole spectrum of techniques available.

TL;DR

• If your codebase is written in Modern C++, then consider using std::function over function pointers;
• If that’s not possible, then consider changing the underlying APIs to accept a function context pointer alongside a function pointer itself (approach #1);
• If that’s not possible and the number of unique runtime contexts is…:
  • …unbounded, then consider utilizing JIT compilation (approach #2)
  • …small (~100-10000), then consider using C++ templates to capture runtime contexts as template arguments (approach #3)

Unfortunately, if there’s no flexibility to use std::function, no flexibility to change APIs, no flexibility to JIT and the number of unique runtime contexts is unbounded, then I don’t have a ready solution for you. In this case, my recommendation is to take a step back and reconsider the entire design to try finding an alternative path forward.

---

Thanks for reading! Please reach out with questions and comments!