Callbacks

Configuration

Let’s assume we have a C API that allows for callbacks either into another C function or back into Ruby.

  void callback_func(char *buffer, long count, unsigned char code);
  int do_work(char *buffer, void *completion_callback);

FFI supports the mapping of Ruby closures (Proc, lambda) to C functions. In our wrapping module, we have two choices for setting up the callback.

Proc callback

The simplest way to define a callback is by using a Proc to create an anonymous function. In the example given below, the callback is assigned to a constant Callback so that we never need worry about the garbage collector removing it.

  module LibWrap
    extend FFI::Library
    ffi_lib "some_lib.so"

    callback :completion_function, [:pointer, :long, :uint8], :void
    attach_function :do_work, [:pointer, :completion_function], :int

    Callback = Proc.new do |buf_ptr, count, code|
      # finish up
    end
  end

  LibWrap.do_work( ..., LibWrap::Callback )

Upon execution of the callback by the C API, the FFI library unwraps the Proc to see if an FFI::Function has been allocated for it. If it finds one, it invokes the FFI::Function immediately. If no FFI::Function is found, it allocates an FFI::Function and invokes it.

FFI::Function

You can save a little work and gain a little flexibility by defining your callback as an FFI::Function directly. Check the rdoc page for a complete description.

  module LibWrap
    extend FFI::Library
    ffi_lib "some_lib.so"

    attach_function :do_work, [:pointer, :pointer], :int

    Callback = FFI::Function.new(:void, [:pointer, :long, :uint8]) do |buf_ptr, count, code|
      # finish up
    end
  end

  LibWrap.do_work( ..., LibWrap::Callback )

GIL

There may be situations where the callback function is another C function in the library. In this case it may not make any sense to retain the Ruby GIL when there is no need to protect the Ruby runtime from a thread race. Releasing the GIL will allow the Ruby runtime to (potentially) schedule another thread to run and complete more work. Luckily, FFI::Function allows us to optionally release the GIL by marking the callback as blocking. This setting only effects Ruby-to-native calls; it has no effect for native-to-Ruby calls.

When using a Proc callback setup, the GIL is always retained.

  module LibWrap
    extend FFI::Library
    ffi_lib "some_lib.so"

    attach_function :do_work, [:pointer, :pointer], :int

    Callback = FFI::Function.new(:void, [:pointer, :long, :uint8], 
                                 :blocking => true) do |buf_ptr, count, code|
      # finish up
    end
  end

Looking at the code path for each kind of setup can shed some light on what’s happening under the covers.

FFI::Function, :blocking => true, Ruby to native method call

Ruby -> FFI stub for parameter conversion -> 
  release GIL -> call native function -> 
  reacquire GIL -> FFI stub for result conversion -> Ruby

FFI::Function, :blocking => true, native method to Ruby

Ruby -> FFI stub for parameter conversion ->  
  call native function -> FFI stub for result conversion -> Ruby

(The code path remains the same in this situation for :blocking => false too.)

Callback Proc

Regardless of the direction of the call (native to Ruby or Ruby to native), callback Procs always follow the same code path.

Ruby -> FFI callback stub ->
  if thread.has_gil?
    convert parameters to Ruby
    call Ruby
    convert results to native

  elsif thread.is_ruby_thread?
    acquire GIL
    convert parameters to Ruby
    call Ruby
    convert results to native
    release GIL

  else # not a Ruby-owned thread
    bundle up FFI data
    pass to Ruby callback processing thread
    wait for signal from callback processing thread
  end
-> native code