The Rust FFI Omnibus

Rust functions with slice arguments

Rust slices bundle the concept of a pointer to a chunk of data together with the number of elements. In C, arrays are composed of the same pieces, but there is no standard container that keeps them together.

extern crate libc;

use libc::size_t;
use std::slice;

#[no_mangle]
pub extern "C" fn sum_of_even(n: *const u32, len: size_t) -> u32 {
    let numbers = unsafe {
        assert!(!n.is_null());

        slice::from_raw_parts(n, len as usize)
    };

    numbers
        .iter()
        .filter(|&v| v % 2 == 0)
        .sum()
}

Converting an array is a two-step process:

  1. Assert that the C pointer is not NULL as Rust references may never be NULL.

  2. Use from_raw_parts to convert the pointer and length into a slice. This is an unsafe operation because we may be dereferencing invalid memory.

C

#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>

extern uint32_t
sum_of_even(const uint32_t *numbers, size_t length);

int main(void) {
  uint32_t numbers[] = {1, 2, 3, 4, 5, 6};
  size_t length = sizeof numbers / sizeof *numbers;
  uint32_t sum = sum_of_even(numbers, length);
  printf("%" PRIu32 "\n", sum);
}

Calling from C is straight-forward, as we’ve made the Rust code match the capabilities of C. The only complication is ensuring that the number of elements is the same between the definition and the function call.

Ruby

require 'ffi'

module SliceArgumentsFFI
  extend FFI::Library
  ffi_lib 'slice_arguments'
  attach_function :sum_of_even, [:pointer, :size_t], :uint32
end

class SliceArguments
  extend SliceArgumentsFFI

  def self.sum_of_even(numbers)
    buf = FFI::MemoryPointer.new(:uint32, numbers.size)
    buf.write_array_of_uint32(numbers)
    super(buf, numbers.size)
  end
end

puts SliceArguments.sum_of_even([1,2,3,4,5,6])

Calling from Ruby requires more work than previous examples. This time, we use MemoryPointer to allocate space to store our integers. Once created, we copy the values into it using write_array_of_uint32.

Python

#!/usr/bin/env python3

import sys, ctypes
from ctypes import POINTER, c_uint32, c_size_t

prefix = {'win32': ''}.get(sys.platform, 'lib')
extension = {'darwin': '.dylib', 'win32': '.dll'}.get(sys.platform, '.so')
lib = ctypes.cdll.LoadLibrary(prefix + "slice_arguments" + extension)

lib.sum_of_even.argtypes = (POINTER(c_uint32), c_size_t)
lib.sum_of_even.restype = ctypes.c_uint32

def sum_of_even(numbers):
    buf_type = c_uint32 * len(numbers)
    buf = buf_type(*numbers)
    return lib.sum_of_even(buf, len(numbers))

print(sum_of_even([1,2,3,4,5,6]))

Calling from Python requires more work than previous examples. This time, we create a new type to store our integers and instantiate the type using the values.

Haskell

{-# LANGUAGE ForeignFunctionInterface #-}

import Data.Word (Word32)
import Foreign (Ptr)
import Foreign.Marshal.Array (withArrayLen)

foreign import ccall "sum_of_even"
  sum_of_even :: Ptr Word32 -> Word32 -> Word32

main :: IO ()
main = withArrayLen [1,2,3,4,5,6] $ \len arr ->
    print (sum_of_even arr (fromIntegral len))

For this example, we can use the withArrayLen function, which takes a Haskell array whose contents are Storable (i.e. serializable to byte sequences that C can understand) and produces a packed array of those values, which it then passes, along with the array’s length, to a callback function. In this case, it passes the array’s length as type Int, which we convert into the expected CUInt type using the fromIntegral function.

Node.js

const ffi = require('ffi-napi');
const ref = require('ref-napi');
const array = require('ref-array-di')(ref);

const U32array = array(ref.types.uint32);

const lib = ffi.Library('libslice_arguments', {
  sum_of_even: ['uint32', [U32array, 'size_t']],
});

const numbers = new U32array([1, 2, 3, 4, 5, 6]);
console.log(lib.sum_of_even(numbers, numbers.length));

We need to use the ref-napi and ref-array-di packages to wrap Node.js memory buffers into array-like objects which can be easily manipulated from JavaScript. The U32array type (constructed using primitives from ref.types) can be then used in function signatures.

C#

using System;
using System.Runtime.InteropServices;

class SliceArguments
{
    [DllImport("slice_arguments")]
    private static extern uint sum_of_even(int[] n, UIntPtr len);

    public static uint SumOfEven(int[] n)
    {
        return sum_of_even(n, (UIntPtr)n.Length);
    }

    static public void Main()
    {
        var sum = SliceArguments.SumOfEven(new [] {1,2,3,4,5,6});
        Console.WriteLine(sum);
    }
}

Passing an array is complicated a bit as we need to pass both a pointer to the data as well as the length of the array. Unlike previous examples, we bring in the non-idiomatic snake_case function as a private method. We can then add a public method that wraps the private one and provides the expected interface.

The C code uses a size_t, a type whose size changes depending on the platform. To mirror that, we use a UIntPtr.

Julia

#!/usr/bin/env julia
using Libdl

libname = "slice_arguments"
if !Sys.iswindows()
    libname = "lib$(libname)"
end

lib = Libdl.dlopen(libname)
sumofeven_sym = Libdl.dlsym(lib, :sum_of_even)

sumofeven(a:: Array{UInt32}) = ccall(
    sumofeven_sym,
    UInt32,
    (Ptr{UInt32}, Csize_t),
    a,
    length(a))

println(sumofeven(UInt32[1, 2, 3, 4, 5, 6]))

Passing an array requires both a pointer and the length of the array. The array is implicitly converted to a pointer to the first element, of type Ptr{UInt32}. The type Csize is the Julia equivalent of size_t. We also enforce the function argument type to Array{UInt32}, which is the specific array type for which the native function is effectively compatible with.

This example gets a bit more complicated because there are two primitive pointer types in Julia. Ptr{T} is a plain memory address to a value of type T, whereas Ref{T} is a managed pointer to data that the Julia garbage collector will not free for as long as this Ref is referenced. These types will be converted to a C compatible pointer type in ccall in either case.

While it is preferred to use Ref when passing arguments by pointer to native functions, and even more so when passing them via mutable pointers (so that the native function may modify the pointed object), C-style arrays are an exception to the rule, and should be passed with a plain Ptr and length.