Using libc with dart:ffi

In the earlier versions of Dart, the language had a fairly dogmatic world view: Dart was strictly a cross-platform language, with batteries included for web and server development. In other words, if Dart did not provide a feature, you were out of luck.

For example, to write to stdout in Dart: 


import 'dart:io' as io;

void main() {
  io.stdout.writeln('Hello, World!');
}

This meant that Dart was not designed to let users directly interact with the underlying operating system and was a feature of the language, not a bug; users could not be "trusted" to write safe code (outside of a poorly supported feature called native extensions).

In March 2021, Dart 2.12 added a new library called dart:ffi, which allowed Dart to interact with C libraries. This was a huge step forward for Dart, as it allowed users to write code that interacted with native code in a predictable way, and without using cumbersome techniques like asynchronous plugins or message channels.

So let's get started. First, you need to have a dynamic C library available. A dynamic library (or shared library) is precompiled code that can be loaded into a running program. The default C library is typically exported as global symbols, so we can access it in-process: 


import 'dart:ffi';

void main() {
  final global = DynamicLibrary.process();

  // To show this works, let's use the native 'time' function.
  // https://man7.org/linux/man-pages/man2/time.2.html
  final time = global.lookupFunction<
    Int64 Function(Pointer),
    int Function(Pointer)
  >('time');

  // time() takes an optional argument, which we'll ignore.
  // let's use the equivalent of NULL in C, which is provided by dart:ffi.
  final now = time(nullptr);
  print('The current time is $now.');
}

Nice! We just called a C function from Dart. But what if we wanted to write to stdout instead? The C library provides a function called write that can be used to write to a file descriptor. The file descriptor for standard output is 1, so let's change our code to write to standard output: 


import 'dart:ffi';

void main() {
  final global = DynamicLibrary.process();

  final write = global.lookupFunction<
    Int64 Function(Int32, Pointer, IntPtr),
    int Function(int, Pointer, int)
  >('write');

  // Wait, what is this 'Pointer' type?
}

Writing the signature for the write function is almost as easy as writing the signature for the time function, but now we have a new type: Pointer<Uint8>. This type represents a pointer to an array of 8-bit unsigned integers, or a buffer of bytes. We need to convert a Dart string into a native buffer of bytes, which is not possible with the Dart string type.

For this example, we'll create our own simple Allocator that uses the C malloc and free functions. We'll use this allocator to allocate a buffer of bytes, copy the string into the buffer, and then write the buffer to standard output. Here is the above example ammended to include the allocator: 


import 'dart:ffi';

void main() {
  final global = DynamicLibrary.process();

  final write = global.lookupFunction<
    Int64 Function(Int32, Pointer, IntPtr),
    int Function(int, Pointer, int)
  >('write');
  final malloc = global.lookupFunction<
    Pointer Function(IntPtr),
    Pointer Function(int)
  >('malloc');
  final free = global.lookupFunction<
    Void Function(Pointer),
    void Function(Pointer)
  >('free');

  final allocate = _Allocator(malloc, free);
}

final class _Allocator implements Allocator {
  const _Allocator(this._malloc, this._free);

  final Pointer Function(int) _malloc;
  final void Function(Pointer) _free;

  @override
  Pointer allocate(int byteCount, {int? alignment}) {
    final pointer = _malloc(byteCount);
    if (pointer.address == 0) {
      throw ArgumentError('Could not allocate $byteCount bytes.');
    }
    return pointer.cast();
  }

  @override
  void free(Pointer pointer) {
    _free(pointer);
  }
}

Now that we have an allocator, we can allocate a buffer of bytes, copy the string into the buffer, and then write the buffer to standard output. Here is a snippet of code that does just that: 


import 'dart:ffi';

void main() {
  // ...

  // Allocate a buffer of bytes and copy the string into the buffer.
  final bytes = 'Hello, World!\n'.codeUnits;
  final buffer = allocate(bytes.length);
  buffer.asTypedList(bytes.length).setAll(0, bytes);

  // Write the buffer to standard output.
  const stdout = 1;
  write(stdout, buffer, bytes.length);

  // Free the buffer.
  allocate.free(buffer);
}

Of course, this is not terribly useful (see also the complete example), but I'm hoping it will be a good introduction to the topic. In the future, I plan to write more about using dart:ffi to access native APIs otherwise unavailable in dart:io. In the meantime, check out these other great resources on the topic: