External Code

Calls to external code are useful in three situations:

Among examples belonging to the third case, one can mention updating the parameters of a real-time controller running with a real process, collecting experimental data to obtain a model, or changing the coefficients of an audio filter to add a new dimension to what the user perceives.

Implementation

Calls to external code are performed by calling functions in a shared library, also known as dynamic link library. Several shared libraries can be used simultaneously, and each of them can contain several functions. Each function must have the following prototype, given here in ISO (ANSI) C using the header file LME_Ext.h. Other languages can also be used, provided that the same calling conventions are used. In C++, for instance, prototypes must be preceded by extern "C" to disable name mangling.

lme_err fn(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)

Its three arguments are:

lme_ref lme
Pointer to a reference to the LME instance which calls the function, to be used with callbacks, and to the callbacks themselves. Definitions in LME_Ext.h assume this argument is named lme. It should be passed to all sub-functions which use callbacks.
lme_int32 nargin
Number of input arguments
lme_int32 nargout
Number of output arguments

The output of the function is 1 for success or 0 for failure.

Retrieving the value of the input arguments and setting the output arguments are performed with the help of callback functions (i.e. functions implemented in LME which are called back by the extension; the header file LME_Ext.h hide the implementation details). Currently, arguments can be real or complex matrices, arrays of any dimension and type supported by LME, strings, lists, structures, and binary objects. Callback functions which manipulate arguments return 1 if the call is successful, or 0 otherwise. Failures are not fatal; for example, if a string or numerical argument is expected, you may try to retrieve a string, then try to get a number (or a numeric matrix) if it fails.

Callbacks

There are 24 callback functions to get input arguments, set output arguments, throw errors, and output information.

Get input arguments

lme_err LMECB_GetMatrix(lme_int32 i, lme_int32 *m, lme_int32 *n, lme_float64 **re, lme_float64 **im)
Retrieves the i:th input argument as a double matrix (a 2-d array). The index i must be between 1 and nargin inclusive. *m and *n are set to the number of rows and the number of columns of the matrix, respectively (1 and 1 mean a scalar number); *re is set to a pointer to the real part, and *im to a pointer to the imaginary part if it exists, or to a null pointer otherwise. im can be a null pointer (NULL or 0) if the imaginary part is not needed. Values are stored row-wise; i.e. the real part of the 3rd element of the second row is (*re)[2*n+3].
lme_err LMECB_GetScalar(lme_int32 i, lme_float64 *re, lme_float64 *im)
Retrieves the i:th input argument as a scalar number. The index i must be between 1 and nargin inclusive. *re is set to the real part, and *im (in im is not null) to the imaginary part if it exists, or to 0 otherwise. The argument can be any numeric type (double, single, or any integer type).
lme_err LMECB_GetArray(lme_int32 i, lme_int8 *ndims, lme_int32 **size, lme_int32 *nbytes, lme_int32 *type,void **data)
Retrieves the i:th input argument as an array. The index i must be between 1 and nargin inclusive. *ndims is set to the number of dimensions (2 or larger); *size to a pointer to a vector of the *ndims dimensions; *nbytes to the number of bytes per element; *type to k_lme_type_signed_int, k_lme_type_unsigned_int, k_lme_type_realfloat, k_lme_type_complexfloat, k_lme_type_char, or k_lme_type_logical; and *data, to a pointer to the data. For complex numbers, imaginary part is stored as a separate array, after the real part.
lme_err LMECB_GetString(lme_int32 i, lme_string8 *str, lme_int32 *length)
Retrieves the i:th input argument as a string. *str is set to a pointer to the beginning of the string, and *length to the string length. Note that the string is not null-terminated.
lme_err LMECB_GetBinaryObject(lme_int32 i, lme_int32 *size, void **data)
Retrieves the i:th input argument as a binary object. *size, if size is not null, is set to its size in bytes, and *data to its address. Each extension has its own, unique binary object; an extension cannot retrieve a binary object created by another extension.
lme_err LMECB_GetObject(lme_int32 i, lme_object *o)
Retrieves the i:th input argument as a generic object. *o is set to a reference to the object. It is a structure whose first field, o->objtype, is public and describes the type of the object:
EnumValueObject type
k_lme_obj_unknown0Unknown (other)
k_lme_obj_array1Array of any type
k_lme_obj_list2List
k_lme_obj_struct3Structure
k_lme_obj_structarray4Structure array
Other fields are private. Functions below permit to extract the object contents.
lme_err LMECB_ObjectToArray(lme_object const *o, lme_int32 *ndims, lme_int32 **size, lme_int32 *nbytes, lme_int32 *type, void **data)
Gets an array of any type from a generic object. No conversion is performed; the object must be an array. Arguments have the same meaning as those of LMECB_GetArray.
lme_err LMECB_ObjectLength(lme_object const *o, lme_int32 *length)
Gives the length of a list or the number of fields of a structure from a generic object.
lme_err LMECB_GetElementFromListObject(lme_object const *o, lme_int32 i, lme_object *el)
Gets an element of a list as a generic object. *el is set to a reference to element i (first is i=1) of the list object referenced by *o.
lme_err LMECB_GetFieldNameFromStructObject(lme_object const *o, lme_int32 i, lme_char8 name[])
Gets the name of field i (first is i=1) of the structure object or structure array object referenced by *o. The name is stored in string name which must contain at least k_lme_fieldname_maxlength (32) characters. It is terminated by the null character.
lme_err LMECB_GetFieldFromStructObject(lme_object const *o, lme_string8 name, lme_object *fld)
Gets a field of structure *o as a generic object. *fld is set to a reference to the field whose name is the null-terminated string name.
lme_err LMECB_GetFieldFromStructArrayObject(lme_object const *o, lme_int i, lme_string8 name, lme_object *fld)
Gets a field of structure object *o as a generic object. *fld is set to a reference to the field whose name is the null-terminated string name of element i (first is i=1).

Set output arguments

lme_err LMECB_PushMatrix(lme_int32 m, lme_int32 n, lme_float64 **re, lme_float64 **im)
Pushes a matrix output argument on the stack. Output arguments must be pushed in reverse order, beginning with the last one. Exactly nargout matrices, arrays, and/or strings must be pushed. m and n are the number of rows and the number of columns of the matrix, respectively; *re is set to a pointer to the real part of the matrix, and *im to a pointer to its imaginary part. To push a real matrix, set im to a null pointer (NULL or 0). After the call, you should store the value of the matrix to the place pointed by *re and *im.
lme_err LMECB_PushArray(lme_int32 ndims, lme_int32 *size, lme_int32 nbytes, lme_int32 type, void **data)
Pushes an array output argument on the stack. ndims is the number of dimensions; size, a vector of ndims dimensions; nbytes, the number of bytes per element; type, the array type (cf. LMECB_GetArray); and *data is set to a pointer to the place where the array must be stored. nbytes must be 1 for k_lme_type_logical; 2 for k_lme_type_char; 8 for k_lme_type_realfloat and k_lme_type_complexfloat; or 1, 2, or 4 for k_lme_type_signed_int and k_lme_type_unsigned_int.
If one does not know the size of the array before filling it, one can replace one (and only one) dimensions in size with -1; LMECB_PushArray will replace it with the largest possible value, which depends on the memory available. Then the array can be filled (with the element layout determined by the final size), and a second call to LMECB_PushArray with the final size must be performed before pushing other output arguments (if any) and returning.
lme_err LMECB_StartPushString(lme_int32 length, lme_string8 *str)
Begins to push a string output argument on the stack, containing 8-bit characters (LMECB_PushArray should be used to push strings with characters whose code is larger that 255). The string length is specified in length; *str is set to a pointer to the buffer where you should store the string itself. The string must not be null-terminated. Once the string is stored, and before pushing anything else, call LMECB_EndPushString() to convert the string to the LME internal format.
lme_err LMECB_EndPushString()
Finishes the string pushing operation.
lme_err LMECB_PushBinaryData(lme_int32 size, void **data)
Pushes an uninitialized binary object with room for size bytes on the stack. *data is set to its address, so that it can be filled. In addition to functions which create and use binary objects, you can overload existing functions, operators such as plus or mtimes, subscript and field access such as subsref and subsasgn, and function disp to display the value.
lme_err LMECB_PushEmptyList()
Pushes an empty list on the stack. Elements can be added by pushing them and appending them to the list with LMECB_AddListElement.
lme_err LMECB_AddListElement()
Adds the object at the top of the stack to the end of the list below it.
lme_err LMECB_PushEmptyStructure()
Pushes an empty structure on the stack. Fields can be added by pushing them and appending them to the structure with LMECB_AddStructureField.
lme_err LMECB_AddStructureField(lme_string8 fieldName)
Adds the object at the top of the stack to the end of the structure below it as a field with name fieldName. fieldName is a null-terminated string.
lme_err LMECB_ConvertStructListToStructArray(lme_int ndims, lme_int const *size)
Converts the list of structures which has just been pushed to a structure array of the specified size, or to a one-column structure array if ndims is zero or size is NULL. This is the only way to create a structure array: first build a list of (scalar) structures with LMECB_PushEmptyList, LMECB_PushEmptyStructure, LMECB_AddStructureField and LMECB_AddListElement, then convert it to a structure array with LMECB_ConvertStructListToStructArray. No more fields or elements can be added to the structure arry afterwards.

Memory allocation

void *LMECB_AllocTemp(lme_int32 n)
Allocates n bytes of temporary memory. Allocating memory must be done after all output arguments have been pushed. Except for strings, this is not a problem, because matrices may be filled later. A null pointer is returned if the allocation fails. The memory needs not be freed.

Output and error handling

void LMECB_Write(lme_int32 fd, lme_string8 ptr, lme_int32 len, lme_int32 textMode)
Writes the data of size len pointed by ptr to the output channel identified by the file descriptor fd. If len is negative, data must be null-terminated. The file descriptor must have a value compatible with those used by LME functions like fprintf and fwrite. If textMode is non-zero, characters '\n' (10) are converted to the end-of-line sequence valid for the file descriptor.
lme_err LMECB_Error(lme_string8 identifier, lme_string8 message)
Throws an error with the specified identifier and message, both null-terminated strings. Null pointers are valid. The function which throws an error should return with the value returned by LMECB_Error (i.e. the usual code to throw an error is return LMECB_Error(...);).
lme_err LMECB_CheckAbort()
Checks if the user interrupts the computation, typically by pressing Control-Break on Windows, Command-. on Mac or Control-C on Linux. If the status code it returns is non-zero, computation should be aborted. This function can be called during lengthy computation to avoid blocking the application.
void LMECB_DbgWriteStr(lme_string8 str)
Writes the null-terminated string str to the standard error, followed by a new line. This is typically used during development for debugging purposes.

Start up and shut down

Functions are added to the set of built-in functions when LME starts up. They effectively extend LME. To permit LME to load them, you must provide the following function, named InstallFn:

lme_int32 InstallFn(lme_ref lme, lme_fn **fnarray)
{
    /* initialize any resource necessary for the
       functions */
    *fnarray = [array of function descriptions];
    return [number of elements in *fnarray];
}

The essential purpose of this function, which must be exported with whatever mechanism is available on your platform (__declspec(dllexport) for DLL on Windows or the PEF export options on Mac OS 9), is to refer LME to an array of function descriptions. This array, which is typically defined as static, has elements of type lme_fn:

typedef struct
{
    char name[32];
    lme_extfn fn;
    lme_int32 minnargin, maxnargin;
    lme_int32 minnargout, maxnargout;
} lme_fn;

The field name is the name of the function (what you will use in your SQ files), fn is a pointer to the function which implements the behavior of the function, minnargin and maxnargin are the minimum and maximum number of input arguments your function is ready to accept, and minnargout and maxnargout are the minimum and maximum number of output arguments your function is ready to provide. Typically, if your function can provide output argument(s), you should set minnargout to 1; LME will display a result if you omit the semicolon at the end of a call to your function.

You can implement new types of object (binary objects), at most one per extension. Functions can overload existing functions in a similar way as for objects defined with class. Overloaded functions must begin with the prefix lme_k_binary_overload_str_prefix; for example to define a function plus for the addition of your binary objects, the entry in the array of functions would be

{lme_k_binary_overload_str_prefix "plus",
    overloadedPlus, 2, 2, 1, 1}

This prefix must not be used for functions unless they take binary objects as input arguments. See the example 3 below for a complete example.

You can also allocate resources in InstallFn (such as opening files); in that case, you want to define and export a function named ShutdownFn to release these resources when LME terminates:

void ShutdownFn(lme_ref lme)
{
    /* release all resources allocated by InstallFn */
}

Examples

The following extension adds two functions to LME: plus1 which accepts up to 50 double real or complex matrix arguments and return them in the same order with 1 added, and hi which displays a message if there is no output argument, or returns it as a string if there is one.

#include "LME_ext.h"
#include <string.h>

static lme_err plus1(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  /* same as (x+1), but with multiple arguments */
{
  int i, j;
  lme_float64 *re, *im, *re1, *im1;
  lme_err status;
  lme_int32 m, n;
  
  for (i = nargout; i >= 1; i--)  /* backward */
  {
    if (i <= nargin)
    {
      status = LMECB_GetMatrix(i, &m, &n, &re, &im);
      if (!status)
        return 0;
      status = LMECB_PushMatrix(m, n, &re1, im ? &im1 : 0);
      if (!status)
        return 0;
      for (j = 0; j < m * n; j++)
        re1[j] = re[j] + 1;
      if (im)
        for (j = 0; j < m * n; j++)
          im1[j] = im[j];
    }
    else
      if (!LMECB_PushMatrix(0, 0, &re1, 0))
        return 0;
  }
  
  return 1;
}

static lme_err hi(lme_ref lme, 
          lme_int32 nargin, lme_int32 nargout)
  /* hello world */
{
  char *msg = "Hello, World!";
  
  if (nargout == 1)
  {
    lme_string8 str;
    int i;
    
    if (!LMECB_StartPushString(strlen(msg), &str))
      return 0;
    for (i = 0; i < strlen(msg); i++)  /* without the '\0' */
      str[i] = msg[i];
    if (!LMECB_EndPushString())
      return 0;
  }
  else
    LMECB_DbgWriteStr(msg);
  
  return 1;
}

static lme_fn fn[] =
{
  {"plus1", plus1, 0, 50, 0, 50},
  {"hi", hi, 0, 0, 0, 1}
};

lme_int32 InstallFn(lme_ref lme, lme_fn **fnarray)
{
  LMECB_DbgWriteStr("Installing test functions.");
  *fnarray = fn;
  return 2;
}

The extension below implements displayobject which displays the skeleton of its input argument. It shows how to scan all elements of a list or a structure.

#include "LME_ext.h"
#include <stdio.h>

static lme_err displayRec(lme_ref lme, lme_object *o)
  /* called recursively */
{
  lme_int32 status = 1, ndims, *size, len, i;
  lme_object el;
  lme_char8 str[k_lme_fieldname_maxlength];
  
  switch (o->objtype)
  {
    case k_lme_obj_unknown:
      LMECB_Write(1, "unknown", -1, 1);
      break;
    case k_lme_obj_array:
      LMECB_Write(1, "array(", -1, 1);
      status = LMECB_ObjectToArray(o, &ndims, &size,
                  NULL, NULL, NULL);
      if (!status)
        return 0;
      for (i = 0; i < ndims; i++)
      {
        sprintf(str, i > 0 ? "x%d" : "%d", size[i]);
        LMECB_Write(1, str, -1, 1);
      }
      LMECB_Write(1, ")", -1, 1);
      break;
    case k_lme_obj_list:
      LMECB_Write(1, "{", -1, 1);
      status = LMECB_ObjectLength(o, &len);
      for (i = 1; status && i <= len; i++)
      {
        if (i > 1)
          LMECB_Write(1, ",", -1, 1);
        status = LMECB_GetElementFromListObject(o, i, &el);
        if (status)
          status = displayRec(lme, &el);
      }
      LMECB_Write(1, "}", -1, 1);
      break;
    case k_lme_obj_struct:
      LMECB_Write(1, "struct(", -1, 1);
      status = LMECB_ObjectLength(o, &len);
      for (i = 1; status && i <= len; i++)
      {
        if (i > 1)
          LMECB_Write(1, ",", -1, 1);
        status = LMECB_GetFieldNameFromStructObject(o, i, str);
        if (!status)
          break;
        LMECB_Write(1, str, -1, 1);
        LMECB_Write(1, "=", -1, 1);
        status = LMECB_GetFieldFromStructObject(o, str, &el);
        if (status)
          status = displayRec(lme, &el);
      }
      LMECB_Write(1, ")", -1, 1);
      break;
  }
  return status;
}

static lme_err displayobject(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  /* display argument */
{
  lme_object o;
  if (!LMECB_GetObject(1, &o) || !displayRec(lme, &o))
    return 0;
  LMECB_Write(1, "\n", -1, 1);
  return 1;
}

static lme_fn fn[] =
{
  {"displayobject", displayobject, 1, 1, 0, 0}
};

lme_int32 InstallFn(lme_ref lme, lme_fn **fnarray)
{
  LMECB_DbgWriteStr("Installing displayobject.");
  *fnarray = fn;
  return 1;
}

The extension below implements a new type for integer arithmetic modulo n. The function modint(i,n) creates a new object of this type. Operators +, - (binary and unary), and * are overloaded to support expressions like modint(2,7)*3+5, whose result would be 4 (mod 7). The function disp is also overloaded; it can be called explicitly, but also implicitly to display the result of an expression which is not followed by a semicolon.

#include "LME_Ext.h"
#include <stdio.h>

typedef struct
{
  long i, n;
} Data;

static lme_err modint(lme_ref lme,
        lme_int32 nargin, lme_int32 nargout)
  // modint(i, n) -> create a binary object
  // for arithmetic modulo n
{
  lme_float64 x, y;
  Data *result;
  
  if (!LMECB_GetScalar(1, &x, NULL)
      || !LMECB_GetScalar(2, &y, NULL))
    return 0;
  if (!LMECB_PushBinaryData(sizeof(Data), (void **)&result))
    return 0;
  
  result->n = (long)y;
  result->i = (long)x;
  return 1;
}

static lme_err getTwoArgs(lme_ref lme,
        Data *data1, Data *data2)
  // get two numbers with at least one binary object
{
  Data *d;
  lme_float64 x;
  
  if (LMECB_GetBinaryData(1, NULL, (void **)&d))
  {
    *data1 = *data2 = *d;
    if (LMECB_GetBinaryData(2, NULL, (void **)&d))
    {
      // binary, binary
      *data2 = *d;
      return 1;
    }
    else if (LMECB_GetScalar(2, &x, NULL))
    {
      // binary, scalar
      data2->i = (long)x;
      return 1;
    }
    else
      return 0;
  }
  else
  {
    // 1st arg is not binary, hence 2nd should be
    if (LMECB_GetBinaryData(2, NULL, (void **)&d))
      *data1 = *data2 = *d;
    else
      return 0;
    if (!LMECB_GetScalar(1, &x, NULL))
      return 0;
    data1->i = (long)x;
    return 1;
  }
}

static lme_err plus(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  // overloaded operator + for arithmetic modulo n
{
  Data data1, data2, *result;
  
  if (!getTwoArgs(lme, &data1, &data2)
      || !LMECB_PushBinaryData(sizeof(Data), (void **)&result))
    return 0;
  
  result->n = data1.n;
  result->i = (data1.i + data2.i) % data1.n;
  return 1;
}

static lme_err minus(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  // overloaded operator - for arithmetic modulo n
{
  Data data1, data2, *result;
  
  if (!getTwoArgs(lme, &data1, &data2)
      || !LMECB_PushBinaryData(sizeof(Data), (void **)&result))
    return 0;
  
  result->n = data1.n;
  result->i = (data1.n + data1.i - data2.i) % data1.n;
  return 1;
}

static lme_err mtimes(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  // overloaded operator * for arithmetic modulo n
{
  Data data1, data2, *result;
  
  if (!getTwoArgs(lme, &data1, &data2)
      || !LMECB_PushBinaryData(sizeof(Data), (void **)&result))
    return 0;
  
  
  result->n = data1.n;
  result->i = (data1.i * data2.i) % data1.n;
  return 1;
}

static lme_err uminus(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  // overloaded unary operator - for arithmetic modulo n
{
  Data *data, *result;
  
  if (!LMECB_GetBinaryData(1, NULL, (void **)&data)
      || !LMECB_PushBinaryData(sizeof(Data), (void **)&result))
    return 0;
  
  result->n = data->n;
  result->i = (data->n - data->i) % data->n;
  return 1;
}

static lme_err disp(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  // overloaded "disp" function to display binary object
{
  Data *data;
  char str[64];
  
  if (!LMECB_GetBinaryData(1, NULL, (void **)&data))
    return 0;
  
  sprintf(str, "%ld (mod %ld)\n", data->i, data->n);
  LMECB_Write(1, str, -1, 1);
  return 1;
}

static lme_err subsref(lme_ref lme,
             lme_int32 nargin, lme_int32 nargout)
  // overloaded subsref (field access) to get
  //  the contents of binary object
  // b.i === subsref(b, struct('type','.','subs','i') -> value
  // b.n === subsref(b, struct('type','.','subs','n') -> modulo
{
  Data *data;
  lme_object o, fld;
  unsigned short *str;
  lme_int32 ndims, *size, nbytes, type;
  lme_float64 *res;
  
  if (!LMECB_GetBinaryData(1, NULL, (void **)&data)
      || !LMECB_GetObject(2, &o))
    return 0;
  
  // extract field name from 2nd arg
  if (!LMECB_GetFieldFromStructObject(&o, "type", &fld)
      || !LMECB_ObjectToArray(&fld, &ndims, &size,
              &nbytes, &type, (void **)&str)
      || ndims != 2 || size[0] * size[1] != 1
      || type != k_lme_type_char
      || (char)str[0] != '.'
      || !LMECB_GetFieldFromStructObject(&o, "subs", &fld)
      || !LMECB_ObjectToArray(&fld, &ndims, &size,
              &nbytes, &type, (void **)&str)
      || ndims != 2 || type != k_lme_type_char)
    return LMECB_Error("LME:wrongType", NULL);
  if (size[0] * size[1] != 1
      || (char)str[0] != 'i' && (char)str[0] != 'n')
    return LMECB_Error("LME:undefField", NULL);
  
  // push result
  if (!LMECB_PushMatrix(1, 1, &res, NULL))
    return 0;
  *res = (char)str[0] == 'i' ? data->i : data->n;
  
  return 1;
}

static lme_fn fn[] =
{
  {"modint", modint, 2, 2, 1, 1},
  {lme_k_binary_overload_str_prefix "plus", plus, 2, 2, 1, 1},
  {lme_k_binary_overload_str_prefix "minus", minus, 2, 2, 1, 1},
  {lme_k_binary_overload_str_prefix "mtimes", mtimes, 2, 2, 1, 1},
  {lme_k_binary_overload_str_prefix "uminus", uminus, 1, 1, 1, 1},
  {lme_k_binary_overload_str_prefix "disp", disp, 1, 1, 0, 0},
  {lme_k_binary_overload_str_prefix "subsref", subsref, 2, 2, 1, 1}
};

lme_int32 InstallFn(lme_ref lme, lme_fn **fnarray)
{
  LMECB_DbgWriteStr("modint: modint, disp, minus, mtimes, plus, "
                    "subsref, uminus");
  *fnarray = fn;
  return 7;
}

Remarks

We have three suggestions to make the development of your external functions easier:


Copyright 1998-2008, Calerga.
All rights reserved.