f2py¶
Rather than calling into a shared library via ctypes or
cffi, the f2py tool can be used to to wrap Fortran
interfaces in Python functions. It does this by generating
a custom CPython C extension and compiling it with a
Fortran shared library.
f2py has many limitations, chief of which is absence of support
for user defined types or derived types (equivalent of a C struct).
The examples below demonstrate a few ways of getting around these
limitations, including manual conversion of a custom typed variable
to raw bytes.
Usage¶
First, the f2py tool must be used to create an extension module:
$ f2py \
> --verbose \
> -c \
> --opt='-O3' \
> -m example \
> fortran/example.f90 \
> skip: make_container view_knob
$ ls *.so
example.cpython-37m-x86_64-linux-gnu.so*
As we can see, this interacts directly with the Fortran source rather than with an object file or a shared library.
Inside the example.so module we’ve created, the only attribute is
example, which represents the Fortran module in example.f90:
>>> import example
>>> example
<module 'example' from '.../example.cpython-37m-x86_64-linux-gnu.so'>
>>> [name for name in dir(example) if not name.startswith("__")]
['example']
>>> example.example
<fortran object>
It is within this wrapped Fortran module that our actual routines live:
>>> example_ns = example.example
>>> for name in dir(example_ns):
... if not name.startswith("__"):
... print(name)
...
foo
foo_array
foo_by_ref
just_print
make_udf
turn_knob
udf_ptr
The first task we’ll accomplish is to call foo() and
foo_by_ref():
# foo()
bar = 1.0
baz = 16.0
msg_foo = "foo ({}, {}) = {}".format(
bar, baz, example_ns.foo(bar, baz)
)
print(msg_foo)
msg_foo_by_ref = "foo_by_ref({}, {}) = {}".format(
bar, baz, example_ns.foo_by_ref(bar, baz)
)
print(msg_foo_by_ref)
As can be seen in the output below. Calling by reference results in the
correct answer while calling by value (foo()) does not work correctly
with f2py.
Next, we invoke the make_udf() routine to “smuggle” out a
UserDefined value as raw bytes:
# make_udf()
buzz = 1.25
broken = 5.0
how_many = 1337
quuz_as_bytes = example_ns.make_udf(buzz, broken, how_many)
quuz = np_to_udf(quuz_as_bytes)
msg = MAKE_UDF_TEMPLATE.format(buzz, broken, how_many, quuz)
print(msg, end="")
In particular, this uses the np_to_udf helper to convert those bytes into
a UserDefined object as defined in ctypes:
def np_to_udf(arr):
address = arr.ctypes.data
return UserDefined.from_address(address)
For udf_ptr(), the other routine which deals with a user defined type,
we use the prepare_udf helper from ctypes. This allocates the memory
for the UserDefined value in Python and then passes a void*
pointer (as an integer) to the Fortran routine:
# udf_ptr()
made_it, ptr_as_int = prepare_udf()
ptr_as_int = ptr_as_int.value
example_ns.udf_ptr(ptr_as_int)
msg = UDF_PTR_TEMPLATE.format(ptr_as_int, ptr_as_int, made_it)
print(msg, end="")
Since f2py is included with NumPy, it has nicer support for NumPy arrays
than either ctypes or cffi. This means we can call foo_array()
directly with a NumPy array:
# foo_array()
val = np.asfortranarray([[3.0, 4.5], [1.0, 1.25], [9.0, 0.0], [-1.0, 4.0]])
two_val = example_ns.foo_array(val)
print(MSG_FOO_ARRAY.format(val, two_val))
Finally, we call just_print() to mix Python and Fortran usage of
STDOUT:
# just_print()
print("just_print()")
example_ns.just_print()
Output¶
$ python f2py/check_f2py.py
------------------------------------------------------------
example: <module 'example' from '.../f2py/example...so'>
dir(example.example): foo, foo_array, foo_by_ref, just_print, make_udf, udf_ptr
------------------------------------------------------------
foo (1.0, 16.0) = 0.0
foo_by_ref(1.0, 16.0) = 61.0
------------------------------------------------------------
quuz = make_udf(1.25, 5.0, 1337)
= UserDefined(buzz=1.25, broken=5.0, how_many=1337)
------------------------------------------------------------
val =
[[ 3. 4.5 ]
[ 1. 1.25]
[ 9. 0. ]
[-1. 4. ]]
two_val = foo_array(val)
two_val =
[[ 6. 9. ]
[ 2. 2.5]
[ 18. 0. ]
[ -2. 8. ]]
------------------------------------------------------------
ptr_as_int = address(made_it) # intptr_t / ssize_t / long
ptr_as_int = 139859191412464 # 0x7f33816c36f0
udf_ptr(ptr_as_int) # Set memory in ``made_it``
made_it = UserDefined(buzz=3.125, broken=-10.5, how_many=101)
------------------------------------------------------------
just_print()
======== BEGIN FORTRAN ========
just_print() was called
======== END FORTRAN ========
What is Happening?¶
f2py actually generates a wrapped {modname}module.c file (so in our
case examplemodule.c) and utilizes a fortranobject C library to create
CPython C extensions:
>>> import os
>>> import numpy.f2py
>>>
>>> f2py_dir = os.path.dirname(numpy.f2py.__file__)
>>> f2py_dir
'.../lib/python3.7/site-packages/numpy/f2py'
>>> os.listdir(os.path.join(f2py_dir, 'src'))
['fortranobject.c', 'fortranobject.h']
It uses a C compiler with flags determined by distutils to
link against NumPy and Python headers when compiling {modname}module.c
and fortranobject.c and uses a Fortran compiler for {modname}.f90.
Then it uses the Fortran compiler as linker:
$ gfortran \
> -Wall \
> -g \
> -shared \
> ${TEMPDIR}/.../{modname}module.o \
> ${TEMPDIR}/.../fortranobject.o \
> ${TEMPDIR}/{modname}.o \
> -lgfortran \
> -o \
> ./{modname}.so
When trying to convert a Fortran subroutine to Python via f2py, a
problem occurs if the subroutine uses a user defined type. For example, if
we tried to use the make_container() routine:
$ f2py \
> --verbose \
> -c \
> --opt='-O3' \
> -m example \
> fortran/example.f90 \
> only: make_container
...
Building modules...
Building module "example"...
Constructing F90 module support for "example"...
Skipping type unknown_type
Skipping type unknown_type
Constructing wrapper function "example.make_container"...
getctype: No C-type found in "{'typespec': 'type', 'typename': 'datacontainer', 'attrspec': [], 'intent': ['out']}", assuming void.
getctype: No C-type found in "{'typespec': 'type', 'typename': 'datacontainer', 'attrspec': [], 'intent': ['out']}", assuming void.
getctype: No C-type found in "{'typespec': 'type', 'typename': 'datacontainer', 'attrspec': [], 'intent': ['out']}", assuming void.
Traceback (most recent call last):
...
File ".../numpy/f2py/capi_maps.py", line 412,in getpydocsign
sig = '%s : %s %s%s' % (a, opt, c2py_map[ctype], init)
KeyError: 'void'
This is because make_container() returns the DataContainer
user defined type.
References¶
- (Lack of) support for user defined types in
f2py - The
f90wrapinterface generator adds support for user defined types tof2py. However, the author of this document has no experience withf90wrap.
