I finally had a chance to sit back down with PureDocs, after getting the build process straightened out, and figure out how to load C plugins. Turns out, it's pretty simple with ctypes. Let's imagine I have a C library, compiled and linked as libtest, containing the following (a simple modification of my original example):
#include <stdio.h>
#include "puredocs.h"
Resource * test_function() {
Resource * r = puredocs_createResource("foo");
Resource * c = puredocs_createResource("bar");
puredocs_setDocString(r, "omg some docs lol");
puredocs_setDocString(c, "some bar docs omg lol");
puredocs_setProperty(r, "feh", "omg");
if ((puredocs_addChild(r, c)) < 0) {
printf("OH NOES!\n");
}
return r;
}
This function can easily be called when loading the libary with``ctypes``'s PyDLL type (won't release the Python interpreter's GIL, which is important if we're creating Python objects we want to use). The restype of the function needs to be set to ctypes.py_object, since we're returning a Python object. The whole process looks like this:
>>> from ctypes import *
>>> t = pydll.LoadLibrary("libtest.dylib")
>>> t.test_function.restype=py_object
>>> r = t.test_function()
>>> r
<Resource foo [<Resource bar []>]>
The whole point of this exercise is basically to make it possible to write C libraries that serve as plugins for PureDocs that don't necessarily have any knowledge of Python. They'll be linked against a PureDocs library that provides the glue-code. So, for PureDocs's purpose, I now just need to wrap this around some library loading and registration code, and we'll be ready to go!
After much trial and tribulation with Pyhton's distutils and setuptools, I decided it wasn't really appropriate to be building a C library with the Python tools. While the capability exists in distutils, it's pretty obvious that it suffers from some pretty substantial code rot.
So, I've gone the autoconf route. I didn't really want to, but really there's no better solution that I can find that doesn't involve setting PureDocs aside while I start yet another project.
In any case, the autoconf'ing isn't complete, and it doesn't quite work, but I've committed what I have so far. Once I get it all working well on multiple platforms, I'll finish off the C API. Then Kate should be able to add some C-based plugins using the SID parser generator.
PureDocs seems reasonably unique in being a Python system that provides an external C library, the kind of thing that would probably be happiest with autoconf and make. Yes, I dislike autoconf, automake, and everything in between, but I'm writing an API doc-generation system, I don't want to maintain a build system.
But, I naturally want a unified build system for the entire kit and kaboodle. The C library should come with the Python system, and the Python system could come with the C library. This includes Python egg files, which can serve as the basis for other package files. So, anything I choose needs to integrate with the existing distutils/setuptools-based Python distribution system.
I had the opportunity to sit down and design a C API for creating PureDocs Resource objects this weekend. For the C API, Resource is an opaque type, and the API provides the ability to create a Resource, add properties, children, etc, without having to worry about it being a Python type.
The reason behind C plugin support for an API documentation generation utility that parses files based on grammars should be pretty obvious. Because most higher-level languages, at the very least, have implementations in C, there are readily available parsers. Having to reinvent the wheel just because we're using Python, or at the very least, write Python-C support code, would be painful and reduce the usefulness of PureDocs. Having a simple C API that can be used to wrap one of these existing parsers seems to be the optimal situation.
The C API itself is defined in a library, libpuredocs, that links against Python to provide access to the Python data types. C plugins will be loadable modules linked against libpuredocs. They will be dyanmically loaded by PureDocs (perhaps by a C Python module) into the same address space, so that Resource objects ca easily be passed back and forth between PureDocs and the C plugin.
It took me a little while to come up with just the right way to do this to avoid C plugin authors having to have any interaction with Python at all. I still haven't figured out how loading and registration of C plugins will work, but I think the datatypes were the hard part. I'll write more about this once the entire thing is finalized. For testing purposes, I have passed a Resource object back and forth between a pure-C loadable module and a C python module, with success.
A quick test showing that Resource objects created using the libpuredocs library can be used in Python looks something like the following, implemented in C:
#include <Python.h>
#include <stdio.h>
#include "puredocs.h"
PyObject * test_function(PyObject * self, PyObject * args) {
Resource * r = puredocs_createResource("foo");
Resource * c = puredocs_createResource("bar");
puredocs_setDocString(r, "omg some docs lol");
puredocs_setDocString(c, "some bar docs omg lol");
puredocs_setProperty(r, "feh", "omg");
if ((puredocs_addChild(r, c)) < 0) {
printf("OH NOES!\n");
}
return (PyObject *)r;
}
The C API (along with the Python API) is documented in The PureDocs Plugin Architecture. Again, I'm going to provide more concrete examples once I have loading and registration worked out, and have some working C plugins as examples.