Multi script langyage Stream Processing Offload Agent
-----------------------------------------------------
This agent receive SPOP message and process it with script languages. The
language register callback with a message. Each callback receive the list
of arguments with types according with the language capabilities. The
callback write variables which are sent as response when the processing
is done.
Prerequirement
----------------
You have to install the development packages, either from the
distribution repositories or from the source.
CentOS/RHEL: sudo yum install python3-devel
The current minimal python version compatible with this library is 2.7.
It's recommended to use python version 3 where possible due to python 2 deprecation.
Compilation
---------------
The server currently supports Lua and Python. Type "make" with the options:
USE_LUA=1 and/or USE_PYTHON=1.
You can add LUA_INC=.. LUA_LIB=.. to the make command to set the paths to
the lua header files and lua libraries.
Similarly, you can add PYTHON_INC=.. PYTHON_LIB=.. to the make command to set the paths to
the python header files and python libraries.
By default, it will try to compile by detecting the default python 3 parameters.
It will fall back to python 2 if python 3 is not available.
Start the service
---------------------
After you have compiled it, to start the service, you just need to use "spoa"
binary:
$> ./spoa -h
Usage: ./spoa [-h] [-d] [-p <port>] [-n <num-workers>]
-h Print this message
-d Enable the debug mode
-p <port> Specify the port to listen on (default: 12345)
-n <num-workers> Specify the number of workers (default: 5)
-f <file> Load script according with the supported languages
The file processor is recognized using the extension. .lua or .luac for lua and
.py for python. Start example:
$> ./spoa -d -f ps_lua.lua
$> ./spoa -d -f ps_python.py
Configure
-------------
Sample configuration are join to this server:
spoa-server.conf : The HAProxy configuration file using SPOE server
spoa-server.spoe.conf : The SPOP description file used by HAProxy
ps_lua.lua : Processing Lua example
ps_python.py : Processing Python example
Considerations
------------------
This server is a beta version. It works fine, but some improvement will be
welcome:
Main process:
* Improve log management: Today the log are sent on stdout.
* Improve process management: The dead process are ignored.
* Implement systemd integration.
* Implement threads: It would be fine to implement thread working. Shared
memory is welcome for managing database connection pool and something like
that.
* Add PHP support and some other languages.
Python:
* Improve reporting: Catch python error message and report it in the right
place. Today the error are dumped on stdout. How using syslog for logging
stack traces ?
Maybe some other things...
bdd034947c