API Reference¶
main¶
Main module to run the Gordon service.
The service expects a gordon.toml
and/or a gordon-user.toml
file for configuration in the current working directory, or in a
provided root directory.
Any configuration defined in gordon-user.toml
overwrites those in
gordon.toml
.
Example:
$ python gordon/main.py
$ python gordon/main.py -c /etc/default/
$ python gordon/main.py --config-root /etc/default/
-
gordon.main.
setup
(config_root='')[source]¶ Service configuration and logging setup.
Configuration defined in
gordon-user.toml
will overwritegordon.toml
.Parameters: config_root (str) – Where configuration should load from, defaults to current working directory. Returns: A dict for Gordon service configuration.
router¶
Core message routing logic for the plugins within Gordon Service.
Messages received on the success channel will be routed to the next
designated plugin phase. For example, a message that has a consume
phase will be routed to the installed enricher provider (or publisher
provider if no enricher provider is installed).
If a message fails its next phase, its phase will be updated to drop
and routed to the event consumer provider for cleanup.
Attention
The GordonRouter
only supports the following two phase
routes:
- consume -> enrich -> publish -> done
- consume -> publish -> done
Future releases may support more configurable phase routings.
-
class
gordon.router.
GordonRouter
(phase_route, success_channel, error_channel, plugins, metrics)[source]¶ Route messages to the appropriate plugin destination.
Attention
error_channel is currently not used in this router, and may be removed entirely from all interface definitions.
Parameters: - phase_route (dict(str, str)) – The route messages should follow.
- success_channel (asyncio.Queue) – A sink for successfully
processed
gordon.interfaces.IEventMessage
s. - error_channel (asyncio.Queue) – A sink for
gordon.interfaces.IEventMessage
s that were not processed due to problems. - plugins (list) – Instantiated message handling plugins.
- metrics (obj) – Implemented
IMetricRelay
interface to emit metrics.
plugins_loader¶
Module for loading plugins distributed via third-party packages.
Plugin discovery is done via entry_points
defined in a package’s
setup.py
, registered under 'gordon.plugins'
. For example:
# setup.py
from setuptools import setup
setup(
name=NAME,
# snip
entry_points={
'gordon.plugins': [
'gcp.gpubsub = gordon_gcp.gpubsub:EventClient',
'gcp.gce.a = gordon_gcp.gce.a:ReferenceSourceClient',
'gcp.gce.b = gordon_gcp.gce.b:ReferenceSourceClient',
'gcp.gdns = gordon_gcp.gdns:DNSProviderClient',
],
},
# snip
)
Plugins are initialized with any config defined in gordon-user.toml
and gordon.toml
. See Configuring the Gordon Service for more details.
Once a plugin is found, the loader looks up its configuration via the
same key defined in its entry point, e.g. gcp.gpubsub
.
The value of the entry point (e.g. gordon_gcp.gpubsub:EventClient
)
must point to a class. The plugin class is instantiated with its config.
A plugin will not have access to another plugin’s configuration. For
example, the gcp.gpusub
will not have access to the configuration
for gcp.gdns
.
See Gordon’s Plugin System for details on how to write a plugin for Gordon.
-
gordon.plugins_loader.
load_plugins
(config, plugin_kwargs)[source]¶ Discover and instantiate plugins.
Parameters: Returns: list of names of plugins, list of instantiated plugin objects, and any errors encountered while loading/instantiating plugins. A tuple of three empty lists is returned if there are no plugins found or activated in gordon config.
Return type: Tuple of 3 lists
interfaces¶
-
interface
gordon.interfaces.
IEventMessage
[source]¶ A discrete unit of work for Gordon to process.
Gordon expects plugins to return or accept objects that implement this interface in order to route them to other plugins, and handle retries or cleanup in case of errors.
-
msg_id
¶ Identifier for the event message instance.
-
phase
¶ Variable phase of the event message.
-
__init__
(msg_id, data, history_log, phase=None)¶ Initialize an EventMessage.
Parameters:
-
append_to_history
(entry, plugin_phase)¶ Append entry to the IEventMessage’s history_log.
Parameters:
-
-
interface
gordon.interfaces.
IRunnable
[source]¶ Extends:
gordon.interfaces.IGenericPlugin
Runnable plugin to produce event messages for Gordon to process.
The plugin also has the ability to send
gordon.interfaces EventMessage
objects to both success and error channels. At least one runnable plugin is required to run Gordon.-
start_phase
¶ Starting phase for event messages.
-
__init__
(config, success_channel, error_channel, metrics, **kwargs)¶ Initialize a runnable plugin.
Parameters: - config (dict) – Plugin-specific configuration.
- success_channel (asyncio.Queue) – A sink for successfully processed IEventMessages.
- error_channel (asyncio.Queue) – A sink for IEventMessages that were not processed due to problems.
- metrics (obj) – Optional obj used to emit metrics.
-
run
()¶ Begin consuming messages using the provided event loop.
-
-
interface
gordon.interfaces.
IMessageHandler
[source]¶ Extends:
gordon.interfaces.IGenericPlugin
Plugin which performs some operation on an event message.
The Gordon core router will use its phase_route to direct messages produced by any runnable plugins the appropriate message handling plugins, identified by their phase attribute. At least one message handling plugin is required to run Gordon.
-
phase
¶ Plugin phase
-
__init__
(config, metrics, **kwargs)¶ Initialize a message handler.
Parameters: - config (dict) – Plugin-specific configuration.
- metrics (obj) – Obj used to emit metrics.
-
handle_message
(event_message)¶ Perform some operation on or triggered by an event message.
Parameters: event_message (IEventMessage) – Message on which to operate.
-
-
interface
gordon.interfaces.
IGenericPlugin
[source]¶ Do not implement this interface directly.
Use
gordon.interfaces.IRunnable
, orgordon.interfaces.IMessageHandler
instead.-
shutdown
()¶ Perform any actions required to gracefully shutdown plugin.
-
-
interface
gordon.interfaces.
IMetricRelay
[source]¶ Manage Gordon metrics.
-
incr
(metric_name, value=1, context=None, **kwargs)¶ Increase a metric by 1 or a given amount.
Parameters:
-
timer
(metric_name, context=None, **kwargs)¶ Get a timer object which implements ITimer.
Parameters:
-
set
(metric_name, value, context=None, **kwargs)¶ Set a metric to a given value.
Parameters:
-
cleanup
(**kwargs)¶ Perform cleanup tasks related to metrics handling.
-