{"id":628,"date":"2017-11-12T20:46:07","date_gmt":"2017-11-12T20:46:07","guid":{"rendered":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=628"},"modified":"2017-12-08T10:50:29","modified_gmt":"2017-12-08T10:50:29","slug":"the-generic-module","status":"publish","type":"page","link":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=628","title":{"rendered":"The generic module"},"content":{"rendered":"

The generic<\/code> module is a dissection module<\/a>: its purpose is to take an Orchids event, parse one of its text fields and return a refined Orchids events, with additional fields. Typically, the generic<\/code> module is meant to dissect text coming from the syslog<\/code><\/a> module.<\/p>\n

The generic<\/code> module is however very special.\u00a0 In a sense, it is a meta<\/em>-module, one that allows you to define new, so-called virtual<\/em> modules, whose parsing methods are defined by regular expressions.<\/p>\n

We explain how that works on an example.\u00a0 The syslog<\/code><\/a> module may have to dissect messages such as:<\/p>\n

Jan 27 11:32:06 laramie dhclient: DHCPREQUEST on eth0 to 192.168.30.1 port 67\r\n<\/pre>\n
In that case, it will produce a last field .syslog.msg<\/code> equal to the block of text \"DHCPREQUEST on eth0 to 192.168.30.1 port 67\"<\/code>. We would like to dissect that further.
\nThe generic<\/code> module allows you to do that, based on regular expressions.<\/p>\n
First, one should plug the generic<\/code> module so that it dissects that kind of message. \u00a0 The generic<\/code> module defines a so-called virtual module<\/em> called\u00a0dhclient<\/code> and which does just that. \u00a0We shall describe it below. \u00a0The plugging is done by\u00a0the following line in\u00a0$OCONF\/orchids-inputs.conf<\/code><\/a>, instructing Orchids to let the\u00a0dhclient<\/code> virtual module parse the last field,\u00a0.syslog.msg<\/code> (in the example, \"DHCPREQUEST on eth0 to 192.168.30.1 port 67\"<\/code>), provided by the\u00a0syslog<\/code>\u00a0module, on the condition that\u00a0its next-to-last field,\u00a0.syslog.prog<\/code>, is equal to \"dhclient\"<\/code> (which is the case in our example).<\/p>\n
DISSECT dhclient syslog \"dhclient\"<\/pre>\n
 <\/p>\n
Configuration options<\/h3>\n
<module generic><\/code><\/p>\n
contains a list of declarations of virtual modules<\/em>.\u00a0 Virtual modules work exactly like actual modules, except you do not need to load any shared library at start-up time to start them.\u00a0 Instead, virtual modules work by parsing sets of regular expressions.<\/p>\n
We describe the standard dhclient<\/code> virtual module, which is part of the standard configuration file for the generic<\/code> module, $OCONF\/conf.d\/12_mod_generic.conf<\/code>.<\/p>\n
That module is meant to parse the syslog<\/code>\u00a0messages produced by the\u00a0dhclient<\/code>\u00a0program. \u00a0That means that you should see<\/p>\n
    <vmod dhclient>\r\n<\/pre>\n
This declares a new virtual module with name dhclient<\/code>. Every virtual module consists of a list of <fieldmatch \u27e8<\/code>regular-expression\u27e9><\/code> … <\/fieldmatch><\/code> declarations.\u00a0 Here are some of them, and we start with the first one. The order is important: the first matching regular expression is the one that is considered to parse the given block of text.<\/p>\n
      <fieldmatch \"^(bound) to ([0-9.]+) -- renewal in ([0-9]+) seconds\\\\.\">\r\n        str_field action 1 Action\r\n        ipv4_field ip 2 Ip address\r\n        int_field renewal 3 Renewal time\r\n      <\/fieldmatch>\r\n<\/pre>\n
Here the regular expression matches any message starting with \"bound to \"<\/code> followed by an IP address, followed by \" -- renewal in \"<\/code> , followed by a number, followed by \" seconds.\"<\/code> (and possibly other text: if we wanted to match that exactly, we woud add “$\"<\/code> at the end of the regular expression, to match the end of text).<\/p>\n
The parenthesized subexpressions, (bound)<\/code>, ([0-9.]+)<\/code>, and ([0-9]+)<\/code> are recognized as subexpressions 1, 2, and 3 respectively, and converted to types str<\/code>,\u00a0 ipv4<\/code>, and int<\/code>, respectively.\u00a0 The possible field declarations are of the form:<\/p>\n