{"id":87,"date":"2015-01-09T09:14:38","date_gmt":"2015-01-09T09:14:38","guid":{"rendered":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=87"},"modified":"2017-12-08T10:55:48","modified_gmt":"2017-12-08T10:55:48","slug":"the-textfile-module","status":"publish","type":"page","link":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=87","title":{"rendered":"The textfile module"},"content":{"rendered":"<p>The <code>textfile<\/code> module was originally designed to allow Orchids to read data from a text file, and split it into lines.\u00a0 It can now also read from Unix pipes and Unix (<code>AF_UNIX<\/code>) TCP sockets.<\/p>\n<p>The <code>textfile<\/code> module is\u00a0 an <a title=\"Input modules\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=126\">input module<\/a>, meaning that its purpose is to read data from some sources, and convert it to Orchids events.\u00a0 The sources\u00a0 should be declared in the <a title=\"orchids-inputs.conf\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=145\"><code>orchids-inputs.conf<\/code><\/a> file, by directives such as:<\/p>\n<pre>INPUT textfile \/var\/log\/messages<\/pre>\n<p>Admissible sources are regular files, Posix pipes and Unix (<code>AF_UNIX<\/code>) TCP sockets (<code>SOCK_STREAM<\/code>).\u00a0 The latter are needed to interact with <a title=\"The auditd module\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=103\"><code>auditd<\/code><\/a>, for example.<\/p>\n<p>Internet (<code>AF_INET<\/code>) sockets are not handled, and should be dealt with using the <a title=\"The udp module\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=171\"><code>udp<\/code><\/a> module.\u00a0 Unix UDP sockets (<code>SOCK_DGRAM<\/code>) should be dealt with using the <a title=\"The sockunix module\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=210\"><code>sockunix<\/code><\/a> module.<\/p>\n<p>The <code>textfile<\/code> module reads from these sources and produces Orchids events, one per text line.\u00a0 The contents of each line will then be found in the <code>.textfile.line<\/code> field of the event.\u00a0 The name of the source, as given in the <a title=\"orchids-inputs.conf\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=145\"><code>orchids-inputs.conf<\/code><\/a> file, will be in <code>.textfile.file<\/code>.<\/p>\n<h3>Configuration options<\/h3>\n<p><code>&lt;module textfile&gt;<\/code><\/p>\n<ul>\n<li><code>PollPeriod<\/code> <em>secs<\/em>: instructs Orchids that it should poll files read by the <em>textfile<\/em> module every secs seconds. There is no way to specify a polling period of less than 1 second. This takes precedence over the general <a href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=68\"><code>PollPeriod<\/code><\/a> directive.<\/li>\n<li><code>ProcessAll<\/code> <em>n<\/em>: if <em>n<\/em> is non-zero, read all lines from the start of the file. This only applies to files, not pipes or Unix TCP sockets, and is meant to analyze log files, offline. The default behavior (<em>n<\/em>=0) is to read only lines added after Orchids started, through polling.<\/li>\n<li><code>ExitAfterProcessAll<\/code> <em>n<\/em>: if <em>n<\/em> is non-zero, shutdown Orchids after last line has been read. On pipes and sockets, this will cause Orchids to shutdown when the pipe or socket is closed. This is meant to analyze a single log file, offline. If different input sources are specified, Orchids will exit regardless of the fact that there are still events pending from the other sources. The default behavior (<em>n<\/em>=0) is to continue polling, waiting for extra lines, once the end of file has been reached.<\/li>\n<\/ul>\n<p><code>&lt;\/module&gt;<\/code><\/p>\n<p>The <code>textfile<\/code> module also understands options named <code>INPUT<\/code> and <code>AddInputFile<\/code>.\u00a0 They are synonyms, take a file, pipe or Unix socket name as argument, which they open and use as input to the <code>textfile<\/code> module.\u00a0 They should <em>not<\/em> be used inside the <code>textfile<\/code> module configuration file.\u00a0 Instead, input should be specified in the <a title=\"orchids-inputs.conf\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=145\"><code>orchids-inputs.conf<\/code><\/a> file.<\/p>\n<h3>Fields<\/h3>\n<p>The last field is <code>.textfile.line<\/code>, available for further dissection; the dissection key is <code>.textfile.file<\/code>.<\/p>\n<table style=\"border: solid 1px black;\">\n<tbody>\n<tr style=\"background-color: lightsteelblue;\">\n<th>Field<\/th>\n<th><a title=\"Types\" href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=98\">Type<\/a><\/th>\n<th><a href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=719\">Mono<\/a>?<\/th>\n<th>Description<\/th>\n<\/tr>\n<\/tbody>\n<tbody>\n<tr style=\"background-color: lightgrey;\">\n<td><code>.textfile.line_num<\/code><\/td>\n<td><code>uint<\/code><\/td>\n<td>\u00a0\u2713<\/td>\n<td>line number<\/td>\n<\/tr>\n<tr style=\"background-color: white;\">\n<td><code>.textfile.file<\/code><\/td>\n<td><code>str<\/code><\/td>\n<td><\/td>\n<td>source file name<\/td>\n<\/tr>\n<tr style=\"background-color: lightgrey;\">\n<td><code>.textfile.line<\/code><\/td>\n<td><code>str<\/code><\/td>\n<td><\/td>\n<td>current line<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The textfile module was originally designed to allow Orchids to read data from a text file, and split it into lines.\u00a0 It can now also read from Unix pipes and Unix (AF_UNIX) TCP sockets. The textfile module is\u00a0 an input module, meaning that its purpose is to read data from some sources, and convert it &hellip; <a href=\"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/?page_id=87\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">The textfile module<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"open","template":"","meta":{"footnotes":""},"class_list":["post-87","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=\/wp\/v2\/pages\/87","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=87"}],"version-history":[{"count":25,"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=\/wp\/v2\/pages\/87\/revisions"}],"predecessor-version":[{"id":734,"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=\/wp\/v2\/pages\/87\/revisions\/734"}],"wp:attachment":[{"href":"https:\/\/projects.lsv.ens-paris-saclay.fr\/orchidsdoc\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=87"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}