Enable SSH

sudo apt install openssh-server

sudo systemctl status ssh

sudo systemctl start ssh

sudo systemctl enable ssh

ip a

Step 1)  Configure Rsyslog on the server

$ sudo apt install -y rsyslog

$ sudo apt install -y rsyslog

rsyslog-service-status-debian11

$ sudo nano /etc/rsyslog.conf

Proceed and uncomment the following lines that allow for UDP and TCP syslog reception from remote clients.

# provides UDP syslog reception

module(load=”imudp”)

input(type=”imudp” port=”514″)

# provides TCP syslog reception

module(load=”imtcp”)

input(type=”imtcp” port=”514″)

$template RemoteLogs,”/var/log/%HOSTNAME%/%PROGRAMNAME%.log”

*.* ?RemoteLogs

& stop

Define Ruleset for Rsyslog-Logging

Looking at the above ruleset, the first rule is “$template RemoteLogs,”/var/log/%HOSTNAME%/%PROGRAMNAME%.log””.

The directive $template tells rsyslog daemon to gather and write all of the received remote messages to distinct logs under /var/log, based on the hostname (client machine name) and remote client facility (program/application) that generated the messages as defined by the settings present in the template RemoteLogs.

The second line “*.* ?RemoteLogs” means record messages from all facilities at all severity levels using the RemoteLogs template configuration.

The final line “& stop” instructs rsyslog to stop processing the messages once it is written to a file. If you don’t include “& stop”, messages will instead be written to the local files.

There are many other templates that you can use, for more information, see the rsyslog configuration man page (man rsyslog.conf) or refer to the Rsyslog online documentation.

$ sudo systemctl restart rsyslog

$ sudo systemctl status rsyslog

Step 2) Configure firewall rules for rsyslog

Rsyslog daemon is running on the server as expected. If you are running a UFW firewall, be sure to allow port 514 so as to allow incoming incoming log messages.

$ sudo ufw allow 514/tcp

$ sudo ufw allow 514/udp

Then reload the firewall to apply the firewall rule as follows.

$ sudo ufw reload

Setp 3) Message Properties

These are extracted by rsyslog parsers from the original message. All message properties start with a letter.

The following message properties exist:

msg

the MSG part of the message (aka “the message” ;))

rawmsg

the message “as is”. Should be useful for debugging and also if a message should be forwarded totally unaltered. Please notice EscapecontrolCharactersOnReceive is enabled by default, so it may be different from what was received in the socket.

rawmsg-after-pri

Almost the same as rawmsg, but the syslog PRI is removed. If no PRI was present, rawmsg-after-pri is identical to rawmsg. Note that the syslog PRI is header field that contains information on syslog facility and severity. It is enclosed in greater-than and less-than characters, e.g. “<191>”. This field is often not written to log files, but usually needs to be present for the receiver to properly classify the message. There are some rare cases where one wants the raw message, but not the PRI. You can use this property to obtain that. In general, you should know that you need this format, otherwise stay away from the property.

hostname

hostname from the message

source

alias for HOSTNAME

fromhost

hostname of the system the message was received from (in a relay chain, this is the system immediately in front of us and not necessarily the original sender). This is a DNS-resolved name, except if that is not possible or DNS resolution has been disabled.

fromhost-ip

The same as fromhost, but always as an IP address. Local inputs (like imklog) use 127.0.0.1 in this property.

syslogtag

TAG from the message

programname

the “static” part of the tag, as defined by BSD syslogd. For example, when TAG is “named[12345]”, programname is “named”.

Precisely, the programname is terminated by either (whichever occurs first):

  • end of tag
  • nonprintable character
  • ‘:’
  • ‘[’
  • ‘/’

The above definition has been taken from the FreeBSD syslogd sources.

Please note that some applications include slashes in the static part of the tag, e.g. “app/foo[1234]”. In this case, programname is “app”. If they store an absolute path name like in “/app/foo[1234]”, programname will become empty (“”). If you need to actually store slashes as part of the programname, you can use the global option

global(parser.permitSlashInProgramName=”on”)

to permit this. Then, a syslogtag of “/app/foo[1234]” will result in programname being “/app/foo”. Note: this option is available starting at rsyslogd version 8.25.0.

pri

PRI part of the message – undecoded (single value)

pri-text

the PRI part of the message in a textual form with the numerical PRI appended in brackets (e.g. “local0.err<133>”)

iut

the monitorware InfoUnitType – used when talking to a MonitorWare backend (also for Adiscon LogAnalyzer)

syslogfacility

the facility from the message – in numerical form

syslogfacility-text

the facility from the message – in text form

syslogseverity

severity from the message – in numerical form

syslogseverity-text

severity from the message – in text form

syslogpriority

an alias for syslogseverity – included for historical reasons (be careful: it still is the severity, not PRI!)

syslogpriority-text

an alias for syslogseverity-text

timegenerated

timestamp when the message was RECEIVED. Always in high resolution

timereported

timestamp from the message. Resolution depends on what was provided in the message (in most cases, only seconds)

timestamp

alias for timereported

protocol-version

The contents of the PROTOCOL-VERSION field from IETF draft draft-ietf-syslog-protocol

structured-data

The contents of the STRUCTURED-DATA field from IETF draft draft-ietf-syslog-protocol

app-name

The contents of the APP-NAME field from IETF draft draft-ietf-syslog-protocol

procid

The contents of the PROCID field from IETF draft draft-ietf-syslog-protocol

msgid

The contents of the MSGID field from IETF draft draft-ietf-syslog-protocol

inputname

The name of the input module that generated the message (e.g. “imuxsock”, “imudp”). Note that not all modules necessarily provide this property. If not provided, it is an empty string. Also note that the input module may provide any value of its liking. Most importantly, it is not necessarily the module input name. Internal sources can also provide inputnames. Currently, “rsyslogd” is defined as inputname for messages internally generated by rsyslogd, for example startup and shutdown and error messages. This property is considered useful when trying to filter messages based on where they originated – e.g. locally generated messages (“rsyslogd”, “imuxsock”, “imklog”) should go to a different place than messages generated somewhere else.

Template: Example

$template class, “Time: %timestamp%, Facility: %syslogfacilitytext%, Priority: %syslogpriority-text%, Hostname: %hostname%,

Message: %msg%\n”

*.* /var/log/logfile;class

String-based Template Samples

This section provides some sample of what the default formats would look as a text-based template. Hopefully, their description is self-explanatory. Note that each $Template statement is on a single line, but probably broken accross several lines for display purposes by your browsers. Lines are separated by empty lines.

$template FileFormat,”%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n”

$template TraditionalFileFormat,”%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n”

$template ForwardFormat,”<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%”

$template TraditionalForwardFormat,”<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%”

$template StdSQLFormat,”insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (‘%msg%’, %syslogfacility%, ‘%HOSTNAME%’, %syslogpriority%, ‘%timereported:::date-mysql%’, ‘%timegenerated:::date-mysql%’, %iut%, ‘%syslogtag%’)”,SQL

sql – format the string suitable for a SQL statement in MySQL format. This will replace single quotes (“‘”) and the backslash character by their backslash-escaped counterpart (“\'” and “\\”) inside each field. Please note that in MySQL configuration, the NO_BACKSLASH_ESCAPES mode must be turned off for this format to work (this is the default).

stdsql – format the string suitable for a SQL statement that is to be sent to a standards-compliant sql server. This will replace single quotes (“‘”) by two single quotes (“””) inside each field. You must use stdsql together with MySQL if in MySQL configuration the NO_BACKSLASH_ESCAPES is turned on.

Available Properties

propname is the name of the property to access. It is case-insensitive (prior to 3.17.0, they were case-senstive). Currently supported are:

msg the MSG part of the message (aka “the message” ;))
rawmsg the message excactly as it was received from the socket. Should be useful for debugging.
hostname hostname from the message
source alias for HOSTNAME
fromhost hostname of the system the message was received from (in a relay chain, this is the system immediately in front of us and not necessarily the original sender). This is a DNS-resolved name, except if that is not possible or DNS resolution has been disabled.
fromhost-ip The same as fromhost, but alsways as an IP address. Local inputs (like imklog) use 127.0.0.1 in this property.
syslogtag TAG from the message
programname the “static” part of the tag, as defined by BSD syslogd. For example, when TAG is “named[12345]”, programname is “named”.
pri PRI part of the message – undecoded (single value)
pri-text the PRI part of the message in a textual form with the numerical PRI appended in brackes (e.g. “local0.err<133>”)
iut the monitorware InfoUnitType – used when talking to a MonitorWare backend (also for phpLogCon)
syslogfacility the facility from the message – in numerical form
syslogfacility-text the facility from the message – in text form
syslogseverity severity from the message – in numerical form
syslogseverity-text severity from the message – in text form
syslogpriority an alias for syslogseverity – included for historical reasons (be careful: it still is the severity, not PRI!)
syslogpriority-text an alias for syslogseverity-text
timegenerated timestamp when the message was RECEIVED. Always in high resolution
timereported timestamp from the message. Resolution depends on what was provided in the message (in most cases, only seconds)
timestamp alias for timereported
protocol-version The contents of the PROTCOL-VERSION field from IETF draft draft-ietf-syslog-protcol
structured-data The contents of the STRUCTURED-DATA field from IETF draft draft-ietf-syslog-protocol
app-name The contents of the APP-NAME field from IETF draft draft-ietf-syslog-protocol
procid The contents of the PROCID field from IETF draft draft-ietf-syslog-protocol
msgid The contents of the MSGID field from IETF draft draft-ietf-syslog-protocol
inputname The name of the input module that generated the message (e.g. “imuxsock”, “imudp”). Note that not all modules necessarily provide this property. If not provided, it is an empty string. Also note that the input module may provide any value of its liking. Most importantly, it is not necessarily the module input name. Internal sources can also provide inputnames. Currently, “rsyslogd” is defined as inputname for messages internally generated by rsyslogd, for example startup and shutdown and error messages. This property is considered useful when trying to filter messages based on where they originated – e.g. locally generated messages (“rsyslogd”, “imuxsock”, “imklog”) should go to a different place than messages generated somewhere.
$bom The UTF-8 encoded Unicode byte-order mask (BOM). This may be useful in templates for RFC5424 support, when the character set is know to be Unicode.
$now The current date stamp in the format YYYY-MM-DD
$year The current year (4-digit)
$month The current month (2-digit)
$day The current day of the month (2-digit)
$hour The current hour in military (24 hour) time (2-digit)
$hhour The current half hour we are in. From minute 0 to 29, this is always 0 while from 30 to 59 it is always 1.
$qhour The current quarter hour we are in. Much like $HHOUR, but values range from 0 to 3 (for the four quater hours that are in each hour)
$minute The current minute (2-digit)
$myhostname The name of the current host as it knows itself (probably useful for filtering in a generic way)

Properties starting with a $-sign are so-called system properties. These do NOT stem from the message but are rather internally-generated.

Character Positions

FromChar and toChar are used to build substrings. They specify the offset within the string that should be copied. Offset counting starts at 1, so if you need to obtain the first 2 characters of the message text, you can use this syntax: “%msg:1:2%”. If you do not whish to specify from and to, but you want to specify options, you still need to include the colons. For example, if you would like to convert the full message text to lower case, use “%msg:::lowercase%”. If you would like to extract from a position until the end of the string, you can place a dollar-sign (“$”) in toChar (e.g. %msg:10:$%, which will extract from position 10 to the end of the string).

There is also support for regular expressions. To use them, you need to place a “R” into FromChar. This tells rsyslog that a regular expression instead of position-based extraction is desired. The actual regular expression must then be provided in toChar. The regular expression must be followed by the string “–end”. It denotes the end of the regular expression and will not become part of it. If you are using regular expressions, the property replacer will return the part of the property text that matches the regular expression. An example for a property replacer sequence with a regular expression is: “%msg:R:.*Sev:. \(.*\) \[.*–end%”

It is possible to specify some parametes after the “R”. These are comma-separated. They are:

R,<regexp-type>,<submatch>,<nomatch>,<match-number>

regexp-type is either “BRE” for Posix basic regular expressions or “ERE” for extended ones. The string must be given in upper case. The default is “BRE” to be consistent with earlier versions of rsyslog that did not support ERE. The submatch identifies the submatch to be used with the result. A single digit is supported. Match 0 is the full match, while 1 to 9 are the acutal submatches. The match-number identifies which match to use, if the expression occurs more than once inside the string. Please note that the first match is number 0, the second 1 and so on. Up to 10 matches (up to number 9) are supported. Please note that it would be more natural to have the match-number in front of submatch, but this would break backward-compatibility. So the match-number must be specified after “nomatch”.

nomatch specifies what should be used in case no match is found.

The following is a sample of an ERE expression that takes the first submatch from the message string and replaces the expression with the full field if no match is found:

%msg:R,ERE,1,FIELD:for (vlan[0-9]*):–end%

and this takes the first submatch of the second match of said expression:

%msg:R,ERE,1,FIELD,1:for (vlan[0-9]*):–end%

Please note: there is also a rsyslog regular expression checker/generator online tool available. With that tool, you can check your regular expressions and also generate a valid property replacer sequence. Usage of this tool is recommended. Depending on the version offered, the tool may not cover all subleties that can be done with the property replacer. It concentrates on the most often used cases. So it is still useful to hand-craft expressions for demanding environments.

Also, extraction can be done based on so-called “fields”. To do so, place a “F” into FromChar. A field in its current definition is anything that is delimited by a delimiter character. The delimiter by default is TAB (US-ASCII value 9). However, if can be changed to any other US-ASCII character by specifying a comma and the decimal US-ASCII value of the delimiter immediately after the “F”. For example, to use comma (“,”) as a delimiter, use this field specifier: “F,44”.  If your syslog data is delimited, this is a quicker way to extract than via regular expressions (actually, a *much* quicker way). Field counting starts at 1. Field zero is accepted, but will always lead to a “field not found” error. The same happens if a field number higher than the number of fields in the property is requested. The field number must be placed in the “ToChar” parameter. An example where the 3rd field (delimited by TAB) from the msg property is extracted is as follows: “%msg:F:3%”. The same example with semicolon as delimiter is “%msg:F,59:3%”.

Please note that the special characters “F” and “R” are case-sensitive. Only upper case works, lower case will return an error. There are no white spaces permitted inside the sequence (that will lead to error messages and will NOT provide the intended result).

Each occurence of the field delimiter starts a new field. However, if you add a plus sign (“+”) after the field delimiter, multiple delimiters, one immediately after the others, are treated as separate fields. This can be useful in cases where the syslog message contains such sequences. A frequent case may be with code that is written as follows:

int n, m;

syslog(LOG_ERR, “%d test %6d”, n, m);

This will result into things like this in syslog messages: “1 test      2”, “1 test     23”, “1 test  234567”

As you can see, the fields are delimited by space characters, but their exact number is unknown. They can properly be extracted as follows:

“%msg:F,32:2%” to “%msg:F,32+:2%”.

This feature was suggested by Zhuang Yuyao and implemented by him. It is modeled after perl compatible regular expressions.

Property Options

property options are case-insensitive. Currently, the following options are defined:

This feature was introduced in rsyslog 4.6.2 and v4 versions above and 5.5.3 and all versions above.

uppercase convert property to lowercase only
lowercase convert property text to uppercase only
csv formats the resulting field (after all modifications) in CSV format as specified in RFC 4180. Rsyslog will always use double quotes. Note that in order to have full CSV-formatted text, you need to define a proper template. An example is this one:
$template csvline,”%syslogtag:::csv%,%msg:::csv%”
Most importantly, you need to provide the commas between the fields inside the template.
This feature was introduced in rsyslog 4.1.6.
drop-last-lf The last LF in the message (if any), is dropped. Especially useful for PIX.
date-mysql format as mysql date
date-rfc3164 format as RFC 3164 date
date-rfc3164-buggyday similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands that a space is written for single-digit days. With this option, a zero is written instead. This format seems to be used by syslog-ng and the date-rfc3164-buggyday option can be used in migration scenarios where otherwise lots of scripts would need to be adjusted. It is recommended not to use this option when forwarding to remote hosts – they may treat the date as invalid (especially when parsing strictly according to RFC 3164).
date-rfc3339 format as RFC 3339 date
date-subseconds just the subseconds of a timestamp (always 0 for a low precision timestamp)
escape-cc replace control characters (ASCII value 127 and values less then 32) with an escape sequence. The sequnce is “#<charval>” where charval is the 3-digit decimal value of the control character. For example, a tabulator would be replaced by “#009”.
Note: using this option requires that $EscapeControlCharactersOnReceive is set to off.
space-cc replace control characters by spaces
Note: using this option requires that $EscapeControlCharactersOnReceive is set to off.
drop-cc drop control characters – the resulting string will neither contain control characters, escape sequences nor any other replacement character like space.
Note: using this option requires that $EscapeControlCharactersOnReceive is set to off.
sp-if-no-1st-sp This option looks scary and should probably not be used by a user. For any field given, it returns either a single space character or no character at all. Field content is never returned. A space is returned if (and only if) the first character of the field’s content is NOT a space. This option is kind of a hack to solve a problem rooted in RFC 3164: 3164 specifies no delimiter between the syslog tag sequence and the actual message text. Almost all implementation in fact delemit the two by a space. As of RFC 3164, this space is part of the message text itself. This leads to a problem when building the message (e.g. when writing to disk or forwarding). Should a delimiting space be included if the message does not start with one? If not, the tag is immediately followed by another non-space character, which can lead some log parsers to misinterpret what is the tag and what the message. The problem finally surfaced when the klog module was restructured and the tag correctly written. It exists with other message sources, too. The solution was the introduction of this special property replacer option. Now, the default template can contain a conditional space, which exists only if the message does not start with one. While this does not solve all issues, it should work good enough in the far majority of all cases. If you read this text and have no idea of what it is talking about – relax: this is a good indication you will never need this option. Simply forget about it 😉
secpath-drop Drops slashes inside the field (e.g. “a/b” becomes “ab”). Useful for secure pathname generation (with dynafiles).
secpath-replace Replace slashes inside the field by an underscore. (e.g. “a/b” becomes “a_b”). Useful for secure pathname generation (with dynafiles).

BSD-Style blocks

Let’s look at this example:

!prog1

*.* /var/log/prog1.log

*.* /var/log/prog1again.log

!prog2

*.* /var/log/prog2.log

*.* /var/log/prog2again.log

This code can very simply be replaced by:

if $programname == ‘prog1’ then {

/var/log/prog1.log

/var/log/prog1again.log

}

if $programname == ‘prog2’ then {

/var/log/prog2.log

/var/log/prog2again.log

}

And if you prefer the more powerful action statments (probably not so much benefit for this use case…), you can write:

if $programname == ‘prog1’ then {

action(type=”omfile” file=”/var/log/prog1.log”)

action(type=”omfile” file=”/var/log/prog1again.log”)

}

if $programname == ‘prog2’ then {

action(type=”omfile” file=”/var/log/prog2.log”)

action(type=”omfile” file=”/var/log/prog2again.log”)

}

I expect that usually these easy cases happen. HOWEVER, if I had kept support for BSD-style blocks, one could configure things like this:

!prog1

if $msg contains ‘test’ then  {

action(type=”omfile” file=”/var/log/somefile”)

!prog2

mail.* :mmjsonparse:

& action(type=”omfile” file=”/var/log/somefile2″)

!prog3

& ~

!prog4

if $msg contains ‘test2’ then

/var/log/logfile

else

/var/log/logfile2

}

 

By Rayhan

My name is Rayhan and I'm an IT professional with over 10 years of experience in the field. I'm passionate about all things tech, and I love helping people solve their IT problems. In my free time, I enjoy tinkering with new gadgets and software, and I'm always on the lookout for the latest tech trends. I believe that technology has the power to make our lives easier and more enjoyable, and I'm excited to be a part of this ever-evolving field. Thank you for taking the time to visit my page.

Leave a Reply

Your email address will not be published. Required fields are marked *