General system logs on port 8322-non secure port
All the actions regarding your system are stored in /var/log/syslog ( Ubuntu ) by default . For example which cron was executed at what time etc.
These metrics are useful in order to see the state of your system.
Cloudlytics helps you to monitor your system using the syslog.
Next we will go through a few steps, to help you configure System Logs of your system using Rsyslog.
Steps to configure system logs for your system using Rsyslog :
Step 1: Create a stream for System Logs
Log into your Cloudlytics account. Go to Create Stream System Page.
Select the type as System Logs
Select the Timezone according to your respective region.
Save and Name the stream.
Check for successful stream creation.
You will receive a token ( Stream Token ) once the Stream is successfully created. This token is very important since all the future configurations will be done using this token only.
Step 2 : Configure Rsyslog on your system to forward your logs to Cloudlytics
The following configuration is required on each of the systems you want to monitor.
Create an empty file “/etc/rsyslog.d/21-system-cloudlytics.conf” under “/etc/rsyslog.d” and add the details below, while replacing TOKEN_HERE with the token you received after creating a Stream. (Pleaser refer the above image )
Please pay attention towards the tag “Cron-Server” after %HOSTNAME%.
Tags are provided by you which will help you to identify your resources when you start monitoring your systems on Cloudlytics. You can provide as many as tags you want.
Just provide a “,” after each tag. Example :%HOSTNAME% ,MyTag_1,MyTag_2,Mytag_3
What did we achieve in the above step ??
We told Rsyslog to forward all the System log data to the Stream which you configured in Cloudlytics.
Your data will be pushed to the Stream for which you have configured the TOKEN.
Please make sure that you provide the correct token for against the stream which you want to configure, else the data will be pushed but will not be processed and no data will be available.
You have now successfully configured System log with Cloudlytics.
Step 3 : Restart rsyslog
SSH Logs
SSH logs help you get details about the login activities in your system. It is quite useful to monitor your SSH logs in order and get details about any attacks.
We will be using Rsyslog in order to send SSH SSH logs to the endpoints.
Next we will go through steps, to help you configure the SSH log file of your system using Rsyslog.
Steps to configure SSH logs fo your system using Rsyslog using port 8322 – non secure port :
Step 1: Create a stream for SSHLogs
Log into your Cloudlytics account.
Go to Create Stream AWS Page.
Select the type as SSH Logs
Select the Timezone according to your respective region.
Save and Name the stream.
Check for successful stream creation.
You will receive a token ( Stream Token ) once the Stream is successfully created. This token is very important since all the future configurations will be done using this token only.
Step 2: Configure Rsyslog on your system to forward your logs to Cloudlytics
The following configuration is required on each of the systems you want to monitor.
Create an empty file “/etc/rsyslog.d/21-system-sshcloudlytics.conf” under “/etc/rsyslog.d” and add the details below, while replacing TOKEN_HERE with the token you received after creating a Stream.
Please pay attention towards the tag “Server-1-SSH” after %HOSTNAME%.
Tags are provided by you which will help you to identify your resources when you start monitoring your systems on Cloudlytics. You can provide as many as tags you want.
Just provide a “,” after each tag. Example : %HOSTNAME% ,MyTag_1,MyTag_2,Mytag_3
What did we do in the above step ??
We told Rsyslog to forward all the SSH log data to the Stream which you configured in Cloudlytics.
Your data will be pushed to the Stream for which you have configured the TOKEN. So please make sure that you provide the correct token for the appropriate log type which you want to configure, else the data will be pushed but it will not be processed and you will not be able to make any sense out of your data.
The above configuration will work perfectly fine for any Ubuntu System on which you did not have made any changes and all the settings are default.
Change ??
Look at above configuration and pay attention to the line $InputFileName /var/log/auth.log.
This line describes the location of your SSH log file , the location where your OS is storing all your SSH activities , by default on Ubuntu systems all the SSH activities are logged into “/var/log/auth.log”.
Above configuration will work perfectly fine and all your logs will be pushed to your Stream without any error, if you have not made any changes to the default log location, but this will fail if you have changed your auth.log path or you are working on any other OS, Example : Amazon Linux, CentOS etc.
Log file locations are specific to OS,so please make sure that you provide the correct path to $InputFileName , if you have made any changes to the default log location or your OS stores SSH logs at a different location.
You have now successfully configured System log with Cloudlytics.
Step 2: Restart rsyslog
Send Logs securely using Rsyslog
Step 1: Install rsyslog-gnutls on the server (linux)
For Amazon Linux : yum -y install rsyslog-gnutls
For Ubuntu: apt-get install rsyslog-gnutls
Step 2: Download SSL certificate
mkdir /etc/rsyslog.d/keys
mkdir /etc/rsyslog.d/keys/ca.d
cd /etc/rsylog.d/keys/ca.d/
wget https://s3.amazonaws.com/cloudlytics-2/SSL+Certificate/data-cloudlytics.cert
Step 3: Add Configuration to encrypt log events in transit
Add following lines after defining workDirectory
#RsyslogGnuTLS
$DefaultNetstreamDriverCAFile /etc/rsyslog.d/keys/ca.d/data-cloudlytics.cert
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer data.cloudlytics.com
Step 4: Configure rsyslog to send log files to Cloudlytics
Configuration files can be downloaded from following links Apache, Nginx, System Log, SSH Log
Download configuration files from the links and replace “TOKEN_HERE” with the stream token from your Cloudlytics account.
Replace “TAGS_HERE” with the comma separated tags that you want to assign to events.
Restart the rsyslog service and check your Cloudlytics for processed events/logs.
