{"id":351,"date":"2025-02-12T21:31:53","date_gmt":"2025-02-12T21:31:53","guid":{"rendered":"https:\/\/blog.lottabytes.com\/?p=351"},"modified":"2025-02-13T17:17:34","modified_gmt":"2025-02-13T17:17:34","slug":"different-logging-approaches","status":"publish","type":"post","link":"https:\/\/blog.lottabytes.com\/index.php\/2025\/02\/12\/different-logging-approaches\/","title":{"rendered":"Different Logging Approaches"},"content":{"rendered":"\n

There are many common approaches to logging. However, since my home environment is running SecurityOnion (SO), I’m going to focus on three common ones and show how to do an end-to-end configuration with SO. If you’re curious about Filebeat, I’ve already written about it here<\/a>.<\/p>\n\n\n\n

Fluentd<\/h2>\n\n\n\n

Fluentd is extremely popular, while providing many options and granular configuration. That said, it’s also more work to get started as opposed to the other options that we’ll discuss.<\/p>\n\n\n\n

To get this working you will need to do a few things on the Security Onion side:<\/p>\n\n\n\n

    \n
  1. Configure an Elastic role with minimal permissions to allow event ingestion.<\/li>\n\n\n\n
  2. Create an Elastic user and assign to the newly created role.<\/li>\n\n\n\n
  3. Open up the SO firewall to allow Elasticsearch on port 9200. This is accomplished using the Administration > Configuration > Firewall<\/em> settings.<\/li>\n<\/ol>\n\n\n\n

    Here’s the role definition, with associated permissions.<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n

    You can now deploy your Fluentd agent. I’m using Ubuntu in this example.<\/p>\n\n\n\n

    # Download the binary\nwget https:\/\/s3.amazonaws.com\/packages.treasuredata.com\/lts\/5\/ubuntu\/jammy\/pool\/contrib\/f\/fluent-package\/fluent-package_5.0.5-1_amd64.deb\n\n# Install Fluentd, which becomes the fluend.service\nsudo dpkg -i fluent-package_5.0.5-1_amd64.deb\n\n# Create the td-agent directory to store position files\nsudo mkdir \/var\/log\/td-agent\nsudo chown _fluentd:_fluentd \/var\/log\/td-agent\/\n\n# Grant the _fluentd user access to read system log files as part of the adm group\nsudo usermod _fluentd -aG adm<\/code><\/pre>\n\n\n\n
    Next, we need to configure Fluentd (\/etc\/fluentd\/fluentd.conf).  Here’s a working configuration.<\/p>\n\n\n\n
    <system> \n  log_level error\n<\/system>\n\n<source>\n  @type tail\n  path \/var\/log\/syslog\n  pos_file \/var\/log\/td-agent\/syslog.pos\n  tag system.log\n  <parse>\n    @type syslog\n    keep_time_key true\n  <\/parse>\n<\/source>\n\n<source>\n  @type tail\n  path \/var\/log\/auth.log\n  pos_file \/var\/log\/td-agent\/auth.pos\n  tag system.auth\n  <parse>\n    @type syslog\n    keep_time_key true\n  <\/parse>\n<\/source>\n\n<filter system.log>\n  @type record_transformer\n  <record>\n    file_location \"\/var\/log\/syslog\"\n  <\/record>\n<\/filter>\n\n<filter system.auth>\n  @type record_transformer\n  <record>\n    file_location \"\/var\/log\/auth.log\"\n    security_event \"true\"\n  <\/record>\n<\/filter>\n\n# The timestamp for syslog doesn't work, as there isn't a year.  The below is a quick workaround to append a new, slightly less accurate timestamp that allows SO to parse it and show time-series data.\n<filter system.*>\n  @type record_transformer\n  enable_ruby true\n  <record>\n    timestamp ${Time.now.utc.iso8601}\n  <\/record>\n<\/filter>\n\n## DOCS: https:\/\/docs.fluentd.org\/output\/elasticsearch\n<match system.* wazuh.*>\n  @type elasticsearch\n  host sec-onion-master.server.com\n  port 9200\n  # This causes it to break, because the permissions are required on the logstash index\n  # logstash_format true\n  user \"filebeat_user\"\n  password \"L0gging123!\"\n  scheme https\n  reconnect_on_error true\n  index_name \"fluentd-\"\n  <buffer tag, time>\n    flush_interval 60s\n    timekey 1m\n    timekey_wait 1m\n  <\/buffer>\n  ssl_verify false\n<\/match>\n<\/code><\/pre>\n\n\n\n
    When you run into issues you can view the troubleshooting logs at \/var\/log\/fluent\/fluentd.log.  However, be cautioned, often it’s not very informative, even at a debug log level.<\/p>\n\n\n\n
    Once things are working, you should be able to create a new Data View against the index_name that you specified in the above file.  This should then show you your incoming logs.<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    <\/p>\n\n\n\n
    rSyslog<\/h2>\n\n\n\n
    Syslog has been around for a long time, and in my experience is primarily used by networking devices to send events.  However, it’s still fully supported to use on a client OS.  <\/p>\n\n\n\n
    First, open the firewall.<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    Once this is opened, you can update the \/etc\/rsyslog\/rsyslog.conf file to begin forwarding log events.<\/p>\n\n\n\n
    # \/etc\/rsyslog.conf configuration file for rsyslog\n#\n# For more information install rsyslog-doc and see\n# \/usr\/share\/doc\/rsyslog-doc\/html\/configuration\/index.html\n#\n# Default logging rules can be found in \/etc\/rsyslog.d\/50-default.conf\n\n\n#################\n#### MODULES ####\n#################\n\nmodule(load=\"imuxsock\") # provides support for local system logging\nmodule(load=\"imklog\" permitnonkernelfacility=\"on\")\n\n###########################\n#### GLOBAL DIRECTIVES ####\n###########################\n\n$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat\n$RepeatedMsgReduction on\n\n#\n# Set the default permissions for all log files.\n#\n$FileOwner syslog\n$FileGroup adm\n$FileCreateMode 0640\n$DirCreateMode 0755\n$Umask 0022\n$PrivDropToUser syslog\n$PrivDropToGroup syslog\n\n$WorkDirectory \/var\/spool\/rsyslog\n\n$IncludeConfig \/etc\/rsyslog.d\/*.conf\n\n# Add in your Security Onion Server here:\n*.* @@sec-onion-master.server.com:514\n<\/code><\/pre>\n\n\n\n
    All the incoming logs (which can be configured via the files in \/etc\/rsyslog.d\/) will show up in the logs-* <\/em>view, and can easily be filtered using the data_stream.dataset = syslog<\/em> parameter.<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    That’s it!  Now you have three different methods to forward logs to your Security Onion (or other Elastic-based) server.<\/p>\n","protected":false},"excerpt":{"rendered":"
    There are many common approaches to logging. However, since my home environment is running SecurityOnion (SO), I’m going to focus on three common ones and show how to do an end-to-end configuration with SO. If you’re curious about Filebeat, I’ve already written about it here. Fluentd Fluentd is extremely popular, while providing many options and … <\/p>\n
    Read more “Different Logging Approaches”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[69,83,84,86,82,51,85],"class_list":["post-351","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-elasticsearch","tag-filebeat","tag-fluentd","tag-getting-started","tag-logging","tag-security-onion","tag-syslog"],"_links":{"self":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/351","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/comments?post=351"}],"version-history":[{"count":9,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/351\/revisions"}],"predecessor-version":[{"id":366,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/351\/revisions\/366"}],"wp:attachment":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/media?parent=351"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/categories?post=351"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/tags?post=351"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}