{"id":251,"date":"2023-03-17T15:34:06","date_gmt":"2023-03-17T15:34:06","guid":{"rendered":"https:\/\/blog.lottabytes.com\/?p=251"},"modified":"2025-02-07T20:41:09","modified_gmt":"2025-02-07T20:41:09","slug":"ingesting-external-logs-via-security-onions-elasticsearch","status":"publish","type":"post","link":"https:\/\/blog.lottabytes.com\/index.php\/2023\/03\/17\/ingesting-external-logs-via-security-onions-elasticsearch\/","title":{"rendered":"Ingesting External Logs via Security Onion’s Elasticsearch"},"content":{"rendered":"\n

So, you’ve got Security Onion (SO) running from the Security-Appliance-in-a-Box via Ansible<\/a>. Now what? How do you begin to ingest logs from your other devices into the included Elastic instance? I’m glad you asked! There’s a couple steps you’ll need to follow.<\/p>\n\n\n\n

Allow Access<\/h2>\n\n\n\n

First you’re going to need to open the firewall to allow incoming TCP traffic to port 9200. You can limit it by host\/subnet, or just open it to all using the below.<\/p>\n\n\n\n

For the 2.4 version, you can do this via the Administration > Configuration, but need to enable the advanced settings.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

On the legacy version you do this as shown below.<\/p>\n\n\n\n

sudo firewall-cmd --permanent --add-port=9200\/tcp<\/code><\/pre>\n\n\n\n
On legacy versions, you’ll also need to authorize the subnet\/host with SO (using the so-allow command) to use the Elasticsearch REST API.<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Configure Elasticsearch<\/h2>\n\n\n\n
There’s a couple things that need to be done to allow ingestion, as well as viewing your data. The first is to create a role for publishing events<\/a>. I’ve granted it the below privileges, which include the ability to create indexes. This is the minimum set of requirements that I’ve found, removal of any will prevent the ability to send logs.<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Next, you’ll need to create a user that your agents will use.<\/p>\n\n\n\n
\"\"
 <\/figcaption><\/figure>\n\n\n\n
Data Views and Indices<\/h2>\n\n\n\n
By default, you won’t see filebeat as a dataview, which isn’t useful as you can’t see your incoming logs. To remedy that situation, click on the Create a data view<\/em> button. <\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Using filebeat-*<\/em> as both the Name and Index Pattern, you should see that you have a source already mapped. If you don’t see that, it’s probably because your agents aren’t sending logs yet.<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Agent Configuration<\/h2>\n\n\n\n
There’s already good documentation around installing the Filebeat Agent<\/a>, so I won’t cover that except to note that this docs page<\/a> is important for SSL settings as HTTPS is the default.  After that, I do want to call out that you need to go into the appropriate module (e.g., nginx) configuration, otherwise you won’t see logs, even after enabling the module.<\/p>\n\n\n\n
Example: I want to collect nginx logs.<\/p>\n\n\n\n
Step 1: Update the nginx module configuration, located at \/etc\/filebeat\/modules.d\/nginx.yml.  Mark sections (e.g., access vs. error) as enabled.  Provide path details to the files as required.<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Finally, enable the module:<\/p>\n\n\n\n
sudo filebeat modules enable nginx<\/code><\/pre>\n\n\n\n
There you have it.  Now your SO appliance is beginning to ingest log data from your remote hosts.<\/p>\n\n\n\n
\"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"
So, you’ve got Security Onion (SO) running from the Security-Appliance-in-a-Box via Ansible. Now what? How do you begin to ingest logs from your other devices into the included Elastic instance? I’m glad you asked! There’s a couple steps you’ll need to follow. Allow Access First you’re going to need to open the firewall to allow … <\/p>\n
Read more “Ingesting External Logs via Security Onion’s Elasticsearch”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[69,68,53,54,51],"class_list":["post-251","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-elasticsearch","tag-nginx","tag-security","tag-security-appliance","tag-security-onion"],"_links":{"self":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/251","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/comments?post=251"}],"version-history":[{"count":9,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/251\/revisions"}],"predecessor-version":[{"id":349,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/posts\/251\/revisions\/349"}],"wp:attachment":[{"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/media?parent=251"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/categories?post=251"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lottabytes.com\/index.php\/wp-json\/wp\/v2\/tags?post=251"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}