Browsed by
Category: Wavefront

Playing with Wavefront – Network Packet Loss

Playing with Wavefront – Network Packet Loss

Now that we know when an agent goes offline, let’s create a query to detect when our devices experience an increased rate of dropped packets. To do that we’ll create 2 queries, the first is our data on all dropped packets per source. [code language=”bash”] sum(mavg(5m,ts("net.drop.*", source="FQDN,sub.domain.com" )),sources) [/code] This value is represented with the blue line in the below chart. That’s great, but we want to detect a change in trends, not just alert on a static threshold. To…

Read More Read More

Playing with Wavefront – Missing Agents

Playing with Wavefront – Missing Agents

One of the first things that we need to detect when using Wavefront is if one of our endpoints goes silent. To do that, anyone can use the Query Wizard to create a basic alerting query. In this example I’m going to use the “system.uptime” metric as my base in the Alert Wizard’s magic sauce with a 1 minute time window. The Query Wizard shows me quite nicely that I’ve had a couple outages in the past.  But, if you…

Read More Read More

Getting Started – Wavefront by VMware – Queries

Getting Started – Wavefront by VMware – Queries

Ok, let’s chat about Wavefront’s UI and getting value from our data! This is the most user-friendly product that I’ve used for time series data! Let’s explore a quick example using disk space to showcase some of that functionality. Telegraf only sends raw values for: Total Space Free Space Used Space This is seen below where I have intentionally limited the results to a single host and single disk object. We have ~55 GB Total and ~30 GB Free. What…

Read More Read More

Wavefront by VMware – Missing Metrics – Point outside of reasonable timeframe

Wavefront by VMware – Missing Metrics – Point outside of reasonable timeframe

I ran into an issue where metrics were not showing up in Wavefront and wanted to share the solution. Bottom line, if you send any metrics to the Wavefront Proxy, they absolutely must be in nanoseconds. Here’s why: I have an application that sends telemetry via Telegraf’s Socket Listener Input Plugin, and the telemetry timestamp was in milliseconds. What happened is that none of the data was recorded in Wavefront. Instead, inside of the /var/log/wavefront/wavefront.log there were entries stating that…

Read More Read More

Getting Started – Wavefront by VMware – Telegraf Agents

Getting Started – Wavefront by VMware – Telegraf Agents

Wavefront, as a TSDB utilizes a wide range of Collectors to gather time series data from various devices. Most of these collectors utilize a, currently forked, version of the popular Telegraf Agent. A preview of this out-of-box functionality is shown below. This forked agent includes and Output Plugin for the Wavefront Protocol and the changes can be seen in the Github Pull Request which is currently in version 1.5 RC1 of the native Telegraf Agent! Once Telegraf 1.5 releases, there…

Read More Read More

Getting Started – Wavefront by VMware – Automated Proxy Installation

Getting Started – Wavefront by VMware – Automated Proxy Installation

As I mentioned in a previous post, we are beginning to use Wavefront, which is a Time Series Database (TSDB) that has an great user experience. Here’s a brief Getting Started guide that covers a bit of reverse engineering I did on the out-of-box installation process so that we can use automation to deploy the Wavefront Proxy. The first step towards beginning to use Wavefront is to deploy local Wavefront Proxies inside of your environment that will ingest time series…

Read More Read More

Comparision of Time Series Database Options

Comparision of Time Series Database Options

We have been investigating how to mature our Time Series Database architecture and options. Towards this end, I have completed an assessment of a couple of the most popular TSDB options as well as exploring Wavefront. In our team we heavily depend on Open Source, but Wavefront is very interesting since it was recently acquired by VMware. Here’s a quick burndown of the assessment. Every comparison has some assumptions, here are the major ones that I made during this comparison…

Read More Read More