Serilog and the new Splunk Sink
Over the last couple of years, like most in software I have become more and more involved with the “DevOps” landscape. One product that has been a great help when working with the IoT is Splunk. In my opinion, when dealing with distributed systems that utilise many devices and operating systems, the architecture of Splunk simply stacks up. This post will not go into the details of how Splunk works, rather how you can start utilising it in the world of .Net.
Day to day I am primarily in the Microsoft space, and as such .Net is a large part of what I do. In the .Net logging space there have always been the constant libraries (NLog, Log4Net etc.), however with the introduction of libraries such as Serilog the .Net community is better understanding the benefits of structured logging. This is not to say that this was not possible prior to Serilog, however the barrier to entry and understanding is much lower.
##.Net Options for logging to Splunk
There are a number of options out of the box that can get you up and running with Splunk using either a traditional logging provider or something like Serilog.
Index a file
The simplest example of getting up and running is to write to a log file and then use a forwarder or indexer on that file. Check out Configuring Data Inputs on how to set up data inputs in Splunk.
When using Serilog, life is quite simple. Target a rolling file or whatever variant you desire, log the file out to a location that can be accessed by Splunk.
e.g.
var log = new LoggerConfiguration()
.WriteTo.RollingFile("AFile.txt")
.CreateLogger();
Below is an example output of an indexed file.
##Splunk Sink for Serilog
There is currently a Splunk sink available for Serilog however it is built on top of the old Splunk SDK package (e.g. Install-Package SplunkSDK).
Recently, Glenn Block, Grigori Melnik and the rest of the dev team @ Splunk have re-written the libraries from the ground up.
Get the new awesome sauce Install-Package Splunk.Client and Install-Package Serilog.Sinks.Splunk
NOTE: Currently the examples depend on a prerelease package (e.g. Install-Package Splunk.Client -Pre). This is due to the Splunk PCL package being a beta however it should be generally available sometime this month.
There are a number of options available to an app targeting either PCL or .net 4.5 when using the new Splunk Serilog sink.
- TCP (not currently available in PCL)
- UDP (not currently available in PCL)
- HTTP/HTTPS (available to both full and PCL)
The sinks follow the Serilog convention in logging the rendered message, message template and all enriched properties.
As a result, in Splunk you get something like the following.
So to get going, pull in the new sink
> Install-Package Serilog.Sinks.Splunk -Pre
Install-Package Serilog.Sinks.Splunk
Via TCP
The first thing to do is setup TCP. To do this, check out the steps located in this link.
For my example, I am using 10001 as my listening TCP port within Splunk. From there I need to configure Serilog.
Via UDP
In a very similar fashion to setting up TCP, you need to configure UDP. The following link will get you started.
In my example, I have used 10000 as my listening UDP port within Splunk. After that, I configure Serilog as below.
Below is an example output when logging using TCP.  ### Via HTTP/HTTPS The final option, and the only one currently available to [Serilog][Serilog] in `PCL` land, is HTTP/HTTPS. This is great for mobile apps and other occasionally connected devices. Generally, the [Splunk][Splunk] team recommend logging via a persistent mechanism (file combined with a forwarder) rather than HTTP/HTTPS. This is normally due to the volume of data being logged. If using a default [Splunk][Splunk] setup, no extra configuration is required. By default the management port of [Splunk][Splunk] is configured to 8089. The following example illustrates how to configure [Serliog][Serilog] to use HTTPS.
What Next
This is just a quick tale indicating how you might get up and running with Splunk. Word is the new Splunk PCL package will be available very soon.
GA is OUT!