Creating an OSS Splunk App Part 1: Setting up

Octocat by jglovier & JohnCreek

This is the next instalment of series of posts related to building your our Open Source app for [Splunk](). My initial thoughts are over here, and I hope to structure the series as follows:

[Part 1 - Setting up a repository and build on Github for a Splunk app]().
Part 2 - Getting data into Splunk via an AddOn/Modular Input
Part 3 - Securing credentials for an AddOn/Modular Input
Part 4 - Dashboards Redux
Part 5 - Deployment Considerations
Part 6 - Splunk App Certification

Getting Started

Many people would probably start hacking away at this point, however I thought I would take a step back and tackle this from a quality point of view. The idea is to first create a deployable artefact with the bare minimum content, prior to crunching out any code.

Overall the goal of this blog series is to produce a high quality app that can be submitted to Splunkbase for the wider public to use.

Tools & Languages

Everyone has their favourite tools, languages and preferences to build cool ideas. For this I will keep it reasonablly straight forward and so I am going to use the following setup.

Code Repository

I chose the defacto for Open Source hosting… GitHub. A new repository exists for the Add On and the existing repository will be used for an updated dashboard app. Once you create a new repository will also want to establish a licence, contribution guidelines along with other bits and pieces. Also don’t forget a .gitignore file.

I structured the folders in the following layout in attempt to keep the code clean from the build and documentation files.

├── .gitignore
├── docs
├── src
│   ├── bin
│   ├── default
│   └── static
└── test

Doco

For the documentation, I opted for Read The Docs, I wanted to establish a documentation output early in the process that was easy to maintain. In this case I went with ReStructuredText that can also be generated locally with a Make file. More details on how to setup ReadTheDocs can be found over here.

The documentation produced can be found at http://splunk-octopusdeploy-add-on.readthedocs.io/en/latest/.

The Build

The next item on the list is the build. For this I have utilised TravisCI. The first step I took however was to create a build file that allowed me to package the app locally and test the process. The current build script can be viewed here.

In this file I attempt to do the following:

Establish a release version resembling SemVer.
Create a clean temp workspace.
Copy any required files for the app package.
Remove any unwanted files (in this case python cache).
Bump the version using bumpversion.
Package the file as a tgz.
Run pytest for any Unit tests in place.
Run Splunk AppInspect to ensure quality.

Next I needed to establish a TravisCI build. After connecting your account to TravisCI (I use a build account in GitHub) you then create a .travis.yml in the root of the GitHub repo.

In TravisCI I am going to treat this as a Python app, so I start with the following block.

language: python
sudo: false
python:
  - '2.7'

Then I need some requirements such as Splunk AppInspect and bumpversion, so I add some before_script steps.

before_install:
  - wget --output-document splunk-appinspect.tar.gz http://dev.splunk.com/goto/appinspectdownload
  - pip install splunk-appinspect.tar.gz
install:
  - pip install bumpversion
  - pip install -r src/requirements.txt

The next set of commands relate to the build itself and the releases. In this case, releasing to GitHub on a tagged commit.

script:
  - "./build.sh"
deploy:
  provider: releases
  skip_cleanup: true
  on:
    tags: true

The complete file is [here]() and TravisCI docs with more detailed configuration can be found at their docs site.

Finally the build is at https://travis-ci.org/merbla/splunk-octopusdeploy-add-on with badges and other integrations possible.

You Broke the build!

50%

That’s right, at the time of writing I wanted to have the build broken as a segue into the next post. I also wanted to illustrate how Splunk AppInspect can be used in the build pipeline.

If you have a look at Build 15 you will see the output of the current build process. At the bottom of the build output is the Splunk AppInspect result. This is key to getting a high quality app ready for a Splunk deployment.

octpopus_deploy_addon Report Summary:

       skipped:  0
       success: 65
  manual_check: 23
       failure:  4
       warning:  0
         error:  0
not_applicable: 92
-------------------
         Total: 184

Recap

So in this post, we set up a repository, established a build pipeline with some quality control, knocked out some basic documentation and broke the build.

Next time, we will start building out the Splunk Add On along with tagging in Git to produce a release, docs and a new version of the app.

Get Amongst It!

Tags