Creating an OSS Splunk App Part 1: Setting up
It's the little things...
Octocat by jglovier & JohnCreek
This is the next instalment of series of posts related to building your our Open Source app for [Splunk](). My initial thoughts are over here, and I hope to structure the series as follows:
- [Part 1 - Setting up a repository and build on Github for a Splunk app]().
- Part 2 - Getting data into Splunk via an AddOn/Modular Input
- Part 3 - Securing credentials for an AddOn/Modular Input
- Part 4 - Dashboards Redux
- Part 5 - Deployment Considerations
- Part 6 - Splunk App Certification
Many people would probably start hacking away at this point, however I thought I would take a step back and tackle this from a quality point of view. The idea is to first create a deployable artefact with the bare minimum content, prior to crunching out any code.
Overall the goal of this blog series is to produce a high quality app that can be submitted to Splunkbase for the wider public to use.
Tools & Languages
Everyone has their favourite tools, languages and preferences to build cool ideas. For this I will keep it reasonablly straight forward and so I am going to use the following setup.
I chose the defacto for Open Source hosting… GitHub. A new repository exists for the Add On and the existing repository will be used for an updated dashboard app. Once you create a new repository will also want to establish a licence, contribution guidelines along with other bits and pieces. Also don’t forget a .gitignore file.
I structured the folders in the following layout in attempt to keep the code clean from the build and documentation files.
├── .gitignore ├── docs ├── src │ ├── bin │ ├── default │ └── static └── test
For the documentation, I opted for Read The Docs, I wanted to establish a documentation output early in the process that was easy to maintain. In this case I went with ReStructuredText that can also be generated locally with a Make file. More details on how to setup ReadTheDocs can be found over here.
The documentation produced can be found at http://splunk-octopusdeploy-add-on.readthedocs.io/en/latest/.
The next item on the list is the build. For this I have utilised TravisCI. The first step I took however was to create a build file that allowed me to package the app locally and test the process. The current build script can be viewed here.
In this file I attempt to do the following:
- Establish a release version resembling SemVer.
- Create a clean temp workspace.
- Copy any required files for the app package.
- Remove any unwanted files (in this case python cache).
- Bump the version using bumpversion.
- Package the file as a tgz.
- Run pytest for any Unit tests in place.
- Run Splunk AppInspect to ensure quality.
Next I needed to establish a TravisCI build. After connecting your account to TravisCI (I use a build account in GitHub) you then create a .travis.yml in the root of the GitHub repo.
In TravisCI I am going to treat this as a Python app, so I start with the following block.
language: python sudo: false python: - '2.7'
Then I need some requirements such as Splunk AppInspect and
bumpversion, so I add some
before_install: - wget --output-document splunk-appinspect.tar.gz http://dev.splunk.com/goto/appinspectdownload - pip install splunk-appinspect.tar.gz install: - pip install bumpversion - pip install -r src/requirements.txt
The next set of commands relate to the build itself and the releases. In this case, releasing to GitHub on a tagged commit.
script: - "./build.sh" deploy: provider: releases skip_cleanup: true on: tags: true
The complete file is [here]() and TravisCI docs with more detailed configuration can be found at their docs site.
Finally the build is at https://travis-ci.org/merbla/splunk-octopusdeploy-add-on with badges and other integrations possible.
You Broke the build!
That’s right, at the time of writing I wanted to have the build broken as a segue into the next post. I also wanted to illustrate how Splunk AppInspect can be used in the build pipeline.
If you have a look at Build 15 you will see the output of the current build process. At the bottom of the build output is the Splunk AppInspect result. This is key to getting a high quality app ready for a Splunk deployment.
octpopus_deploy_addon Report Summary: skipped: 0 success: 65 manual_check: 23 failure: 4 warning: 0 error: 0 not_applicable: 92 ------------------- Total: 184
So in this post, we set up a repository, established a build pipeline with some quality control, knocked out some basic documentation and broke the build.
Next time, we will start building out the Splunk Add On along with tagging in Git to produce a release, docs and a new version of the app.
- GitHub Repository - Octopus Deploy Add On
- GitHub Repository - Existing Dashboard app
- Read The Docs - Octopus Deploy Add On
- TravisCI - Octopus Deploy Add On