The first question one may ask is why the heck would a post like this appear? Well, for my sins, I spent considerable time learning the Python 3 stack, the Splunk SDK and how it connects to Splunk itself and authenticates etc. To be fair - the documentation at Splunk HQ is poor, it's probably ok for Python ninjas but to me it didn't work out well and made zero sense. Besides, the SOAR solution I was working on, it was a bash solution that used search.py from the examples, so what need is there to know/learn python? ;)
After a hard battle getting a SOAR box migrated over to a fresh and upgraded platform - we now have this image.
I've seen various people struggle and I've struggled myself getting this sorted, the backstory is a 32-bit debian box was running this on-site and in prod against a Splunk 6 server, and it all needed migrating to a new Debian box with no internet access or any packages in the loca repo - I won't even go in to how I got that bit working... Then zoning in to an 8.2.7 indexer cluster.
I had to build this (out of hours too!) in order to prove to my boss I can get this working, 3 attempts in total!
It turned out that pip install wasn't working well, and then I had to grab pip for python and invoke that within python too (see my build below), contrary to the official documents. As well as this "utils.py" needs to be in the same path as "search.py" and the error about "parse" being unavailable is nothing to do with the parse utils/libs you can download for Python. Absolutely SH*T, but eventually I figure it all out. Not to forget the .env file location, numerous FML moments, but I carried on, as we do!
Download here: https://download.sqeye.net/Debian11-SplunkPythonSDK.zip Then Unzip & import, fire her up!
OS: Debian 11.
Splunk version: Choice between V8.2.9 or V9.0.3.
SDK Version: 1.7.3
VM Username: devuser (or root)
VM Password: Python123 (same for both users)
*You set up the Splunk admin account on install with Python123 as password.
Getting Started
Open terminal in the desktop environment:-
> sudo su -
> cd /opt
# For Splunk V9:
> dpkg -i splunk-9.0.3-dd0128b1f8cd-linux-2.6-amd64.deb
# For splunk V8:
> dpkg -i splunk-8.2.9-4a20fb65aa78-linux-2.6-amd64.deb
# Note that when running for first time, we are assuming "admin" and "Python123" for the Splunk admin account:
> /opt/splunk/bin/splunk start --accept-license --answer-yes
> export PYTHONPATH=~/splunk-sdk-python
# Testing:
> python3 /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/kvstore.py
> python3 /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/search.py "search index=_internal" --username="admin" --password="Python123"
> python3 /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/search.py "search *" --username="admin" --password="Python123"
N.B. always sudo when starting Splunk! You know this well lol.
Extra Info
- Want to change my supplied credentials for SDK/API login? Go in to Splunk (http://127.0.0.1:8000) and then update the account details, then edit /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/.env. Alternately just change them on the .py command line if supplied there.
- Want to plug this in to a live system? Add this node as a search head (independent) and it will start working straight away, don't forget to change usernames/passwords FFS...
- How to do poor-mans SOAR? Maybe I'll follow up in a separate post, I'm not fully understanding the bash side of things myself but I'll be sure to post a simple example here that uses search.py from the examples, that's how the customer is doing it and it works well with cron.
My standalone build script (needs to have internet connection)
#Usernames: devuser, root
#Password: Python123
sudo apt-get update
sudo apt-get install python3
sudo apt-get install curl
sudo apt-get install pip
sudo su -
cd /opt
#Choose Splunk 8 or 9 below (wget & dpkg):
#Splunk 9:
wget -O splunk-9.0.3-dd0128b1f8cd-linux-2.6-amd64.deb "https://download.splunk.com/products/splunk/releases/9.0.3/linux/splunk-9.0.3-dd0128b1f8cd-linux-2.6-amd64.deb"
dpkg -i splunk-9.0.3-dd0128b1f8cd-linux-2.6-amd64.deb
#Splunk 8:
wget -O splunk-8.2.9-4a20fb65aa78-linux-2.6-amd64.deb "https://download.splunk.com/products/splunk/releases/8.2.9/linux/splunk-8.2.9-4a20fb65aa78-linux-2.6-amd64.deb"
dpkg -i splunk-8.2.9-4a20fb65aa78-linux-2.6-amd64.deb
/opt/splunk/bin/splunk start --accept-license --answer-yes
#Splunk Username: admin
#Splunk Password: Python123
python3 -m pip install splunk-sdk
python3 -m pip install python-dotenv
cd ~
wget https://files.pythonhosted.org/packages/99/ac/c7287f153e65c28ef4f04fdc2483d5dee7e0c80b9f4b46d0398ab2512446/splunk-sdk-1.7.3.tar.gz
tar -zxvf splunk-sdk-1.7.3.tar.gz
mv splunk-sdk-1.7.3 splunk-sdk-python
cd splunk-sdk-python
export PYTHONPATH=~/splunk-sdk-python
python3 setup.py install
mkdir examples
cd examples/
wget https://github.com/splunk/splunk-app-examples/archive/refs/heads/master.zip
unzip master.zip
cd /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/
nano .env
#Change password to Python123
python3 /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/kvstore.py --username="admin" --password="Python123"
python3 /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/search.py "search index=_internal" --username="admin" --password="Python123"
python3 /opt/splunk-sdk-python/examples/splunk-app-examples-master/python/search.py "search *" --username="admin" --password="Python123"