Halley's Blog

Extracting fields from a Splunk Search

Posted in Technical by Halley on June 17, 2017

Splunk has a nifty command, which can be used to extract fields from your splunk searches. In our case, we were logging an entire json request of a service call which did not go through due to some errors and we wanted to extract a specific field from the request for reporting purposes.

That’s where the rex command came into picture.

To use rex, you perform your regular splunk query operation, and then apply the rex command on your search.

Search Query

index="my_rest_services" status="failure" httpStatusCode="503" operation="myRestOperation"

Result

2017-06-17T 09:03:06,052+0000 LogLevel=ERROR [ServerName=Tatooine] [HostIP=127.0.0.1] [HostPort=7443] [ClientIP=127.0.0.1] [SessionID=8fpc5oolxzic6wjoe3uzrk7i] [RequestId=a1391174-098a-4108-814b-e4ea8e0ff1ff] [TransactionID=Tatooine-1497690184463] [ThreadId=719] [UserID=SOME_USER] [ModuleName=com.myapp.test.myservice.MyServiceManager] [EventCode=MY_API_EVENT] [Request={ "id": "0001", "type": "donut", "name": "Cake", "image": { "url": "images/0001.jpg", "width": 200, "height": 200 }, "thumbnail": { "url": "images/thumbnails/0001.jpg", "width": 32, "height": 32 }}]

Now, let us see how we can apply rex command the search result.

The syntax of the rex command is pretty simple.

... | rex [field=] ( [max_match=] [offset_field=] )

Here, if I want to extract the id from my request and render them in a table, the splunk query that I will use will be as follows.

... | rex field=_raw "{\s+\"id":\s+\"(?\d{4})\""

Lets try to break up how rex works into simple steps.

  • It tries to identify a string which matches the regular expression  specified in the beginning. – {\s+\"id":\s+\"
  • Then it extracts the 4 digits that comes after the above expression to the field called idfield. – (?\d{4})
  • And it will stop when it encounters a double quote. – \"

And voila.. We have the required data, extracted. To render it in a table, just use the table command.

... | table idfield

Advertisements
Tagged with: ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: