Extracting fields from a Splunk Search

Splunk has a nifty command, which can be used to extract fields from your splunk searches. In our case, we were logging an entire json request of a service call which did not go through due to some errors and we wanted to extract a specific field from the request for reporting purposes.

That’s where the rex command came into picture.

To use rex, you perform your regular splunk query operation, and then apply the rex command on your search.

Search Query

index="my_rest_services" status="failure" httpStatusCode="503" operation="myRestOperation"

Result

2017-06-17T 09:03:06,052+0000 LogLevel=ERROR [ServerName=Tatooine] [HostIP=127.0.0.1] [HostPort=7443] [ClientIP=127.0.0.1] [SessionID=8fpc5oolxzic6wjoe3uzrk7i] [RequestId=a1391174-098a-4108-814b-e4ea8e0ff1ff] [TransactionID=Tatooine-1497690184463] [ThreadId=719] [UserID=SOME_USER] [ModuleName=com.myapp.test.myservice.MyServiceManager] [EventCode=MY_API_EVENT] [Request={ "id": "0001", "type": "donut", "name": "Cake", "image": { "url": "images/0001.jpg", "width": 200, "height": 200 }, "thumbnail": { "url": "images/thumbnails/0001.jpg", "width": 32, "height": 32 }}]

Now, let us see how we can apply rex command the search result.

The syntax of the rex command is pretty simple.

... | rex [field=] ( [max_match=] [offset_field=] )

Here, if I want to extract the id from my request and render them in a table, the splunk query that I will use will be as follows.

... | rex field=_raw "{\s+\"id":\s+\"(?\d{4})\""

Lets try to break up how rex works into simple steps.

• It tries to identify a string which matches the regular expression  specified in the beginning. – {\s+\"id":\s+\"
• Then it extracts the 4 digits that comes after the above expression to the field called idfield. – (?\d{4})
• And it will stop when it encounters a double quote. – \"

And voila.. We have the required data, extracted. To render it in a table, just use the table command.

... | table idfield