I im newbie in splunk.
I have this json:
"request": {
"headers": [
{
"name": "x-real-ip",
"value": "10.31.68.186"
},
{
"name": "x-forwarded-for",
"value": "10.31.68.186"
},
{
"name": "x-nginx-proxy",
"value": "true"
}
I need to pick a value when the property name has "x-real-ip" value.
There are a couple ways to do this - here's the one I use most often (presuming you also want the value along side the name):
index=ndx sourcetype=srctp request.headers{}.name="x-real-ip"
| eval combined=mvzip(request.headers{}.name,request.headers{}.value,"|")
| mvexpand combined
| search combined="x-real-ip*"
This skips all events that don't have "x-real-ip" somewhere in the request.headers{}.name multivalue field
Next, it combines the two multivalue fields (name & value) into a single mv field, separated by the | character
Then expand the resultset so you're looking at one line at a time
Finally, you look for only results that have the value "x-real-ip" in them
If you'd like to then extract the value from the combined field, add the following line:
| rex field-combined "\|(?<x_real_ip>.+)"
And, of course, you can do whatever other SPL operations on your data you wish
I tried #Warren's answer but I got the following error:
Error in 'eval' command: The expression is malformed. Expected ).
You need to add a rename because the {} charcters in mvzip causes problems.
This is the query that works:
index=ndx sourcetype=srctp request.headers{}.name="x-real-ip"
| rename request.headers{}.name AS headerName, request.headers{}.value AS headerValue
| eval reviewers=mvzip(headerName,headerValue ,"|")|
| mvexpand combined
| search combined="x-real-ip*"
your search
| rex max_match=0 "name\":\s\"(?<fieldname>[^\"]+)"
| rex max_match=0 "value\":\s\"(?<fieldvalue>[^\"]+)"
| eval tmp=mvzip(fieldname,fieldvalue,"=")
| rename tmp as _raw
| kv
| fields - _* field*
When you ask a question, please present the correct information.
You've run out of logs in the process.
Related
This question already has an answer here:
Having a POJO like feature in KarateAPI?
(1 answer)
Closed 1 year ago.
I am a newbie at both JavaScript and Karate. This may not be a Karate centric question per se, however, I am wondering if the solution can be done in Karate natively by any chance.
I have looked at existing questions on here but they don't seem to work likely due to my unique input. This answer looked promising, but it didn't work out for me: Adding new key-value pair into json using karate
I have a JAVA method that produces a payload consisting of a JSON object (which has a secondary json object in it) for a POST call. The payload looks something like this:
[
{
"keyId": "s123",
"clientId": "c0909",
"regionInfo": {
"geoTag": "3s98d238",
"locationId": 32
}
}
]
Now I am doing a test where I have to insert a bogus key/value pair into the payload and make sure it is ignored in the POST call itself and we return a 200. I have tried using karate.merge and karate.append, but they have not worked thus far.
The key value pair looks like this:
{'bogusfield': 'ABC', 'bogusfield': '123', 'bogusfield': 'abc123', 'bogusfield': 'abc123!$%'}
So in total, there will be four POST calls , each with a different value from above .Is ttherea way to get this done? I apologize if I missed on giving any crucial details/and/or if this too newb of a question. Thank you in advance for all the help!
Here's a simple example that makes 4 requests with the payload edits you want which should get you on your way:
Feature:
Scenario Outline:
* url 'https://httpbin.org'
* path 'anything'
* def body = { foo: 'bar' }
* body.bogusField = bogusValue
* request body
* method post
Examples:
| bogusValue |
| ABC |
| 123 |
| abc123 |
| abc123!$% |
I want to make interface/any other way for this so that can use that datatype instead of this long string or string pattern.i want to remove this incomplete or submitted pattern and put some small variable there
export interface IWithdraw{
applicationType: "incomplete" | "submitted" | "interview" | "inactive" | "offer" | "preonboard" | "hired",
jobApplicatonId:number,
startedOn: Date,
inProgress:boolean,
error?:string
}
Initially I was working with a CSV, where each row contained data e.g.
------------------------------------------------------
123 | cat | dog |
------------------------------------------------------
456 | cat | rabbit |
------------------------------------------------------
789 | snake | dog |
------------------------------------------------------
I am now getting more data with different structure, so I can no longer use a csv. Instead I am using JSON file. The JSON file looks something like this
[
[
{
"ID": 123,
"animal_one": "cat",
"animal_two": "dog"
},
{
"ID": 456,
"animal_one": "cat",
"animal_two": "rabbit"
},
{
"ID": 789,
"animal_one": "snake",
"animal_two": "dog"
}
],
[
2222
],
[
12345
],
[
"2012-01-02"
],
[
"2012-12-20"
]
]
So you can see the additional data. For this part of the application, I only want to work with the first part of the JSON, the part containing animals. The function I have basically works on values only, so I need to make this JSON like the original CSV file, whereby I only have the values, no keys.
So I am loading the JSON file, which is then contained within the variable data
I am then trying something like this
var key = "CUST_ID";
delete data[0][key];
console.log(JSON.stringify(data[0]))
Although this doesnt even work, I think it is the wrong approach anyway. I dont want to define the keys I want removed, I just want it to remove all keys, and keep the values.
How would I go about doing this? I am using data[0] to get the first section of the JSON, the part that contains animals. Not sure if this is correct either?
Thanks
You can simply do this, if you dont care what keys you are getting:
var collection = "";
data[0].forEach(function(row) {
var line = [];
Object.keys(row).forEach(function(key) {
line.push(row[key])
});
collection = collection + line.join(',') + '\n';
})
You will get csv string out of collection
I dont want to define the keys I want removed
You still will need to define which keys you want the values from. One could just take all the values from the object, in arbitrary order, but that's not robust against changes to the data format. Use something like
const table = data[0].map(value => [value.ID, value.animal_one, value.animal_two]);
data[0].forEach(function(row,index) {
data[0][index] = Object.values(data[0][index]);
});
I have a MySQL table with the structure below
+------------------+------------------+----------------+
| comp_id | name | parent |
|------------------|------------------|----------------+
| 1 | comp1 | NULL |
+------------------+------------------+----------------+
| 2 | comp2 | 1 |
+------------------+------------------+----------------+
| 3 | comp3 | 2 |
+------------------+------------------+----------------+
| 4 | comp4 | 2 |
+------------------+------------------+----------------+
Assuming that no data has been inserted into the table. In other words, assuming that the table is empty how should i go about the following:
traverse the JSON data below for entry into the table:
{
"org_name":"paradise island",
"daughters" : [
{
"org_name": "banana tree",
"daughters": [
{"org_name":"Yellow Banana"},
{"org_name":"Brown Banana"}
]
},
{
"org_name": "big banana tree",
"daughters": [
{"org_name":"green banana"},
{"org_name":"yellow banana"},
{
"org_name": "Black banana",
"daughters": [
{"org_name": "red spider"}
]
}
]
}
]
}
what effective SQL query can I write to insert the JSON above into MYSQL database at once.
I've researched a host of resources on adjacency list model and nested models but none has been exhaustive on how inserts should be done via JSON input
If using uuidv4's as id's would be ok for you, you could just normalize your object with a recursive function, add uuid's as id's and construct the parental relationships.
After That you just bulk insert your data with whatever database client you are using.
This is an ideal use case for UUIDV4's. The chance of a collision is very unlikely, so you can consider it production safe.
You just have to make sure, that your uuid generator is random enough. Bad implementations could lead to much higher probabillities of collisions.
I suggest using the uuid package. You can look up it's sources on github. It uses nodes crypto library as random data generator which is cryptographically strong according to the documentation
Nice to know:
MongoDB's ObjectId's are created in a similar way. They also don't provide 100 percent security against collision. But they are so unlikely that they consider this chance irrelevant.
EDIT: I assumed your code runs server side in node.js. Generating uuid's in the browser is not safe, because the crypto api is still experimental and users might try to intentionally cause collisions.
I am building a system where users associate tags with posts, not unlike SO. I am having a spot of bother implementing tag synonyms.
Here I have a table called Tags:
| TagName |
|------------|
| Python |
| JavaScript |
| Node |
And I have another called TagSynonyms:
| SynonymId | SourceTagName | TargetTagName |
|-----------|---------------|---------------|
| 1 | Py | Python |
| 2 | Python2 | Python |
The server is implemented using Node and the user enters some tags as a comma-delimited string:
var input = 'Py,Flask'
var tags = request.tags.split(',');
In this case, the user has entered the tag Py which, according to the TagSynonyms table, should be mapped to the tag Python. The second tag, Flask has no synonym and should remain the same.
I managed to implement this functionality using imperative code:
tags.forEach(function (tag) {
connection.query('SELECT TargetTagName FROM TagSynonyms WHERE SourceTagName = ?', tag, function(err, rows) {
if (rows.length !== 0) {
console.log(rows[0].TargetTagName);
} else {
console.log(tag);
}
});
});
Here, the user input
['Py','Flask']
results in the following output
Python
Flask
What I want to do is, defer this logic to the database engine, as I think using a loop is a code smell. I also think the database engine will be more performant. What is an appropriate query to do this?
You need a UNION and a join:
select TagName
from Tags
where TagName in (?,?,?,...)
union
select TagName
from Tags
join TagSynonyms
on Tags.TagName = TagSynonyms.TargetTagName
where TagSynonyms.SourceTagName in (?,?,?,...)
Note that union can be slow since it will try to remove duplicates. If that's the case for you, use union all and remove duplicates in the application code.
(?,?,?,...) stands for a list of input values; check your DB driver documentation for the exact syntax that you need to use to avoid SQL injection.
Update: Here is what the implementation will look like in Node:
var query =
'SELECT TagName \
FROM Tags \
WHERE TagName IN (?) \
UNION \
SELECT TagName \
FROM Tags \
JOIN TagSynonyms \
ON Tags.TagName = TagSynonyms.TargetTagName \
WHERE TagSynonyms.SourceTagName IN (?)'
connection.query(query, [tags, tags], function(err, rows) {
tags = rows.map(function(row) {
return row.TagName
});
});