Trying to build a query for a postgreSQl DB based on a keyword. LIKE doesn't work as it matches any row that contains any of the letters. For Example:
SELECT * FROM table WHERE column ilike '%jeep%';
This returns any row that a j,e or p in the column (and the same row multiple times for some reason). Not the word 'jeep'.
Below is my query structure. Using Knex and queuing multiple tables:
searchAllBoardPosts(db, term) {
return db
.select('*')
.from({
a: 'messageboard_posts',
b: 'rentals',
c: 'market_place',
d: 'jobs'
})
.where('a.title', 'ilike', `%${term}%`)
.orWhere('b.title', 'ilike', `%${term}%`)
.orWhere('c.title', 'ilike', `%${term}%`)
.orWhere('d.title', 'ilike', `%${term}%`);
},
Thanks in advance!
UPDATE:
Here is the SQL output:
select *
from "messageboard_posts" as "a",
"rentals" as "b",
"market_place" as "c",
"jobs" as "d"
where "a"."title" ilike '%jeep%'
or "b"."title" ilike '%jeep%'
or "c"."title" ilike '%jeep%'
or "d"."title" ilike '%jeep%'
This query is a cross join
(But the Knex syntax masks that, a little).
This returns any row that a j,e or p in the column (and the same row multiple times for some reason).
It's not returning the same row multiple times. It's returning everything from each table named in a CROSS JOIN. This is the behaviour of Postgres when more than one table is named in the FROM clause (see: docs). This:
db
.select('*')
.from({
a: 'table_one',
b: 'table_two'
})
will return the entire row from each of the named tables every time you get an ILIKE match. So at minimum you'll always get an object consisting of two rows joined (or however many you name in the FROM clause).
The tricky part is, Knex column names have to map to JavaScript objects. This means that if there are two column results named, say, id or title, the last one will overwrite the first one in the resulting object.
Let's illustrate (with wombats)
Here's a migration and seed, just to make it clearer:
table_one
exports.up = knex =>
knex.schema.createTable("table_one", t => {
t.increments("id");
t.string("title");
});
exports.down = knex => knex.schema.dropTable("table_one");
table_two
exports.up = knex =>
knex.schema.createTable("table_two", t => {
t.increments("id");
t.string("title");
});
exports.down = knex => knex.schema.dropTable("table_two");
Seed
exports.seed = knex =>
knex("table_one")
.del()
.then(() => knex("table_two").del())
.then(() =>
knex("table_one").insert([
{ title: "WILLMATCHwombatblahblahblah" },
{ title: "WILLMATCHWOMBAT" }
])
)
.then(() =>
knex("table_two").insert([
{ title: "NEVERMATCHwwwwwww" },
{ title: "wombatWILLMATCH" }
])
)
);
Query
This allows us to play around a bit with ILIKE matching. Now we need to make the column names really explicit:
return db
.select([
"a.id as a.id",
"a.title as a.title",
"b.id as b.id",
"b.title as b.title"
])
.from({
a: "table_one",
b: "table_two"
})
.where("a.title", "ilike", `%${term}%`)
.orWhere("b.title", "ilike", `%${term}%`);
This produces:
[
{
'a.id': 1,
'a.title': 'WILLMATCHwombatblahblahblah',
'b.id': 1,
'b.title': 'NEVERMATCHwwwwwww'
},
{
'a.id': 1,
'a.title': 'WILLMATCHwombatblahblahblah',
'b.id': 2,
'b.title': 'wombatWILLMATCH'
},
{
'a.id': 2,
'a.title': 'WILLMATCHWOMBAT',
'b.id': 1,
'b.title': 'NEVERMATCHwwwwwww'
},
{
'a.id': 2,
'a.title': 'WILLMATCHWOMBAT',
'b.id': 2,
'b.title': 'wombatWILLMATCH'
}
]
As you can see, it's cross-joining both tables, but I suspect you were only seeing results that appeared not to match (because the match was in the other table, and the title column name was a duplicate).
So, what should the query be?
I think your (or Ry's) plan to use UNION was correct, but it's probably worth using UNION ALL to avoid unnecessary removal of duplicates. Something like this:
return db
.unionAll([
db("market_place")
.select(db.raw("*, 'marketplace' as type"))
.where("title", "ilike", `%${term}%`),
db("messageboard_posts")
.select(db.raw("*, 'post' as type"))
.where("title", "ilike", `%${term}%`),
db("rentals")
.select(db.raw("*, 'rental' as type"))
.where("title", "ilike", `%${term}%`),
db("jobs")
.select(db.raw("*, 'job' as type"))
.where("title", "ilike", `%${term}%`)
]);
A similar query against our test data produces the result set:
[
{ id: 1, title: 'WILLMATCHwombatblahblahblah', type: 'table_one' },
{ id: 2, title: 'WILLMATCHWOMBAT', type: 'table_one' },
{ id: 2, title: 'wombatWILLMATCH', type: 'table_two' }
]
Using .union works and returns the correct values, however using the key tag from the first table in the query. Have ended up just making four separate queries in the end but hope this can help some else!
searchAllBoardPosts(db, term) {
return db
.union([db
.select('id', 'market_place_cat')
.from('market_place')
.where('title', 'ilike', `%${term}%`)
])
.union([db
.select('id', 'board_id')
.from('messageboard_posts')
.where('title', 'ilike', `%${term}%`)
])
.union([db
.select('id', 'rental_cat')
.from('rentals')
.where('title', 'ilike', `%${term}%`)
])
.union([db
.select('id', 'job_cat')
.from('jobs')
.where('title', 'ilike', `%${term}%`)
]);
},
This expression:
WHERE column ilike 'jeep'
Only matches rows where the value is lower(column) = 'jeep', such as:
JEEP
jeep
JeeP
It does not match any other expression.
If you use wildcards:
WHERE column ilike '%jeep%'
then is looks for 'jeep' anywhere in lower(column). It is not searching character by character. For that, you would use regular expressions and character classes:
WHERE column ~* '[jep]'
If you want to find a word in the field, you would normally use regular expressions, not like/ilike.
Related
how are you? I'm trying to move a filter inside the fetch I'm doing to bring my data from Builder.io and I'm struggling with one of them here. The title search works fine, but the second one don't. My objective is to filter between the entries to catch only the ones that match at least one of the annotationArray items.
The annotationArray can be, for example:
const annotationArray = ['Video', 'Image', 'GPS']
or just
const annotationArray = ['Video']
or whatever.
And the entries have an annotation field that consists in a string where I pass the annotations, like this:
const entries = [{title: 'First', annotation: 'Video, GPS'}, {title: 'Second', annotation: 'GPS'}, {title: 'Third', annotation: 'Video, Image'}]
So, for example, if the annotationArray is ['Video', 'GPS'], I want to fetch all of them. But if it's ['Video'], only the third and first, and so.
Currently I have this code
const sets = await builder.getAll('open-dataset', {
options: { noTargeting: true },
omit: 'data.blocks',
limit: 100,
query: {
data: {
title: { $regex: search, $options: 'i' },
annotation: { $regex: annotationArray && annotationArray.join(' '), $options: 'i' },
}
}
});
The result of annotationArray.join(' ') can be, for example, Video Image GPS or just Image. And annotation Video Image or whatever.
So I need to filter between the entries and fetch only the ones that contain at least one of the annotationArray strings.
My code is failing because currently it only fetches the ones that have all the annotationArray items, and not the ones that have at least one. I don't know how to do it with MondoDB query operators... previously, I had this code with javascript and it worked fine.
const filtered = entries.filter(item => annotationArray.some(data => item.annotation.includes(data)));
can somebody help me? thanks
I've a collection of countries with country calling code in the country object. How can I find a country using calling code with a mobile number?
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
const number = '971524500000'; // Input
How can I find country for the given mobile using regex in mongoose javascript;
[https://en.wikipedia.org/wiki/List_of_country_calling_codes][1]
Take a look at the link above on country calling codes, and specifically see the section "Tree List".
One solution would be to implement a schema in Mongo based on this tree in order to decode the country codes.
So, a table could be created to store Mongo documents containing a field "1x" "2x" "21x" etc (the Y axis in the Tree List table).
Each of these documents could contain an array of sub-documents from x=0 to x=9 (the x axis in the Tree List table). The sub-document can contain the country name/code you are looking for. You can use a direct index into the array in the Mongo document for an efficient lookup.
I think you'll find this to be a pretty efficient implementation and should cover all the bases.
If you can restructure your array to an object this would be the fastest
const countries =
{
971: 'UAE',
1: 'USA',
44: 'UK',
}
;
var code = 44;
console.log(countries[code]);
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
var myFound =countries.filter(myFunc.bind(this,44));
function myFunc(code,element) {
if(element.callingCode == code){
return element;
}
}
console.log(myFound);
On MongoDB v 4.2 - you can use $let & $regexFind to do this :
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$let: {
vars: {
value: {
$regexFind: {
input: "971524500000", // pass in as string
regex: { $toString: "$callingCode" }
}
}
},
in: "$$value.idx",
}
},
0
]
}
}
}
]);
Test : MongoDB-Playground
Explanation :
General Use-case :
In general regex searches - Will have an input which will be sub-string of actual string, Ex.:-
Actual string in DB : 'We can do it in mongo'
Input : mongo (/mongo/ - will work)
Your Use-case :
From above case as mongo exists in actual string/value of db field then you can get that document, But your case is different :
Actual string in DB : mongo
Input : 'We can do it in mongo'
This doesn't work that way, So using normal /We can do it in mongo/ isn't going help you here (Also doing few tricks with regex). So we need to make a trick to $regexFind operator. Unlike mongo documentation we need take 971524500000 into input field & regex as string value of callingCode field which is vice-versa to what's given in documentation.
So once we do that, We would get something like below :
{
"match" : "971", /** value of callingCode field */
"idx" : 0, /** Index of `971` in '971524500000' */
"captures" : []
},{
"match" : "1",
"idx" : 2,
"captures" : []
},
null /** 3rd doc no match */
As country code has to be at first of given number we need docs where "idx" : 0 - So we're using $let to get index of returned object & checking against 0 & eventually getting respective docs using $match.
Note :
There is something you need to look into, Just in case if you've docs like below :
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade1"),
"name": "USA",
"callingCode": 1.0
},
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade3"),
"name": "somecountry",
"callingCode": 197.0
}
& input is 1971524500000, then this query will bring both docs in result. This will be the case you need to check on. Anyhow I would suggest to try this query, rather than getting all documents for collection to the code & extract required o/p this might be better to do.
I have a JSONB column in DB.
I'd like to have request to DB where I can check if some value in this JSON it true or false:
SELECT *
FROM table
WHERE ("json_column"->'data'->>'data2')::boolean = true AND id = '00000000-1111-2222-3333-456789abcdef'
LIMIT 1
So, my sequelize request:
const someVariableWithColumnName = 'data2';
Model.findOne({
where: {
[`$("json_column"->'data'->>'${someVariableWithColumnName}')::boolean$`]: true,
id: someIdVariable,
},
order: [/* some order, doesn't matter */],
})
And sequelize generate bad result like:
SELECT *
FROM table
WHERE "(json_column"."->'data'->>'data2')::boolean" = true AND id = '00000000-1111-2222-3333-456789abcdef'
LIMIT 1
Split my column by . and add " to every element.
Any idea how to get rid of adding " to the column in where condition?
Edit:
Here is my query with sequelize.literal():
const someVariableWithColumnName = 'data2';
Model.findOne({
where: {
[sequelize.literal(`$("json_column"->'data'->>'${someVariableWithColumnName}')::boolean$`)]: true,
id: someIdVariable,
},
order: [/* some order, doesn't matter */],
})
You can use Sequelize.literal() to avoid spurious quotes. IMHO, wrapping the json handling in a db function might also be helpful.
I just came across a similar use case.
I believe you can use the static sequelize.where method in combination with sequelize.literal.
Here is the corresponding documentation in sequelize API reference: https://sequelize.org/master/class/lib/sequelize.js~Sequelize.html#static-method-where
And here is an example (although I will admit hard to find) in the regular documentation:
https://sequelize.org/master/manual/model-querying-basics.html#advanced-queries-with-functions--not-just-columns-
In the end for your specific sit try something like this:
const someVariableWithColumnName = 'data2';
Model.findOne({
where: {
[Op.and]: [
// We provide the virtual column sql as the first argument of sequelize.where with sequelize.literal.
// We provide the matching condition as the second argument of sequelize.where, with the usual sequelize syntax.
sequelize.where(sequelize.literal(`$("json_column"->'data'->>'${someVariableWithColumnName}')::boolean$`), { [Op.eq]: true }),
{ id: someIdVariable }
]
})
I am using a ColumnSet and the helper.insert function for a multi row insert.
I have a table column where I want to use the Postgres Date/Time now() function.
const cs = new helpers.ColumnSet([
'lastname',
{
name: 'rental_date',
def: 'now()'
}
], { table: { table: 'book_rental', schema: 'public' } })
let rentals = [
{
lastname: 'Mueller'
},
{
lastname: 'Johnson'
}
]
let insert = helpers.insert(rentals, cs)
db.result(insert)
.then(data => res.json({ message: 'Ok!' }))
.catch(err => res.json({ message: 'Not ok!' }))
It seems to be working by using def: 'now()', but I want to make sure that I am using it the right way.
Edit:
Regarding the answer in the comment. I tried to do the insert manually and it looks like Postgres is converting the 'now()' string into the now() function.
INSERT INTO book_rental (lastname, rental_date) VALUES ('Mueller', 'now()');
To involve your answer, am I right that this should be the correct code then?
const cs = new helpers.ColumnSet([
'lastname',
{
name: 'rental_date',
mod: ':raw',
def: 'now()'
}
], { table: { table: 'book_rental', schema: 'public' } })
Your code doesn't look right, for the following reasons:
You want to use now() without any condition, but the def value is only used when the property doesn't exist in the source object (see Column). The init callback is what should be used instead to guarantee the right value override.
You return now() as an escaped string, while the query needs it as a raw-text string.
First, let's declare a reusable Raw Text string, as per Custom Type Formatting:
const rawText = text => ({toPostgres: () => text, rawType: true});
Then you can define the column like this:
{
name: 'rental_date',
init: () => rawText('now()')
}
And make sure you are using the latest version of pg-promise (v7.2.1 as of this writing).
Or alternatively, you can declare it like this:
{
name: 'rental_date',
mod: ':raw', // same as mode: '^'
init: () => 'now()'
}
This syntax however will work in all versions of the library, and perhaps is even simpler to use ;)
I'm writing a node.js app that uses the pg package for accessing a PostgreSQL database. The issue I'm running into is that if I do a query like this:
select * from posts p inner join blogs b on b.id = p.blog_id
When I get the results, they're all in the same namespace, so any field repeated in the blogs table will overwrite those in the posts table.
My question is, what's the best way of binding these results to objects?
Ideally, I'd like a result like:
{
id: 1,
name: 'A post name',
published_at: (some date object),
blog_id: 1,
b: {
id: 1,
name: 'A blog name'
}
}
But I'm open to any convenient solution short of adding an alias for every column manually.
http://www.postgresql.org/docs/9.3/static/functions-json.html
http://www.postgresql.org/docs/9.4/static/functions-aggregate.html
You may want to look at the json features of Postgres. If I'm understanding you right, and without a test database something like this may be close to what you're looking for:
SELECT
p.*, /* Select all the post fields */
row_to_json(blogs.*) as b /* Use the row_to_json function on the blogs results */
FROM
posts p
INNER JOIN
blogs ON (blogs.id=p.blog_id); /* Join blogs on the proper fields */
Returns:
{
id: 3,
name: 'test',
published_at: 2015-10-08,
blog_id: 2,
b: {
id:2,
name:"test 2"
}
}
Here's a great tutorial on them:
http://bender.io/2013/09/22/returning-hierarchical-data-in-a-single-sql-query/
If you change your query to
'SELECT * FROM posts, blogs WHERE posts.id = blogs.id;'
you should have your column names prefixed with either 'posts' or 'blogs'
If you want a nested result like above, you'll have to run some manual processing.
res.map(d => {
return {
id: d.posts_id
b : {
id: d.blogs_id
}
};
});