This is something that has intrigued me a lot recently. It's a general SQL/Relational Database problem coming from a guy who prefers Mongo.
What I want is, to, as such, associate data from different tables in the most efficient, easiest way, without using associations and assuming I can't restructure or re-model the db.
So, for example, with FQL (which doesn't have associations), if I asked for the name and eid of all the events my current user has been invited to, I'd also like to know whether my current user is going, but that info is in the 'event_member' table.
In this instance I've an interest in another column (rsvp_status) in event_member, one that I'd like to be associated with the columns from event, i.e eid and name.
In this case the instinct may be to say that since every event has a name, an eid and a rsvp_status then we could say sort by eid and then match each nth item (for n=1 to whatever), because there's guaranteed to be the same number, but there are many cases when we can't do that.
And I know I could do separate queries and then iterate through and match them by eid, but basically I'm looking for a generic, simple,efficient solution for the associations idea if one exists. Preferably in javascript.
What you are looking for here is a simple JOIN of two or more tables. http://www.w3schools.com/sql/sql_join.asp
You do not have to have any relations between tables in order to perform JOINS. The relations are just a constraint to ensure that bad/invalid data can't propagete to the tables. For example an event_member with eid of unexisting user. Anyway you are free to JOIN tables as you like :)
Here is a way to connect to Sql Server using javascript How to connect to SQL Server database from JavaScript in the browser?
Related
I need to find all documents in a MongoDB database that have a property containing a string that is similar to the search term but allows for a certain % in divergence.
In plain javascript I could for example use https://www.npmjs.com/package/string-similarity and then basically match all documents that have > 90% similarity score.
I'd like do to this as MongoDB query and be as performant as possible as the database contains millions of documents.
What possible options do I have in this situation?
I found something about $text search, but it doesn't seem it helps a lot
I was thinking about creating some sort of signature for each document, like some sort of hash that allows for some sort of divergence.
I am really happy for every idea to get this solved in the best possible way.
The common solution to this problem is to use a search engine database, like Elasticsearch or Atlas search (by Mongodb team). I will not go into too much detail on how these databases work but generally speaking they are an inverse index database, this means you tokenize your data on insert and then your queries run on the tokenized data and not on the raw data set.
This approach is very powerful and can help with many "search engine" problems like autocomplete or in your case what is called a "fuzzy" search.
Let's see how elasticsearch deals with this by reading about their fuzzy feature:
To find similar terms, the fuzzy query creates a set of all possible variations, or expansions, of the search term within a specified edit distance. The query then returns exact matches for each expansion.
Basically what they do is create all "possible" permutations of the query within the given parameters. I would personally recommend you just use one of these databases that give this ability OOTB, however if you want to do a "pseudo" search engine in Mongo you can just use this approach ( with the downside of Mongo's indexes being a tree so you force a tree scan for these queries instead of a db designed for this )
I want to write data into a specific location in the database. Let's say, I have a couple of users in the database. Each of them has their own personal information, including their e-mails. I want to find the user based on the e-mail, that's to say by using his e-mail (but I don't know exactly whose e-mail it is, but whoever it is do something with that user's information). To be more visible, here is my database sample.
Now, while working on one of my javascript files, when the user let's say name1 changes his name, I update my object in javascript and want to replace the whole object under ID "-LEp2F2fSDUt94SRU0cx". To cut short, I want to write this updated object in the path ("Users/-LEp2F2fSDUt94SRU0cx") without doing it by hand and just "knowing" the e-mail. So the logic is "Go find the user with the e-mail "name1#yahoo.com" and replace the whole object with his new updated object". I tried to use orderByChild("Email").equalTo("name1#yahoo.com").set(updated_object), but this syntax does not work I guess. Hopefully I could explain myself.
The first part is the query, that is separate from the post to update. This part is the query to get the value:
ref.child('users').orderByChild("Email").equalTo("name1#yahoo.com")
To update, you need to do something like this once you have the user id from the query result:
ref.child('users').child(userId).child("Email").update(newValue);
firebase.database.Query
A Query sorts and filters the data at a Database location so only a
subset of the child data is included. This can be used to order a
collection of data by some attribute (for example, height of
dinosaurs) as well as to restrict a large list of items (for example,
chat messages) down to a number suitable for synchronizing to the
client. Queries are created by chaining together one or more of the
filter methods defined here.
// Find all dinosaurs whose height is exactly 25 meters.
var ref = firebase.database().ref("dinosaurs");
ref.orderByChild("height").equalTo(25).on("child_added", function(snapshot) {
console.log(snapshot.key);
});
In 10 Common Misconceptions about CouchDB, Joan Touzet is asked (30:16) if CouchDB will have a way to secure/validate reads on specific documents and/or specific fields of a document.
Joan says that if someone has access to the database, he/she can access all documents in that database.
So she says that there are a few ways to accomplish that:
(30:55) Cloudant was working on field level security access. Have they implemented it yet? Is it open-sourced?
(32:10) You should create separate document in a separate database.
(32:20) Filtered replications. She mentions that it slows 'things' down. She means that the filter slows the replication, correct?
Also, according to rcouch wiki (https://github.com/rcouch/rcouch/wiki/Validate-documents-on-read), it implements a validate_doc_read function (I haven't tested it, though). Does CouchDB has anything like it?
As far as I can see, the best approach is to model the database according to my problem (one database for this, another for that, one for this person, another for that person) and do filtered replications when necessary. Any suggestions?
I am working with a database that was handed down to me. It has approximately 25 tables, and a very buggy query system that hasn't worked correctly for a while. I figured, instead of trying to bug test the existing code, I'd just start over from scratch. I want to say before I get into it, "I'm not asking anyone to build the code for me". I'm not that lazy, all I want to know is, what would be the best way to lay out the code? The existing query uses "JOIN" to combine the results of all the tables in one variable, and spits it into the query. I have been told in other questions displaying this code, that it's just too much, and far too many bugs to try to single out what is causing the break.
What would be the most efficient way to query these tables that reference each other?
Example: Person chooses car year, make, model. PHP then gathers that information, and queries the SQL database to find what parts have matching year, vehicle id's, and parts compatible. It then uses those results to pull parts that have matching car model id's, OR vehicle id's(because the database was built very sloppily, and compares all the different tables to produce: Parts, descriptions, prices, part number, sku number, any retailer notes, wheelbase, drive-train compatibility, etc.
I've been working on this for two weeks, and I'm approaching my deadline with little to no progress. I'm about to scrap their database, and just do data entry for a week, and rebuild their mess if it would be easier, but if I can use the existing pile of crap they've given me, and save some time, I would prefer it.
Would it be easier to do a couple queries and compare the results, then use those results to query for more results, and do it step by step like that, or is one huge query comparing everything at once more efficient?
Should I use JOIN and pull all the tables at once and compare, or pass the input into individual variables, and pass the PHP into javascript on the client side to save server load? Would it be simpler to break the code up so I can identify the breaking points, or would using one long string decrease query time, and server loads? This is a very complex question, but I just want to make sure there aren't too many responses asking for clarification on trivial areas. I'm mainly seeking the best advice possible on how to handle this complicated situation.
Rebuild the database then make a php import to bring over the data.
well, i am creating a network that allows users creating posts and like them.
Asking on stackoverflow i've understood how to structure my database:
A collection which includes a document for each post.
A collection which includes a document for each like, in each of these documents there is a reference to post is referenced to.
When i want to get ALL likes about a post i can query the like collection looking for the reference to that post.
And till here i am ok. But assuming i'll have millions documents in like collection, i wondered how could i query and search among them in not too long time.
And i was advised of ensureIndex, in this case, i have to ensureindex of the field which contains reference to a post.
But when do i have to create this index? is enough to create it once (for example when i set up my database) and it will be as default in mongodb or do i have to do it during application life-time? thank you
But assuming i'll have millions documents in like collection, i wondered how could i query and search among them in not too long time.
I assume you would most likely want to do a count on the likes as an example?
You can't, instead you use optimizations to combat this. A count on millions of rows might get a bit slow.
A typical scenario are counters in SQL techs that you use to amend the parent row with a sum figure of its children.
Same applies to MongoDB.
You would aggregate important data to the top.
If you require to actually query the likes to show some who have liked it then you limit those likes. Google+ and other networks tend to limit the amount of likes they show to about 1,000.
And i was advised of ensureIndex,
Adding indexes to a database does help with actually searching for documents.
But when do i have to create this index? is enough to create it once
Yes, MongoDB will manage the index itself. You only need to ensure it once.