If I want to allow a user to upload a video in a post, then what's the best way to store it so that I can access it later? I am currently just using a INSERT query to upload it to a PostGres SQL database field, but it's extremely slow.
Is there another method or best practice to store large files like this and access it quickly and efficiently?
var video = videoFile ? "'\\x" + videoFile.buffer.toString('hex') + "'" : "NULL";
let insertQuery = "INSERT INTO posts (video) VALUES(" + video + ") RETURNING *";
Above is the code I'm using right now to insert the file into the database.
Do not store the whole video into the database. These 2 steps below might be a better solution:
1) Upload the video file to the cloud storage. See Google Cloud Strage and Amazon S3. There are many other options as well. Go do some comparison yourself.
2) You will have a link or ID (or key in S3) to access a file. Store all those links or ID into your database. Please take a look at this GCS doc and this S3 question.
Also, make use of the non-blocking I/O which is a good thing of node.js. In your case, you may try something like this:
let insertQuery;
let promisesList=[];
for(let vid of videoIds){
insertQuery = "INSERT INTO posts (videoId) VALUES($1) RETURNING *";
promisesList.push(client.query(query,[vid]));
}
await Promise.all();
So I've been using Firebase as a database for my website (this is a web based project, using HTML, CSS and JS) and I'm running into a problem retrieving data from it.
Basically this site allows users to create a profile for a character (they can fill in the name, the characters stats etc...) and when they click submit, it'll save the values they filled out to the database.
The values are saved perfectly fine, but when I go to retrieve the data the command doesn't seem to do anything.
So in order to get the profiles, I've been trying to use this bit of code to get whatever is stored at the specified .ref(path):
var uid = firebase.auth().currentUser.uid;
var getChar = firebase.database().ref('/users/' + uid + '/chars/').orderByKey();
Which according to the Firebase docs should return a list of keys at the path that I specified in .ref(). However whenever I try to access whatever is in the var, it just gives me the string that contains a link to the database that looks like this:
https://#mydatabaseurlhere.firebaseio.com/users/uid/chars
Where #mydatabaseurlhere is the url I created on the Firebase app, and the uid is the authenticated user's ID.
I've been reading the docs, and its telling me that the above code should return a list of whatever is at the path that I specified, but so far it just gives me a link. Is there something I've been missing from the Docs that'll allow me to access whatever data is currently in the database? Because I've tried to take a snapshot using .once() to no avail either. I've also set the rules on /users/ to allow anyone to read/write to the database but I'm still not able to access the data (or maybe I am accessing, I'm just missing how to retrieve it).
Either way, I'm wondering how one can go about accessing this data, as I'm extremely confused as to why I can't seem to retrieve the data that has been successfully written to the database.
You're defining a query. But that doesn't yet retrieve the data.
To retrieve the data, you need to attach a listener. For example:
var uid = firebase.auth().currentUser.uid;
var getChar = firebase.database().ref('/users/' + uid + '/chars/').orderByKey();
getChar.on('value', function(snapshot) {
snapshot.forEach(function(child) {
console.log(child.key, child.val());
});
});
I created the Database in firebase and used for my hybrid application, Let you explain the application scenarios below.
I have a hybrid application in this app, We need to insert, update, delete, kind of operations using firebase.
As of now, I did the insert method kind of queries alone using code. So the thing is I want to create multiple tables in firebase is it possible ?
Because For example, I've userTable, adminTable and guestTable, So If I need to insert one userData before that I need to check already the user found or not in Admin table this kind of scenarios How can I do in firebase ?
How can I maintain multiple tables in the firebase Database dashboard?
I have the table name called "Inmate_Data" now I want to add one more Table called "Admin_Data" and How can I do a joint operation like connecting two table using Firebase.
And, I tried to add one more table using Import JSON option But while I inserting new JSON the old Table got deleted like the "Inmate_Data" deleted and new JSON got added.
Please guide me in this.
#FrankvanPuffelen - Please find the below coding part.
Actually, I created the form and saving the data into firebase using below code.
$scope.addInmateDataToFirebase = function() {
alert('Firebase')
var newPostKey = null;
// Get a key for a new Post.
newPostKey = firebase.database().ref().child('Tables/Inmate_Data').push().key;
alert(newPostKey);
var dateofjoin= $filter('date')($scope.IMDOJ, 'yyyy-MM-dd');
var dateofbirth= $filter('date')($scope.IMDOB, 'yyyy-MM-dd');
console.log("Result"+ newPostKey);
var admissionId="KAS"+pad(newPostKey,5);
console.log("Padded number"+ pad(newPostKey,5));
alert(admissionId);
// A post entry.
var postInmateData = {
Inmate_ID: admissionId,
Inmate_Name: $scope.IMfirstnameText,
Inmate_Perm_Address1: $scope.IMAddress1 ,
Inmate_Perm_Address2: $scope.IMAddress2 ,
Inmate_Perm_City: $scope.IMCity,
Inmate_Perm_State: $scope.IMState,
Inmate_Perm_Pincode: $scope.IMPincode,
Inmate_ProfilePhotoPath: $scope.IMProfilePhotoPath,
Inmate_Temp_Address1: $scope.IMLocAddress1,
Inmate_Temp_Address2: $scope.IMLocalAddress2,
Inmate_Temp_City:$scope.IMGcity,
Inmate_Temp_State: $scope.IMGstate,
Inmate_Temp_Pincode: $scope.IMGpincode,
Inmate_Mobile: $scope.IMMobile,
Inmate_DOB: dateofbirth,
Inmate_EmpOrStudent: $scope.IMEmpStudent,
Inmate_DOJ: dateofjoin,
Inmate_ID_Type: $scope.IMIdcardType,
Inmate_ID_No: $scope.IMIdcardno,
Inmate_ProffPhotoPath: $scope.IMProofPhotoPath,
Inmate_Status:$scope.IMStatus
};
// Write the new post's data simultaneously in the list.
var updates = {};
updates['/Tables/Inmate_Data/' + newPostKey] = postInmateData;
return firebase.database().ref().update(updates).then(function() {
//$scope.ClearInMateDetails();
alert(admissionId);
$scope.statusMessage = "Welcome to Kasthuri Hostel. Your Admission ID is" + admissionId +" . Enjoy your Stay.";
//sendSMS('9488627377',$scope.statusMessage);
alert('Inmate data stored in cloud!');
$scope.ClearInMateDetails();
}, function(error) {
alert (error);
});
}
Now the Inmate_data got stored. and I need to save the Inmate_alloc_data into firebase but before that, I need to check whether the Inmate_Id available in the Inmate_Data table in firebase.
Please find the below snap :
My suggestions :
For example like above screenshot now I've multiple tables like "Inmate_Data", "Inmate_Alloc", and more Shall I get all the data and store it in a local SQLite Db like same as "Inmate_Alloc_tbl" and "Inmate_Data_tbl" and finally If I update any values finally I want to get all values in the local database and store it in the firebase. Is it possible to handle ?
If so I can manage lots of SQL queries, right ? Because via firebase we can't able to manage all queries right ?
Please suggest your ideas.
I am new to Web SQL database and I use it to save data in a local database in a web page.
I can create a database by
var db = openDatabase('database', '1.0', 'my database', 2 * 1024 * 1024);
and I can create a table by doing this
db.transaction(function (tx) {
tx.executeSql('CREATE TABLE IF NOT EXISTS mytable (blah,blah)');
});
I can delete the table by
db.transaction(function (tx) {
tx.executeSql('DROP TABLE mytable');
});
but is there a way to delete the database programmatically?
Using PersistenceJS there is a persistence.reset API which will wipe the database clean.
PersistenceJS Site
For developing / testing purposes, you can view content and delete webSQL, IndexedDB, cookies, etc by searching for your domain name at this URL in Chrome:
chrome://settings/cookies
There, you can delete all the storage for a domain or just certain local storage entities. Yes, the URL implies just 'cookies', but the interface at this URL includes all types of offline storage.
It would be great I think if the Chrome developer tools interface had the ability to right-click and delete a data storage entity in the Resources tab along with inspecting the content. But for now, all I know of is the settings/cookies URL.
Spec says:
4.1 Databases
Each origin has an associated set of databases. Each database has a name and a current version. There is no way to enumerate or delete the databases available for an origin from this API.
I am developing a phonegap+jquery-mobile+KO app with offline storage using web sql via persistencejs, and jasmine js for BDD.
I'm working on some sort of "database cleaner" to be executed after each spec. When I was searching on how to drop a web sql database I read the reply https://stackoverflow.com/a/10929725/667598 (in this thread/question), and went to see what's in that directory (Mac OS X).
cd ~/Library/Application\ Support/Google/Chrome/Default/databases
Inside you will see a Databases.db SQLite3 database, and directories for each origin. These directories are named with the pattern protocol_host_somenumber (I don't know what that number is). So for example, in my case, since my apps are just files I open in Google Chrome with the file:/// … protocol, I can see a file__0 directory. And for twitter and I can also see a http_twitter.com_0 and a https_twitter.com_0.
Inside this directories all file names are just numbers. For example inside file__0 I found a file named 8 and another named 9. In my case, these files are websql database. I don't know if there also Indexed DB databases in chrome's Default/databases dir.
With this names it is a little hard to guess what database is what. You can open the database and you'll have to infer the app or site via its tables and data.
Luckily, the Databases.db I mentioned before is a mapping between those files named with numbers and the databases.
You can open the Databases.db and any other web sql file with the sqlite3 command
sqlite3 Databases.db
Obviously, once inside the sqlite3 shell, is handy to have some SQL knowledge. Anyway, it is also always handy some help, which is available via the command
.help
With the command .tables you can list tables in the database. Inside this Databases.db we can find the tables Databases and meta. The important one is Databases, so with a
select * from Databases;
we can see the mapping between the databases and their files. For example
7|http_jquerymobile.com_0|testdb|html5 test db|200000
8|file__0|elfaro_dev|Base de datos de ElFaro para desarrollo|734003200
The first column is the id of the table which is the number used for db file names, the second is the origin (the directory) the other columns are the db name, the db description and the estimated size used when creating the db from the Javascript API.
So to actually delete a database what I did was to delete it from this table, for example:
delete from Databases where id = 8
And then delete the actual file from the filesystem (outside sqlite3 shell)
rm file__0/8
And that's it.
PS: I know this is a too long answer for a simple subject but I just needed to flush this from my system and back it up somewhere like SO or a blog.
The developer options
There is no way to enumerate or delete the databases programmatically (yet).
Chrome developers can navigate to chrome://settings/cookies search and delete any database
Opera developers can navigate to opera://settings/cookies
The only way to truly delete a database (and everything else)
A new Spec says this might be possible in the feature with both response header and javascript.
The disadvantages is that you can't control what is being deleted, So you would need to create a backup first of everything else unless you want to clear everything
2.1.3. The storage parameter
The storage parameter indicates that the server wishes to remove locally stored data associated with the origin of a particular response’s url. This includes storage mechansims such as (localStorage, sessionStorage, [INDEXEDDB], [WEBDATABASE], etc), as well as tangentially related mechainsm such as service worker registrations.
Js:
navigator.storage.clear({
types: [ "storage" ],
includeSubdomains: true // false by default
});
Response header:
res.header("Clear-Site-Data", "storage; includeSubdomains");
But this is not avalible to any browser yet...
Best solution for clients (not the developers)
/* This will fetch all tables from sqlite_master
* except some few we can't delete.
* It will then drop (delete) all tables.
* as a final touch, it is going to change the database
* version to "", which is the same thing you would get if
* you would check if it the database were just created
*
* #param name [string] - the database to delete
* #param cb [function] - the callback when it's done
*/
function dropDatabase(name, cb){
// empty string means: I do not care what version, desc, size the db is
var db = openDatabase(name, "", "", "");
function error(tx, err){
console.log(err);
}
db.transaction(ts => {
// query all tabels from sqlite_master that we have created and can modify
var query = "SELECT * FROM sqlite_master WHERE name NOT LIKE 'sqlite\\_%' escape '\\' AND name NOT LIKE '\\_%' escape '\\'";
var args = [];
var success = (tx, result) => {
var rows, i, n, name;
rows = result.rows;
n = i = rows.length;
// invokes cb once it’s called n times
function after(){
if (--n < 0) {
// Change the database version back to empty string
// (same as when we compear new database creations)
db.changeVersion(db.version, "", function(){}, error, cb);
}
}
while(i--){
// drop all tabels and calls after() each time
name = JSON.stringify(rows.item(i).name);
tx.executeSql('DROP TABLE ' + name, [], after, error);
}
// call it just 1 more extra time incase we didn't get any tabels
after();
};
ts.executeSql(query, args, success, error);
});
}
Usage
dropDatabase("database", function(){
console.log("done")
});
The localdatabase files are stored in your Windows user settings under Application Data > Google > Chrome > User Data > Default > databases.
So manually deleting them is theoretically possible. This is only useful while testing / developing on your own computer, since when another user opens your app/site, it is unlikely to have file system access.
However, even though you can find the files and delete them, the data sticks around. I've tried it with Chrome both open and closed and all chrome processes ended, and yet the browser inspector keeps showing me my old database with all the unwanted fields and data in it.
This is answered in HTML5 database storage (SQL lite) - few questions.
To summarize:
Currently no way to drop a WebSQL database.
Probably use Indexed DB or localStorage instead.
In my library implementation, I just delete all tables. Which, indeed, delete the database. List of tables are select * from sqlite_master.
Please note that if you use multiple
tx.executeSql('DROP TABLE mytable');
statements in the same transaction callback then make sure that they all exist or consider using DROP TABLE IF EXISTS syntax instead. If even one table doesn't exist when you try to drop it will result in the entire transaction failing. This failure results in a rollback of the transaction and means that the data will stay in your database even when you thought that it should have been deleted. There is no error reported unless you're specifically listening for it in the executeSql's 4th argument which is an error callback. This is intended behavior but is, in my experience, confusing.
No method to delete the existing database in websql it will clear when the cache is cleared or
The browser is closed. If you want to create a database with the same name Just use openDatabase Method It will first check for the existence of the database with the same name. If not exists it will create one otherwise it will open the existing one
please follow this link http://html5doctor.com/introducing-web-sql-databases/
I want to make a blog post in my web application. Initially I was using mysql as DB. In which i will get the post entered in the text area of blog as an object in JS and send that object to java server side. There I will write mysql query and get the object in resultset and save in database. But now i want to use mongoDB for the same. I am able to understand the basic stuff through many tutorials which I learnt. But I am unable to implement that in my application. I want to know how the object that comes from the JS will be sent inside the loop and how I should query to save the object also similarly if I need to send a object from server side to JS. How should i do.?
My Server side code:
public DB MongoConnection(Blog blog) throws UnknownHostException, MongoException{
Mongo m = new Mongo("localhost" , 27017); //mongo object
DB db = m.getDB("myblog");
System.out.println("Connected");
//making a collection object which is table when compared to sql
DBCollection items = db.getCollection("items");
System.out.println("items got");
//to work with document we need basicDbObject
BasicDBObject query = new BasicDBObject();
System.out.println("Created mongoObject");
//Cursor, which is like rs in sql
DBCursor cursor = items.find();
System.out.println("items got");
while(cursor.hasNext()){
System.out.println(cursor.next());
}
In the above code i understood everything such as how a mongo connection, documents, collections and cursor's work. Now how should i save the value coming as an object from JS and save in mongoDB. Any Suggestions Please?
Use method save of DBCollection class something like that:
while(cursor.hasNext()){
DBObject doc = cursor.next();
doc.put("name", "Leo-vin");
items.save(doc);
}
method cursor.next() returns object of type DBObject. It is your BSONObject.
Update:
to modify document (BSON) use method put of class BSONObject