Knex.js migration issue: Fails with ` relation "knex_migrations" does not exist` - javascript

I've been using knex.js successfully for a while now on this project, first time I've come accross this. I deleted all my migration files and dropped my db (locally) and instead used pg_dump to get the DDL and all data from my prod database (not including knex_migrations or knex_migrations_lock tables).
I created a new migration script and used knex.raw to paste all the SQL in. When I ran the migration script, I get the following:
error: insert into "knex_migrations" ("batch", "migration_time", "name") values ($1, $2, $3) - relation "knex_migrations" does not exist
This migration script is being run on a brand new database. When I change what's in the script to just be basic DDL, it works fine. So something about the result of pg_dump and trying to run it using knex is causing it to bomb out.
This isn't a ton of data either really, so I'm not too sure where I'm going wrong - knex is responsible for creating the migration tables and I've made sure there's no mention of knex or migration tables in the DDL.
Any suggestions would be greatly appreciated :)

The table knex_migrations and knex_migrations_lock will be created on running the command
knex migrate:make <name of migration>
Could you please confirm you have executed above knex command.
Once that is done running
knex migrate:latest
should create all schema you have mention. knex.raw

Using pd_dump to dump a database to an sql file, there occurs some additional lines in the beginning of the sql such as:
SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SELECT pg_catalog.set_config('search_path', '', false);
SET check_function_bodies = false;
SET xmloption = content;
SET client_min_messages = warning;
SET row_security = off;
comment those out and the migration should work. You can uncomment one statement at a time to see which one is causing the problem.

I just went down this hole. The following line is the culprit.
SELECT pg_catalog.set_config('search_path', '', false);
It's removing your search path so knex doesn't know where to find anything.

Related

Giveaway commands does not save (Heroku) discord.js

I am making a giveaway command, but whenever I restart all dynos in heroku it seems the giveaway just froze(Never ends the giveaway) and when I do !gdelete {messageid} It says there is no giveaway for {messageid} any idea why and how to fix it. I have tried using quick.db but still the same and I am quite new to heroku and coding discord bot. Im using node.js
const { GiveawaysManager } = require("discord-giveaways");
const manager = new GiveawaysManager(bot, {
storage: "./giveaways.json",
updateCountdownEvery: 10000,
default: {
botsCanWin: false,
embedColor: "#FF0000",
reaction: "🎉"
}
})
bot.giveawaysManager = manager;
Heres the code
And heres the gstart command: https://pastebin.com/9tBjpVEY
The issue is caused by Heroku, which doesn't store local files when you're not running the app. Every time you restart a dyno Heroku deletes everything and rebuilds it: that means that if you save your files locally when it restarts they'll get deleted.
To solve this issue you need either to switch to another service or to create some form of backup for your file.
You could also use a remote database, but I don't know how that could be implemented with the discord-giveaways package.
I had the same issue and I think that it can be solved by doing this:
Instead of using quick.db, you can use quickmongo which just the same as quick.db and discord-giveaways also has an example of it. Although there is one change that you need to make. The example of quickmongo also shows a local way to store the files but instead of typing the localhost string, replace it with the MongoDB Compass connection string of your MongoDB cluster and give the new collection the same name which is giveaways.
In order to get the connection string, log in to your MongoDB account and create a cluster. After creating the cluster, click the connect button on the cluster and then select Connect using MongoDB Compass. From there you will see a connection string. Copy and paste that string in the place where there was the localhost string. Then replace <password> with your account's password which is your password with your username. Also, replace the test at the end with giveaways and you are good to go. After running the code, you would also see a collection named giveaways in the Collections Tab inside your cluster.
Example:
const db = new Database('connectionLink/giveaways');
db.once('ready', async () => {
if ((await db.get('giveaways')) === null) await db.set('giveaways', []);
console.log('Giveaway Database Loaded');
});

Retrieving data from Firebase (web) not working

I just got into Firebase and I´m trying to retrieve data from the realtime database, but I can't get anything to show up on the webpage.
This is what I got, before the JS I got the initialize Firebase code. (IF I delete the val() I receive [object Object] so something is at least working).
JS:
var heading = document.getElementById("head");
var firebaseHeadingRef = firebase.database().ref().child("Heading");
firebaseHeadingRef.on('value', function(datasnapshot) {
heading.innerText = datasnapshot.val();
});
HTML:
<div id="table_body"> <h1 id="head">Some Text</h1>
This is a screenshot from the database:
I appreciate all help :)
The Realtime Database is case sensitive. In your DB screenshot you have Heading but your JS code has .child('heading'). Try making those the same case and it should work. Note that you must also have appropriate Security Rules to allow access for unauthenticated users.
I sort of faced a similar issue, everything was fine including the database rules but no data was being fetched.
My solution
My configuration containing the apiKey & projectId was missing the databaseURL portion, so I recopied the correct configuration & it worked.
P.S
Try checking your console for any warnings from Firebase

can not get data with MysqlSubscription of numtel:mysql for meteor

I am using meteor to do full stack javascript work.And I want to use mysql instead of mongodb.I find numtel:mysql on github, which is a Reactive MySQL for Meteor.
On the server side
Meteor.publish('test', function(){
let array = liveDb.select(
'select * from tasks',
[ { table: 'tasks' } ]
);
console.log('publish test mysql')
console.log(array);
return array;
});
on the client side
let mysqlData = new MysqlSubscription('test');
console.log('subscribe mysql data at client');
console.log(mysqlData);
console.log(mysqlData.length);
console.log(mysqlData.subscriptionId);
console.log(mysqlData[0]);
console.log(mysqlData[1]);
However, I can not get data on the client.And there is a strange phenomenon.From the logs, I find data of mysqlData.However, mysqlData.length is 0, mysqlData[0] and mysqlData[1] are undefined.
Who can help me?
You might want to call mysqlData.reactive() and check if it is ready by using mysqlData.ready() first.
In case the subscription does not work (it does not update the data automatically), you should make sure that you config your MySQL server correctly following the installation instruction.
For me, I could not make it work when I used capital letter on the database name, you should check it also.

Finding saved data from mongo shell (no output)

Here is the code for initialization
mongoose.connect('mongodb://localhost/gpsdb');
var db = mongoose.connection;
db.on('open', function () {
// now we can start talking
});
After successful opening, I am saving data like this, it's giving me no errors.
function saveGPSData(data){
var newData = new GPSData(data);
newData.save(function(err){
if(err)
return console.error(err);
});
}
Now in mongo shell, I am trying to retrieve that data but it's giving me empty output.
> use gpsdb
> db.GPSData.find();
>
It's giving me no output. Also can I found what models are there in gpsdb?
Here is the full source code http://pastebin.com/K7QPYAx8
JUST FOUND THAT in db folder there these files for my db created by mongodb
/data/db/gpsdb.0
/data/db/gpsdb.1
/data/db/gpsdb.n
A good place to start to get a quick answer is
https://groups.google.com/forum/#!forum/mongoose-orm
the community is very responsive :)
In the shell I did the following
>use gpsdb
switched to gpsdb
>db show collections
gpsdatas
From here I found that collection name is gpsdatas...... Not sure why its adding extra (s) to my modal, although you can see from the code that I am setting Modal to
var GPSData = mongoose.model('GPSData', GPSDataSchema);
Now using the shell its working like this
>db.gpsdatas.find()

How to delete a database in WebSQL programmatically?

I am new to Web SQL database and I use it to save data in a local database in a web page.
 I can create a database by
var db = openDatabase('database', '1.0', 'my database', 2 * 1024 * 1024);
 and I can create a table by doing this
db.transaction(function (tx) {
tx.executeSql('CREATE TABLE IF NOT EXISTS mytable (blah,blah)');
});
 I can delete the table by
db.transaction(function (tx) {
tx.executeSql('DROP TABLE mytable');
});
 but is there a way to delete the database programmatically?
Using PersistenceJS there is a persistence.reset API which will wipe the database clean.
PersistenceJS Site
For developing / testing purposes, you can view content and delete webSQL, IndexedDB, cookies, etc by searching for your domain name at this URL in Chrome:
chrome://settings/cookies
There, you can delete all the storage for a domain or just certain local storage entities. Yes, the URL implies just 'cookies', but the interface at this URL includes all types of offline storage.
It would be great I think if the Chrome developer tools interface had the ability to right-click and delete a data storage entity in the Resources tab along with inspecting the content. But for now, all I know of is the settings/cookies URL.
Spec says:
4.1 Databases
Each origin has an associated set of databases. Each database has a name and a current version. There is no way to enumerate or delete the databases available for an origin from this API.
I am developing a phonegap+jquery-mobile+KO app with offline storage using web sql via persistencejs, and jasmine js for BDD.
I'm working on some sort of "database cleaner" to be executed after each spec. When I was searching on how to drop a web sql database I read the reply https://stackoverflow.com/a/10929725/667598 (in this thread/question), and went to see what's in that directory (Mac OS X).
cd ~/Library/Application\ Support/Google/Chrome/Default/databases
Inside you will see a Databases.db SQLite3 database, and directories for each origin. These directories are named with the pattern protocol_host_somenumber (I don't know what that number is). So for example, in my case, since my apps are just files I open in Google Chrome with the file:/// … protocol, I can see a file__0 directory. And for twitter and I can also see a http_twitter.com_0 and a https_twitter.com_0.
Inside this directories all file names are just numbers. For example inside file__0 I found a file named 8 and another named 9. In my case, these files are websql database. I don't know if there also Indexed DB databases in chrome's Default/databases dir.
With this names it is a little hard to guess what database is what. You can open the database and you'll have to infer the app or site via its tables and data.
Luckily, the Databases.db I mentioned before is a mapping between those files named with numbers and the databases.
You can open the Databases.db and any other web sql file with the sqlite3 command
sqlite3 Databases.db
Obviously, once inside the sqlite3 shell, is handy to have some SQL knowledge. Anyway, it is also always handy some help, which is available via the command
.help
With the command .tables you can list tables in the database. Inside this Databases.db we can find the tables Databases and meta. The important one is Databases, so with a
select * from Databases;
we can see the mapping between the databases and their files. For example
7|http_jquerymobile.com_0|testdb|html5 test db|200000
8|file__0|elfaro_dev|Base de datos de ElFaro para desarrollo|734003200
The first column is the id of the table which is the number used for db file names, the second is the origin (the directory) the other columns are the db name, the db description and the estimated size used when creating the db from the Javascript API.
So to actually delete a database what I did was to delete it from this table, for example:
delete from Databases where id = 8
And then delete the actual file from the filesystem (outside sqlite3 shell)
rm file__0/8
And that's it.
PS: I know this is a too long answer for a simple subject but I just needed to flush this from my system and back it up somewhere like SO or a blog.
The developer options
There is no way to enumerate or delete the databases programmatically (yet).
Chrome developers can navigate to chrome://settings/cookies search and delete any database
Opera developers can navigate to opera://settings/cookies
The only way to truly delete a database (and everything else)
A new Spec says this might be possible in the feature with both response header and javascript.
The disadvantages is that you can't control what is being deleted, So you would need to create a backup first of everything else unless you want to clear everything
2.1.3. The storage parameter
The storage parameter indicates that the server wishes to remove locally stored data associated with the origin of a particular response’s url. This includes storage mechansims such as (localStorage, sessionStorage, [INDEXEDDB], [WEBDATABASE], etc), as well as tangentially related mechainsm such as service worker registrations.
Js:
navigator.storage.clear({
types: [ "storage" ],
includeSubdomains: true // false by default
});
Response header:
res.header("Clear-Site-Data", "storage; includeSubdomains");
But this is not avalible to any browser yet...
Best solution for clients (not the developers)
/* This will fetch all tables from sqlite_master
* except some few we can't delete.
* It will then drop (delete) all tables.
* as a final touch, it is going to change the database
* version to "", which is the same thing you would get if
* you would check if it the database were just created
*
* #param name [string] - the database to delete
* #param cb [function] - the callback when it's done
*/
function dropDatabase(name, cb){
// empty string means: I do not care what version, desc, size the db is
var db = openDatabase(name, "", "", "");
function error(tx, err){
console.log(err);
}
db.transaction(ts => {
// query all tabels from sqlite_master that we have created and can modify
var query = "SELECT * FROM sqlite_master WHERE name NOT LIKE 'sqlite\\_%' escape '\\' AND name NOT LIKE '\\_%' escape '\\'";
var args = [];
var success = (tx, result) => {
var rows, i, n, name;
rows = result.rows;
n = i = rows.length;
// invokes cb once it’s called n times
function after(){
if (--n < 0) {
// Change the database version back to empty string
// (same as when we compear new database creations)
db.changeVersion(db.version, "", function(){}, error, cb);
}
}
while(i--){
// drop all tabels and calls after() each time
name = JSON.stringify(rows.item(i).name);
tx.executeSql('DROP TABLE ' + name, [], after, error);
}
// call it just 1 more extra time incase we didn't get any tabels
after();
};
ts.executeSql(query, args, success, error);
});
}
Usage
dropDatabase("database", function(){
console.log("done")
});
The localdatabase files are stored in your Windows user settings under Application Data > Google > Chrome > User Data > Default > databases.
So manually deleting them is theoretically possible. This is only useful while testing / developing on your own computer, since when another user opens your app/site, it is unlikely to have file system access.
However, even though you can find the files and delete them, the data sticks around. I've tried it with Chrome both open and closed and all chrome processes ended, and yet the browser inspector keeps showing me my old database with all the unwanted fields and data in it.
This is answered in HTML5 database storage (SQL lite) - few questions.
To summarize:
Currently no way to drop a WebSQL database.
Probably use Indexed DB or localStorage instead.
In my library implementation, I just delete all tables. Which, indeed, delete the database. List of tables are select * from sqlite_master.
Please note that if you use multiple
tx.executeSql('DROP TABLE mytable');
statements in the same transaction callback then make sure that they all exist or consider using DROP TABLE IF EXISTS syntax instead. If even one table doesn't exist when you try to drop it will result in the entire transaction failing. This failure results in a rollback of the transaction and means that the data will stay in your database even when you thought that it should have been deleted. There is no error reported unless you're specifically listening for it in the executeSql's 4th argument which is an error callback. This is intended behavior but is, in my experience, confusing.
No method to delete the existing database in websql it will clear when the cache is cleared or
The browser is closed. If you want to create a database with the same name Just use openDatabase Method It will first check for the existence of the database with the same name. If not exists it will create one otherwise it will open the existing one
please follow this link http://html5doctor.com/introducing-web-sql-databases/

Categories