I'm writing some tests agains MongoDb GridFs and want to make sure that I start each test with a clean slate. So I want to remove everything, both from fs_files and fs_chunks.
What's the easiest way to do that?
If GridFs has its own database then I would just drop the database via mongo shell with db.dropDatabase(). Alternately if there are collections you would like to keep apart from fs_files and fs_chunks in the database you could drop the collections explicitly with db.collection.drop(). For both you can run via command from a driver rather than through the shell if desired.
If you want to remove GridFS, try it like (pyMongo Case)
for i in fs.find(): # or fs.list()
print fs.delete(i._id)
The simple way to remove all files and chunks is
let bucket = new GridFSBucket(db, { bucketName: 'gridfsdownload' });
bucket.drop(function(error) { /* do something */ });
This will remove the collections as well. If you want to keep the collections, you need to
to do something like this:
let allFiles = db.collection('uploaded.file').find({}).toArray();
for (let file of allFiles) {
await bucket.delete(file._id);
}
Related
I am making a giveaway command, but whenever I restart all dynos in heroku it seems the giveaway just froze(Never ends the giveaway) and when I do !gdelete {messageid} It says there is no giveaway for {messageid} any idea why and how to fix it. I have tried using quick.db but still the same and I am quite new to heroku and coding discord bot. Im using node.js
const { GiveawaysManager } = require("discord-giveaways");
const manager = new GiveawaysManager(bot, {
storage: "./giveaways.json",
updateCountdownEvery: 10000,
default: {
botsCanWin: false,
embedColor: "#FF0000",
reaction: "🎉"
}
})
bot.giveawaysManager = manager;
Heres the code
And heres the gstart command: https://pastebin.com/9tBjpVEY
The issue is caused by Heroku, which doesn't store local files when you're not running the app. Every time you restart a dyno Heroku deletes everything and rebuilds it: that means that if you save your files locally when it restarts they'll get deleted.
To solve this issue you need either to switch to another service or to create some form of backup for your file.
You could also use a remote database, but I don't know how that could be implemented with the discord-giveaways package.
I had the same issue and I think that it can be solved by doing this:
Instead of using quick.db, you can use quickmongo which just the same as quick.db and discord-giveaways also has an example of it. Although there is one change that you need to make. The example of quickmongo also shows a local way to store the files but instead of typing the localhost string, replace it with the MongoDB Compass connection string of your MongoDB cluster and give the new collection the same name which is giveaways.
In order to get the connection string, log in to your MongoDB account and create a cluster. After creating the cluster, click the connect button on the cluster and then select Connect using MongoDB Compass. From there you will see a connection string. Copy and paste that string in the place where there was the localhost string. Then replace <password> with your account's password which is your password with your username. Also, replace the test at the end with giveaways and you are good to go. After running the code, you would also see a collection named giveaways in the Collections Tab inside your cluster.
Example:
const db = new Database('connectionLink/giveaways');
db.once('ready', async () => {
if ((await db.get('giveaways')) === null) await db.set('giveaways', []);
console.log('Giveaway Database Loaded');
});
i'm trying to create collection from a script in mongodb, i found on this link a useful script to do that Make a script to create MongoDB collections from Mongo shell? (thanks for that script).
I'm using now this kind of script and it work fine:
db = db.getSiblingDB('emanueledb');
var collectionList = ["collection1", "collection2", "collection3"]
collectionList.forEach(function(collName) {
db.createCollection(collName, {autoIndexId: true});
});
I'm trying to cycle also some db but with this kind of code:
var dblist = ["emanueledb", "db_ema_dv"];
var collectionList = ["ab111", "bc111"];
dblist.forEach(function(dbname) { db.getSiblingDB(dbname)
collectionList.forEach(function(collName) {
db.createCollection(collName, {autoIndexId: true});
})
});
it works but create only the two collection on db_ema_dv. What is wrong?
Is also possible to get collection and db from an external csv to automate everything??
CSV example:
dbname,collectionname
emanueledb,collection1
emanueledb,collection2
testdb,testcollection
Thanks
In the code you posted this expression does nothing (it selects the database but you don't do anything with the database):
db.getSiblingDB(dbname)
Try:
db.getSiblingDB(dbname).createCollection(collName, {autoIndexId: true});
Perfect, thank you! Now works!
var dblist = ["emanueledb", "db_ema_dv"];
var collectionList = ["ab111111", "bc111111"];
dblist.forEach(function(dbname) {
collectionList.forEach(function(collName) {
db.getSiblingDB(dbname).createCollection(collName, {autoIndexId: true});
})
});
as requested before, is possibile to get db and collection from a csv?
CSV Example:
dblist,collectionList
emanueledb,collection1
emanueledb,collection2
testdb,testcollection
To get a database from a .csv you can use the mongoimport command. This will read a CSV file into a collection.
I am making a discord bot in Node.js mostly for fun to get better at coding and i want the bot to push a string into an array and update the array file permanently.
I have been using separate .js files for my arrays such as this;
module.exports = [
"Map: Battlefield",
"Map: Final Destination",
"Map: Pokemon Stadium II",
];
and then calling them in my main file. Now i tried using .push() and it will add the desired string but only that one time.
What is the best solution to have an array i can update & save the inputs? apparently JSON files are good for this.
Thanks, Carl
congratulations on the idea of writing a bot in order to get some coding practice. I bet you will succeed with it!
I suggest you try to split your problem into small chunks, so it is going to be easier to reason about it.
Step1 - storing
I agree with you in using JSON files as data storage. For an app that is intended to be a "training gym" is more than enough and you have all the time in the world to start looking into databases like Postgres, MySQL or Mongo later on.
A JSON file to store a list of values may look like that:
{
"values": [
"Map: Battlefield",
"Map: Final Destination",
"Map: Pokemon Stadium II"
]
}
when you save this piece of code into list1.json you have your first data file.
Step2 - reading
Reading a JSON file in NodeJS is easy:
const list1 = require('./path-to/list1.json');
console.log(list.values);
This will load the entire content of the file in memory when your app starts. You can also look into more sophisticated ways to read files using the file system API.
Step3 - writing
Looks like you know your ways around in-memory array modifications using APIs like push() or maybe splice().
Once you have fixed the memory representation you need to persist the change into your file. You basically have to write it down in JSON format.
Option n.1: you can use the Node's file system API:
// https://stackoverflow.com/questions/2496710/writing-files-in-node-js
const fs = require('fs');
const filePath = './path-to/list1.json';
const fileContent = JSON.stringify(list1);
fs.writeFile(filePath, fileContent, function(err) {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
Option n.2: you can use fs-extra which is an extension over the basic API:
const fs = require('fs-extra');
const filePath = './path-to/list1.json';
fs.writeJson(filePath, list1, function(err) {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
In both cases list1 comes from the previous steps, and it is where you did modify the array in memory.
Be careful of asynchronous code:
Both the writing examples use non-blocking asynchronous API calls - the link points to a decent article.
For simplicity sake, you can first start by using the synchronous APIs which is basically:
fs.writeFileSync
fs.writeJsonSync
You can find all the details into the links above.
Have fun with bot coding!
I'm trying to set up a history command for when a player gets warned with a moderation bot. I have it where it saves it in a JSON file however I don't know how to save multiple cases for one player. Right now it just replaces the last warn.
//--Adds To Logs
let warnhist = JSON.parse(fs.readFileSync("./historywarn.json", "utf8"));
warnhist[wUser.id] = {
Case: `${wUser} was warned by ${message.author} for ${reason}`
};
fs.writeFile("./historywarn.json", JSON.stringify(warnhist), (err) => {
if (err) console.log(err)
});
It saves like this without adding onto it every time:
{"407104392647409664":{"Case":"<#407104392647409664> was warned by <#212770377833775104> for 2nd Warn"}}
I need it to save like this:
{
"407104392647409664":{"Case":"<#407104392647409664> was warned by <#212770377833775104> for 1st Warn", "Case 2":"<#407104392647409664> was warned by <#212770377833775104> for 2nd Warn" }
}
You want an array instead of an object. Try structuring your data a bit more too, so your JSON looks like this:
{ "<user id>": { "warnings": [...] }
Then instead of overriding the history every time with an assignment warnhist[wUser.id] = ... you can use Array.prototype.push to add a new one.
Taking a step back though, I don't think JSON is a good strategy to store this data, especially if it contains all warnings for all users. You need to load the entire JSON and read the file and write the file every single time you add a warning, and there could be conflicts if you have multiple instances of your service doing the same for a large number of users.
For a super similar API and storage format but better performance and consistency, try MongoDB instead of a plain JSON file.
I just start learning Meteor. I put all my images in the root of "/public" folder. The images are named as "1.jpg","2.jpg","3.jpg".... I want to insert all the images to a collection "Images" by using for loop without putting a certain number as the for loop limit. So how can I make it detect how many images are in the public folder automatically?
You would need a mechanism to get the list of images in the public folder, using the fs package will do this for you. Consider the following example which uses the package to read the public directory which is /web.browser/app/ from the server (total unix path /home/user/your_app_name/.meteor/local/build/programs/web.browser/app/).
After getting the folder list, filter the list to only get image files with the extension .jpg, then insert the images to your collection. For illustrative purposes the image document saved to the collection has the following simple schema example
{ _id: "v2PrCTPea6tM6JNHn", name: "1.jpg" }
To aid you with your final goal of inserting the images to the mongo collection, you can use the batch-insert Meteor package which enables the underlying mongo driver to insert of multiple documents. It works just like insert() but takes an array of objects to be inserted rather than a single object.
#Installation
In your meteor app directory run:
meteor add mikowals:batch-insert
On server
var fs = Npm.require('fs'),
public_path = path.join(__meteor_bootstrap__.serverDir, "../web.browser/app"),
files = fs.readdirSync(public_path),
images = _(files).reject( function(fileName) {
return fileName.indexOf('.jpg') < 0;
}),
imagesList = images.map(function (image){
return { name: image };
});
Images = new Meteor.Collection('images');
Images.allow({
insert: function(){ return true };
});
var newIds = Images.batchInsert(imagesList); // returns array of created _id values