I have schema that take two fields - name and file.
file is Binary type and name is a String
I use this schema to save images as documents in my db
While the image is getting saved without issues - I am not sure how can I read it later. The main propose is to be able to search a file by its name using findOne and then to stream it to the response using pipe or any other solution that might work - to show the image to the client inside <img/> tag. I tried to convert the file to base64 and then to send something like res.send('data:image/png;base64,${file.file}') without much luck .
* Note that because I have inside the document two fields (name and file) I need to stream only the file for obvious reasons
This is my GET request to fetch the file :
router.get('/:url',(req, res) => {
const url = req.params.url;
console.log(url)
console.log('streaming file')
File.
findOne({ name:url}, 'file').
cursor().
on('data', (doc) => {
console.log('doc', doc.file);
res.send(doc.file)
}).
on('end', () => { console.log('Done!'); });
})
this not helps because its for streaming file with path which I dont have. My file stored in a db
Image of the saved file inside the db:
Related
I was trying to create a new website using nodejs and expressjs. Basically I have a page (/ home) where there is a form with an input file. I have already created the code to show the preview of the image once loaded ... what I would like to do is essentially transfer the "link" of the image to nodejs, where I will then take this "link" and insert it in the src of the img tag in the EJS end. can someone help me? thanks
if you are using express you can add this attribute to form tag enctype="multipart/form-data". Then your image will be sent from frontend . To receive the image in the server you need to configure a middleware called express-fileupload
var fileupload = require("express-fileupload");
then you can register the middleware by app.use(fileupload()); in your app.js file
in your route file
router.post('/something', async (req, res) => {
let image=req.files.fieldName
}))
To save the image here i am storing the image in server itself but this is not recommended .
You want to rename the image with a connection . for eg the mongo document inserted id
image.mv('./public/images/foldername/' + id + '.jpg', (err) => {
if (err) {
console.log(err)
} else {
res.redirect('/)
}
})
once it is done when you submit your form you can see the uploaded image in your server /public/images/foldername .
To view in ejs you can basically loop through the mongo id or some connection that i have mentioned earlier
I have a bulk amount of data in CSV format. I am able to upload that data with python by converting them to the dictionary (dict) with loop. the whole data is getting uploaded.
but now I want to upload bulk data to firebase and images to storage and I want to link between each document and image because i am working on e-commerce react app. so that I can retrieve documents along with images.
which is a good way to do this? should I do this with javascript or python?
I uploaded data manually to firebase by importing from there but again I am unable to upload bulk images to storage and also unable to create references between them. please give me a source where I can find this solution
This is tough, because it's hard to fully understand how exactly your images and CSV's are linked, but generally if you need to link something to items stored in Firebase, you can get a link either manually (go into storage, click and item, and the 'Name' Field on the right hand side is a link), or you can get it when you upload it. So for example, I have my images stored in firebase, and a postgres database with a table storing the locations. In my API (Express), when I post the image to blob storage, I create the URL of the item, and post that as an entry in my table, as well as setting it to be the blobs name. I'll put the code here, but obviously it's a completely different architecture to your problem, so i'll try and highlight the important bits (it's also JS, not Python, sorry!):
const uploadFile = async () => {
var filename = "" + v4.v4() + ".png"; //uses the uuid library to create a unique value
const options = {
destination: filename,
resumable: true,
validation: "crc32c",
metadata: {
metadata: {
firebaseStorageDownloadTokens: v4.v4(),
},
},
};
storage
.bucket(bucketName)
.upload(localFilename, options, function (err, file) {});
pg.connect(connectionString, function (err, client, done) {
client.query(
`INSERT INTO table (image_location) VALUES ('${filename}')`, //inserts the filename we set earlier into the postgres table
function (err, result) {
done();
if (err) return console.error(err);
console.log(result.rows.length);
}
);
});
console.log(`${filename} uploaded to ${bucketName}.`);
};
Once you have a reference between the two like this, you can just get the one in the table first, then use that to pull in the other one using the location you have stored.
Got stuck while trying to download an attachment of a work item (Azure DevOps).
I'm Using node.js 'azure-devops-node-api' client (https://www.npmjs.com/package/azure-devops-node-api) to interact with ADO API. I get a certain workItem using WorkItemTracking client (wit):
let workItem = await witApi.getWorkItem(1234, undefined, undefined, WorkItemExpand.All);
let attachment = await witApi.getAttachmentContent(attachmentId, fileName, projectId, true);
Documentation states that getAttachmentContent method downloads an attachment (https://github.com/microsoft/azure-devops-node-api/blob/ff820b2dd0c9a09cf09e64e94d3f95818a77249d/api/WorkItemTrackingApi.ts#L392), as a return value I'm getting a ReadableStream which i tried to write to a file using standard fs module:
fs.writeFile('WordDoc.docx', attachment, function (err) {if (err) return console.log(err);});
File is created but is empty. While debugging i also see that attachment variable has type ReadableStream but inside has lot's of properties and values, among them there is a buffer which i actually would like to extract and pass to fs.writeFile but can't reach them
What am i doing wrong ?
I believe you should write using WritableStream. As the getAttachmentContent is returning a ReadableStream. Below is the pseudo code. It might work
let readableStream = await witApi.getAttachmentContent(attachmentId, fileName, projectId, true);
let writableStream = fs.createWriteStream('./WordDoc.docx');
readableStream.pipe(writableStream);
I am using MySQL to get urls I have for a web scraper. Currently I get the data using csv-stringify as shown below:
conn.query('SELECT id, URL FROM company_table')
.stream()
.pipe(stringifier).pipe(process.stdout);
I see the data in the process.stdout but how can I use the data itself as I need to pass the url and id to another function for processing. I am unsure what I need to put in place of process.stdout for my needs.
The data basically looks like 38391,ysu.edu
I have another scraper where I stream in a csv file like
let fname = '/Users/tom/Scrappers/companies1k.csv';
fs.createReadStream(fname).pipe(csv()).on('data',
(row) => cluster.queue({url: `http://www.${row.domain}`}));
and that works fine but I just can't see how I pipe the db stream the same way to puppeteer-cluster.
Any ideas?
I've a use case, in which I need to save registration form field values to the data base and also upload the user image too.
According to my knowledge, I need to use 2 different apis for storing form field values and one for uploading file. As images uploaded like, request.files and raw data be like, request.body.
Is it possible to upload both images and form field values in a single api? I need to get the id of user so that I can map the userId to the image.
Yes, it is possible to do that.
For image you can use it multer to store image(file), and form value can be stored as well.
For multer reference you can visit link
You have to use multer as a middleware as I have used in my router
router.post('/addtype', upload.single("image"), (req, res) => {
const newExample = new Example ({
examplename: req.body.name,
imagePath:'/images/' + req.file.filename
})
newExample .save()
.then(result => {
res.json(result)
})
.catch(error => {
console.log(error)
})
})
I'm using mongoose in this example you can do same with mongodbClient