I am trying to unzip a gzipped file in Node but I am running into the following error.
Error: incorrect header check
at Zlib._handle.onerror (zlib.js:370:17)
Here is the code the causes the issue.
'use strict'
const fs = require('fs');
const request = require('request');
const zlib = require('zlib');
const path = require('path');
var req = request('https://wiki.mozilla.org/images/f/ff/Example.json.gz').pipe(fs.createWriteStream('example.json.gz'));
req.on('finish', function() {
var readstream = fs.createReadStream(path.join(__dirname, 'example.json.gz'));
var writestream = fs.createWriteStream('example.json');
var inflate = zlib.createInflate();
readstream.pipe(inflate).pipe(writestream);
});
//Note using file system because files will eventually be much larger
Am I missing something obvious? If not, how can I determine what is throwing the error?
The file is gzipped, so you need to use zlib.Gunzip instead of zlib.Inflate.
Also, streams are very efficient in terms of memory usage, so if you want to perform the retrieval without storing the .gz file locally first, you can use something like this:
request('https://wiki.mozilla.org/images/f/ff/Example.json.gz')
.pipe(zlib.createGunzip())
.pipe(fs.createWriteStream('example.json'));
Otherwise, you can modify your existing code:
var gunzip = zlib.createGunzip();
readstream.pipe(gunzip).pipe(writestream);
Related
const jsonServer = require('json-server')
const cors = require('cors')
const path = require('path')
const server = jsonServer.create()
const router = jsonServer.router(path.join(__dirname, 'db.json'))
const middlewares = jsonServer.defaults()
server.use(cors())
server.use(jsonServer.bodyParser)
server.use(middlewares)
server.use(router)
const PORT = 8000
server.listen(PORT, () => {
console.log(`JSON Server is running on http://localhost:${PORT}`)
})
I am using cyclic.sh for deployment. How to solve this error?
I was trying to deploy a json-server. And while doing a post request I got this error.
Taking as a starting point that you shouldn't use a tool like json-server in production environments, I could understand you might be using it for demo purposes. That case, there are some serverless solutions which prevents your code from open files in write mode, so that's why the jsonServer.router(...) line fails its execution.
If you don't matter about that db.json file being updated by the requests (because anyway, your deployment solution doesn't seem to allow it), you could just use instead a js object, that can be modified in memory (but, of course, the json file will still keep intact). So, instead of:
const router = jsonServer.router(path.join(__dirname, 'db.json'))
try using something like:
const fs = require('fs')
const db = JSON.parse(fs.readFileSync(path.join(__dirname, 'db.json')))
const router = jsonServer.router(db)
I'm trying to download a file (without saving it) and convert it, but readFileSync throws an error that the file can't be found: "Error: ENOENT: no such file or directory, open 'https://s3.amazonaws.com/appforest_uf/f1631452514756x615162562554826200/testdoc.txt'"
Code:
var path = require('path');
const fs = require('fs');
const https = require('https');
const file_x = 'https://s3.amazonaws.com/appforest_uf/f1631452514756x615162562554826200/testdoc.txt';
var filename = path.basename(file_x);
const FileBuffer = fs.readFileSync(file_x);
const fileParam = {
value: FileBuffer,
options: {
filename: filename,
contentType: 'application/octet-stream'
}
};
console.log(fileParam);
And I get the error the file can't be found... The URL is working OK, do I need to do something else to download from URL ?
Did you try looking for similar questions? For example, readFileSync not reading URL as file. There it is clearly stated that the fs API only deals with local files.
You will need to use the https module to stream the file.
The package fs only reads files, not URLs. Alternatively, you can use a package like axios. Axios allows you to make HTTP requests within node.js.
Example usage:
const axios = require('axios');
axios.get('https://s3.amazonaws.com/appforest_uf/f1631452514756x615162562554826200/testdoc.txt').then(response => {
console.log(response);
...
});
I build a Node.js tool in order to change GCP config quickly. But I'm a bit stuck on the parsing of the config_nameOfConfig file.
It looks like this :
[core]
account = email#example.fr
project = project-example
disable_usage_reporting = False
[compute]
region = europe-west1-b
(This file is available in '/Users/nameOfUser/.config/gcloud')
And I want to convert it in an object like this :
const config = {
account:"email#example.fr",
project:"project-example",
disable_usage_reporting:false,
region:"europe-west1-b",
};
I get the content of this file with the fs.readFileSync function who converts it into a string.
An idea ?
This seems to be an ini file format. Unless you want to do the parsing yourself, you'd better just download a library to your project, for example:
npm install ini
Then in the code:
var ini = require('ini')
var fs = require('fs')
var config = ini.parse(fs.readFileSync('./config_nameOfConfig', 'utf-8'))
In config you should now have object containing the data in the file.
You can console.log(config) to see how it exactly looks like.
I have created a zip file of a .txt file using node.js zlib and while trying to unzip the file i used the same node_module zlib but its throwing error.Please check my code snippet and please feel free to edit code.I also checked the file location its same as the
Error :
Error: unexpected end of file
at Unzip.zlibOnError (zlib.js:153:15)
Code :
const fs=require('fs');
const zlib = require('zlib');
const iii = fs.createReadStream('test.txt.gz');
const oo = fs.createWriteStream('test1.txt');
const unzip = zlib.createUnzip()
iii.pipe(unzip).pipe(oo)
I am unable to understand my mistake.Can anyone help me with the same?
Thanks in advance!!
Used below code to zip the file.
code to zip:
const fs=require('fs');
const zlib = require('zlib');
const gzip = zlib.createGzip();
const inp = fs.createReadStream('test.txt');
const out = fs.createWriteStream('test.txt.gz');
inp.pipe(gzip).pipe(out);
I was getting an error because the file got corrupted while trying the zip-unzip.
There was no error in the code.
So after deleting and recreating zip file, i was able to unzip it.
Note:
Whenever you try something like this,if you get such error, then always check file size it should not be zero. If it is zero,then recreate the zip and try again.
I am trying to write data to a text file in Node.js. When initializing the write stream I get the following error message:
Be sure you have included at the top of your node program
var fs = require('fs');
Also It looks like you spelled 'Stream' incorrectly
fs.createWriteSteam > fs.createWriteStream
Another issue that may come up is trying to get createWriteStream from 'fs/promises'.
It doesn't exist on 'fs/promises'. (At least on NodeJS version 14);
My solution was to import both:
var fs = require('fs');
var fsPromises = require('fs/promises');