I am trying to load JSON data from https://blockchain.info/ticker into Node like so: const btc = require(https://blockchain.info/ticker) Obviously, this does not work. How can one do this?
You can't pass require() a URL. It needs a filename.
If you want to load some JSON from a remote server, you can use the request or request-promise packages. The loading will be asynchronous though so you will need to use the result in the appropriate callback. Here's an example:
const rp = require('request-promise');
rp({json: true}, "https://blockchain.info/ticker").then(data => {
// use data here
console.log(data);
}).catch(err => {
// process error here
console.log(err);
});
Related
Good Evening,
I have a function that contains a route that is a call to the Auth0 API and contains the updated data that was sent from the client. The function runs, but the app.patch() does not seem to run and I am not sure what I am missing.
function updateUser(val) {
app.patch(`https://${process.env.AUTH0_BASE_URL}/api/v2/users/${val.id}`,(res) => {
console.log(val);
res.header('Authorization: Bearer <insert token>)
res.json(val);
})
app.post('/updateuser', (req, ) => {
const val = req.body;
updateUser(val);
})
app.patch() does NOT send an outgoing request to another server. Instead, it registers a listener for incoming PATCH requests. It does not appear from your comments that that is what you want to do.
To send a PATCH request to another server, you need to use a library that is designed for sending http requests. There's a low level library built into the nodejs http module which you could use an http.request() to construct a PATCH request with, but it's generally a lot easier to use a higher level library such as any of them listed here.
My favorite in that list is the got() library, but many in that list are popular and used widely.
Using the got() library, you would send a PATCH request like this:
const got = require('got');
const options = {
headers: {Authorization: `Bearer ${someToken}`},
body: someData
};
const url = `https://${process.env.AUTH0_BASE_URL}/api/v2/users/${val.id}`;
got.patch(url, options).then(result => {
console.log(result);
}).catch(err => {
console.log(err);
});
Note: The PATCH request needs body data (the same that a POST needs body data)
My task is to download json-file from website (pubchem) using only the query string (h2o for example) and JS. I know it's possible to do with parsing, but this is too much code because of number of pages i need to parse for getting destination. Is there any other options to solve the problem?
Using google didnt give me any of idea ):
You will still need to do some parsing if you really want to automate this, since only using a query parameter will get you to the main page that lists the 'articles' and you need to go in to find the URL that will give you the JSON format. But! I think you can "reverse engineer" it since the URLS for the article and its JSON format are very similar.
I checked out the website and tried to download one of the files that they have for https://pubchem.ncbi.nlm.nih.gov/compound/3076959 and it turns out to get the JSON representation this was the URL https://pubchem.ncbi.nlm.nih.gov/rest/pug_view/data/compound/748328/JSON/
As you can see they are very similar and you might be able to figure out how different topics such as compound for example construct the JSON output endpoint.
To download the JSON files using NodeJS is to use the node-fetch module or axios library to send your http requests to the JSON endpoint and from there you can save the response to a file on your machine.
Here is an example of how you can do this with axios and the NodeJS fs module in order to save the file to your machine.
const fs = require("fs");
const fetch = require("node-fetch");
async function downloadASJson(url, fileName) {
const response = await fetch(url);
const jsonContent = await response.buffer();
fs.writeFile(`${fileName}.json`, jsonContent, "utf8", function (err) {
if (err) {
console.log("An error occured while writing JSON Object to File.");
return console.log(err);
}
console.log("JSON file has been saved.");
});
}
try {
downloadASJson(
"https://pubchem.ncbi.nlm.nih.gov/rest/pug_view/data/compound/748328/JSON/",
"2-Methyl-3-(5'-bromobenzofuroyl-2')-4-dimethylaminomethyl-5-hydroxybenzofuran HCl H20"
);
} catch (err) {
console.log(error);
}
You save the following code in a file called app.js for example, and you can use node app.js to run it. Don't forget to install the dependencies.
I create server (node.js / express / boby-parser)
and I need to get array of objects 'users'.
its part of code from server.js file:
let users = [{
name: 'user1',
}];
app.get('/users/', (req, res) => {
const filePath = path.join(pth.dir, 'build', 'index.html');
res.json(users);
res.sendFile(filePath);
});
Its my code from frontend:
const handleResponse = (response) => {
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
};
const getAll = (baseUrl) => {
const requestOptions = {
method: 'GET'
};
return fetch(baseUrl, requestOptions).then(handleResponse);
};
Something wrong with my code at server. (I just dodnt know how to use express server).
when I use getAll function I got JSON text replace my page. Can anyone help? How should I write app.get() in server.js. Or do I need write in server part one app.get() to get page or another app.get() to get JSON data?
Why are you trying to send a file in the response?:
res.sendFile(filePath);
For starters, the response content can either be JSON or a file (or any of a variety of other things of course), but not both. With data like JSON or XML it's possible to combine multiple objects into one larger object for a single response, but this won't work if the content types are entirely different as it is with a file.
Looking at your client-side code, you're not even doing anything with that file. You only read the JSON data:
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
So the simplest approach here would just be to not try to send back the file:
app.get('/users/', (req, res) => {
res.json(users);
});
Edit: Based on comments below, you seem to be struggling with the different requests the client makes to the server. The browser loading the page is one request with one response. If that page includes JavaScript that needs to fetch data, that would be a separate AJAX request with its own response containing just that data.
It’s possible to use JSON (or any data) server-side to populate a page template and return a whole page with the data. For that you’d need to use (or build) some kind of templating engine in the server-side code to populate the page before returning it.
The res.json() represents the HTTP response that an Express app sends when it gets an HTTP request. On the other hand, res.sendFile() transfers the file at the given path.
In both cases, the flow is essentially transferred to client who might have made the request.
So no, you cannot use res.sendFile and res.json together.
var options = {
headers: {
'name': 'user1',
}
};
res.sendFile(path.join(__dirname, 'build', 'index.html'), options);
Thats really the closest you can do to achieve the desired task.
I am trying to submit a image file using a POST Request, to server, where in the body of the front end fetch request i added body as FormData like this
let formdata = new FormData(form)
async function finalFetch(formdata){
let postReq = await fetch('/api/fileupload', {method : 'POST', body : formdata})
let result = await postReq.json()
return result.url;
}
Now on submit of this form in the backend i am handling the data like this
req.on('data', (chunk)=>{
console.log(chunk);
})
req.on('end', ()=>{
// pseudo code, this will get replaced by something else
res.write(JSON.stringify({
msg : 'File Upload done',
url : '/232'
}));
res.end();
})
and the above implementation gives me a Buffer, i have no idea how to write file in the server form this Buffer
I know there are lots of modules out there to handle the forms like multer, express, formidable, but i don't want to use any of them i am trying to understand how these packages work. I am just using core node js.
There are tons of third party packages why not use them, one of the famous one is https://www.npmjs.com/package/multer, by default node dosnt give you an option to write files.
After the buffer is fully received you should use core node module 'fs' to write buf to file.
const fs = require('fs');
fs.writeFile('file.name', buffer);
see docs fs.writeFile
I've tried all sorts to get this to work. I'm trying to request a PDF from an API on node, then send this back to the client who called it to begin with.
For the minute I just want to successfully save and view the PDF on the node server.
The issue is the PDF file is always empty when I open it (Even though it has a size of 30kb).
The basic flow is like this (removed a few bits, but the below code works and returns me the PDF fine)
// We pass through session ID's, request dates etc through in body
app.post("/getPayslipURL", function(client_request, res) {
// create request, which will simply pass on the data to the database (In order to get the NI number we need for the pay API)
const NI_NUMBER_REQUEST = db_api.createRequestTemplate({
body: JSON.stringify(client_request.body)
});
// Create a chain of HTTPS Requests, Starting with our call to the DB
requestPromise(NI_NUMBER_REQUEST)
.then((db_response) => {
const PAY_API_OPTIONS = /*Code to generate options based on furhter DB info (Includes dates etc)*/
return requestPromise(PAY_API_OPTIONS); // Call pay API
})
.then((pay_pdf_data) => {
console.log(typeof pay_pdf_data); // It's a string
// At this point I can log pay_pdf_data, But if I try to save it to file it's always empty
// No matter how I encode it etc
fs.writeFile("./test.pdf", pay_pdf_data, 'binary', function(err) {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
})
.catch(err => `Error caught: ${console.log}`) // Catch any errors on our request chain
});
}
I've tried saving with / without the binary flag as suggested in other posts in both the file save aswell as within the requests itself. Also various types of decoding methods have been tried, I always get an empty PDF saved.
My return data looks like this (is much bigger, when saved as test.pdf I get a 30kb file as before mentioned)
%PDF-1.4
%����
1 0 obj
0 obj
<
I've found a post which says about piping the data all the way through, I have a feeling my pdf_data is corrupted when getting converted to a string
Any ideas how would I go about doing this with the current setup?
e/ RequestPromise is a library, could also use the standards request library if it's easier
https://github.com/request/request-promise -
https://github.com/request/request
Thanks!
Your code doesn't work because the underlying request library (used by request-promise) requires the option encoding set to null for binary data - see https://github.com/request/request#requestoptions-callback.
Here's how you download binary data using that module -
app.post("/getPayslipURL", function(client_request, res) {
const NI_NUMBER_REQUEST = db_api.createRequestTemplate({
body: JSON.stringify(client_request.body),
encoding: null
});
requestPromise(NI_NUMBER_REQUEST)
.then((db_response) => {
const PAY_API_OPTIONS = /*Code to generate options based on furhter DB info (Includes dates etc)*/
return requestPromise(PAY_API_OPTIONS); // Call pay API
})
.then((pay_pdf_data) => {
fs.writeFile("./test.pdf", pay_pdf_data, 'binary', (err) => {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
})
.catch(err => `Error caught: ${console.log}`) // Catch any errors on our request chain
});
}