Calling an external API with a Node application (KeystoneJS) - javascript

I'm newer to Node and trying to learn how to modify my Keystone.JS app so it can call data from an API (JSON or XML) and display it in the view that is rendered.
The current code in my app is essentially a cloned version of this demo app https://github.com/JedWatson/keystone-demo except the view engine is Handlebars in my app. What I have tried to so far is is installing the request package and played around with code from the documentation in a my keystone.js file with no luck.
Then I created a model/api.js file, routes/api.js, routes/views/api.js and templates/views/api.hbs and again played with code examples in the request documentation but failed to even grasp what I was doing and how all of these new pages even worked within my app.
I would greatly appreciate figuring out how to call an API and display the requested info in one of the apps rendered views. Thank you in advance!

You could hit the api from your model logic like so https://github.com/r3dm/shpe-sfba/blob/master/models/Event.js#L69 You could use node's built in http library http://devdocs.io/node/http
// Below we call the Facebook api to fill in data for our model
Event.schema.pre('save', function(next) {
var myEvent = this;
var apiCall = 'your API string';
https.get(apiCall, function(res) {
var body = '';
res.on('data', function(d) { body += d; });
res.on('end', function() {
body = JSON.parse(body);
if (body.error) {
var err = new Error('There was an error saving your changes. Make sure the Facebook Event is set to "Public" and try again');
next(err);
} else {
next();
});
})
.on('error', function(e) {
console.log(e);
});
});
If you want the data to be fetched in some other scenario try adding the http request to initLocals in routes/middleware.js.

Related

Bi-directional communication between a client page and a service worker

Having a preact app generated by preact-cli (uses workbox), my objective is to register a 'message' event handler on the service worker, post a message from the app and finally receive a response back.
Something like:
/* app.js */
navigator.serviceWorker.postMessage('marco');
const response = ? // get the response somehow
/* sw.js */
addEventListener('message', function (e) { return 'polo' });
I don't have much experience with service workers and there are a lot of moving parts that confuse me, like workbox doing magic on service worker, preact hiding away the code that registers the sercice worker and service workers being tricky to debug in general.
So far I've placed a sw.js file in the src/ directory as instructed by the preact-cli docs here: https://preactjs.com/cli/service-worker/
I know I am supposed to attach an event listener but I can't find documentation on which object to do so.
(Neither Workbox nor Preact have much to do with this question. Workbox allows you to use any additional code in your service worker that you'd like, and Preact should as well for your client app.)
This example page demonstrates sending a message from a client page to a service worker and then responding, using MessageChannel. The relevant helper code used on the client page looks like:
function sendMessage(message) {
return new Promise(function(resolve, reject) {
const messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
// The response from the service worker is in event.data
if (event.data.error) {
reject(event.data.error);
} else {
resolve(event.data);
}
};
navigator.serviceWorker.controller.postMessage(message,
[messageChannel.port2]);
});
}
And then in your service worker, you used the MessageChannel's port to respond:
self.addEventListener('message', function(event) {
// Your code here.
event.ports[0].postMessage({
error: // Set error if you want to indicate a failure.
message: // This will show up as event.data.message.
});
});
You could also do the same thing using the Comlink library to simplify the logic.

Implementing Dropbox API V2 in Cordova Application

I have a Cordova application with previous Dropbox implementation using rossmartin/phonegap-dropbox-sync-android. Now as the API V1 is going to be deprecated I want to upgrade to Dropbox API V2. I have searched for plugins for Cordova applications using Dropbox API V2 but didn't find any.So I am trying to implement it using dropbox/dropbox-sdk-js.
For Authentication, I am using authenticateWithCordova method which returns me the Access token (Full documentation here).This method returns Access token once the user completes authentication with Dropbox and uses the redirect URL to redirect the user to Cordova application.
This method works perfectly when the user clicks the button for the first time, but when the user clicks the button again calling this method shows a blank screen and return a new access token. How to avoid seeing the blank screen?
This is the method from Dropbox-sdk.js file, which I have called from my application,
DropboxBase.prototype.authenticateWithCordova = function (successCallback, errorCallback)
{
var redirect_url = 'https://www.dropbox.com/1/oauth2/redirect_receiver';
var url = this.getAuthenticationUrl(redirect_url);
var browser = window.open(url, '_blank');
var removed = false;
var onLoadError = function(event) {
// Try to avoid a browser crash on browser.close().
window.setTimeout(function() { browser.close() }, 10);
errorCallback();
}
var onLoadStop = function(event) {
var error_label = '&error=';
var error_index = event.url.indexOf(error_label);
if (error_index > -1) {
// Try to avoid a browser crash on browser.close().
window.setTimeout(function() { browser.close() }, 10);
errorCallback();
} else {
var access_token_label = '#access_token=';
var access_token_index = event.url.indexOf(access_token_label);
var token_type_index = event.url.indexOf('&token_type=');
if (access_token_index > -1) {
access_token_index += access_token_label.length;
// Try to avoid a browser crash on browser.close().
window.setTimeout(function() { browser.close() }, 10);
var access_token = event.url.substring(access_token_index, token_type_index);
successCallback(access_token);
}
}
};
Here is my code which I use to call the method,
function authenticateWithCordova()
{
var dbx = new Dropbox({ clientId: CLIENT_ID });
dbx.authenticateWithCordova(AuthSuccess,AuthFail);
}
function AuthSuccess(accessToken)
{
localStorage.accessToken = accessToken;
}
function AuthFail()
{
alert("Auth Fail");
}
I have found an analog issue right yesterday. This is the way I solved it.
First, I have set var dbx as global. In my index.js I put these lines immediately after app.initialize():
var CLIENT_ID = 'xxxxxxxxxxxxxxx';
var dbxt;
var dbx = new Dropbox({clientId: CLIENT_ID});
Then I check if dbxt is null: if it is, I create a new Dropbox object using accessToken, otherwise I go with the dropbox connection already established:
if (dbxt == null) {
dbx.authenticateWithCordova(function (accessToken) {
dbxt = new Dropbox({accessToken: accessToken});
dbxt.filesUpload({
path: '/mydump.sql',
contents: sql,
mode: 'overwrite',
mute: true
}).then(function (response) {
alert('Your backup has been successfully uploaded to your Dropbox!')
}).catch(function (error) {
alert('Error saving file to your Dropbox!')
console.error(error);
});
}, function (e){
console.log("failed Dropbox authentication");
}
}else{//dbxt already created
dbxt.filesUpload... //and the rest
}
This is just to avoid to create a new connection and get a new access token everytime and I confess I'm not sure this is a good practice: I only know that before to apply this code I got a lot of bad requests responses by Dropbox server:)
When I used the above code, after the first login, I started to see the blank page: that's is the inappbrowser page which Dropbox OAuth2 uses as redirect URI (set to https://www.dropbox.com/1/oauth2/redirect_receiver in your Dropbox app page).
So the problem was how to make this page invisible. I found a dirty trick applying a small tweak to inappbrowser.js script.
Near the bottom of the script, immediately before this line:
strWindowFeatures = strWindowFeatures || "";
I have put this small block:
if (strUrl.indexOf('dropbox') > -1){
strWindowFeatures += "location=no,hidden=yes";
}
I would have expected to can just use 'hidden=yes' but surprisingly if I remoce 'location=no' the blkank page appears again.
Notice 1: you don't have to modify the script inappbrowser.js located at plugins\cordova-plugin-inappbrowser\www\ but the one you find in platforms\android\platform_www\plugins\cordova-plugin-inappbrowser\www\
Notice 2: I have found this workaround right now so I'm not 100% sure it works perfectly.
Notice 3: making the inappbrowser page invisible, depending on the Internet connection, it could look like nothing is happening for a while, so you'll have to add some loader to inform your user that the app is working.
Hope this help.
UPDATE
I've just realized we can tweak directly the dropbox-sdk instead of inappbrowser.
If you are using Dropbox with browserify you have to open dropbox-base.js and look for authenticateWithCordova() method (it should be at line 107. Then change the line
var browser = window.open(url, '_blank');
to
var browser = window.open(url, '_blank', "location=no,hidden=yes");
If you are using Dropbox-sdk.min.js, you have to look for 'window.open' using the search function of your code editor. It will be easy because 'window.open' is used only once. So you'll have to change the following:
i=window.open(n,"_blank"),
to
i=window.open(n,"_blank","location=no,hidden=yes"),
And this seems to work fine (I prefer to be careful before I get excited).
UPDATE 2
Forgive previous update. My previous check:
if (strUrl.indexOf('dropbox') > -1){
strWindowFeatures += "location=no,hidden=yes";
}
is wrong because it makes invisible any inappbrowser window which tries to connect to dropbox so it prevent us from even logging into Dropbox. So we need to change it to
if (strUrl == 'https://www.dropbox.com/1/oauth2/redirect_receiver') {
strWindowFeatures += "location=no,hidden=yes";
}
This way we can do the login correctly and next connections won't show the inappbrowser window, as we want.
So summarizing:
Ignore my first update
Use UPDATE 2 to modify the url check in inappbrowser.js
Forgive me for the confusion...

How to inject module from different app in Node.js

I've two node apps/services that are running together,
1. main app
2. second app
The main app is responsible to show all the data from diffrent apps at the end. Now I put some code of the second app in the main app and now its working, but I want it to be decoupled. I mean that the code of the secnod app will not be in the main app (by somehow to inject it on runtime )
like the second service is registered to the main app in inject the code of it.
the code of it is just two modules ,is it possible to do it in nodejs ?
const Socket = require('socket.io-client');
const client = require("./config.json");
module.exports = (serviceRegistry, wsSocket) =>{
var ws = null;
var consumer = () => {
var registration = serviceRegistry.get("tweets");
console.log("Service: " + registration);
//Check if service is online
if (registration === null) {
if (ws != null) {
ws.close();
ws = null;
console.log("Closed websocket");
}
return
}
var clientName = `ws://localhost:${registration.port}/`
if (client.hosted) {
clientName = `ws://${client.client}/`;
}
//Create a websocket to communicate with the client
if (ws == null) {
console.log("Created");
ws = Socket(clientName, {
reconnect: false
});
ws.on('connect', () => {
console.log("second service is connected");
});
ws.on('tweet', function (data) {
wsSocket.emit('tweet', data);
});
ws.on('disconnect', () => {
console.log("Disconnected from blog-twitter")
});
ws.on('error', (err) => {
console.log("Error connecting socket: " + err);
});
}
}
//Check service availability
setInterval(consumer, 20 * 1000);
}
In the main module I put this code and I want to decouple it by inject it somehow on runtime ? example will be very helpful ...
You will have to use vm module to achieve this. More technical info here https://nodejs.org/api/vm.html. Let me explain how you can use this:
You can use the API vm.script to create compiled js code from the code which you want run later. See the description from official documentation
Creating a new vm.Script object compiles code but does not run it. The
compiled vm.Script can be run later multiple times. It is important to
note that the code is not bound to any global object; rather, it is
bound before each run, just for that run.
Now when you want to insert or run this code, you can use script.runInContext API.
Another good example from their official documentation:
'use strict';
const vm = require('vm');
let code =
`(function(require) {
const http = require('http');
http.createServer( (request, response) => {
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end('Hello World\\n');
}).listen(8124);
console.log('Server running at http://127.0.0.1:8124/');
})`;
vm.runInThisContext(code)(require);
Another example of using js file directly:
var app = fs.readFileSync(__dirname + '/' + 'app.js');
vm.runInThisContext(app);
You can use this approach for the conditional code which you want to insert.
You can create a package from one of your apps and then reference the package in the other app.
https://docs.npmjs.com/getting-started/creating-node-modules
There are several ways to decouple two applications. One easy way is with pub/sub pattern (in case you don't need a response).
(Now if you have an application that is very couple, it will be very difficult to decouple it unless you do some refactoring.)
zeromq offers a very good implementation of pub/sub and is very fast.
e.g.
import zmq from "zmq";
socket.connect('tcp://127.0.0.1:5545');
socket.subscribe('sendConfirmation');
socket.on('message', function (topic, message) {
// you can get the data from message.
// something like:
const msg = message.toString('ascii');
const data = JSON.parse(msg);
// do some actions.
// .....
});
//don't forget to close the socket.
process.on('SIGINT', () => {
debug("... closing the socket ....");
socket.close();
process.exit();
});
//-----------------------------------------
import zmq from "zmq";
socket.bind('tcp://127.0.0.1:5545');
socket.send(['sendConfirmation', someData]);
process.on('SIGINT', function() {
socket.close();
});
This way you could have two different containers (docker) for your modules, just be sure to open the corresponding port.
What i don't understand, is why you inject wsSocket and also you create a new Socket. Probably what I would do is just to send the
socket id, and then just use it like:
const _socketId = "/#" + data.socketId;
io.sockets.connected[socketId].send("some message");
You could also use another solution like kafka instead of zmq, just consider that is slower but it will keep the logs.
Hope this can get you an idea of how to solve your problem.
You can use npm link feature.
The linking process consists of two steps:
Declaring a module as a global link by running npm link in the module’s root folder
Installing the linked modules in your target module(app) by running npm link in the target folder
This works pretty well unless one of your local modules depends on another local module. In this case, linking fails because it cannot find the dependent module. In order to solve this issue, one needs to link the dependent module to the parent module and then install the parent into the app.
https://docs.npmjs.com/cli/link

How to use phantomjs in node-js environment for dynamic-page web scraping?

I am working on web scraping for few task to complete.
I have used node-js request module for page scraping.
It is working fine and great for cookie-session and all.
But it fails when time comes to render Dynamic pages build with some javascript framework like ANGULAR or BACKBONE etc.
I am trying for phantomjs to overcome this thing as i found on google that it is helpful to come over such case.
I also found one nodejs bridge for phantomjs phantom
With phantomjs and this bridge module i am able to achieve same thing nothing more.
var phantom = require('phantom');
var fs = require('fs');
var sitepage = null;
var phInstance = null;
phantom.create()
.then(instance => {
phInstance = instance;
console.log("Instance created");
return instance.createPage();
})
.then(page => {
sitepage = page;
console.log("createing page");
return page.open('https://paytm.com/shop/p/carrier-estrella-plus-1-5-ton-3-star-window-ac-LARCARRIER-ESTRPLAN5550519593A34?src=grid&tracker=%7C%7C%7C%7C%2Fg%2Felectronics%2Flarge-appliances%2F1-5-ton-3-star-ac-starting-at-rs-22699%7C88040%7C1');
})
.then(status => {
//console.log(status);
console.log("getting content of page");
return sitepage.property('content');
})
.then(content => {
console.log("success");
//console.log(content);
fs.writeFile("ok.text", content);
sitepage.close();
phInstance.exit();
})
.catch(error => {
console.log("errr");
//console.log(error);
phInstance.exit();
});
Above is code which i am trying for load one of dynamic website page which is build with angular framework.
Can anybody guide me for same or correct in above code where i am missing right things.
You're getting the content of the page before the dynamic code has run, you need to wait for the load to be completed.
The block behind the page.open would need to wait for the page to complete, if there is an element you know is being fetched from the back end you can lie in wait for that element (see the waitfor example in phantomjs doc).

Authorization header using OAuth1.0A for Twitter API v1.1

I am building a search engine as a start-up project in Web Development. The search engine takes a string and queries the Wiki and Twitter API. I have been working on it for days and, I believe, I have found a nice, clean way to implement it.
BoyCook's TwitterJSClient
In his code (which is beautifully written) he has set up a Twitter function which takes in a config variable and sets up the authorization for us. Is there something it is missing? I have been through it and it all looks great. I am new to Javascript and might be missing something..
My code:
var error = function (err, response, body) {
console.log('ERROR [%s]', err);
};
var success = function (data) {
console.log('Data [%s]', data);
};
var config = {
"consumerKey": "{Mine}",
"consumerSecret": "{Mine}",
"accessToken": "{Mine}",
"accessTokenSecret": "{Mine}",
"callBackUrl": ""
}
var Twitter = require('twitter-node-client').Twitter;
var twitter = new Twitter(config);
twitter.getSearch({'q':'Lebron James','count': 10}, error, success);
Can anyone help? Have you done this before and know an easier way? Has anyone been able to get it working using postMessage()?
And yes... the origin is using HTTPS protocol. See: JBEngine. My app permission (on Twitter) is set to read only.
[EDIT] Should probably also mention that I glob'd it all together with browserify and am running the script client-side.

Categories