Cheerio error handler node js - javascript

I start a new project using Node.js and I'm wondering if there is any way to manage what returns from cheerio.load() function.
I've tried to use callback and promises(then and catch) but it did't work.
example:
var $ = cheerio.load(html);
//what if it's falied ???? how I can handle it?
I'm asking this because I tried to run script serval time but sometimes it's work and sometimes not.
BTW:
I'm using cheerio module from npm site -> npm install cheerio.
thanks :-)

You can use a try/catch block since you do not know if the cheerio operation will succeed.
const cheerio = require('cheerio')
try {
const $ = cheerio.load(html)
} catch (e) {
console.log(e) // handle error
}
console.log('continue script')

Related

500 process is not defined ReferenceError: process is not defined

I am getting this problem every time I import a lib or when I use puppeteer and I don't know how to fix it. I am trying to get some data from LinkedIn using https://www.npmjs.com/package/linkedin-client
the code is easy:
import LinkedinClient from 'linkedin-client';
async function getIt() {
const session = supabase.auth.session();
const tok = session?.provider_token;
const token = JSON.stringify(tok);
console.log(token);
const client = new LinkedinClient(token);
const data = await client.fetch('https://www.linkedin.com/in/some-profile/');
console.log(data);
}
at first it gives me this error:Module "util" has been externalized for browser compatibility. Cannot access "util.promisify" in client code
after I install npm i util then it displays the following error:
500 process is not defined ReferenceError: process is not defined
Can you please let me know how to fix it?(I'm using sveltekit)
The library requires to be run on the server. It has be in a server endpoint, it cannot be in a component or a load function.
If this is already the case, this might be an issue with Vite trying to remove server dependencies. There is e.g. a plugin #esbuild-plugins/node-globals-polyfill which polyfills the process variable. It may also be necessary to list packages in resolve.alias in the Vite config, to point to the Node modules.

My Javascript file won't run because of bigint error

I am trying to use #metaplex/js to do some NFT minting. Usually my .js files work properly but when I run the file this error comes up.
bigint: Failed to load bindings, pure JS will be used (try npm run rebuild?)
I don't really get what that means. So, I tried to run npm run rebuild but rebuild is said to be a missing script and I couldn't find a way to install it.
Here is my code:
import { Connection, programs} from "#metaplex/js";
import { Loader } from "#solana/web3.js";
const { metadata: {Metadata}} = programs;
const connection = new Connection("devnet");
const tokenPublicKey = 'my_adress';
const run = async() => {
try{
const ownedMetadata = await Metadata.Loader(connection,tokenPublicKey)
console.log(ownedMetadata)
}
catch{
console.log('Failed to fetch')
}
};
run();
If you have any idea, or simply an explanation of what my error means, I'd be grateful.
You are getting this error because a nested dependency has a compilation step that might not succeed in your platform. This issue provides a good explanation.
[...] This happens because one of our dependencies (bigint-buffer) runs a compilation step on installation and this can step may fail for a couple of reasons. One of the reasons is that your system might not have the build-tools the library is looking for. You can install these build tools on Windows (see https://www.npmjs.com/package/windows-build-tools), but you don't actually need to as it automatically falls back to a pure JS solution instead. Though I agree... that warning is very annoying.
However, this should give you a warning and still allow you to compile your code.
It is worth noting that the current JS SDK from Metaplex is going to be deprecated in favour of the new one: https://github.com/metaplex-foundation/js-next
With the new JS SDK, you can fetch an NFT using the following piece of code.
import { Metaplex } from "#metaplex-foundation/js";
import { Connection, clusterApiUrl } from "#solana/web3.js";
const connection = new Connection(clusterApiUrl("mainnet-beta"));
const metaplex = new Metaplex(connection);
const mintAddress = new PublicKey("ATe3DymKZadrUoqAMn7HSpraxE4gB88uo1L9zLGmzJeL");
const nft = await metaplex.nfts().findByMint({ mintAddress });

Using jquery in a Node.js project to make api request

i am trying to make an api request in a node project using jquery, which I've istalled via npm install jquery but I'm getting the error TypeError: $.post is not a function even after requiring jquery like so: const $ = require('jquery') how can I get Jquery to work? I've seen a lot of examples of people using jquery but the process was not clear to me since I'm new to Node.js
Here is my code
const $ = require('jquery')
$.post("https://ticketsoko.nouveta.co.ke/api/index.php", {
TransactionType: "getEvents"
}, function (data, status) {
var info = JSON.parse(data);
for (var i = 0; i < info.data.length; i++) {
ticketSales[i] = info.data[i].totalTicketsSalesAmount;
events[i] = info.data[i].Events.events_name;
console.log(events[i]);
}
})
Here is step by step hard-to-go-wrong tutorial on how to obtain jQuery into your NodeJs project, then require it in your code and use in a manner similar to what you would do without NodeJs. If you haven't heard of modules and npm and are working with NodeJs, YESTERDAY was the time to have some working knowledge with them.
click here to see how to
On the other hand, WHY are you using jQ for this? You have entire world of modules to your disposal via npm, there are better modules, more fitted to do HTTP requests with like this one:
request module
You have to use a capital Q for jQuery
So it is const $ = require('jQuery') instead of require('jquery').

Execute a JS file (with logs, etc...) inside another NodeJS process

Here is my problem, I want to create a CLI that automatically runs a test. Without the CLI, I'm able to run everything perfectly with the node command:
node test.js
Basically, I want to do the exact same thing as the command before, so I googled for a technique that does this. I found this:
#!/usr/bin/env node
'use strict';
const options = process.argv;
const { execFile } = require('child_process');
const child = execFile('node', ['../dist/test.js'], (error, stdout, stderr) => {
if (error) {
throw error;
}
console.log(stdout);
});
This method doesn't work for me because, in the test.js file, I'm using the ora package. And because this package is making real-time animations, it doesn't come in stdout.
Is there any way of executing in real time (without subprocess) my test.js using Node? I'm open to other methods, but I want to publish the CLI on NPM, so keep in mind that it has to be in JavaScript 😊.
You can find every file that I've talked here on GitHub. Normally, you wouldn't need this link, but I'm giving it to you if you need to have a closer look.
You should simply call your test() function from your CLI code, after requiring the module that defines it. Have a look at mocha and jasmine: you will see that while both tools provide a CLI, they also provide instructions for invoking the test frameworks from arbitrary JS code.
I can't think of a way without a sub-process. but this may help.
The child process exec will not work with the continuous output commands as it buffers the output the process will halt when that buffer is full.
The suitable solution is spwan :
var spwan = require('child_process').spwan
var child = spwan('node', ['../dist/test.js'])
child.stdout.on('data', function(data) {
console.log(data)
})
child.stderr.on('data', function(data) {
console.log(data)
})
Here is my solution, you can use the fs library to get the code of the file, and then, you simply use eval to execute in the same process.
const fs = require("fs");
function run(file) {
fs.readFile(file, (err, data) => {
eval(data.toString('utf8'))
})
}

Can I use jQuery with Node.js? Is it good pair? [duplicate]

Is it possible to use jQuery selectors/DOM manipulation on the server-side using Node.js?
Update (27-Jun-18): It looks like there was a major update to jsdom that causes the original answer to no longer work. I found this answer that explains how to use jsdom now. I've copied the relevant code below.
var jsdom = require("jsdom");
const { JSDOM } = jsdom;
const { window } = new JSDOM();
const { document } = (new JSDOM('')).window;
global.document = document;
var $ = jQuery = require('jquery')(window);
Note: The original answer fails to mention that it you will need to install jsdom as well using npm install jsdom
Update (late 2013): The official jQuery team finally took over the management of the jquery package on npm:
npm install jquery
Then:
require("jsdom").env("", function (err, window) {
if (err) {
console.error(err);
return;
}
var $ = require("jquery")(window);
});
Yes you can, using a library I created called nodeQuery
var Express = require('express')
, dnode = require('dnode')
, nQuery = require('nodeQuery')
, express = Express.createServer();
var app = function ($) {
$.on('ready', function () {
// do some stuff to the dom in real-time
$('body').append('Hello World');
$('body').append('<input type="text" />');
$('input').live('click', function () {
console.log('input clicked');
// ...
});
});
};
nQuery
.use(app);
express
.use(nQuery.middleware)
.use(Express.static(__dirname + '/public'))
.listen(3000);
dnode(nQuery.middleware).listen(express);
At the time of writing there also is the maintained Cheerio.
Fast, flexible, and lean implementation of core jQuery designed
specifically for the server.
A simple crawler using Cheerio
This is my formula to make a simple crawler in Node.js. It is the main reason for wanting to do DOM manipulation on the server side and probably it's the reason why you got here.
First, use request to download the page to be parsed. When the download is complete, handle it to cheerio and begin DOM manipulation just like using jQuery.
Working example:
var
request = require('request'),
cheerio = require('cheerio');
function parse(url) {
request(url, function (error, response, body) {
var
$ = cheerio.load(body);
$('.question-summary .question-hyperlink').each(function () {
console.info($(this).text());
});
})
}
parse('http://stackoverflow.com/');
This example will print to the console all top questions showing on SO home page. This is why I love Node.js and its community. It couldn't get easier than that :-)
Install dependencies:
npm install request cheerio
And run (assuming the script above is in file crawler.js):
node crawler.js
Encoding
Some pages will have non-english content in a certain encoding and you will need to decode it to UTF-8. For instance, a page in brazilian portuguese (or any other language of latin origin) will likely be encoded in ISO-8859-1 (a.k.a. "latin1"). When decoding is needed, I tell request not to interpret the content in any way and instead use iconv-lite to do the job.
Working example:
var
request = require('request'),
iconv = require('iconv-lite'),
cheerio = require('cheerio');
var
PAGE_ENCODING = 'utf-8'; // change to match page encoding
function parse(url) {
request({
url: url,
encoding: null // do not interpret content yet
}, function (error, response, body) {
var
$ = cheerio.load(iconv.decode(body, PAGE_ENCODING));
$('.question-summary .question-hyperlink').each(function () {
console.info($(this).text());
});
})
}
parse('http://stackoverflow.com/');
Before running, install dependencies:
npm install request iconv-lite cheerio
And then finally:
node crawler.js
Following links
The next step would be to follow links. Say you want to list all posters from each top question on SO. You have to first list all top questions (example above) and then enter each link, parsing each question's page to get the list of involved users.
When you start following links, a callback hell can begin. To avoid that, you should use some kind of promises, futures or whatever. I always keep async in my toolbelt. So, here is a full example of a crawler using async:
var
url = require('url'),
request = require('request'),
async = require('async'),
cheerio = require('cheerio');
var
baseUrl = 'http://stackoverflow.com/';
// Gets a page and returns a callback with a $ object
function getPage(url, parseFn) {
request({
url: url
}, function (error, response, body) {
parseFn(cheerio.load(body))
});
}
getPage(baseUrl, function ($) {
var
questions;
// Get list of questions
questions = $('.question-summary .question-hyperlink').map(function () {
return {
title: $(this).text(),
url: url.resolve(baseUrl, $(this).attr('href'))
};
}).get().slice(0, 5); // limit to the top 5 questions
// For each question
async.map(questions, function (question, questionDone) {
getPage(question.url, function ($$) {
// Get list of users
question.users = $$('.post-signature .user-details a').map(function () {
return $$(this).text();
}).get();
questionDone(null, question);
});
}, function (err, questionsWithPosters) {
// This function is called by async when all questions have been parsed
questionsWithPosters.forEach(function (question) {
// Prints each question along with its user list
console.info(question.title);
question.users.forEach(function (user) {
console.info('\t%s', user);
});
});
});
});
Before running:
npm install request async cheerio
Run a test:
node crawler.js
Sample output:
Is it possible to pause a Docker image build?
conradk
Thomasleveil
PHP Image Crop Issue
Elyor
Houston Molinar
Add two object in rails
user1670773
Makoto
max
Asymmetric encryption discrepancy - Android vs Java
Cookie Monster
Wand Maker
Objective-C: Adding 10 seconds to timer in SpriteKit
Christian K Rider
And that's the basic you should know to start making your own crawlers :-)
Libraries used
request
iconv-lite
cheerio
async
in 2016 things are way easier. install jquery to node.js with your console:
npm install jquery
bind it to the variable $ (for example - i am used to it) in your node.js code:
var $ = require("jquery");
do stuff:
$.ajax({
url: 'gimme_json.php',
dataType: 'json',
method: 'GET',
data: { "now" : true }
});
also works for gulp as it is based on node.js.
I believe the answer to this is now yes.
https://github.com/tmpvar/jsdom
var navigator = { userAgent: "node-js" };
var jQuery = require("./node-jquery").jQueryInit(window, navigator);
npm install jquery --save #note ALL LOWERCASE
npm install jsdom --save
const jsdom = require("jsdom");
const dom = new jsdom.JSDOM(`<!DOCTYPE html>`);
var $ = require("jquery")(dom.window);
$.getJSON('https://api.github.com/users/nhambayi',function(data) {
console.log(data);
});
jQuery module can be installed using:
npm install jquery
Example:
var $ = require('jquery');
var http = require('http');
var options = {
host: 'jquery.com',
port: 80,
path: '/'
};
var html = '';
http.get(options, function(res) {
res.on('data', function(data) {
// collect the data chunks to the variable named "html"
html += data;
}).on('end', function() {
// the whole of webpage data has been collected. parsing time!
var title = $(html).find('title').text();
console.log(title);
});
});
References of jQuery in Node.js** :
http://quaintous.com/2015/07/31/jqery-node-mystery/
http://www.hacksparrow.com/jquery-with-node-js.html
You have to get the window using the new JSDOM API.
const jsdom = require("jsdom");
const { window } = new jsdom.JSDOM(`...`);
var $ = require("jquery")(window);
First of all install it
npm install jquery -S
After installing it, you can use it as below
import $ from 'jquery';
window.jQuery = window.$ = $;
$(selector).hide();
You can check out a full tutorial that I wrote here: https://medium.com/fbdevclagos/how-to-use-jquery-on-node-df731bd6abc7
WARNING
This solution, as mentioned by Golo Roden is not correct. It is just a quick fix to help people to have their actual jQuery code running using a Node app structure, but it's not Node philosophy because the jQuery is still running on the client side instead of on the server side. I'm sorry for giving a wrong answer.
You can also render Jade with node and put your jQuery code inside. Here is the code of the jade file:
!!! 5
html(lang="en")
head
title Holamundo!
script(type='text/javascript', src='http://code.jquery.com/jquery-1.9.1.js')
body
h1#headTitle Hello, World
p#content This is an example of Jade.
script
$('#headTitle').click(function() {
$(this).hide();
});
$('#content').click(function() {
$(this).hide();
});
My working code is:
npm install jquery
and then:
global.jQuery = require('jquery');
global.$ = global.jQuery;
or if the window is present, then:
typeof window !== "undefined" ? window : this;
window.jQuery = require('jquery');
window.$ = window.jQuery;
None of these solutions has helped me in my Electron App.
My solution (workaround):
npm install jquery
In your index.js file:
var jQuery = $ = require('jquery');
In your .js files write yours jQuery functions in this way:
jQuery(document).ready(function() {
The module jsdom is a great tool. But if you want to evaluate entire pages and do some funky stuff on them server side I suggest running them in their own context:
vm.runInContext
So things like require / CommonJS on site will not blow your Node process itself.
You can find documentation here. Cheers!
As of jsdom v10, .env() function is deprecated. I did it like below after trying a lot of things to require jquery:
var jsdom = require('jsdom');
const { JSDOM } = jsdom;
const { window } = new JSDOM();
const { document } = (new JSDOM('')).window;
global.document = document;
var $ = jQuery = require('jquery')(window);
Hope this helps you or anyone who has been facing these types of issues.
Yes, jQuery can be used with Node.js.
Steps to include jQuery in node project:-
npm i jquery --save
Include jquery in codes
import jQuery from 'jquery';
const $ = jQuery;
I do use jquery in node.js projects all the time specifically in the chrome extension's project.
e.g. https://github.com/fxnoob/gesture-control-chrome-extension/blob/master/src/default_plugins/tab.js
I did it manually easy way without any additional packages or code.
npm i jquery
then I copy the jquery.min.js file from node_modules/jquery/dist directory to public/js
<script type='text/javascript' src='/js/jquery.min.js'></script>
<script>
$(document).ready(function() { console.log( "ready!" ); });
</script>
And it will work. TEST IT
Note copy/pasting the file is not the ideal thing, you could enable the file as a static file by enabling it as a static so expressJS could read it. But it is easier for me to just copy it to the static public directory.
No. It's going to be quite a big effort to port a browser environment to node.
Another approach, that I'm currently investigating for unit testing, is to create "Mock" version of jQuery that provides callbacks whenever a selector is called.
This way you could unit test your jQuery plugins without actually having a DOM. You'll still have to test in real browsers to see if your code works in the wild, but if you discover browser specific issues, you can easily "mock" those in your unit tests as well.
I'll push something to github.com/felixge once it's ready to show.
You can use Electron, it allows hybrid browserjs and nodejs.
Before, I tried to use canvas2d in nodejs, but finally I gave up. It's not supported by nodejs default, and too hard to install it (many many ... dependeces).
Until I use Electron, I can easily use all my previous browserjs code, even WebGL, and pass the result value(eg. result base64 image data) to nodejs code.
Not that I know of. The DOM is a client side thing (jQuery doesn't parse the HTML, but the DOM).
Here are some current Node.js projects:
https://github.com/ry/node/wiki (https://github.com/nodejs/node)
And SimonW's djangode is pretty damn cool...
An alternative is to use Underscore.js. It should provide what you might have wanted server-side from JQuery.

Categories