We Keep Coding

How to filter Twit (API) JSON response in node.js?

I'm currently writing a small Twitter app using the Twit API. To do what I need to do, I'd like the data to be able to be filtered by user id, and not get all the other garbage JSON spits out. Here's what the response looks like:
{ created_at: 'Sat Jun 23 03:45:13 +0000 2018',
id: 1010368149466697700,
id_str: '1010368149466697728',
text:
'RT #ClassicIsComing: "Let\'s Talk ETC!" Podcast Series by #chris_seberino of #InputOutputHK \nA deep series of powerful intervie
ws with influ…',
truncated: false,
entities:
{ hashtags: [],
symbols: [],
user_mentions: [ [Object], [Object], [Object] ],
urls: [] },
source:
'TweetDeck',
in_reply_to_status_id: null,
in_reply_to_status_id_str: null,
in_reply_to_user_id: null,
in_reply_to_user_id_str: null,
in_reply_to_screen_name: null,
user:
{ id: 759252279862104000,
id_str: '759252279862104064',
name: 'Ethereum Classic',
screen_name: 'eth_classic',
location: 'Blockchain',
description:
'Latest News and Information from Ethereum Classic (ETC). A crypto-currency with smart contracts which respects immutability a
nd neutrality.',
url: ,
entities: { url: [Object], description: [Object] },
protected: false,
followers_count: 216255,
friends_count: 538,
listed_count: 2147,
etc. The code i'm using to get this is:
T.get('statuses/home_timeline', {count: 1, exclude_replies: true},
function(err, data, response){
if (err){
console.log('Uh oh, we got a problem');
}
else{
console.log('We GUUCie bruh');
}
var tweets = data;
/* for (var i = 0; i < tweets.length; i++) {
console.log(tweets[i]);
} */
console.log(data);
});
the last block of code is commented out because I've attempted to define "tweets" as data.id, data.statuses.id, etc, but everything seems to spit out "undefined." I'm a complete noob to javascript and JSON as I'm only currently learning C++ # school, so any help would be appreciated!
edit
I thought I'd add in the error message to show you what happens when I try to treat the data as an object.
If I try to use JSON.parse(data) as the value for my tweet variable:
T.get('statuses/home_timeline', {count: 1, exclude_replies: true}, callBackFunction)
function callBackFunction(err, data, response){
if (err){
console.log('Uh oh, we got a problem');
}
else{
console.log('We GUUCie bruh');
}
var tweets = JSON.parse(data);
//for (var i = 0; i < tweets.length; i++) {
// console.log(tweets[i].id_str);
// }
console.log(tweets.id_str);
}
I get:
$ node crypt.js
the bot is starting
We GUUCie bruh
undefined:1
[object Object]
^
SyntaxError: Unexpected token o in JSON at position 1
at JSON.parse (<anonymous>)
If I try to treat it as an object right away, with:
function callBackFunction(err, data, response){
if (err){
console.log('Uh oh, we got a problem');
}
else{
console.log('We GUUCie bruh');
}
var tweets = data.id_str;
//for (var i = 0; i < tweets.length; i++) {
// console.log(tweets[i].id_str);
// }
console.log(tweets);
}
I get:
$ node crypt.js
the bot is starting
We GUUCie bruh
undefined

Have you tried JSON.parse?
So your line "var tweets = data;" would be "var tweets = JSON.parse(data);"
From there you should be able to interact with the data as if it were an object and grab specifically the id or whatever you're looking for.
I'm also a noob, so I don't have an in depth explanation as to why this works, but it helped fix an issue I had when pulling data from API.

Node requestify fails when body includes special chars

Im using node.js and a library called requestify.
The code looks like this:
console.log('body');
console.log(body);
return new Promise(function (f, r) {
requestify.request(myurl, {
method: 'POST',
body : body,
headers : {
"Content-Type" : "application/json; charset=utf-8"
}
})
.then(function (response) {
f({messge : "Success"});
}, function (err) {
console.log(err);
err.code == 405 ? r(err.body) : r({ success: false, message: err });
});
});
When the request contains special characters (in this case swedish characters) then it throws an error.
Sending exactly the same request without the swedish characters, the request succedes.
I dont really have access to the server that is handling the request that's why it is hard to debug.
But I print the body before sending the request and I cannot see any strange things in it.
Here are the 2 different body-requests:
//FAILURE
{ eventType: 'createUser',
eventId: '1e2ca3b4-0f6b-4ed9-ae79-2e112c8d693c',
version: '0.1',
date: 2016-10-07T06:42:26.209Z,
attributes:
{ userName: 'akeandersson#gmail.se',
email: 'akeandersson#gmail.se',
firstName: 'Åke',
lastName: 'Andresson',
phoneNumber: '1',
identifier: '198509120328',
assignment: 'se.me',
role: [ 'se.me.role.user' ] },
metadata: {} }
//SUCCESS
{ eventType: 'createUser',
eventId: '1e2ca3b4-0f6b-4ed9-ae79-2e112c8d693c',
version: '0.1',
date: 2016-10-07T06:44:09.857Z,
attributes:
{ userName: 'akeandersson212#gmail.se',
email: 'akeandersson212#gmail.se',
firstName: 'Ake',
lastName: 'Andresson',
phoneNumber: '9',
identifier: '196606095823',
assignment: 'se.me',
role: [ 'se.me.role.user' ] },
metadata: {} }
I also put "; charset=utf-8" in the header's content-type (by default it is already utf-8 according to requestify documentation).
But still the same result. Any idea how I could debug this?
This is what the server returns when the request fails:
{
code: 400,
headers:
{ connection: 'close',
'content-type': 'text/plain',
'content-length': '244',
date: 'Fri, 07 Oct 2016 06:42:27 GMT' },
body: 'Unexpected end-of-input: expected close marker for OBJECT (from [Source: ServletInputStreamImpl#75eb88e4; line: 1, column: 0])\n at [Source: ServletInputStreamImpl#75eb88e4; line: 1, column: 767]' }

I guess, you need to upload your requestify module to the most recent version. I use version 0.2.3 which doesn't throw me any error.
Assuming that you encounter an error while trying to read a norwegian (or) swedish character, I have tried reading a website which has norwegian characters in it, and with the below code, I'm able to output it to the console without any error. Same with a swedish page too. I'm using 0.2.3 version though (which is the most recent version). I have not set the encoding to UTF-8 too.
var requestify = require('requestify');
requestify.get('http://www.norwegian4people.com/lesson.php?id=41').then(function(response) {
// Get the response body
console.log(response.getBody());
});
Hope this helps!

Python - Parse requests response

I've an endpoint where I post the data.
r = requests.post(url, data=data, headers=headers)
I get the following response in Javascript:-
throw 'allowScriptTagRemoting is false.';
(function() {
var r = window.dwr._[0];
//#DWR-INSERT
//#DWR-REPLY
r.handleCallback("1", "0", {
msg: "",
output: {
msg: "Showing from city center",
resultcode: 1,
scrpresent: true,
srclatitude: "28.63244546123956",
srclongitude: "77.21981048583984",
},
result: "success"
});
})();
How do I parse the response? I basically want the output json. HOw can I get the same?

The problem is - output is not a valid JSON string and cannot be loaded via json.loads().
I would use regular expressions to locate the output object and then use findall() to locate key-value pairs. Sample working code:
import re
data = """
throw 'allowScriptTagRemoting is false.';
(function() {
var r = window.dwr._[0];
//#DWR-INSERT
//#DWR-REPLY
r.handleCallback("1", "0", {
msg: "",
output: {
msg: "Showing from city center",
resultcode: 1,
scrpresent: true,
srclatitude: "28.63244546123956",
srclongitude: "77.21981048583984",
},
result: "success"
});
})();
"""
output_str = re.search(r"output: (\{.*?\}),\n", data, re.DOTALL | re.MULTILINE).group(1)
d = dict(re.findall(r'(\w+): "?(.*?)"?,', output_str))
print(d)
Prints:
{'msg': 'Showing from city center', 'resultcode': '1', 'srclongitude': '77.21981048583984', 'srclatitude': '28.63244546123956', 'scrpresent': 'true'}

Why data received is different to sent by Electron event?

I'm trying make CRUD operation in Electron, where using MongoDB, and Mongoose for Electron connect to database.
I have a function list for return the "return" of Mongoose. When I debugging return data, is show correct in my terminal
ipc.on('products.list', function(event, query) {
Service.list(query, function(err, data) {
console.log(data); // this will be print correct result
event.sender.send('product.list', data);
});
});
Result of code upper part:
[ { _id: 55c15f2981260a6f0ef8a657,
__v: 0,
deleted_at: '',
updated_at: '',
created_at: Tue Aug 04 2015 21:51:52 GMT-0300 (BRT),
measure: '',
aplication: '',
model: 'Mode XBA',
reference: 'Reference test 123',
code: '2028' } ]
But when I debug return data by client side with fallowing code, result is not correct and is not equal server side print.
Client side code:
ipc.on('product.list', function(data) {
window.data = data;
console.log(data); // this will print incorrect result
});
ipc.send('products.list', {});
Result of client side received:
{
"$__":{....},
"_doc":{...},
"_posts":{...},
"_pres":{...},
"isNew":false
}
Contains more objects in each element.
Why this happens? How to can I resolve this problem?

I resolved this problem!
This return is result from Mongoose
{
"$__":{....},
"_doc":{...},
"_posts":{...},
"_pres":{...},
"isNew":false
}
For return only property, is need add options {lean : true}
Ex:
Model.find(this.query)
.lean()
.exec(function(err, data) {}
);
More detail in Mongoose Doc

emailjs not working [node js]

i'm trying to use the emailjs (https://github.com/eleith/emailjs) for send emails with nodejs, but, it gives me an error: { [Error: timedout while connecting to smtp server] code: 4, smtp: undefined }
I'm trying to use hotmail, but I can use other, I only want this working. Any idea please?
var email = require("emailjs");
var server = email.server.connect({
user: "xxx#hotmail.com",
password:"xxxyyyy",
host: "smtp-mail.live.com",
ssl: true
});
// send the message and get a callback with an error or details of the message that was sent
server.send({
text: "i hope this works",
from: "you <xxxxxx#hotmail.com>",
to: "someone <zzzzz#hotmail.com>",
//cc: "else <else#your-email.com>",
subject: "testing emailjs"
}, function(err, message) { console.log(err || message); });

I tested with the code below (using gmail) and it is working:
var email = require('emailjs');
var server = email.server.connect({
user: 'nodejsiscool#gmail.com',
password: 'stackoverflow',
host: 'smtp.gmail.com',
ssl: true
});
server.send({
text: 'Hey howdy',
from: 'NodeJS',
to: 'Wilson <wilson.balderrama#gmail.com>',
cc: '',
subject: 'Greetings'
}, function (err, message) {
console.log(err || message);
});
In my console the output is:
{
attachments: [],
alternative: null,
header: {
'message-id': '<1433256538447.0.5970#Wilsons-MacBook-Pro.local>',
date: 'Tue, 02 Jun 2015 10:48:58 -0400',
from: '=?UTF-8?Q?NodeJS?= <>',
to: '=?UTF-8?Q?Wilson?= <wilson.balderrama#gmail.com>',
cc: '',
subject: '=?UTF-8?Q?Greetings?='
},
content: 'text/plain; charset=utf-8',
text: 'Hey howdy'
}
Indeed I have received in my inbox the email message.
I suspect that you should use the host smtp.live.com instead of smtp-mail.live.com
Note: The account (nodejsiscool#gmail.com) used is a valid one I just created for testing purposes.

Using Gmail
First: Required activate the use less secure apps at your own risk as indicated in the Gmail documentation. I use an account not very important for my tests:
https://myaccount.google.com/u/2/lesssecureapps?pageId=none
After:
var email = require("emailjs");
var server = email.server.connect({
user: "YOUR_ACCOUNT#gmail.com",
password:"YOUR_PASSWORD",
host: "smtp.gmail.com",
ssl: true
});
server.send({
text: "Hello world",
from: "YOUR_NAME <YOUR_ACCOUNT#gmail.com>",
to: "FRIEND_NAME <FRIEND_ACCOUNT#gmail.com>",
subject: "Hello"
}, function(err, message) { console.log(err || message); });
Additional Note: About how set "from" and displayed name in Gmail inbox:
Example 1:
Set "from" with only name.
from: "YOUR_NAME", // Only name
In inbox display <>
YOUR_NAME <> Hello 18:11
Example 2:
Set "from" with name and emial.
from: "YOUR_NAME <YOUR_ACCOUNT#gmail.com>", // Name and email
In inbox not display <>
YOUR_NAME Hello 18:11

Make sure you are linking the emailjs files correctly (check var email) or it won't work.
//Email stuff
var email = require("./node_modules/emailjs/email");
var server = email.server.connect({
user: "EMAIL#gmail.com",
password:"INSERTPASSWORDHERE",
host: "smtp.gmail.com",
ssl: true
});
//If button is clicked then send email
server.send({
text: 'Hello World! \n\n Regards,\n INSERTNAMEHERE \n',
from: 'Name',
to: 'Name <EMAIL#gmail.com>',
cc: '',
subject: ''
}, function (err, message) {
console.log(err || message);
});

make sure you have installed exact package with same version
npm i emailjs#1.0.11
and try below code
var email = require("emailjs");
var server = email.server.connect({
user: "your-email#gmail.com",
password: "your-password",
host: "smtp.gmail.com",
ssl: true
});
server.send(
{
text: "Hey howdy",
from: "NodeJS",
to: "Wilson <your-email#gmail.com>",
cc: "",
subject: "Greetings"
},
function(err, message) {
console.log(err || message);
}
);
seems latest version is not working its using es6 exports "emailjs#1.0.11" but version still works fine .
also don't forget to enable less secure setting for email address you are using with password .

I think you can use this version
emailjs#2.2.0
,
For me, this is working in version 2.2.0
where version 3.0 and above will show the error.
npm i emailjs#2.2.0
here is the example