Maybe this is simple, maybe this is a bug on Parse - would like to know if anyone has had the same problem and a possible solution.
What I'm trying to do:
I'm sending a JSON request from an app called FormEntry to my Parse app
The body comes in like this: json={"someLabel" : "someValue"}
I would like to take the entire body and create a Parse.Cloud.httpRequest over to Zapier to perform some functions.
Now, the problem seems to be this:
On random occasions (i.e. I have no idea why), the body is sent (as shown by the logs) where there is a trailing comma at the end of the last pair in the JSON object. e.g. like this json={"lastLabel" : "lastValue",}
The number of elements in 'normal' and 'incorrect' objects seem to be the same, so it's simply just another comma added. And I have no idea why.
My setup:
Using app.use(parseExpressRawBody()); only and not the standard app.use(express.bodyParser()); which doesn't provide access to the raw body.
Because parseExpressRawBody converts the body to a buffer I need to turn it back into a string to send it in the HTTP request in a meaningful way. Therefore I use: var body = req.body.toString();
When logging this var to the Parse console it looks to be format back from the buffer fine.
And that's about it. Nothing complex going on here but a real annoying bug that I just haven't found a sensible way of understanding. Would SUPER appreciate anyone who has seen this before or who could point me in a direction to focus on.
Just an update on this. Not a solution that answers why there is malformed JSON but a hack to get the right result.
The purpose of the HTTP request was to point over to Zapier so I wrote a Zapier script that would deal with the malformed JSON. Added here for anyone else who needs it.
"use strict";
var Zap = { newSubmission_catch_hook: function(bundle) {
var body = bundle.request.content;
var cleanTop = body.substring(5,body.length);
var cleanChar = cleanTop.length;
var condition = cleanTop.substring(cleanChar-2,cleanChar);
function testCase(condition,cleanTop) {
if (condition != ",}"){
console.log("Everything is fine, returning JSON");
return cleanTop;
}
else {
console.log("Nope! We have an error, cleaning end");
var cleanEnd = cleanTop.substr(0,cleanChar-2) + '}';
console.log("The object now ends with: " + cleanEnd.substr(-10));
return cleanEnd;
}
}
var newBody = JSON.parse(testCase(condition,cleanTop));
return newBody;
}
};
Related
I have written a POSTMAN call to a server that responds with a list of items in JSON like below:-
{
"count": 6909,
"setIds": [
"1/7842/0889#001",
"2/4259/0166#001",
"ENT0730/0009",
"9D/11181#002",
"1/9676/0001#001",
"2/4718/0001#004",
"2/1783/0044#001",
"1/4501/0001#001",
"1/2028/0002#002",
"2/3120/0079#001",
"2/1672/0024#001",
"2/3398/0064#001"
}
I want to make calls to another server using the value of the setID each time and iterate through all of these so that I end up calling the server thousands of times to verify the response from that server. The problem I have is that the second server expects the set id to be in a form where the forward slashes are converted to underscores and the hashes to dots, so
"1/7842/0889#001"
becomes
"1_7842_0889.001"
I have code that converts one to the other in POSTMAN
var jsonData = pm.response.json()
for (var i=0; i<jsonData.setIds.length; i++)
{
var new_j = jsonData.setIds[i].replace (/#/g, ".");
var new_i = new_j.replace (/\//g, "_");
}
})
This works fine line by line it creates the right thing in the console of POSTMAN but obviously what I really need to do is save the entire JSON in the right form to a file and then read from that file line by line using the corrected data. I don't seem to be able to save the data in a file in the right form using my code and I suspect I am missing something simple. Is there a way to write a file line by line from in side postman or in a script and manipulate the data as I'm creating it?
Alternatively I guess I could read from the JSON I have saved i.e. the full response and iterate through that manipulating the data as a pre-request script?
I have tried to do something like this using environmental variables - so in my first call I do:
var jsonData = JSON.parse(responseBody);
postman.setEnvironmentVariable('setIds', JSON.stringify(jsonData));
and then in my second call to the express server where I want to send my payload I run a pre-request script that I thought would work using the env variable but this fails as it doesn't seem to like the {...
SyntaxError: Unexpected token {
I think there are probably some neat ways of solving this either doing all of this outside of POSTMAN in javascript but I'm a little lost where to start. Any help appreciated
Would tell you are plaing with content, but not setting it back to JSON object ??
jsonData.setIds[i] = new_i;
can help or you can use 2x replace it in a string and convert back to make it easier (in case there are no / or # somewhere else).
var src = {
"count": 6909,
"setIds": [
"1/7842/0889#001",
"2/4259/0166#001",
"ENT0730/0009",
"9D/11181#002",
"1/9676/0001#001",
"2/4718/0001#004",
"2/1783/0044#001",
"1/4501/0001#001",
"1/2028/0002#002",
"2/3120/0079#001",
"2/1672/0024#001",
"2/3398/0064#001"
//...
]
}
var str = JSON.stringify(src, null, 4);
str = str.replace(/\//g,'_').replace(/#/g,'.');
console.log(str);
var res = JSON.parse(str);
console.log(res);
Goal
Transform HTML extracted from Telligent (an extranet platform) to plain text and send to Slack
Setup
A Telligent webhook is triggered when an event occurs. An Azure Logic App receives the event JSON. The JSON value is in HTML. A JavaScript Azure Function inside the Azure Logic App pipeline transforms the JSON value to plain text. The final step in the pipeline posts the plain text in Slack.
Example of incoming code to the Azure Function
"body": "<p>" '</p><div style=\"clear:both;\"></div>"
Transformation method
This is the basic code in the Azure Function. I have left out parts that seem irrelevant to this question but can provide the entire script if that is necessary.
module.exports = function (context, data) {
var html = data.body;
// Change HTML to plain text
var text = JSON.stringify(html.body);
var noHtml = text.replace(/<(?:.|\n)*?>/gm, '');
var noHtmlEncodeSingleQuote = noHtml.replace(/'/g, "'");
var noHtmlEncodeDoubleQuote = noHtmlEncodeSingleQuote.replace(/"/g, "REPLACEMENT");
// Compile body for Slack
var readyString = "Slack text: " + noHtmlEncodeDoubleQuote;
// Response of the function to be used later
context.res = {
body: readyString
};
context.done();
};
Results
The single quote is replaced successfully and resolves accurately when posted in Slack.
The following replacement methods for the double quote throw a Status: 500 Internal Server Error within the Azure Function.
Unsuccessful replacement methods
"\""
'"'
"
"'"'"
"["]"
"(")"
Putting these replacement methods in their own var also throws the same error. E.g.:
var replace = "\""
...
var noHtmlEncodeDoubleQuote = noHtmlEncodeSingleQuote.replace(/"/g, replace);
The code appears to be correct because when I replace " with something like abc, the replacement is successful.
Thank you
Please forgive my JavaScript as I am not a programmer and am seeking to streamline a process for my job. However I am grateful for any advice both about the code or my entire approach.
Generally, you don't want to try to parse HTML with regular expressions or string replacement. There are just too many things that can go wrong. See this now famous StackOverflow answer. (It was even made into a T-Shirt.)
Instead, you should use a technique that is purposefully built for this purpose. If you were in a web browser, you could use the techniques described in the answers to this question. But in Azure Functions, your JavaScript doesn't run in a browser, it runs in a Node JS environment. Therefore, you need will need to use a library such as Cheerio or htmlparser2 (and others).
Here is an example using Cheerio:
var cheerio = require('cheerio');
var text = cheerio.load(html.body).text();
Also, regarding this part:
... as I am not a programmer ...
Yes you are. You are clearly programming right now. Anyone who writes code is a programmer. There is no club or secret handshake. We all start out exactly like this. Good job asking questions, and good luck on your journey!
Im building a windows 8 app (html)
And have a api im fetching data from.
I keep getting this error however
0x800a138f - JavaScript runtime error: Unable to get property 'json' of undefined or null reference
in my scripts1.js file. then my program crashes -_-.
Here is the the code i use
$(function () {
startRefresh();
});
function startRefresh() {
setTimeout(startRefresh, 10000);
var url = 'http://pubapi.cryptsy.com/api.php?method=singlemarketdata&marketid=132';
$.getJSON('http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20json%20where%20url%3D%22' + encodeURIComponent(url) + '%22&format=json', function (data) {
jQuery('#ticker').html(data['query'].results.json.return.markets.DOGE.lasttradeprice);
jQuery('#ticker').append(' ' + data['query'].results.json.return.markets.DOGE.secondarycode);
jQuery('#ticker2').html(data['query'].results.json.return.markets.DOGE.lasttradetime);
jQuery('#ticker3').html(data['query'].results.json.return.markets.DOGE.volume);
});
}
This is located in scripts1.js Then i use ect.
It works everything comes and displays just get that error. Not sure what to do.
Seems like data['query'].results is undefined. Pasting the JSON you get will help.
Also, one small piece of advice: If you are going to access an in-depth attribute and, specially, a DOM node several times, you might want to recycle a reference to it rather than travelling again and again to fetch it for performance reasons. Something like this:
var ticker = jQuery('#ticker');
var DOGE = data['query'].results.json.return.markets.DOGE;
ticker.html(DOGE.lasttradeprice);
...
It looks like, occasionally, the API will send back some JSON that, when parsed, doesn't contain a results object. To mitigate this you should put a condition in to catch this eventuality.
if (data.query.results) {
jQuery('#ticker').html(data['query'].results.json.return.markets.DOGE.lasttradeprice);
// rest of DOM update code
}
Demo.
Having issues with Jade and the way the data is passed to it when it is rendered.
I am trying to save the data which is in [{key1: "val1", key2: "val2"}, ...}];
format but having issues as it shows up as the result below.
Result
key: xyz value:[{"artist":"Lady Gaga",...
This is the code I am working with on the server-side Node.js which is passing it fine ...
res.render('musics', {
title: 'site',
result: JSON.stringify(result)
});
This is the code I am having issues with because of the way I have to call result in jade...
script
function local (arr) {
var i;
i = "#{result}";
localStorage.setItem('xyz', i);
}
console.log('stored');
local();
The quotes around result are messing it up but without them I get an error for unexpected identifier...
Any suggestions or if it might be better to go an ajax route through Backbone(which is what I am using with the client-side) I am willing to, just to put some pointers out - the data is being scraped and through selections of a form post - so the data comes back after the post and is a on time transfer, so if I did an ajax call it would have to include the post and the get, otherwise i am not sure of how to receive it... maybe res.json(result) on the server side, but then the page needs to render somehow... Open to suggestions. Thanks! Ultimately I want it to go into localStorage without the " around everything.
your jade snippet should look like this then:
script!= "(function() {localStorage.setItem('xyz',JSON.stringify(" +result + ");})();"
by using != you tell jade to not escape the following content, and on the clientside you have to stringify again before puting your data to local storage.
As an improvement to #greelgork's answer:
This is for JSON array
script!= "(function() {var items = []; items = JSON.parse(localStorage.getItem('Stored_items')); console.log(JSON.stringify(items)); items.push(" + JSON.stringify(product) + "); localStorage.setItem('Stored_items', JSON.stringify(items)); })();"
Anyways, pushing an item into localStorage needs to be stringified before inserted into localStorage hence, #greelgorke's answer should be modified so:
single item
script!= "(function() {localStorage.setItem('xyz',JSON.stringify(result)); })();"
So the JSON.stringify is outside the string just like all the other javascript code is,
This is what I use in my project and it worx
Credit Push JSON Objects to array in localStorage
if usersList.length
script.
const userList = !{JSON.stringify(usersList)}
localStorage.setItem('xyz',JSON.stringify(userList))
const loader = document.querySelector(".loader");
loader.className +=" hidden";
I have a server side function which returns content of HTML page:
if (Meteor.isServer) {
Meteor.startup(function () {
// code to run on server at startup
Meteor.methods({
sayHello: function() {
var response = Meteor.http.call("GET", "http://google.com");
return response;
}
});
});
And I have client code where I am trying to get title from this HTML page:
'click .add_tag' : function(e,t) {
//Session.set('editing_tag_id', e.target.id);
Meteor.call("sayHello", function(err, response) {
var title = $(response.content).find("title").text();
var title2 = $(response).find("title").text();
var title3 = response.content.match(/<title[^>]*>([^<]+)<\/title>/)[1];
alert(title3);
});
I would like to get jQuery version ('title' or 'title2'), but it doesn't works. It returns empty string.
'Title3' - version works fine, but I don't like regexps. :)
Is there any way to make 'jQuery'-versions works ?
As requested, I will reiterate my comment as an answer...
I would stick with the regex, even though you don't like it. There is a huge overhead of constructing a DOM element that is essentially an entire page, purely for the purpose of parsing a small amount of text. The regex is more lightweight and will perform adequately in slower browsers or on slower machines.
Wrap response.content in a <div> and then do a selection off of that. This way you have a proper structure to start from rather than an array that you might actually be getting.
var $wrap = $("<div></div>").html(response.content);
$wrap.find("title").text();
An example of what is probably going on: http://jsfiddle.net/UFtJV/
Don't forget one thing : you should never return HTML to client. You should return Json (or even Xml) that your client will transform into Html using Template.
You are doing like a lot of dev doing Bad Ajax.
Don't forget : "only data on wire, not display".
So there should not be any problem coz, on response you just have to take data from Json formatted response and inject it inside your Template.