I am unable to post a yaml throught http in js. The code works when I used JSON.stringify(text) for body and have content-type application/json. If I use YAML stringify, body on server side is just {}.
I have areatext where I enter a text (like yaml format) in html e.g.
martin:
name: Martin D'vloper
job: Developer
skill: Elite
Client:
$('#create-yaml2').on('click',async function () {
var text = $('#yaml-create').val();
// console.log(text)
var textYAML = YAML.stringify(text);
var options = {
hostname: 'myhostname',
port: 80,
path: `/api/postyaml`,
method: 'POST',
body: textYAML,
headers: {
'Content-Type': 'text/x-yaml'
}
};
var executeReq = http.request(options);
executeReq.write(textYAML);
executeReq.end();
});
EDIT:
Function in server side, that prints not an empty {} when posting JSON.
exports.functionCalled = async function (req, res) {
console.log('\n\n\n\n')
console.log(req.body)
console.log('\n\n\n\n')
try {
res.send(`RECEIVED`);
} catch (upgradeErr) {
res.status(422).json({
message: `Failed`,
error: upgradeErr
});
}
}
You already have a string coming from HTML, so you don't need to call YAML.stringify once again - text is already a string.
$('#create-yaml2').on('click',async function () {
var text = $('#yaml-create').val();
// console.log(text)
var textYAML = text;
var options = {
hostname: 'myhostname',
port: 80,
path: `/api/postyaml`,
method: 'POST',
body: textYAML,
headers: {
'Content-Type': 'text/x-yaml'
}
};
var executeReq = http.request(options);
executeReq.write(textYAML);
executeReq.end();
});
You may want to do something like
$('#create-yaml2').on('click',async function () {
var text = $('#yaml-create').val();
try {
YAML.parse(text)
} catch(e) {...}
...
send request
to make sure there was a valid YAML provided
YAML.stringify converts a JavaScript data structure to a string containing YML.
You don't have a JavaScript data structure, you just a string containing YML.
Almost. You have an error in it. You can't use a raw ' in a YML string that isn't quoted.
So:
Fix your YML:
martin:
name: "Martin D'vloper"
job: Developer
skill: Elite
Don't double encode it:
var textYAML = $('#yaml-create').val();
Related
I'm trying to use the JS Mailgun API to send emails. Have it working fine, until I throw template variables into 'h:X-Mailgun-Variables' like so, where jsonString is very large (17000+ characters):
const mailData = {
from: 'Insights <insights#hello.net>',
to: mailAddress,
subject: `Insights: ${DAYS_OF_WEEK[date.getDay()]}, ${MONTHS[date.getMonth()]} ${ordinal_suffix_of(date.getDate())} ${date.getFullYear()}`,
template: "template1",
'h:X-Mailgun-Variables': jsonString,
};
Looking at the documentation here states the following:
Note The value of the “X-Mailgun-Variables” header must be valid JSON string,
otherwise Mailgun won’t be able to parse it. If your X-Mailgun-Variables
header exceeds 998 characters, you should use folding to spread the variables
over multiple lines.
Referenced this post, which suggested I "fold" up the JSON by inserting CRLF characters at regular intervals. This led me here, which still does not work, though logging this does show regular line breaks and is compliant JSON:
const jsonString = JSON.stringify(templateVars).split('},').join('},\r \n');
Any insight into how to properly "fold" my JSON so I can use large template variables in my MailGun emails?
UPDATE:
As requested, adding my code. This works when data only has a few companies/posts, but when I have many companies each with many posts, I get the 400 error:
function dispatchEmails(data) {
const DOMAIN = 'test.net';
const mg = mailgun({apiKey: API_KEY, domain: DOMAIN});
const templateVars = {
date: theDate,
previewText: 'preview',
subject: 'subject',
subhead: 'subhead',
companies: data.companies.map(company => {
return {
url: company.url,
totalParts: data.totalParts,
currentPart: data.currentPart,
companyData: {
name: company.name,
website: company.website,
description: company.description
},
posts: _.map(company.news, item => {
return {
category: item.category,
date: new Date(item.date),
url: item.sourceUrl,
title: item.title,
source: item.publisherName,
description: item.description,
}
})
}
})
};
const jsonString = JSON.stringify(templateVars).split('},').join('},\r \n');
const mailData = {
from: 'test#test.com',
to: 'recipient#test.com',
subject: 'subject',
template: 'template',
'h:X-Mailgun-Variables': jsonString
};
return mg.messages().send(mailData)
.then(body => {
return body;
})
.catch(err => {
return {error: err};
});
}
I think your problem may be overall payload size rather than the string folding. The folding for strings exceeding 998 characters seems to be handled by the node.js client, possibly by the form-data package.
I ran the following test:
test_big_data (template created in mailgun through the UI)
<!DOCTYPE html>
<html>
<body>
<h2>An HTML template for testing large data sets.</h2>
<ul>
<li>{{param_name_0}}</li>
... other lis ...
<li>{{param_name_999}}</li>
</ul>
</body>
</html>
send_email.js
const API_KEY = 'MY_KEY';
const DOMAIN = 'MY_DOMAIN.COM';
const formData = require('form-data');
const Mailgun = require('mailgun.js');
const mailgun = new Mailgun(formData);
const client = mailgun.client({ username: 'api', key: API_KEY });
const bigData = {};
for (let i = 0; i < 400; i++) {
bigData[`param_name_${i}`] = `param_value_${i}`;
}
const dataString = JSON.stringify(bigData);
console.log(dataString.length);
const messageData = {
from: 'Mailgun Sandbox <postmaster#DOMAIN>',
to: 'test#MY_DOMAIN.COM',
subject: 'Big Data Test',
template: 'test_big_data',
'h:X-Mailgun-Variables': dataString,
};
client.messages
.create(DOMAIN, messageData)
.then((res) => {
console.log(res);
})
.catch((err) => {
console.error(err);
});
The length of the dataString in this case is 13781 characters and the email queues successfully.
However, if I bump the for loop condition to i < 1000 I get the following error when queueing the email:
[Error: Bad Request] {
status: 400,
details: '{"message":"Send options (parameters starting with o:, h:, or v:) are limited to 16 kB total"}\n'
}
When I asked Mailgun support about the folding warning form the documentation they pointed me to RFC 2822 section "3.2.3. Folding white space and comments". But like I said, I don't think folding is the issue here.
Cheers!
https://datatracker.ietf.org/doc/html/rfc2822#page-11
Just thinking outside the box but why pass that much data in an email header? I assume you have something on the receiving end which is going to parse the email headers. What if instead of sending them that data you send them a key that they can call back into an API on your end to get the data
I am having a problem using Axios with node js. Here is my code
let callResult = await axios.get(urlData, config)
Where config is
let config = {
headers: {
'X-Token': token
}
};
And the urlData is
let urlData = 'https://api.regulaforensics.com/webapi/Transaction2/GetTransactionResultJson?transactionId=<IDVariable>&resultType=15'
I am trying to add my IDVariable to the URL but it does not work. If I take the variable and put it directly in the URL I get a response.
I have also tried this
let config = {
headers: {
'X-Token': token
},
params: {
transactionId: IDVariable,
resultType: 15
}};
And this
let querys = querystring.stringify({ transactionId: keyId, resultType: 15 })
let path = 'https://api.regulaforensics.com/webapi/Transaction2/GetTransactionResultJson?'
let urlData = path.concat("", querys)
This is the complete URL
https://api.regulaforensics.com/webapi/Transaction2/GetTransactionResultJson?transactionId=05cc6ccc-3ae6-4185-b2c9-1e1aba01d705&resultType=15
When using {params: } or concatenation
When putting the whole URL. As the URL I pasted above
This is my whole function
If you want to pass some parameters as a query string you can use the following syntax for the GET request:
axios.get('/user', { params: { ID: 12345 } });
Tath will be translated in the following request:
/user?ID=12345
As explained in the documentation here: https://github.com/axios/axios#note-commonjs-usage
Note
You don't need to add the ? char at the end of URL and you don't need to have the parameters in the URL, so the part ?transactionId=<IDVariable>&resultType=15 must be removed
I found the solution.
I am calling a service first and the response of that service is the key I need for the second service.
The solution was to just put a sleep() between those two services.
Thanks you for your response, guys!
I'm trying the Wikipedia client login flow depicted in the API:Login docs, but something wrong happens:
1) I correctly get a token raised with the HTTP GET https://en.wikipedia.org/w/api.php?action=query&meta=tokens&type=login&format=json
and I get a valid logintoken string.
2.1) I then try the clientlogin like:
HTTP POST /w/api.php?action=clientlogin&format=json&lgname=xxxx&lgtoken=xxxx%2B%5C
and the POST BODY was
{
"lgpassword" : "xxxxx",
"lgtoken" : "xxxxx"
}
But I get an error:
{
"error": {
"code": "notoken",
"info": "The \"token\" parameter must be set."
},
"servedby": "mw1228"
}
If I try to change lgtoken to token I get the same result.
2.2) I have then tried the old method i.e. action=login and passing the body, but it does not work, since it gives me back another login token: HTTP POST https://en.wikipedia.org/w/api.php?action=login&format=json&lgname=xxxx
and the same POST BODY
I then get
{
"warnings": {}
},
"login": {
"result": "NeedToken",
"token": "xxxxx+\\"
}
where the docs here states that
NeedToken if the lgtoken parameter was not provided or no session was active (e.g. your cookie handling is broken).
but I have passed the lgtoken in the json body as showed.
I'm using Node.js and the built-in http module, that is supposed to pass and keep session Cookies in the right way (with other api it works ok).
I have found a similar issue on a the LrMediaWiki client here.
[UPDATE]
This is my current implementation:
Wikipedia.prototype.loginUser = function (username, password) {
var self = this;
return new Promise((resolve, reject) => {
var cookies = self.cookies({});
var headers = {
'Cookie': cookies.join(';'),
'Accept': '*/*',
'User-Agent': self.browser.userAgent()
};
// fetch login token
self.api.RequestGetP('/w/api.php', headers, {
action: 'query',
meta: 'tokens',
type: 'login',
format: 'json'
})
.then(response => { // success
if (response.query && response.query.tokens && response.query.tokens['logintoken']) {
self.login.logintoken = response.query.tokens['logintoken'];
self.logger.info("Wikipedia.login token:%s", self.login);
return self.api.RequestPostP('/w/api.php', headers, {
action: 'login',
format: 'json',
lgname: username
},
{
lgpassword: password,
lgtoken: self.login.logintoken
});
} else {
var error = new Error('no logintoken');
return reject(error);
}
})
.then(response => { // success
return resolve(response);
})
.catch(error => { // error
self.logger.error("Wikipedia.login error%s\n%#", error.message, error.stack);
return reject(error);
});
});
}//loginUser
where this.api is a simple wrapper of the Node.js http, the source code is available here and the api signatures are like:
Promise:API.RequestGetP(url,headers,querystring)
Promise:API.RequestPostP(url,headers,querystring,body)
If the currently accepted answer isn't working for someone, the following method will definitely work. I've used the axios library to send requests. Any library can be used but the key lies in formatting the body and headers correctly.
let url = "https://test.wikipedia.org/w/api.php";
let params = {
action: "query",
meta: "tokens",
type: "login",
format: "json"
};
axios.get(url, { params: params }).then(resp => {
let loginToken = resp.data.query.tokens.logintoken
let cookie = resp.headers["set-cookie"].join(';');
let body = {
action: 'login',
lgname: 'user_name',
lgpassword: 'password',
lgtoken: loginToken,
format: 'json'
}
let bodyData = new URLSearchParams(body).toString();
axios.post(url, bodyData, {
headers: {
Cookie: cookie,
}
}).then(resp => {
// You're now logged in!
// You'll have to add the following cookie in the headers again for any further requests that you might make
let cookie = resp.headers["set-cookie"].join(';')
console.log(resp.data)
})
})
And you should be seeing a response like
{
login: { result: 'Success', lguserid: 0000000, lgusername: 'Username' }
}
The second post request was where I got stuck for several hours, trying to figure out what was wrong. You need to send the data in an encoded form by using an API like URLSearchParams, or by just typing up the body as a string manually yourself.
I think from what you are saying you have lgtoken and lgname in the URL you are using, and then lgpassword and lgtoken (again!) in a JSON-encoded POST body.
This is not how the Mediawiki API works.
You submit it all as POST parameters. JSON is never involved, except when you ask for the result to come back in that format. I can't help you fix your code as you don't provide it, but that's what you need to do. (If you edit your question with your code, I'll do my best to help you.)
After seeing your code, I'll presume (without knowing the detail of your code) that you want something like this:
return self.api.RequestPostP('/w/api.php', headers, {
action: 'login',
format: 'json',
lgname: username,
lgpassword: password,
lgtoken: self.login.logintoken
});
I am making a web app With Polymer. I make a POST request and then fill an object, then I'm making a databind to fill the data attribute needed in the google-chart tag I'm using.
<dom-module id="leads-chart" bind>
<template>
<google-chart
id="leadschart"
type='column'
data={{datagraph}}>
</google-chart>
</template>
<script>
Polymer({ is: "leads-chart",
properties:{
params:{
type: Object
},
command:{
type: String,
value: "SP_GetLead12Meses"
},
datagraph:{
type: Object,
computed: 'getBargraph(command,params)'
}
},
getBargraph: function(command,params){
var options = {
hostname: 'localhost',
path: '/',
port: 8081,
secure: false,
method: 'POST',
headers: {
'x-powered-by': 'HTTPClient.js'
},
'Content-Type': 'application/json'
}
var dictionary = new Object();
var outter = [];
const that = this;
var example1 = new HTTPClient(options)
example1.post("/executeGraph1?command="+ command + "¶m1=" + params.user, function (err, res, body) {
body = JSON.parse(body);
outter.push(['Mes','Leads',{"role":"style"}]);
for(var k in body){
var inner = [];
inner.push(dictionary[body[k]['MES']]);
inner.push(body[k]['LEADS']);
inner.push("#00A6d4");
outter.push(inner);
}
});
return outter;
}
});
The problem is the chart is not showing any data. The browser shows the following message:
The weirdest thing is that this error at the beggining was present in rare ocations but now I get it every time I load the page. Does someone know why is it happening and how to solve it?
I have made some debugging and found out the post message is answering the request with the correct information.
the object is being filled with data but the message is still there and chart is still empty
I am trying to create one Node js server with http package. I want to receive only POST request which I have already implemented it. The problem which I am facing is that I am not able to parse JSON correctly (I am expecting one JSON to be attached).
I tried using JSON.parse but that doesn't parse whole json content. It leaves some values as [Object] which is wrong. I saw few packages which is JSONStream but I am not sure how to implement in this case.
server.on('request', function(req, res){
if(req.method == 'POST')
{
var jsonString;
req.on('data', function (data) {
jsonString = JSON.parse(data);
});
req.on('end', function () {
serverNext(req, res, jsonString);
});
}
else
{
res.writeHead(405, {'Content-type':'application/json'});
res.write(JSON.stringify({error: "Method not allowed"}, 0, 4));
}
res.end();
});
Request example:
Here d = JSON file content. (I did this in Python to make this example request)
r = requests.post('http://localhost:9001', headers = {'content-type': 'application/json'}, data = json.dumps(d))
Note: I am able to parse JSON correctly but there are some cases when it shows something like this:
{ 'Heading':
{ 'Content':
{ sometext: 'value',
List: [Object], // Wrong
test: [Array] } } } // Wrong
Update:
Inside serverNext() I am getting few values like:
var testReq = Object.keys(jsonData)[0];
var testId = Object.keys(jsonData[testRequest])[0];
var test = jsonData[testRequest][testId]
Further if I keep on extracting values then at some point it encounters [Objects] value and get crashed.
I can reproduce this "problem" with data as { "Foo": {"Bar": {"Some data": [43, 32, 44]} } } -- it returns the following result: { Foo: { Bar: { 'Some data': [Object] } } }.
As OP mentioned in question, the JSON is parsed correctly, the reason why [Object] is displayed in result is: when JavaScript Object is returned to display, it would be converted to String first by toString() automatically, which will make all object (including array) as [Object] in text.
To display the real content, JSON.stringify() need to be invoked. In your case, the code would be:
req.on('end', function () {
serverNext(req, res, JSON.stringify(jsonString));
});
Please note it is better to rename variable jsonString as jsonObject.