We Keep Coding

Can't get JavaScript and Python strings to be the same because of backslash escaping

I'm trying to make a POST web request work for 24 hours now. But something is messing up my body string. I have to call a JavaScript function within a Python Program (to run the encryption function which is used by the webiste I'm trying to request) but the string processed by JavaScript isn't the same as the string returned by JavaScript (using js2py library for executing JavaScript within Python).
I have an object like this one:
let enc = {
data: {
"endpointId": 17273,
params: {
"DEBUG_": false,
"userId": String(userId),
"userIsOnline": true,
"lang": "en",
"locale": "en#US"
}
}
The final body for the POST request must look like this (and exactly like this because on the server side the data will be decrypted by using the same key as I use within the JavaScript function):
{"endpointId":17273,"params":"{\"DEBUG_\":false,\"userId\":\"338de884bc1f41818dfae7a8b19f5613\",\"userIsOnline\":false,\"lang\":\"en\",\"locale\":\"en#US\"}"}
I don't know how the put just one backslash (like "DEBUG_") everything I'm trying it will be two or three backslashes (and the furthermore the backslashes should be only within the params object, not at the appId).
And another problem is that if I return the string to python console.log(n.data) is not matching return n.data:
console.log("Encrypting: " + n.data);
> Encrypting: {"endpointId":17273,"params":"{\\"DEBUG_\\":false,\\...
(as you can see two backslashes here!)
JavaScript:
return n.data
Python:
body = js2py.eval_js(Constants.sign_function + f"make('{input}');");
print(f"# Returned: {body}")
Returned: {"endpointId":17273,"params":"{\"DEBUG_\":false,\\..
And the problem here is now if I send the web request with the returned body like:
req = requests.post(url, data=body, headers=headers)
The response will be 403 forbidden because the sent data is not the encrypted data (because both differ in backslashes within the string)
I tried a lot of things so far (build the body from a string, build the body from dict, build the body within JavaScript, build the body within Python...) but can't make it work. The backslashes always kill my efforts in the end.

POSTMAN - EXPRESS - Modify JSON before writing to file

I have written a POSTMAN call to a server that responds with a list of items in JSON like below:-
{
"count": 6909,
"setIds": [
"1/7842/0889#001",
"2/4259/0166#001",
"ENT0730/0009",
"9D/11181#002",
"1/9676/0001#001",
"2/4718/0001#004",
"2/1783/0044#001",
"1/4501/0001#001",
"1/2028/0002#002",
"2/3120/0079#001",
"2/1672/0024#001",
"2/3398/0064#001"
}
I want to make calls to another server using the value of the setID each time and iterate through all of these so that I end up calling the server thousands of times to verify the response from that server. The problem I have is that the second server expects the set id to be in a form where the forward slashes are converted to underscores and the hashes to dots, so
"1/7842/0889#001"
becomes
"1_7842_0889.001"
I have code that converts one to the other in POSTMAN
var jsonData = pm.response.json()
for (var i=0; i<jsonData.setIds.length; i++)
{
var new_j = jsonData.setIds[i].replace (/#/g, ".");
var new_i = new_j.replace (/\//g, "_");
}
})
This works fine line by line it creates the right thing in the console of POSTMAN but obviously what I really need to do is save the entire JSON in the right form to a file and then read from that file line by line using the corrected data. I don't seem to be able to save the data in a file in the right form using my code and I suspect I am missing something simple. Is there a way to write a file line by line from in side postman or in a script and manipulate the data as I'm creating it?
Alternatively I guess I could read from the JSON I have saved i.e. the full response and iterate through that manipulating the data as a pre-request script?
I have tried to do something like this using environmental variables - so in my first call I do:
var jsonData = JSON.parse(responseBody);
postman.setEnvironmentVariable('setIds', JSON.stringify(jsonData));
and then in my second call to the express server where I want to send my payload I run a pre-request script that I thought would work using the env variable but this fails as it doesn't seem to like the {...
SyntaxError: Unexpected token {
I think there are probably some neat ways of solving this either doing all of this outside of POSTMAN in javascript but I'm a little lost where to start. Any help appreciated

Would tell you are plaing with content, but not setting it back to JSON object ??
jsonData.setIds[i] = new_i;
can help or you can use 2x replace it in a string and convert back to make it easier (in case there are no / or # somewhere else).
var src = {
"count": 6909,
"setIds": [
"1/7842/0889#001",
"2/4259/0166#001",
"ENT0730/0009",
"9D/11181#002",
"1/9676/0001#001",
"2/4718/0001#004",
"2/1783/0044#001",
"1/4501/0001#001",
"1/2028/0002#002",
"2/3120/0079#001",
"2/1672/0024#001",
"2/3398/0064#001"
//...
]
}
var str = JSON.stringify(src, null, 4);
str = str.replace(/\//g,'_').replace(/#/g,'.');
console.log(str);
var res = JSON.parse(str);
console.log(res);

Passing objects through Django template to javascript

I am making a website using Django and I want to pass a python object from my view (where it is created) through the Django template and to a Dajax call. The problem is that by the time it gets to dajax it has been turned into type unicode.
In my Template
<script>
var emailer = "{{emailer|safe}}"; <---If I omit the quotes here then I get a javascript error.
sessionStorage.setItem('emailer',emailer);
$(document).ready(function(){
$('.send').on('click', function(e){
var emailer = sessionStorage.getItem('emailer');
Dajaxice.InterfaceApp.sendEmail(submitverify,{'emailer':emailer});
});
});
</script>
The dajax function
#dajaxice_register(method='GET')
def sendEmail(emailer):
logger.warning("type: %s, %s" % (type(emailer),emailer))
email_body = "message"
emailer.addToMessage(email_body)
emailer.send()
message = "Email Sent"
return json.dumps({'message':message})
Here the logger statement returns: type: <type 'unicode'>, <Utils.SIMPL_Emailer instance at 0x103142ab8>. Is there any way to fix this so that I get my emailer object instead of a unicode string?

First try to understand what is happening:
On your template you're trying to save a Python object to a Javascript var:
var emailer = "{{emailer|safe}}";`
But it's not possible. When your template is rendered by Django what you really get is a call to object __str__() method and your Javascript will store the <Utils.SIMPL_Emailer instance at 0x103142ab8> value on your emailer var. And remember: this code run in the client browser. That's why you get an error when you remove the quotes.
To solve it you need to first serialize your emailer object (Turn it into something that could be represented as a String, for example, and then turned back to Python Object). But as pointed by Peter DeGlopper it is a very insecure approach. Never, ever deserialize an whole object that was public accessible. Instead send only the email data to your template. You can create a dictionary with this data, turn it into JSON (it's a serialization too, but this time you are serializating only data) and then pass it to your template.
So do not put your emailer on the template context. Instead create a dictonary and pass it to the template.
Then in your Python sendEmail(emailer) method you'll need to instanciate a new Emailer object and feed it with the data, like:
#dajaxice_register(method='GET')
def sendEmail(email_json):
email = json.loads(email_json) # email_json is a json with your email data only
logger.warning("type: %s, %s" % (type(email_json),email_json))
emailer = Emailer("<with all your params...>")
emailer.addToMessage(email.get('body'))
emailer.send()
message = "Email Sent"
return json.dumps({'message':message})

Codeigniter - sending json to script file

I query the db i my model like so
function graphRate($userid, $courseid){
$query = $this->db->get('tblGraph');
return $query->result();
}
My controller gets data back from my model and I json encode it like so
if($query = $this->rate_model->graphRate($userid, $courseid)){
$data['graph_json'] = json_encode($query);
}
$this->load->view('graph', $data);
And thats returns me a json object like so
[
{"id":"1","title":"myTitle","score":"16","date":"2013-08-02"},
{"id":"2","title":"myTitle2","score":"17","date":"2013-09-02"},
{"id":"3","title":"myTitle3","score":"18","date":"2013-10-02"}
]
In my view graph I'm loading an js file
<script type="text/javascript" src="script.js"></script>
Now I want to use $data that is being sent from my controller to my view, to my external script.js to use as labels and data to feed my chart. But How do I get that Json data to my external script.js so I can use it?
1 more thing about the json data, isn't it possible to get the output of the json data as
{
"obj1":{"id":"1","title":"myTitle","score":"16","date":"2013-08-02"},
"obj2":{"id":"2","title":"myTitle2","score":"17","date":"2013-09-02"},
"obj3":{"id":"3","title":"myTitle3","score":"18","date":"2013-10-02"}
}

The problem isn't a Codeigniter problem, it's a javascript scope/file inclusion/where-do-i-get-my-data-from problem.
I run into this all the time and have used these solutions:
naming my php files with .php extensions and loading them as if they're views.
Just putting the script that needs data from a view IN the view file where it's used
Using an ajax request in my included js file to hit a controller and get json data.
I use #2 most frequently (for things like datatables where I WANT the js code right there next to the table it's referencing.
I use #1 occasionally, but try NOT to do that because it means some .js files are in my webroot/js dir and some are in teh application/views directory, making it confusing for me or anyone else who wants to support this project.
#3 is sometimes necessary...but I like to avoid that approach to minimize the number of requests being made and to try to eliminate totally superfluous requests (which that is).

You need to print the result of the output json string to the html generated file.
But you need to parse the string with some script. I would recommend you: http://api.jquery.com/jQuery.parseJSON/
For the second question. It is possible by doing:
$returnValue = json_encode(
array (
"obj1" => array("id"=>"1","title"=>"myTitle","score"=>"16","date"=>"2013-08-02"),
"obj2" => array("id"=>"2","title"=>"myTitle2","score"=>"17","date"=>"2013-09-02"),
"obj3" => array("id"=>"3","title"=>"myTitle3","score"=>"18","date"=>"2013-10-02"),
)
);

Print the output using PHP like:
echo json_encode($query);
Then from the client-side (where JavaScript resides) load that JSON that you printed using PHP. This can be done easily using JQuery.
Like this:
$.get("test.php", function(data) {
alert("Data Loaded: " + data);
});
You can find more information about this here: http://api.jquery.com/jQuery.get/
Now you'll need to parse this data so that JavaScript can understand what you got as text from the server. For that you can use the JSON.parse method on the "data" object in the aforementioned example. Once parsed, you can use the object like any other object in JavaScript. You can find more information about JSON.parse here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse
I hope that helps.

Using JSON data in a javascript function

I am returing JSON data as a response in a web service I am writing.
I am wondering what is the best way to consume the data, returned to a Javascript function (passed as an argument).
Do I:
use eval()
use Doug Crawfords jSon parser to parse the JSON data into a Javascript object
simply use the returned data as an object literal
BTW, I am still learning Javascript, (just started reading the Definitive Guide 2 days ago) so if my question seems a bit naive or asinine - bear with me.

I would suggest a few things, first, on the web service side:
If a callback=functionName parameter is passed in the query string, then return the properly serialized JSON as the only parameter to the callback function (In this case functionName
If no callback was requested, then simply return the properly serialized JSON string. (This is helpful for when other languages consume the JSON. PHP or Ruby for instance)
Then, as far as consuming it, if you are on the same domain as the web service, you can retrieve the code and use the json2.js parser to parse it. If you are on a separate domain, use the callback parameter when requesting the data, and make sure you set up a function to handle the callback.
Simple example (Using PHP):
<?php
$callback = null;
if( isset($_REQUEST['callback']) ){
$callback = $_REQUEST['callback'];
}
$fakeData = array( 'key' => 'value');
$json = json_encode( $fakeData );
echo $callback ? "$callback($json)" : $json;
?>
JS (On different domain than the web service):
function handleCallback( data ){
alert( data.key );
}
function get_json(){
// Dynamically create the script element
var script = document.createElement('script');
script.type = "text/javascript";
script.src = "http://yourwebservice.com/service/?callback=handleCallback";
document.getElementsByTagName('head')[0].appendChild( script );
}
Calling get_json would dynamically create the request (as a script element due to cross domain restrictions. You can't use XMLHttpRequest to make cross domain requests in most browsers), and the PHP returned script would correctly call the handleCallback function when the script returns.

If you are using jQuery you can set it up so it returns as a JSON object.
http://api.jquery.com/jQuery.getJSON/

Assuming that your webservice is decorated with ScriptService attribute:
I would suggest using json2.js to handle the json clientside. It defers to native implementations when they are available.

(1). use eval()
Only if you're certain the data is clean. I'd never trust anything from an outside source.
(2). use Doug Crawfords jSon parser to parse the JSON data into a Javascript object
Best idea overall. Even jQuery uses plain old eval() to parse JSON.
(3). simply use the returned data as an object literal
When it's returned, it's just a string, not an object unless passed through one of the two aforementioned functions, or wrapped in a function ala JSONP.