I would like to run some rules against pages. These rules are essentially functions that check the page for information. They can be simple as in 'check if the page has a title tag' or more complex like 'check if all links on the pages are whitelisted based on example.com/allowed_links.json'.
The rules would be run on the page on-demand only and come from a trusted source.
My first approach has been to create a rule service that sends back an javascript array of rules. All the client then has to do is go over the array and run each function in it. The response is a standard object {rule: [name], pass: [true|false], message: [some message about success/failure]}
Since this is on demand only, we fetch the rules from the service and run 'eval' on it.
EDIT: The response from 'mysite/rules' looks like this
RULESYSTEM.rules.push(function1() {...});
RULESYSTEM.rules.push(function2() {...});
...
const RULESYSTEM = {
rules: [],
};
let rules = fetch('mysite/rules')
let rulesscript = await rules.text();
eval(rulescript)
...
//eval will populate the previously declared rules array.
let pass = true;
for(let i=0; i < RULESYSTEM.rules.length; i++) {
let rule = RULESYSTEM.rules[i];
//This obj has only one property. Get that one.
let result = rule();
pass = pass && result.pass;
}
...
This works perfectly fine. However it is receiving a lot of pushback as 'eval' is considered evil and to be avoided at any cost. The security is not an issue here since the source is within the organization itself and thus trusted.
Are there any other ways to implement a system like this.
It would appear that all you're attempting to do is to retrieve JSON data and transforming it into a javascript object.
fetch('mysite/rules')
.then(res=>res.json())
.then(data=>{
//TODO: handle data which is your object/array
console.log(data)
})
Thus no need for eval. Also you need to remember that fetch returns a promise and that rules.text() and rules.json() also returns a promise. The way you've currently written it won't work anyway.
According to MDN
The json() method of the Body mixin takes a Response stream and reads it to completion. It returns a promise that resolves with the result of parsing the body text as JSON.
To answer your question:
Is it possible to return javascript code as JSON
That's clearly a no, however there are alternatives ! You can simply fetch javacsript as text (as you've done) and programmatically create a script tag, load your javascript text in it and append it to your HTML file.
Or even better, you can simply dynamically create a script tag with the URL of your server endpoint sending javascript and append it to your HTML file.
Something like:
const script = document.createElement("script");
script.onload = function(){
console.log("script loaded");
}
script.src = '/some/url/here';
document.body.appendChild(script);
I am going to add this as an answer. I will use some dummy data you can query based on an endpoint
Route("get-functions")
Response getJSFunctions(List<string> js_to_load){
var options = getData(); //returns a list
var guid = new Guid()
var fp = File.open(guid.toString() + ".js", "w+")
var out = "var fns = [" + options.join("\n") + "];";
fp.write(out);
fp.write(" var runner = options => fns.forEach(fn => fn(options) );");
fp.close()
return new Response({url: guid.toString() + ".js"})
}
Js:
$.json("get-functions", data => {
let script = document.createElement("script");
script.src = data.url;
document.head.appendChild(script)
runner(options);
});
So what is happening is that you build a Temp JS FIle containing all JS Functions you want run, then add that file dynamically to the runtime. Then you have a function which will always be available called runner, which you can immediately call.
Runner will iteratively call each function with a global list of options across the functions, which you can define on the clientside.
Instead of using C#, you can use any serverside or even Javascript if you are using node as your backend. You need DB Access and file creation access on the host machine. You will also want to delete the GUID files ocassionally as they are just one and done use files, so you can delete them all every 5 minutes or something.
I dont have the means right now to create a running sample. I can create something a little later with Python if you like as the backend.
Related
I am looking for a better way of passing data to my index.js file in a webdev application. Note I really only have about a month of webdev experience so this is likely due to lack of experience. here is my software flow:
I query data in a route handler before the page is rendered. I then pass this data to the rendered page (note i need to keep some of the key-vals hidden. However aggregation works).
exports.getPlotView = async(req, res, next) =>{
//grab the module to query from, stored as var.testModel
const qParse = new PlotQueryParse(req.query).parseObj();
// console.log(qParse)
// const testblockName = qParse.testblock+"Name" ;
// const limitName = qParse.limitname;
const aggregationObj = {$match:
{'jobId':qParse.jobId, '<key2>':<val2>, "<key>":"<val>"}
}
const data = await qParse.testModel.aggregate([aggregationObj]);
console.log(data[0])
const dataString = JSON.stringify(data[0]);
//parse the url to make the query
res.status(200).render('testPlotView', {
pageHeader: "Test",
subHead: "Test summary",
IPn: "IPn",
inData:dataString
});
}
data is passed to pug template. The template uses this as a hidden element
extends base
block content
div.hide_data #{inData}
div#dataviz
now in my index.js script (listens for evenets), the data is loaded from the page and then stored for post processing. I would like to directly access the variable instead of having it hidden then accessing the DOM element.
window.addEventListener('load', function(){
if(window.location.href.includes('testplotdata')){
console.log('its a me mario')
//if we are in a test plot data page, lets plot
var jsonObject = JSON.parse(document.querySelector('.hide_data').innerHTML);
console.log(jsonObject['testData'])
//post processing code ....
}
})
Again, I want a way to grab my queried data without saving it as a DOM element then accessing it in my external event listener script.
Thanks!
Instead of storing data in HTML, add inline script to your template to store it in a global variable instead. So replace div.hide_data #{inData} with:
script.
var inData = !{inData}; // inData passed by backend must be a string
// representing a valid JS object (JSON will do)
Now you just access inData as a ready native object in your external script (which you need to make sure load after the above script, putting it at the end of <body> will do)
// No need: var jsonObject = JSON.parse(document.querySelector('.hide_data').innerHTML);
console.log(inData); // Go ahead with the data
Goal
Transform HTML extracted from Telligent (an extranet platform) to plain text and send to Slack
Setup
A Telligent webhook is triggered when an event occurs. An Azure Logic App receives the event JSON. The JSON value is in HTML. A JavaScript Azure Function inside the Azure Logic App pipeline transforms the JSON value to plain text. The final step in the pipeline posts the plain text in Slack.
Example of incoming code to the Azure Function
"body": "<p>" '</p><div style=\"clear:both;\"></div>"
Transformation method
This is the basic code in the Azure Function. I have left out parts that seem irrelevant to this question but can provide the entire script if that is necessary.
module.exports = function (context, data) {
var html = data.body;
// Change HTML to plain text
var text = JSON.stringify(html.body);
var noHtml = text.replace(/<(?:.|\n)*?>/gm, '');
var noHtmlEncodeSingleQuote = noHtml.replace(/'/g, "'");
var noHtmlEncodeDoubleQuote = noHtmlEncodeSingleQuote.replace(/"/g, "REPLACEMENT");
// Compile body for Slack
var readyString = "Slack text: " + noHtmlEncodeDoubleQuote;
// Response of the function to be used later
context.res = {
body: readyString
};
context.done();
};
Results
The single quote is replaced successfully and resolves accurately when posted in Slack.
The following replacement methods for the double quote throw a Status: 500 Internal Server Error within the Azure Function.
Unsuccessful replacement methods
"\""
'"'
"
"'"'"
"["]"
"(")"
Putting these replacement methods in their own var also throws the same error. E.g.:
var replace = "\""
...
var noHtmlEncodeDoubleQuote = noHtmlEncodeSingleQuote.replace(/"/g, replace);
The code appears to be correct because when I replace " with something like abc, the replacement is successful.
Thank you
Please forgive my JavaScript as I am not a programmer and am seeking to streamline a process for my job. However I am grateful for any advice both about the code or my entire approach.
Generally, you don't want to try to parse HTML with regular expressions or string replacement. There are just too many things that can go wrong. See this now famous StackOverflow answer. (It was even made into a T-Shirt.)
Instead, you should use a technique that is purposefully built for this purpose. If you were in a web browser, you could use the techniques described in the answers to this question. But in Azure Functions, your JavaScript doesn't run in a browser, it runs in a Node JS environment. Therefore, you need will need to use a library such as Cheerio or htmlparser2 (and others).
Here is an example using Cheerio:
var cheerio = require('cheerio');
var text = cheerio.load(html.body).text();
Also, regarding this part:
... as I am not a programmer ...
Yes you are. You are clearly programming right now. Anyone who writes code is a programmer. There is no club or secret handshake. We all start out exactly like this. Good job asking questions, and good luck on your journey!
I'm trying to pass initial state from server to client by stringifying that state and storing it in global variable. Below is simplified version of what I'm doing.
// home.tsx
res.render('home', { appState: {script: '<script></script>'} });
// home.pug
doctype html
html
head
script.
window.__initialState__ = !{JSON.stringify(appState)};
body
#root
Sadly the result looks like this
<script>window.__initialState__ = {"script":"<script></script>
which is not valid Javascript. My endgoal is to have server's appState object assigned to window.__initialState__.
The 1st part of your code is very confusing. However, as a general rule, you should always encode any HTML passed from the server to the client as string.
So you need to change the part of your server code that generated the line below:
let scriptObject = {script: '<script></script>'};
to this (assuming that you're using pug and node.js):
var htmlencode = require('htmlencode');
let scriptObject = {script: htmlencode.htmlEncode('<script></script>')};
I have a server side function which returns content of HTML page:
if (Meteor.isServer) {
Meteor.startup(function () {
// code to run on server at startup
Meteor.methods({
sayHello: function() {
var response = Meteor.http.call("GET", "http://google.com");
return response;
}
});
});
And I have client code where I am trying to get title from this HTML page:
'click .add_tag' : function(e,t) {
//Session.set('editing_tag_id', e.target.id);
Meteor.call("sayHello", function(err, response) {
var title = $(response.content).find("title").text();
var title2 = $(response).find("title").text();
var title3 = response.content.match(/<title[^>]*>([^<]+)<\/title>/)[1];
alert(title3);
});
I would like to get jQuery version ('title' or 'title2'), but it doesn't works. It returns empty string.
'Title3' - version works fine, but I don't like regexps. :)
Is there any way to make 'jQuery'-versions works ?
As requested, I will reiterate my comment as an answer...
I would stick with the regex, even though you don't like it. There is a huge overhead of constructing a DOM element that is essentially an entire page, purely for the purpose of parsing a small amount of text. The regex is more lightweight and will perform adequately in slower browsers or on slower machines.
Wrap response.content in a <div> and then do a selection off of that. This way you have a proper structure to start from rather than an array that you might actually be getting.
var $wrap = $("<div></div>").html(response.content);
$wrap.find("title").text();
An example of what is probably going on: http://jsfiddle.net/UFtJV/
Don't forget one thing : you should never return HTML to client. You should return Json (or even Xml) that your client will transform into Html using Template.
You are doing like a lot of dev doing Bad Ajax.
Don't forget : "only data on wire, not display".
So there should not be any problem coz, on response you just have to take data from Json formatted response and inject it inside your Template.
I'm looking at putting together a good way of caching results of AJAX queries so that the same user doesn't have to repeat the same query on the same page twice. I've put something together using a Hash which works fine but i'm not sure if there's a better method i could be using. This is a rough snippet of what i've come up with which should give you a general idea:
var ajaxresults;
document.observe("dom:loaded", function() {
ajaxresults = new Hash();
doAjaxQuery();
});
function doAjaxQuery(){
var qs = '?mode=getSomething&id='+$('something').value;
if(ajaxresults.get(qs)){
var vals = (ajaxresults.get(qs)).evalJSON();
doSomething(vals);
}else{
new Ajax.Request('/ajaxfile.php'+qs,{
evalJSON: true,
onSuccess: function(transport){
var vals = transport.responseText.evalJSON();
ajaxresults.set(qs,transport.responseText);
},
onComplete: function(){
doSomething(vals);
}
});
}
}
Did you try caching the AJAX requests by defining the cache content headers. Its another way and your browser will take care of caching. You dont have to create any hash inside your libraray to maintaing cache data.
High performance websites discussed lot about this. I dont know much about the PHP, but there is a way in .Net world to setting cache headers before writing the response to stream. I am sure there should be a similar way in PHP too.
If you start building a results tree with JSON you can check of a particular branch (or entry) exists in the tree. If it doesn't you can go fetch it from the server.
You can then serialize your JSON data and store it in window.name. Then you can persist the data from page to page as well.
Edit:
Here's a simple way to use JSON for this type of task:
var clientData = {}
clientData.dataset1 = [
{name:'Dave', age:'41', userid:2345},
{name:'Vera', age:'32', userid:9856}
]
if(clientData.dataset2) {
alert("dataset 2 loaded")
}
else {
alert("dataset 2 must be loaded from server")
}
if(clientData.dataset1) {
alert(clientData.dataset1[0].name)
}
else {
alert("dataset 1 must be loaded from server")
}
Well, I guess you could abstract it some more (e.g. extend Ajax by a cachedRequest() method that hashes a combination of all parameters to make it universally usable in any Ajax request) but the general approach looks fine to me, and I can't think of a better/faster solution.