We Keep Coding

How do you access a model attribute with javascript variable

I'm adding an attribute to my ModelAndView in spring and after this i forwarding it to my thymeleaf view.
In the view i have the following code:
<script th:inline="javascript">
/*<![CDATA[*/
var applicationName = /*[[${T(com.sample.constants.ApplicationConstants).MODEL_ATTR_COLLECTED_VALUES}]]*/ "Test";
var collectedValueJson = [[${collectedValues}]];
console.log(collectedUserJson);
/*]]>*/
</script>
Result from this is
var applicationName = 'collectedValues';
var collectedUserJson = '[{\"givenname\":\"Muster\",\"surname\":\"Peter\"}]';
That's fine. Now my wish for this is, that i can take the var application and access with this variable the modelattribute, but that's not working.
Result is this:
var tmp2 = ${applicationName};
An other try was, that i have access to the modelattribute with the syntax /*[[ ]]*/ from the first try:
var applicationName = ${/*[[${T(com.sample.constants.ApplicationConstants).MODEL_ATTR_COLLECTED_VALUES}]]*/};
But result will be:
var tmp = ${'collectedValues'
i have no idea what can i try.
Any other suggestions?
Thank's in advance.

There is a workaround worth mentioning: Write out the attribute to
<span id="myvar" th:text="${attributeName}"></span>
and then read it in with JavaScript using
document.getElementById("myvar").value
or some similar jQuery call:
$('#myvar').text()

there is no way you can access model atributes hence they are on server side and they where lost when rendering jsp into HTML

You can:
<span id="span_id" style="display: none;" th:text="${attributeName}"></span>
and then in your Javascript:
document.getElementById("span_id").innerHTML

awesomium web scraping certain parts

I asked this earlier but I wanted to rephrase the question. I am trying to make a scraper for my project. I would like to have it display a certain part of a link. The only part of the link that changes is the number. This number is what I would like to scrape. The link looks like this:
<a href="/link/player.jsp?user=966354" target="_parent" "="">
As mentioned I am trying to scrap only the 966354 part of the link. I have tried several ways to do this but cant figure it out. When I add
<a href="/link/player.jsp?user="
to the code below it breaks
List<string> player = new List<string>();
string html = webControl2.ExecuteJavascriptWithResult("document.getElementsByTagName('a')[0].innerHTML");
MatchCollection m1 = Regex.Matches(html, "<a href=\\s*(.+?)\\s*</a>", RegexOptions.Singleline);
foreach (Match m in m1)
{
string players = m.Groups[1].Value;
player.Add(players);
}
listBox.DataSource = player;
So I removed it, it shows no errors until I go to run the program then I get this error:
"An unhandled exception of type 'System.InvalidOperationException' occurred in Awesomium.Windows.Forms.dll"
So I tried this and it some what works:
string html = webControl2.ExecuteJavascriptWithResult("document.getElementsByTagName('html')[0].innerHTML");
This code scraps but not the way I would like, Could someone lend a helping hand please.

I would use HtmlAgilityPack (install it via NuGet) and XPath queries to parse HTML.
Something like this:
string html = webControl2.ExecuteJavascriptWithResult("document.getElementsByTagName('html')[0].innerHTML");
var htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(html);
var playerIds = new List<string>();
var playerNodes = htmlDoc.DocumentNode.SelectNodes("//a[contains(#href, '/link/profile-view.jsp?user=')]");
if (playerNodes != null)
{
foreach (var playerNode in playerNodes)
{
string href = playerNode.Attributes["href"].Value;
var parts = href.Split(new char[] { '=' }, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length > 1)
{
playerIds.Add(parts[1]);
}
}
id.DataSource = playerIds;
}
Also you may find these two simple helper classes useful: https://gist.github.com/AlexP11223/8286153
The first one is extension methods for WebView/WebControl and the second one has some static methods to generate JS code for retrieving elements (JSObject) by XPath + getting coordinates of JSObject)

Using a sample html file such as below, I was unable to duplicate the exception.
<html>
test
</html>
However, the javascript
document.getElementsByTagName('a')[0].innerHTML
will return "test" in my example. What you probably want is
document.getElementsByTagName('a')[0].href
which will return the href portion.
The 'innerHTML' property will return everything between the start and end tags (such as <html> </html>). This is probably the reason you have better success when getting the 'html' element - you end up parsing the entire <a> </a> link.
FYI, as a test you can use your browser to test out the javascript output.

Why Node.js function doesn't return JSDom serializeDocument result?

I have PHP script which acts as a DNode client. Then I have Node.js Dnode server which evaluates code which receives from PHP client and it returns DOM as HTML. However, Node.js acts strangely to me (beeing a Node.js newbie). It doesn't return anything, even though the returning string is not empty. My code is below:
PHP client code using DNode-PHP library:
<?php
require(__DIR__.'/../../vendor/autoload.php');
$loop = new React\EventLoop\StreamSelectLoop();
$dnode = new DNode\DNode($loop);
$dnode->connect(7070, function($remote, $connection) {
$js = 'var a = document.createElement("A");';
$js.= 'document.getElementsByTagName("body")[0].appendChild(a);'
$remote->zing($js, function($n) use ($connection) {
print_r($n);
$connection->end();
});
});
$loop->run();
?>
Node.js server code:
var dnode = require('dnode');
var jsdom = require("jsdom");
var server = dnode({
zing: function (n, cb) {
var document = jsdom.jsdom('<!DOCTYPE html>');
var window = jsdom.parentWindow;
eval(n);
var html = jsdom.serializeDocument(document);
// console.log(html);
cb(html);
}
});
server.listen(7070);
Console.log() clearly outputs <!DOCTYPE html><html><head></head><body><a></a></body></html> what is expected result. But it never gets to PHP client. But what is strange, if I change line cb(html); to cb('test');, PHP outputs "test". So the problem must be somewhere on the Node.js side. But I have no idea where to look for.
Thanks in advance for any hints.

How are you viewing the response? Through a web browser? If so, then you're depending on whatever you're evaluating in eval(n) to change the DOM of the document... If nothing changes, then you won't end up seeing anything because you'll have an empty DOM other than the html/head/body tags. It would be worth your time confirming that you're getting an empty response back and it's not just an empty DOM.
That being said, The eval function has any context of you wanting to execute it on the document/window you declare above. As it is, it is just executing in the context of node itself, not on the page you are attempting to create. To fix this, try using:
window.eval(n)
If you take a look at the example Creating a browser-like window object
on the Github page for jsdom, this will give you a better idea of how exactly to use this package.
https://github.com/tmpvar/jsdom
What you have above should look something like this:
var document = jsdom.jsdom("<!DOCUMENT html>");
var window = document.parentWindow;
window.eval(n);
var html = jsdom.serializeDocument(document);
cb(html);
Now you'll be executing the Javascript on the DOM you were previously creating :-)

Your problem is not in Node. When I use the server code you show in your question and try with this client code, I get the expected result:
var dnode = require("dnode");
var d = dnode();
d.on('remote', function (remote) {
var js = 'var a = document.createElement("A");' +
'document.getElementsByTagName("body")[0].appendChild(a);';
remote.zing(js, function (s) {
console.log(s);
});
});
d.connect('localhost', '7070');
I don't do PHP so I don't know what the problem might be on that side.

What is the best way to parse html in google apps script

var page = UrlFetchApp.fetch(contestURL);
var doc = XmlService.parse(page);
The above code gives a parse error when used, however if I replace the XmlService class with the deprecated Xml class, with the lenient flag set, it parses the html properly.
var page = UrlFetchApp.fetch(contestURL);
var doc = Xml.parse(page, true);
The problem is mostly caused because of no CDATA in the javascript part of the html and the parser complains with the following error.
The entity name must immediately follow the '&' in the entity reference.
Even if I remove all the <script>(.*?)</script> using regex, it still complains because the <br> tags aren't closed.
Is there a clean way of parsing html into a DOM tree.

I ran into this exact same problem. I was able to circumvent it by first using the deprecated Xml.parse, since it still works, then selecting the body XmlElement, then passing in its Xml String into the new XmlService.parse method:
var page = UrlFetchApp.fetch(contestURL);
var doc = Xml.parse(page, true);
var bodyHtml = doc.html.body.toXmlString();
doc = XmlService.parse(bodyHtml);
var root = doc.getRootElement();
Note: This solution may not work if the old Xml.parse is completely removed from Google Scripts.

In 2021, the best way to parse HTML on the .gs side that I know of is...
Click + next to Library
Enter 1ReeQ6WO8kKNxoaA_O0XEQ589cIrRvEBA9qcWpNqdOP17i47u6N9M5Xh0
Click "Look up"
Click Add
Sample usage:
const contentText = UrlFetchApp.fetch('https://www.somesite.com/').getContentText();
const $ = Cheerio.load(contentText);
$('.some-class').first().text();
That's it -- this is probably the closest we'll get to doing jQuery-like DOM selection in GAS. The .first() is important or else you may extract more content than you expected (think of it as using querySelector() instead of querySelectorAll()).
Credit where credit is due: https://github.com/tani/cheeriogs

As of May 2020, you can now use the Cheerio library for Google Apps Script to do this.
Returns the content of Wikipedia's Main Page
const content = getContent_('https://en.wikipedia.org');
const $ = Cheerio.load(content);
Logger.log($('#mp-right').text());
Returns the content of the first paragraph <p> of Wikipedia's Main Page
const content = getContent_('https://en.wikipedia.org');
const $ = Cheerio.load(content);
Logger.log($('p').first().text());
To add to your project:
Select Resources - Libraries... in the Google Apps Script editor. Enter the project key 1ReeQ6WO8kKNxoaA_O0XEQ589cIrRvEBA9qcWpNqdOP17i47u6N9M5Xh0 in the Add a library field, and click "Add". Select the highest version number, and click "Save".

I found that the best way to parse html in google apps is to avoid using XmlService.parse or Xml.parse. XmlService.parse doesn't work well with bad html code from certain websites.
Here a basic example on how you can parse any website easily without using XmlService.parse or Xml.parse. In this example, i am retrieving a list of president from "wikipedia.org/wiki/President_of_the_United_States"
whit a regular javascript document.getElementsByTagName(), and pasting the values into my google spreadsheet.
1- Create a new Google Sheet;
2- Click the menu Tools > Script editor... to open a new tab with the code editor window and copy the following code into your Code.gs:
function onOpen() {
var ui = SpreadsheetApp.getUi();
ui.createMenu("Parse Menu")
.addItem("Parse", "parserMenuItem")
.addToUi();
}
function parserMenuItem() {
var sideBar = HtmlService.createHtmlOutputFromFile("test");
SpreadsheetApp.getUi().showSidebar(sideBar);
}
function getUrlData(url) {
var doc = UrlFetchApp.fetch(url).getContentText()
return doc
}
function writeToSpreadSheet(data) {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheets()[0];
var row=1
for (var i = 0; i < data.length; i++) {
var x = data[i];
var range = sheet.getRange(row, 1)
range.setValue(x);
var row = row+1
}
}
3- Add an HTML file to your Apps Script project. Open the Script Editor and choose File > New > Html File, and name it 'test'.Then copy the following code into your test.html
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<input id= "mButon" type="button" value="Click here to get list"
onclick="parse()">
<div hidden id="mOutput"></div>
</body>
<script>
window.onload = onOpen;
function onOpen() {
var url = "https://en.wikipedia.org/wiki/President_of_the_United_States"
google.script.run.withSuccessHandler(writeHtmlOutput).getUrlData(url)
document.getElementById("mButon").style.visibility = "visible";
}
function writeHtmlOutput(x) {
document.getElementById('mOutput').innerHTML = x;
}
function parse() {
var list = document.getElementsByTagName("area");
var data = [];
for (var i = 0; i < list.length; i++) {
var x = list[i];
data.push(x.getAttribute("title"))
}
google.script.run.writeToSpreadSheet(data);
}
</script>
</html>
4- Save your gs and html files and Go back to your spreadsheet. Reload your Spreadsheet. Click on "Parse Menu" - "Parse". Then click on "Click here to get list" in the sidebar.

Xml.parse() has an option to turn on lenient parsing, which helps when parsing HTML. Note that the Xml service is deprecated however, and the newer XmlService doesn't have this functionality.

For simple tasks such as grabbing one value from a webpage, you could use a regular expression. Regex is notoriously bad for parsing HTML as there's all sorts of weird cases it can get tripped up, but if you're confident about the HTML you're accessing this can sometimes be the simplest way.
Here's an example that fetches the contents of the page's <title> tag:
var page = UrlFetchApp.fetch(contestURL);
var regExp = new RegExp("<title>(.*)</title>", "gi");
var result = regExp.exec(page.getContentText());
// [1] is the match group when using parenthesis in the pattern
var value = result ? result[1] : 'No title found';

I know it is not exactly what OP asked, but I found this question when I was looking for some html parsing options - so it might be useful for others as well.
There is an easy to use the library for TEXT parsing. It's useful if you want to get only one piece of information from the html(xml) code.
EDIT 2021: The script library id is:
1Mc8BthYthXx6CoIz90-JiSzSafVnT6U3t0z_W3hLTAX5ek4w0G_EIrNw
It works like in the picture above
function getData() {
var url = "https://chrome.google.com/webstore/detail/signaturesatori-central-s/fejomcfhljndadjlojamaklegghjnjfn?hl=en";
var fromText = '<span class="e-f-ih" title="';
var toText = '">';
var content = UrlFetchApp.fetch(url).getContentText();
var scraped = Parser
.data(content)
.from(fromText)
.to(toText)
.build();
Logger.log(scraped);
return scraped;
}

If you are using
Cheerio library for Google Apps Script
Source code
Library page (⭐ star it!)
Installation by library ID:
1ReeQ6WO8kKNxoaA_O0XEQ589cIrRvEBA9qcWpNqdOP17i47u6N9M5Xh0
A function to get current emojis from unicode.org:
function getEmojis() {
var t = new Date();
var url = 'https://unicode.org/emoji/charts/full-emoji-list.html';
var fetch = UrlFetchApp.fetch(url);
var contentText = fetch.getContentText();
//console.log(new Date() - t);
// Cherio
var $ = Cheerio.load(contentText);
var data = [];
$("table > tbody > tr").each((index, element) => {
var row = [];
$(element).find("td").each((index, child) => {
row.push($(child).text());
});
if (row.length > 0) {
data.push(row);
}
});
//console.log(data);
//console.log(new Date() - t);
// Result
return data;
}
↑ Sample code shows how to parse table and put it into [[array]]
May be used as a custom function:
Bonus
Parsing the site may be a time-consuming operation + you may reach the limit.
Here's a test file with a full version of the script:
https://docs.google.com/spreadsheets/d/1iO7YjYWyfseQu_YCfRbGDPg7NskOgMu_iO1iGjr7KxY/edit#gid=93365395
↑ it uses CasheService to reduce the number of calls.

Natively there's no way unless you do what you already tried which wont work if the html doesnt conform with the xml format.

There are two options
a) One is to use JavaScript's string functions. First locate your tag using string.indexOf() and then extract the data you want using string.substring().
b) The other option is to make use of the Xml Service.

It's not possible to create an HTML DOM server-side in Apps Script. Using regular expressions is likely your best option, at least for simple parsing.

Get link to render as html rather than text json

I am responding to clicks on li's by using $.post to post to an action method in my MVC application.
I want to send a link back in Json.
Can I have this link render as html rather than text ? how ?
I tried this, just to test the html:
var link = "<b>Hi</b>";
var encoded = Server.HtmlEncode(link);
that came out as <b>Hi</b>
Surely there is just a Json.encode or visual studio method I can use and I don't have to format it myself? Have googled fairly extensively and can't find anything about an Json.encode
var link = "<b>Hi</b>";
var encoded = new JavaScriptSerializer().Serialize(link);
the page rendered "\u003cb\u003eHi\u003c/b\u003e"
If I send just the link variable, i.e:
var link = "<b>Hi</b>"
<b>Hi</b> renders
This is the line which sends it back:
return Json(new {Title = pTitle, Selection = pSelection, Link = pLink}, JsonRequestBehavior.AllowGet);
Starting to get frustrated, wtf!
Silly me, I didn't post enough code where the problem was:
<script type="text/javascript">
function TreeView_onSelect(e) {
...
$.post(url, id, function (data, textStatus) {
...
$("#panel-link").text(data.Link);
}
$("#panel-link").text(data.Link);
obv has to be
$("#panel-link").html(data.Link);

Try using JavaScriptSerializer:
var link = "<b>Hi</b>";
var encoded = new JavaScriptSerializer().Serialize(link);

Try to use javascript's decodeURI() function.
http://www.w3schools.com/jsref/jsref_decodeuri.asp
<script type="text/javascript">
var uri="mytest.asp?name=ståle&car=saab";
document.write(encodeURI(uri)+ "<br />");
document.write(decodeURI(uri));
</script>
The output of the code above will be:
mytest.asp?name=st%C3%A5le&car=saab
mytest.asp?name=ståle&car=saab

I have the same problem with you and killing me whole day,
I solved this problem by using Json.NET
Sample code is :
Newtonsoft.Json.JsonConvert.SerializeObject(link);
Reference
http://json.codeplex.com/documentation