We Keep Coding

Extending JavaScript syntax in Monaco Editor with full integration

I'm trying to extend Monaco Editor to allow users to write in a hybrid of JavaScript and another language, using delimiters to separate them within the same file, similarly to how Markdown allows writing multiple languages using fenced code blocks.
The difference is that I want to keep all the other IDE features that Monaco has built in for JavaScript, such as linting (done via diagnostics), smart auto-completion, jump-to-definition, auto-formatting helpers, and every other IDE feature that comes with Monaco's built-in JavaScript mode. I'd like these features to still work within the JavaScript portion of the code that Monaco is editing, and be disabled for the sub-language portion.
My first attempt was to call setMonarchTokensProvider, passing in a modified version of TypeScript's tokenizer rules. Specifically, I was able to add the beginning-fence delimiter to the root rule and create a new rule for the sub-language in the same way the documentation for Monarch (Monaco's syntax highligher) describes, using #nextEmbedded. (For testing purposes, I've been hard-coding CSS as the embedded language.)
When I call setMonarchTokensProvider like this for the language "javascript", it completely ignores this syntax highlighting tokenizer, and colors the code-fences of CSS as invalid JavaScript, indicating that you cannot override the built-in JavaScript mode this way.
When I call setMonarchTokensProvider with a new language (e.g. "mylang") and set the editor to use that language, it provides correct syntax highlighting (!) for this CSS-in-JS hybrid language. But all other advanced features that were found in the JavaScript mode are no longer present. The editor didn't have any smart auto-completion for methods defined on classes in the same file, or any in-editor error-reporting for invalid syntax, or any of its trademark JavaScript IDE features.
So my next attempt was to modify the pre-bundled Monaco code's TypeScript definition to include my custom syntax highlighting rules. This correctly highlighted my CSS-in-JS code completely (!), when setting the language to "typescript", and left all the other features intact (!) including diagnostics reporting (live-validation and underlining of errors), auto-completion, all of it! (I didn't try it with "javascript" but it's safe to assume it probably works or is trivial to get it working, since JavaScript is actually implemented as a variant configuration of the TypeScript mode in Monaco.)
Unfortunately, it also considered the entire CSS portion of it, including the fence around it, to be invalid JavaScript code.
I know that this is theoretically doable, because within HTML mode, you can embed CSS or JS with full support for proper validation and auto-completion and every other IDE feature; basically, every sub-language in an HTML file works like it's in its own file: HTML features in the root of the file, CSS features within style tags, JS features within script tags.
But digging into the TypeScript plugin's implementation inside Monaco, it's not clear where to begin editing this, either as a user of Monaco the library, or by forking it and patching it up where necessary. I started at trying to modify the DiagnostcsAdapter [sic] and tracing where it's actually implemented, but I got stuck two function-calls deep, where it seems to push a promise of syntax validation that returns a value that's used later, but the implementation of getSyntacticDiagnostics just shells the work out to some other implementation that I can't find anywhere else in the repo, nor in the monaco-languages repo or the vscode repo.

I make the similar thing. My solution is to place non-JS code inside a block comment:
regularJsCode()
/*
[your-syntax-identifying-start-token]
place any syntax you want here
[your-syntax-identifying-end-token]
*/
regularJsCode()
Then you can process it with your tools, parsers, IDE extension etc. And the coolest part you can make VSCode to syntax-highlight it like you want so it won't seem like some hack.
This approach is preferrable because your JS file still remains a 100% valid JS file.
If you still don't want to put your syntax into comments, then you should create your own file extension like .jsx/.tsx. In practical, VSCode terms this means you need to create VSCode extension with language server and stuff. This is not so easy, but the documentation is good. You could assemble your own JS highlighting code inside your VSCode extension using language server: https://github.com/sourcegraph/javascript-typescript-langserver

According to the creator of Monaco:
Architecturally, you can do the following:
use monaco-editor-core directly
define a new language to the editor
fork monaco-typescript and change it to work with your newly defined langauge id. Then, modify the TS language host code to not pass the original models to TypeScript, but first run a preprocess which strips your custom language out of the text, and then only passes on valid TypeScript to the TS compiler. One idea is to replace each character that you remove with a space. This will leave all the position/offset computation work without any effort on your side.
Good luck!

How do I include global styles/javascript in HTML files displayed as dialogs?

I am developing a Google App Script project that will be used right from within a Google Sheet, with HTML files as dialogs. My project will be a mix of .gs files as well as HTML files for data entry, etc. I am trying to use the methodology explained here:
https://developers.google.com/apps-script/guides/html/best-practices#separate_html_css_and_javascript
to create global JavaScript and CSS modules that I can include in my HTML files rather than cutting and pasting inline code all over the place. This will be mainly useful for the data-saving routines which capture form data, serialize it, then save it to Sheets via the methodology outlined here (and many other places): http://railsrescue.com/blog/2015-05-28-step-by-step-setup-to-send-form-data-to-google-sheets/.
The problem I am having is with trying to call the "include" statement from my HTML files, namely, lines like:
<?!= include('JavaScript'); ?>
It doesn't work when I create a menu on the spreadsheet to display my HTML file as a dialog -- the text of the include line just shows up as literal output on the dialog, and code does not appear to be getting included (not in scope).
I know the Google example is primarily for pages delpoyed via a web app, but I'd like to use my HTML files as dialogs right inside the spreadsheet (e.g. from a menu or sidebar) -- that feels nice and tidy to me. But if I can't get includes to work, my code base is going to be a nightmare and it will be really, really hard to standardize CSS across the whole app. I don't want to be cutting and pasting all the time.
So, what is the secret behind this <?! tag, and why won't it work in my HTML files when they are called as dialogs? It is clear those lines are different from the get-go (maybe not in a bad way, but they don't work), as the Google Scripting console displays those lines oddly, as depicted in the screenshot below:

Please try adding:
<?xml-stylesheet href="chrome://global/skin/" type="text/css"?>
More information can be found in Adding Style Sheets.

Figured it out. I was not properly understanding the way the HTML was being served up as a dialog. I was using this behind a custom menu option:
var html = HtmlService.createHtmlOutputFromFile(htmlFileName);
when I should have been using the more dynamic:
var html = HtmlService.createTemplateFromFile(htmlFileName).evaluate();
The latter generates a user interface object where the server-side script is executed and everything is included properly when I display the object with showModalDialog() (or showSidebar()).
I just had a complete misunderstanding of how the user interface object was being created, so now all scripting works inside my HTML files.

Angular 2 Quickstart: wrong encoding of js files

I'm very new to angular 2, so I'm trying to make this work: https://angular.io/guide/quickstart
The problem which I have really confuses me. All JS files, which I get as a response, have strange encoding - basically, I see a bunch of hieroglyphs. Non-js files (html and css) are ok. Please see below, html is ok. Js - totally not.
Do you guys have any idea, what can it be? Computer culture is not Chinese or Japanese, just in case.

You can use notepad++ or any html editor with encode options available to change the page encoding:
Open your file with Notepad++
Go to "Encoding" menu option in main menu
Select "Convert to UTF-8-BOM"
check if the page display correctly

This can be related to your js files encoding on your editor.

Angular-Dialog-Service and readability of modal code

I'm using the Angular Dialog Service to build pop up forms on my website. This services source can be found here: https://github.com/m-e-conroy/angular-dialog-service/blob/master/README.md
Anyway, the actual angular and JavaScript that describes what the modal contains and does goes after the following where the ... is:
.run(['$templateCache',function($templateCache){
$templateCache.put(...)
My problem is the html and js seem to have to all be one after the other with no spacing or indentation allowed between any of it. This causes the code that describes the modal to essentially just be a wall of text that has become borderline unreadable and editable in my complicated modal. An example of this can be found in the JS portion of the code here: http://codepen.io/m-e-conroy/pen/rkIqv
Is there an easy fix which will allow me to have my modals html and JS in non wall of text format and have it build successfully? If there is no fix this seems like a pretty big flaw in using the Angular Dialog Service to handle modals...
Thanks!

There are tools, like html2js, that can build templateCache for you.
It means, that you can store your html code in html files (where it obviously should be), and then gather it into single javascript file with $templateCache.put(/* content of html file */); in it. Just don't forget to include resulting module into your project, so when one of the services requested html file, it could be found in templateCache.
So you shouldn't edit html in .js files. It's wrong on many levels.

tinymce extended_valid_elements for Microdata

We are currently using TinyMCE (Version: 3.3.9.2 (2010-09-29)) on a CakePHP framework and are trying to modify the Advance Themes file tiny_mce/themes/advanced/editor_template.js to accept the Microdata syntax (for using the Schema.org vocabulary) by adding the following string:
extended_valid_elements : "p[itemtype|itemscope|itemprop|id|class|style|title|dir<ltr?rtl|lang|xml::lang|onclick|ondblclick|onmousedown|onmouseup|onmouseover|onmousemove|onmouseout|onkeypress|onkeydown|onkeyup]",
and/or
extended_valid_elements : "#[itemtype|itemscope|itemprop|id|class|style|title|dir<ltr?rtl|lang|xml::lang|onclick|ondblclick|onmousedown|onmouseup|onmouseover|onmousemove|onmouseout|onkeypress|onkeydown|onkeyup]",
Referencing this debug snippet: http://www.tinymce.com/develop/bugtracker_view.php?id=4469
However, when we modify the file and refresh, even when we put <p itemtype="def">This is some content.</p> or <div itemscope itemtype="http://schema.org/Movie">This is some content.</div> as tests in the HTML source editor, everything is being stripped.
Anyone have any ideas on how to fix or what is making TinyMCE continue to strip structured data?

Before TinyMCE v3.4 there was a config option cleanup and another option verify_html. You might try setting those options to false. This thread seems to suggest that some attribute stripping would still happen even when the attributes are listed in extended_valid_elements.
I have a dim memory of running into this issue myself and reading that TinyMCE was aggressive in fixing invalid HTML even with the configs set to be as liberal as possible. I will edit in some links if I can find those threads on the TinyMCE forum.
Another issue could be validation going on with CakePHP. Older versions of CakePHP have a Sanitize class that could filter out HTML. Confirm that the string is not running through a library like HTML Purifier.