We Keep Coding

Extending JavaScript syntax in Monaco Editor with full integration

I'm trying to extend Monaco Editor to allow users to write in a hybrid of JavaScript and another language, using delimiters to separate them within the same file, similarly to how Markdown allows writing multiple languages using fenced code blocks.
The difference is that I want to keep all the other IDE features that Monaco has built in for JavaScript, such as linting (done via diagnostics), smart auto-completion, jump-to-definition, auto-formatting helpers, and every other IDE feature that comes with Monaco's built-in JavaScript mode. I'd like these features to still work within the JavaScript portion of the code that Monaco is editing, and be disabled for the sub-language portion.
My first attempt was to call setMonarchTokensProvider, passing in a modified version of TypeScript's tokenizer rules. Specifically, I was able to add the beginning-fence delimiter to the root rule and create a new rule for the sub-language in the same way the documentation for Monarch (Monaco's syntax highligher) describes, using #nextEmbedded. (For testing purposes, I've been hard-coding CSS as the embedded language.)
When I call setMonarchTokensProvider like this for the language "javascript", it completely ignores this syntax highlighting tokenizer, and colors the code-fences of CSS as invalid JavaScript, indicating that you cannot override the built-in JavaScript mode this way.
When I call setMonarchTokensProvider with a new language (e.g. "mylang") and set the editor to use that language, it provides correct syntax highlighting (!) for this CSS-in-JS hybrid language. But all other advanced features that were found in the JavaScript mode are no longer present. The editor didn't have any smart auto-completion for methods defined on classes in the same file, or any in-editor error-reporting for invalid syntax, or any of its trademark JavaScript IDE features.
So my next attempt was to modify the pre-bundled Monaco code's TypeScript definition to include my custom syntax highlighting rules. This correctly highlighted my CSS-in-JS code completely (!), when setting the language to "typescript", and left all the other features intact (!) including diagnostics reporting (live-validation and underlining of errors), auto-completion, all of it! (I didn't try it with "javascript" but it's safe to assume it probably works or is trivial to get it working, since JavaScript is actually implemented as a variant configuration of the TypeScript mode in Monaco.)
Unfortunately, it also considered the entire CSS portion of it, including the fence around it, to be invalid JavaScript code.
I know that this is theoretically doable, because within HTML mode, you can embed CSS or JS with full support for proper validation and auto-completion and every other IDE feature; basically, every sub-language in an HTML file works like it's in its own file: HTML features in the root of the file, CSS features within style tags, JS features within script tags.
But digging into the TypeScript plugin's implementation inside Monaco, it's not clear where to begin editing this, either as a user of Monaco the library, or by forking it and patching it up where necessary. I started at trying to modify the DiagnostcsAdapter [sic] and tracing where it's actually implemented, but I got stuck two function-calls deep, where it seems to push a promise of syntax validation that returns a value that's used later, but the implementation of getSyntacticDiagnostics just shells the work out to some other implementation that I can't find anywhere else in the repo, nor in the monaco-languages repo or the vscode repo.

I make the similar thing. My solution is to place non-JS code inside a block comment:
regularJsCode()
/*
[your-syntax-identifying-start-token]
place any syntax you want here
[your-syntax-identifying-end-token]
*/
regularJsCode()
Then you can process it with your tools, parsers, IDE extension etc. And the coolest part you can make VSCode to syntax-highlight it like you want so it won't seem like some hack.
This approach is preferrable because your JS file still remains a 100% valid JS file.
If you still don't want to put your syntax into comments, then you should create your own file extension like .jsx/.tsx. In practical, VSCode terms this means you need to create VSCode extension with language server and stuff. This is not so easy, but the documentation is good. You could assemble your own JS highlighting code inside your VSCode extension using language server: https://github.com/sourcegraph/javascript-typescript-langserver

According to the creator of Monaco:
Architecturally, you can do the following:
use monaco-editor-core directly
define a new language to the editor
fork monaco-typescript and change it to work with your newly defined langauge id. Then, modify the TS language host code to not pass the original models to TypeScript, but first run a preprocess which strips your custom language out of the text, and then only passes on valid TypeScript to the TS compiler. One idea is to replace each character that you remove with a space. This will leave all the position/offset computation work without any effort on your side.
Good luck!

How to include OVRManager in Three.js Scene?

I have created several Three.js/Javascript demo applications that I'm experimenting with in my new Oculus Go. I'm trying to enable the Go Controller to do stuff in my applications, and according to the Oculus Developer Center, the best thing to do is to include OVRManager in my scene so I have access to that API. That sounds good, but for all the documentation (https://developer.oculus.com/documentation/unity/latest/concepts/unity-ovrinput/) I can't see HOW to add OVRManager to my scene! I have not worked with Unity before, but from what I can tell in the documentation there shouldn't be any compatibility issues (should there?)
So what I'd think to do is something like:
<script src="OVRManager.js or something like that"></script>
and then call the functions I need, as I've done with OrbitControls.js and other external dependencies.
But for the life of me, Google searching is just sending me in circles. I see questions posed for C++ and C# but that's of no use to me. How do I get this API working in my Three.js scene? Where do I find it and is there some other way to include it?
Thanks!

Create a unity WebGL build and expose the API you need as public methods in a Unity Script you attach to a GameObject.
Then, you should be able to follow the directions at How to call Unity functions from javascript (copied below) on how to call those methods from your javascript code.
You may be able to use UnityScript, which is vaguely similar to JavaScript, to write the Script if you use an old version of Unity. As of this writing, Oculus recommends version 2017.4.11f1, which I think might still support UnityScript.
One major reason you see so much less UnityScript information is that Unity has been moving away from UnityScript, into only supporting C#.
But regardless of if you code your OVRManager script in C# or UnityScript, Unity will make the methods callable from your JavaScript.
Calling Unity scripts functions from JavaScript
Sometimes you need to send some data or notification to the Unity
script from the browser’s JavaScript. The recommended way of doing it
is to call methods on GameObjects in your content. If you are making
the call from a JavaScript plugin, embedded in your project, you can
use the following code:
SendMessage(objectName, methodName, value);
Where objectName is the name of an object in your scene; methodName is
the name of a method in the script, currently attached to that object;
value can be a string, a number, or can be empty. For example:
SendMessage('MyGameObject', 'MyFunction');
SendMessage('MyGameObject', 'MyFunction', 5);
SendMessage('MyGameObject', 'MyFunction', 'MyString');
If you would like to make a call from the global scope of the
embedding page, see the Code Visibility section below.
Code visibility
Starting from Unity 5.6 all the build code is executed in its own
scope. This approach makes it possible to embed your game on an
arbitrary page without causing conflicts with the embedding page code,
as well as makes it possible to embed more than one build on the same
page.
If you have all your JavaScript code in the form of .jslib plugins
inside your project, then this JavaScript code will run inside the
same scope as the compiled build and your code should work pretty much
the same way as in previous versions of Unity (for example, the
following objects and functions should be directly visible from the
JavaScript plugin code: Module, SendMessage, HEAP8, ccall etc.).
However, if you are planning to call the internal JavaScript functions
from the global scope of the embedding page, you should always assume
that there are multiple builds embedded on the page, so you should
explicitly specify which build you are referencing to. For example, if
your game has been instantiated as:
var gameInstance = UnityLoader.instantiate("gameContainer", "Build/build.json", {onProgress: UnityProgress});
Then you can send a message to the build using
gameInstance.SendMessage(), or access the build Module object using
gameInstance.Module.

how to avoid generation document.write in GWT cache.js file

I am using Gwt 2.4. After compiling my project i am getting some document.write lines in the mymodle.chache.js which is most weired .
So i want to know the places which producing these lines in my GWT code and came across and found some lines of code like
element.setInnerHtml("blah ..blah..");
and
doc.write("blah..blah ..");
So i removed those line of code and compiled again again and still getting some (ex:document.write(df+er+t)) lines .
Anyone suggest a way to find the cause to producing those lines from my GWT ??

(assuming you're talking about the *.nocache.js file rather than the *.cache.* files, as *.cache.* don't contain calls to document.write, unless you're using a crappy third-party lib)
The *.nocache.js file is generated by the primary linker, so its content is independent from your code. It contains calls to document.write() for various things: determine your module base URL (as a last resort, when it cannot infer it from elsewhere), inject scripts and stylesheets referenced from your *.gwt.xml files, and finally inject the GWT application code itself (with the standard linker, this will be an <iframe>, with the newer xsiframe linker it'll be a <script>).
If you're really worried about it, it can be customized quite easily by extending the CrossSiteIframeLinker (xsiframe linker) and providing different code snippets.
Tip: compile with -style PRETTY to make the generated code readable.

You are not supposed to hand-edit/modify mymodle.cache.js. This file gets generated again from GWT Compiler. What issue are you trying to solve by hand-editing it?

what is google map's javascript load strategy

When I use google maps, I am interested in its implemention, so I use the firebug to inspect.
Then I found that its javascript loading strategy is rather interesting. Take this page for example:
The overlay example
Then when I open this page first time, the following js are loaded:
https://maps.googleapis.com/maps/api/js?sensor=false
https://maps.gstatic.com/intl/en_us/mapfiles/api-3/9/13b/main.js
https://maps.gstatic.com/cat_js/intl/en_us/mapfiles/api-3/9/13b/%7Bcommon,map,util,poly%7D.js
https://maps.gstatic.com/cat_js/intl/en_us/mapfiles/api-3/9/13b/%7Bonion,geometry%7D.js
But if I refresh the page(use the ctrl+f5), the following js are loaded:
https://maps.googleapis.com/maps/api/js?sensor=false
https://maps.gstatic.com/intl/en_us/mapfiles/api-3/9/13b/main.js
However the page still works, the overlay is drawn in the map. But where is the poly.js and etc?
Also, can anyone tell me how to load the js by components? For exmaple the common util poly in the example.
What should I know when I write the different components?

1. When poly.js loads, it passes a string to google.maps.__gjsload___.
Here's an excerpt:
google.maps.__gjsload__('common', '\'use strict\';var Ai=isNa...
The rest of the file is just the contents of that string.
My hunch is this function probably stores this string in localStorage or sessionStorage so that it only has to be retrieved once.
2. Also, if you want to learn about loading js files as-needed, look into AMD and/or CommonJS:Modules.
A good imlementation of AMD (my preference) is RequireJS.
Update
I did some poking around, and localStorage and sessionStorage do not appear to be being used on this page. I also can't duplicate your results. In Firebug, poly.js always loads for me. There may be some magic happening somewhere, but I don't see it.
However, it's entirely possible to store a string in localStorage and sessionStorage for retrieval without having to make an extra js call.

Also,any one can tell me how to load the js by components?
this touches on the topic of asynchronous javascript file loading. if you've ever used a language that has a way to "include" a file at any point in a script, you'll understand that javascript does not have this capability. because of that, there is this whole paradigm of "aysnc javascript addition" via script tag injection.
script tag injection: you dynamically make a script tag, and set its source to the file you need, and insert that tag into the DOM, and voila, a new file has been loaded and executed. With javascript heavy applications, this is common, especially when loading third party applications. Google does it alllll the time, just check out google analytics' include script for a good example of this.
Now, since this is a touchy and delicate type of coding to do, some "javascript component / module / asset loading" frameworks have refined it and made it pretty stable. common.js, require.js, etc have all done good jobs at this.
What should I know when I write the different components ?
For what you're doing with google maps, you don't really need to know much. but if you get into javascript module pattern development, you need to know this: make sure you protect your global namespace from being cluttered by your own variables, so encapsulate all of your work in closures when possible, and (recommended but not required) start them all with a ; so they don't break each other if they get loaded out of order.

Add ASP.NET server script to mostly-static .JS / .CSS files without losing IntelliSense?

Using VS2008 and ASP.NET 3.5 (or VS 2010 / .NET 4.0?), how can I include a bit of dynamic ASP.NET server-side code in mostly-static JavaScript and CSS files?
I want to do this to avoid cloning entire JS or CSS files to vary just a small part of them multi-tenant sites. Later, I want to extend the solution to handle localization inside javascript/CSS, dynamic debugging/tracing support, and other cool things you can get by injecting stuff dynamically into JavaScript and CSS.
The hard part is that I don't want to lose all the cool things you get with static files, for example:
JS/CSS code coloring and intellisense
CSS-class "go to definition" support in the IDE
automatic HTTP caching headers based on date of underlying file
automatic compression by IIS
The server-side goodness of static files (e.g. headers/compression) can be faked via an HttpHandler, but retaining IDE goodness (intellisense/coloring/etc) has me stumped.
An ideal solution would meet the following requirements:
VS IDE provides JS/CSS intellisense and code coloring. Giving up server-code intellisense is OK since server code is usually simple in these files.
"go to defintion" still works for CSS classes (just like in static CSS files)
send HTTP caching headers, varying by modified date of the underlying file.
support HTTP compression like other static files
support <%= %> and <script runat=server> code blocks
URL paths (at least the ones that HTTP clients see) end with .JS or .CSS (not .ASPX). Optionally, I can use querystring or PathInfo to parameterize (e.g. choosing a locale), although in most cases I'll use vdirs for this. Caching should vary for different querystrings.
So far the best (hacky) solution I've come up with is this:
Switch the underlying CSS or JS files to be .ASPX files (e.g. foo.css.aspx or foo.js.aspx). Embed the underlying static content in a STYLE element (for CSS) or a SCRIPT element (for JS). This enables JS/CSS intellisense as well as allowing inline or runat=server code blocks.
Write an HttpHandler which:
looks at the URL and adds .aspx to know the right underlying ASPX to call
uses System.Net.HttpWebRequest to call that URL
strips out the containing STYLE or SCRIPT tags, leaving only the CSS or JS
adds the appropriate headers (caching, content type, etc.)
compresses the response if the client suports compression
Map *.CSS and *.JS to my handler.
(if IIS6) Ensure .JS and .CSS file extensions are mapped to ASP.NET
I'm already using a modified version of Darick_c's HttpCompression Module which handles almost all of above for me, so modifying it to support the solution above won't be too hard.
But my solution is hacky. I was wondering if anyone has a more lightweight approach for this problem which doesn't lose Visual Studio's static-file goodness.
I know I can also hack up a client-side-only solution where I split all JS and CSS into "vary" and "won't vary" files, but there's a performance and maintenance overhead to this kind of solution that I'd like to avoid. I really want a server-side solution here so I can maintain one file on the server, not N+1 files.
I've not tried VS10/.NET 4.0 yet, but I'm open to a Dev10/.net4 solution if that's the best way to make this scenario work.
Thanks!

I have handled a similar problem by having a master page output a dynamic generated JSON object in the footer of each page.
I needed to have my js popup login dialog box support localization. So using JSON.NET for serialization, I created a public key/value collection property of the master page that pages could access in order place key/values into such as phrase key/localized phrase pairs. The master page then renders a dynamic JSON object that holds these values so that static js files could reference these dynamic values.
For the js login box I have the masterpage set the localized values. This made sense because the masterpage also includes the login.js file.

I do commend you on your concern over the number of http requests being made from the client and the payload being returned. Too many people I know and work with overlook those easy optimizations. However, any time I run into the same issue you're having (which is actually quite often), I have found I've usually either taken a wrong turn somewhere or am trying to solve the problem the wrong way.
As far as your JS question goes, I think Frank Schwieterman in the comments above is correct. I'd be looking at ways to expose the dynamic parts of your JS through setters. A really basic example would be if you have want to display a customized welcome message to users on login. In your JS file, you can have a setMessage(message) method exposed. That method would then be called by the page including the script. As a result, you'd have something like:
<body onLoad="setMessage('Welcome' + <%= user.FirstName %>);">
This can obviously be expanded by passing objects or methods into the static JS file to allow you the functionality you desire.
In response to the CSS question, I think you can gain a lot from the approach Shawn Steward from the comments makes a good point. You can define certain static parts of your CSS in the base file and then redefine the parts you want to change in other files. As a result, you can then dictate the look of your website by which files you're including. Also, since you don't want to take the hit for extra http requests (keep in mind, if you set those files to be cached for a week, month, etc. it's a one time request), you can do something like combining the CSS files into a single file at compilation or runtime.
Something like the following links may be helpful in pointing you in the right direction:
http://geekswithblogs.net/rashid/archive/2007/07/25/Combine-Multiple-JavaScript-and-CSS-Files-and-Remove-Overheads.aspx
http://www.asp.net/learn/3.5-SP1/video-296.aspx?wwwaspnetrdirset=1
http://dimebrain.com/2008/04/resourceful-asp.html
By utilizing the combining at run or compile time you can gain the best of both world by allowing you to logically separate CSS and JS files, yet also gaining the reduction of payload and requests that comes with compressing and combining files.