We Keep Coding

Retrieving a javascript processed Web page

What am I asking for is the ability to download a rendered / processed page via Google Chrome or Firefox I think.
For example, I don't want:
hendry#x201 ~$ w3m -dump http://hello.dabase.com
FAIL
I want:
$ $answer http://hello.dabase.com
Hello World

You should be able to do it using PhantomJS. It is running WebKit without the visuals, but you get the same fast and native supports for JavaScript, HTML/DOM, CSS, SVG, Canvas, and many others.
Disclaimer: I started PhantomJS.

Probably too early, but someone ported V8 to Go-lang, so now you could write your own client that makes use of this powerful combo:
http://bravenewmethod.wordpress.com/2011/03/30/embedding-v8-javascript-engine-and-go/
Looks quite straightforward, doesn't require an ugly Java/Rhino stack and adopts the next big programming language.

It looks similar to the problem http://simile.mit.edu/wiki/Crowbar is trying to solve.

You could use jsdom:- https://github.com/tmpvar/jsdom
I'd build a node driver for it, but it's supposed to work with Rhino etc.

I'd take a look at Rhino.
I'd use the excellent env.js library in conjunction with Rhino to simulate the browser environment as much as is technically possible. Once you've implemented some web spider bootstrap code you should be able to obtain the result you want above.
I'd be interested in other solutions to this though.

Is Javascript only available for web browsers?

I would like to know about Javascript. Is Javascript available only for web browsers? Because I used some Javascript code for Firefox Plugin development and Thunderbird.
Help me to find out more about this: where can I use JavaScript other than web browsers, and how?

There's node.js which includes a full webserver and runs on Google's V8

No! Have a look at Google's V8

Google Javascript. Click first link (Wikipedia). Click "uses outside webpages".
http://en.wikipedia.org/wiki/JavaScript#Uses_outside_web_pages

There are a number of server-side (or otherwise non-browser) implementations of Javascript, including Node.js, which currently has a lot of momentum behind it, as well as the now-defunct Jaxer, which provided a server-side DOM implementation based on Mozilla Gecko. There are also efforts to bring DOM type functionality to Node.js, primarily from Yahoo.

With Rhino, you can embed Javascript in Java applications.

No, there are definitely other implementations. For instance, JScript.Net is included with the .NET framework.

MS-Windows can execute .js files (from the command prompt).
Since you do not have the browser context, you cannot use any calls to the browser (alert, for example, does not work).
Look for "Windows Scripting Host".
WSH supports scripts written in
Microsoft Visual Basic Scripting
Edition (VBScript) or JavaScript.

There are several implementations of stand-alone Javascript
JSLibs (http://code.google.com/p/jslibs/) is general purpose
jsdb (http://www.jsdb.org/) is more database oriented

Many text editors (including the one I use, ActiveState's Komodo Edit) expose their APIs and let you write macros in Javascript.

Using Adobe AIR you can write almost platform-independent Desktop Applications using JavaScript

PDF files can also contain JavaScript code. See first entry in google on this subject and also Adobe JavaScript virus.

Javascript can be easily embedded in applications written in other programming languages.
For Java there is as already mentioned Rhino, for C++ you can use for example Flusspferd. Here are some tutorials/examples.

Qt has the QtScript module. From Qt 4.7 it uses JavaScriptCore (the Webkit JavaScript engine).

In short, No.
Actually, Javascript and other Web technologies are used these days to create native mobile and desktop applications. (see Titanium Appcelerator and PhoneGap)
If you are familiar with the KDE project you can craft Plasma widgets using JavaScript, see Creating plasmoids with JavaScript
In Qt framework there is a QtScript module for JavaScript (ECMAScript) application scripting to provide much of the application’s functionality.
Also the new technology that is soon to be released in Qt 4.7 is Qt Quick, aka the Qt UI Creation Kit, which allows application developers to declaratively define their user interfaces in QML, for more information go here and an example here
QML is an extension to JavaScript,
that provides a mechanism to
declaratively build an object tree of
QML elements. QML improves the
integration between JavaScript and
Qt's existing QObject based type
system, adds support for automatic
property bindings and provides network
transparency at the language level.
And let's not forget Javascript is used as an embedded scripting language in various applications OpenOffice.org, Google Desktop Widgets, and many others, see Wikipedia's article here
On the server-side Javascript enables back-end access to databases, file systems, etc (see Node.js, Google V8, SpiderMonkey and others here)

Some document databases, such as CouchDB and MongoDB, use JavaScript to interact with the database.

Javascript (and HTML rendering) engine without a GUI for automation?

Are there any libraries or frameworks that provide the functionality of a browser, but do not need to actually render physically onto the screen?
I want to automate navigation on web pages (Mechanize does this, for example), but I want the full browser experience, including Javascript. Thus, I'd like to have a virtual browser of some sort, that I can use to "click on links" programmatically, have DOM elements and JS scripts render within it, and manipulate these elements.
Solution preferably in Python, but I can manage others.

PhantomJS and PyPhantomJS are what I use for tasks like these.
What it is, is a headless WebKit based browser which is fully controllable via JavaScript. There's a C++ implementation (PhantomJS) and a Python one (PyPhantomJS). I prefer the Python one though, because it has a plugin system which allows you to add functionality to the core without actually modifying any code, unlike the C++ one. :)

There is an absolute ton of free software technology now available: take your pick at http://wiki.python.org/moin/WebBrowserProgramming but if you have specific questions join pyjamas-dev on google groups and i'll be happy to give further details, there. brief answer: you can run pywebkitgtk "headless", or you can use xulrunner (via python-hulahop) again using pygtk without actually doing "browserwidget.show()", and there's also pykhtml. also you could use python COM to connect to MSHTML.DLL.
these are all "cheat" methods: using python bindings to a graphical web browser engine without actually firing up the graphical bit. if you really wanted to put some serious hard-core programming in, you could create a "port" of webkit which was not connected to a GUI toolkit: as an experienced webkit programmer i'd put it as around... 2 weeks of full-time effort to make such a "headless" version of webkit.
l.

Looks like http://watin.sourceforge.net/ might be a good way to go.
If you don't have to go pure Python, you could do IronPython since it's a C# project.

take a look at this little doosy on ajaxian
http://ajaxian.com/archives/server-side-rendering-with-yui-on-node-js
It also talks about Aptana Jaxer which I think runs on a headless firefox so is basically the Mozilla browser engine in all it's glory.

There is Kapow. Its pure Java and costs money:
http://kapowtech.com/
And there is Lixto: Its Eclipse based and uses Mozilla Gecko as rendering engine (unless they already changed it to WebKit, as they said they'll do years ago). Its very nice and also costs money:
http://www.lixto.com/?page_id=50
They are both graphical tools where you define the site navigation and what should be extracted by point and click. But you can also write xpath and regular expressions and even JavaScript that runs in the sites context.
I used them both in the lectures web data extraction and applied web data extraction at the technical university Vienna (Lixto is written by the Professor who held the lecture).

HTMLUnit in Java is very good. I think it's only the Java implementations of headless browsers that manage to provide Javascript support.
MaxQ, I read about here, sounds like it might be interesting: "written in Java, generates Jython scripts"

Try HtmlUnit !!!

SpiderMonkey vs JavaScriptCore vs?

I have a C++ desktop application (written in wxWidgets) and I want to add support for some scripting language.
Scripting would mostly be used for run-time conversions of strings, numbers and dates by user supplied JavaScript code.
I'd like to use JavaScript because it is widely used and everyone is familiar with the syntax.
Googling around, it seems I have two options:
SpiderMonkey from Mozilla
JavaScriptCore from WebKit
Has anyone tried those? Which one would be easier to set up?
Do you know of some other implementation that is better for my needs?
BTW, I target Windows and Linux platforms.

There is also Google's V8 JavaScript engine, builds nicely on Linux, embedding API seems quite straightforward too: (Compared to SpiderMonkey's, never looked at the JavaScriptCore API)
http://code.google.com/apis/v8/get_started.html

Of course, you could also use Lua, which not only is designed specifically for this, it's vastly faster than any JS.
Also, it's has well-designed semantics, a very minimal core, simple C API, great portability, a very mature JIT, the most helpful online community I've seen, etc...

JavaScriptCore has a stable C API (and ABI), and has been available (and used as) a standard system framework on macos.
[edit: oh, and it works on linux and windows as a standalone library, although i believe only debian distributes it as such]

Javascript interpreter to replace Python

In terms of quick dynamically typed languages, I'm really starting to like Javascript, as I use it a lot for web projects, especially because it uses the same syntax as Actionscript (flash).
It would be an ideal language for shell scripting, making it easier to move code from the front and back end of a site, and less of the strange syntax of python.
Is there a good, javascript interpreter that is easy to install (I know there's one based on java, but that would mean installing all the java stuff to use),

I personally use SpiderMonkey, but here's an extensive list of ECMAScript shells
Example spidermonkey install and use on Ubuntu:
$ sudo apt-get install spidermonkey
$ js myfile.js
output
$ js
js> var f = function(){};
js> f();

Of course, in Windows, the JavaScript interpreter is shipped with the OS.
Just run cscript or wscript against any .js file.

There are four big javascript interpreters currently. V8, Squirrelfish, Spidermonkey and Rhino. I think more important than performance is how well it integrates into existing infrastructure, and I guess Rhino with its bridge to Java wins here.

Try jslibs, a scripting-focused standalone JS runtime and set of libraries that uses SpiderMonkey (the Gecko JS engine).

On the 'easy to translate' theme, there's also Lua.
It's somewhat similar to Javascript, but more 'orthogonal' (closer to functional roots).
The heavy orientation to 'pure' programming theory has made it really small and fast. It's the fastest scripting language around, and the JIT runs circles around the new JavaScript JITs that are starting to appear.
Also, since it was originally thought as an extension language, it has a very nice and clean interface to C, making it very easy to create bindings to any C library you might want to access.

Google's V8 can be used as a standalone interpreter. Configuring with scons sample=shell will build an executable named shell, that can be called like so: ./shell file.js.

You'll need some server-side JavaScript interpreter. Check out the following blog post. Something such as Rhino might be useful for you.

You might try toying around with SquirrelFish or v8, both should be runnable on the command line.

FYI, there is a built-in one already on modern windows platforms. You need to use JScript, but it's close enough. Same environment also allows for VBScript. To run a program you can execute something like:
cscript foo.js
The windows system API is a bit weird and frustrating if you expect the same flexibility as with basic JS objects, but they do have thorough documentation if you can handle digging through the MSDN pages and seeing all the examples in VBScript.
Not sure what's available for Linux/Mac in terms of js shell.

Well, for safety reasons, javascript had not been provided with file access right by design. So as a scripting language, it's a bit limited.
But still, if you really want to, spider monkey is your best option. Here is a tuto :
http://developer.mozilla.org/en/Introduction_to_the_JavaScript_shell

Node.JS. It's great. Has many modules. you can do all your file scripting with Node.

In my years I've found most Javascript developers find it quite easy to transfer over to PHP and vice versa - it isn't a direct answer to your question, although if you're working in ActionScript and JavaScript then you're best to stick with something like PHP (if you're not willing to move to Java, and stick with the ECMA base)