Problem
I need a way to store and collect JSON data in an entirely offline(!) web application, hosted on a local (shared) machine. Several people will access the app but it will never actually be online.
I'd like the app to:
Read and write JSON data continuously and programmatically (i.e. not using a file-upload type schema)
Preferably not require any software installation other than the browser, specifically I'd like not to use local server. (edit: I may be willing to learn a bit of Python if that helps)
The amount of data I need to store is small so it is very much overkill to use some sort of database.
Solution?
My first thought was to let the html5 file API, just read/parse and write my JSON object to a local txt file, but this appears not to be possible?!
Local storage is not applicable here right, when several people - each with their own browser - need to access the html?
Any ideas?
note
I know this topic is not entirely novel, but I think my situation may be slightly different than in other threads. And I've spent the better part of the last couple hours googling this and I'm none the wiser..
Have you considered Python's Json module? http://docs.python.org/2/library/json.html
Using this you can convert python objects to and from json strings. You can store the strings however you want to
You can't use Localstorage to enable such features because every client will have its own dataset stored.
Have you ever considered using a java applet to handle such informations ?
You could start a java applet using it as a bridge between browser clients and as a store of informations.
Browsers could share such information using websockets.
Some times ago I build demo with such solution.
Check it at: https://github.com/KingRial/SBrower
In this demo I open a browser/tab which starts a java Applet to create a websocket server.
All the browsers/tabs are just clients connecting to the websocket server and sharing informations.
Since python is one of the question tags, I am giving a python solution:
import json
#reading
file_json = open("json.txt")
print file_json
python_json_object = json.loads(file_json)
print python_json_object
file_json.close()
#writing
file_json = open("json.txt", 'w')
file_json.write(json.dumps(python_json_object))
My suggestion would be something like WampServer (Windows, Apache, MySQL, PHP). I've seen a few tutorials about adding Python to that mix.
You would have access to reading and writing JSON data to the local storage or placing your data in a local database.
I know you said you don't want to opt for local server, but nodejs could be the solution. If you know JavaScript, then it's very simple to set one server up and let everybody access to the server from any browser. Since it's entirely JavaScript you don't even have conversion issues with the JSON format.
For storing the JSON you can use the FileSystem built-in library of nodejs which lets you read and write from a file, so you don't even need a database.
This is using Node.js and Express.
const express = require('express');
const http = require("http");
const app = express();
const server = http.createServer(app);
const fs = require('fs');
const newestData = {
"id":123,
"title":"spoon"
}
app.get("/",async(req,res,next)=> {
res.status(200).send('Hello World!');
fs.readFile("Metadata.json", 'utf8', (err,content)=>{
if(err) throw err
let data = JSON.parse(content)
data.push(newestData)
fs.writeFile("Metadata.json",JSON.stringify(data), (err)=>{
if(err) throw err
})
})
})
const port = process.env.PORT || 5000;
require("dotenv").config();
server.listen(port, function() {
console.log('Express server listening on port ' + port);
});
Related
Our workplace has a shared network drive that everyone in our department has access to without explicitly requiring credentials. The link is something like
myworkplace.com/mydept/archive/etc
Anyone from my department can access this link via file explorer or even their browser. Currently, I use it by calling python scripts that exist on the drive by using
> pushd myworkplace.com/.../python/
> ./python.exe scriptPath/script.py arg1 arg2
The result is that it copies a set of files (arg1) from an SFTP server (arg2) onto this drive for everyone in the department to review via a new link/directory path. Now, I want to create a very basic webpage that others in the department can use to essentially pass arg1 and arg2 as input to this script and by the click of a button run this script themselves.
My research suggests that I should use node.js as client-side webpages cannot interact with the server-side scripts/tools without it. This requires serious network configs which is entirely out of the question as that is not our responsibility and the last thing I would want to do is get an angry email from the networking team. However, if me and others can easily pushd into the drive and run scripts from it, shouldn't there be something just as easy that I can do from a webpage? Maybe I can leverage the network config of the network drive itself?
I also believe that I may be misunderstanding the issue entirely, and that maybe I'm calling the script from the drive but actually running it locally, but my knowledge on networking is very slim so I'm not exactly sure what is really possible here. Any suggestions?
So I was able to solve this rather simply
When I pushd into the network drive I can run the command
ipconfig
which returns the IP address of the drive. I used that as the host name and used port 8080. When running the node.js script I made sure to use those details as below
const express = require("express");
const app = express();
const host = '11.111.111.111';
const port = 8080;
app.get('/Home', (req,res) => {
res.sendFile(<UNC Network path to HTML landing page >)
})
app.listen(port, host, () => console.log(`App listening on port ${port}`))
When my colleagues and I typed '11.111.111.111:8080/Home' into our browser we were able to access the page and it's contents. I think this is the standard protocol for creating a website and I may have simply misunderstood the requirements.
The rendered HTML page contained a button that would call a different function in my node.js script which returned the output from my python script. See below
function myFunc(arg1, arg2){
const spawner = require('child_process');
const path = require('path');
var dataToRead;
//declare path to python executable
const python_exe = path.resolve(<UNC path> + 'python/python.exe');
//declare path to python script
const python_py = path.resolve(<UNC path> + 'script.py');
//create a process on server side calling the python executable to initiate python
//and pass the script and arguments as a list of paramaters
const python_call = spawner(python_exe, [python_py, arg1, arg2]);
//read the output from python script
python_call.stdout.on('data', (data) => {
dataToRead = data.toString();
}
}
And that's how I used a network drive to create a webapp that calls a python script and retrieves the results
P.S. I could not change the website name to be a string as that would require configuring the DNS to point to this IP and port, which is not something folks outside the network team have access to.
am very new to both mongodb and java script.
Now i need to connect to my local mongodb instance using java script in order to get the list of documents.
any help?,
thanks in advance.
You can use mongoose to connect to a mongoDB database : http://mongoosejs.com/
Take a look at here,
https://mongodb.github.io/node-mongodb-native/driver-articles/mongoclient.html
var MongoClient = require('mongodb').MongoClient
, Server = require('mongodb').Server;
var mongoClient = new MongoClient(new Server('localhost', 27017));
mongoClient.open(function(err, mongoClient) {
var db1 = mongoClient.db("mydb");
mongoClient.close();
});
In order to connect to database, you need to connect your application to a server( not sure if the application could connect to database directly but from what I learnt, it seems server side language is needed). Thus, if you want to connect to database with javascript only, you may consider using node.js+express to connect to mongoDb. I think this link is quite useful. I also learnt from this website. This website teaches you how to do that from scratch and instructions can be easily followed. If you prefer to use php for server side, I just googled this website which might help. It seems to me that php is easier to learn but if you only need basic CRUD operation, the website mentioned is enough.
I have been experimenting with Node.js on my machine for a little while and I have found my knowledge of HTTP requests, XHR objects, and the like quite lacking. So the main thing that's been nagging at me while using Node is that I can't seem to understand how to communicate from the client to the server (other than simple GET requests) and vice versa. What brings me to this question is my recent project, which is a simple 2 player chess game (no AI opponents).
What I want to do is to be able to send the game board data (a JSON string) to the server and have it save the data to a file. I understand how to get the file contents using an XHR object on the client-side. I also understand how to use Node's fs module to create and read files on the server-side. What I don't understand is how to use the XHR object to send the string to the server and have Node process and save it into a file. Is it even possible to call server-side code using client-side code in this way? Is trying to send an argument through a XHR object to the server an incorrect way of doing this?
If what I have asked is too broad of a topic to answer, I would also be open to links and books on the topic of server and client communication.
Expanding on SLaks answer a little:
Assuming you're using jQuery on the client and express on the server (both are useful frameworks to avoid reinventing low-level stuff), you could do something like this.
Client
$.ajax({
type: "POST",
url: "http://www.yourserver.com:3000/some/path",
data: { ...chessdatahere... }
}).done(function(msg) {
alert("Data Saved: " + msg);
});
Server
var fs = require('fs');
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.json());
app.post('/some/path', function(req, res) {
fs.writeFile('filename', res.body, function(err) {
if (err) {
res.send('Something when wrong');
} else {
res.send('Saved!');
}
})
});
app.listen(3000);
(Note that the code has not been tested, but it should show the general idea at least).
You can can use XHR to send an HTTP request with (eg) JSON data in the POST payload.
You can then write server-side Node.js code that handles this request, reads the payload, and does whatever you want with the data.
As a follow-up to my question yesterday about Node.js and communicating with clients, I'm trying to understand how the following would work.
Case:
So, I have this website where content is updated very frequently. Let's assume this time, this content is a list of locations with temperatures. (yes, a weather service)
Now, every time a client checks for a certain location, he or she goes to a url like this: example.com/location/id where id corresponds to the id of the location in my database.
Implementation:
At the server, checktemps.js loops (every other second or so) through all the locations in my (mySQL) database and checks for the corresponding temperature. It then stores this data is an array within checktemps.js. Because temperatures can change all the time, it's important to keep checking for updates in the database.
When a request to example.com/location/id is made, checktemps.js looks into the array with a record with id = id. Then, it responds with the corresponding temperature.
Question:
Plain text, html or an ajax call is not relevant at the moment. I'm just curious if I have this right? Node.js is a rather unusual thing to get a grasp on, so I try to figure out if this is logical?
At the server, checktemps.js loops
(every other second or so) through all
the locations in my (mySQL) database
and checks for the corresponding
temperature. It then stores this data
is an array within checktemps.js
This is extremely inefficient. You should not be doing looping(every other second or so).
Modules
Below I would try and make a list of the modules(both node.js modules as other modules) I would use to do this efficient:
npm is a package manager for node. You can use it to install and publish your node
programs. It manages dependencies and does other cool stuff.
I hope sincerely that you already know about npm, if not i recommend you to learn about it as soon as possible. In the beginning you just need to learn how to install packages, but that is very easy. You just need to type npm install <package-name>. Later I would really like to advice you to learn to write your own packages to manage the dependencies for you.
Express is a High performance, high class web development for Node.js.
This sinatra-style framework from TJ is really sweet and you should read the documentation/screencasts available to learn it's power.
Socket.IO aims to make realtime apps possible in every browser and
mobile device, blurring the
differences between the different
transport mechanisms.
Redis is an open source, advanced
key-value store. It is often referred
to as a data structure server since
keys can contain strings, hashes,
lists, sets and sorted sets.
Like Raynos said this extremely fast/sexy database has pubsub semantics, which are needed to do your question efficiently. I think you should really play with this database(tutorial) to appreciate it's raw power. Installing is easy as pie: make install
Node_redis is a complete Redis client for node.js. It supports all Redis commands, including MULTI, WATCH, and PUBLISH/SUBSCRIBE.
Prototype
I just remembered I helped another user out in the past with a question about pubsub. I think when you look at that answer you will have a better understanding how to do it correctly. The code has been posted a while back and should be updated(minor changes in express) to:
var PORT = 3000,
HOST = 'localhost',
express = require('express'),
io = require('socket.io'),
redis = require('redis'),
app = module.exports = express.createServer(),
socket = null;
app.use(express.static(__dirname + '/public'));
if (!module.parent) {
app.listen(PORT, HOST);
console.log("Express server listening on port %d", app.address().port)
socket = io.listen(app);
socket.on('connection', function(client) {
var subscribe = redis.createClient();
subscribe.subscribe('pubsub'); // listen to messages from channel pubsub
subscribe.on("message", function(channel, message) {
client.send(message);
});
client.on('message', function(msg) {
});
client.on('disconnect', function() {
subscribe.quit();
});
});
}
I have compressed the updated code with all the dependencies inside, but you while still need to start redis first.
Questions
I hope this gives you an idea how to do this.
With node.js you could do it even better. The request/response thingy is manifested in our heads since the beginning of the web. But you can do just one ajax request if you open the website/app and never end this call. Now node.js can send data whenever you have updates to the client. Search on youtube for Introduction to Node.js with Ryan Dahl (the creator of node.js), there he explains it. Then you have realtime updates without having to do requests by the client all the time.
I want to build an entire web app using only Javascript and MYSQL . How can I go about this if it's possible.
Try something like Jaxer, which will allow you to execute JavaScript on the Web Server and query databases.
Here are some syntax examples and usages:
Database, file, and socket access from JavaScript
alt text http://jaxer.org/images/Picture+4_0.png
Easily create RESTful JSON data services
alt text http://jaxer.org/images/Picture+6.png
Directly call server-side functions from the browser
alt text http://jaxer.org/images/Picture+2_0.png
You can do it with Jaxer. There are some screencasts that'll get you started. Also check out project Phobos. Jaxer integrates nicely in Aptana studio, Phobos in Netbeans.
If you can run javascript on the server, you can build a web-application with it (without the need for any other language like PHP etc.). Search the web for 'connection string mysql' to find out how to connect to your mySQL database and use ADO/ODBC. You'll need the MySQL ODBC-connector on the MySQL server.
Here's an example database connection (where MySQL server resides on the same server as the web server):
function connectDB()
{
var connectStr = "DRIVER={MySQL ODBC 3.51 Driver}; " +
"SERVER=localhost; " +
"PORT=[MySQL server port];" +
"DATABASE=[your database]; " +
"UID=[username];PWD=[password];" +
"OPTION=3",
conection = Server.CreateObject("ADODB.Connection");
//ERRID=>lib::connectDB::open
try {connection.Open(connectStr) }
catch(e) {errAlert(e,'rs::connectDB','connection failed',1) }
return connection;
}
(Where errAlert is a custom function to return the error)
You could write your application entirely in client side javascript with AJAX / REST calls to your database server - using something like CloudKit on your server (or CouchDB, which features a native JSON HTTP interface). On the client side, Dojo or YUI abstract out a great deal of the IO handling…
It's quite possible to write a web application using only javascript. One the key benefits of that is that since all code runs locally, you can make an application which doesn't require online connectivity.
The main detractor though, is that you can't hook it up to a database. But there are alternative data storage hacks you can use.
One example of such a javascript application is TiddlyWiki which is a personal wiki, contained in a single html file. The javascript application rewrites that html file, so you can carry it with you on a USB-drive or something.
You could look at triplify which should expose your database as json and rdf. I haven't actually used this but I would imagine that would let you bypass writing any server side js and talk to the database directly in a language javascript understands, using an ajax request and json.
You can build client-side applications in javascript, with an embedded database. HTML 5 has support for databases, and a couple of browsers have already implemented this part of the spec (safari, firefox with the gears plugin).
But this is only for clientside usage. You wont be able to share the database with other users. Also you can select which database you want to use. I think gears uses sqlite.
You will not be able to use Javascript and MYSQL without using something such as PHP on the server side to bridge the gap between the database and the Javascript on the client side.
Edit: I may be wrong, however I have no idea how you would run Javascript on the server side.