We Keep Coding

BigQuery similarity to "signed urls"

I have the following use case in BigQuery:
A non-trusted user will be querying a BigQuery table. Let's say the query is SELECT * FROM [bigquery.table123].
The query will return a large amount of data, let's say 200MB, which will then be displayed in the user's browser.
Our goal is to provide the most efficient way to get the 200MB data into the user's browser (and the worst way seems to do two trips instead of one -- from BQ to our server and then (compressed) to the client). I think the solution for this would probably be to enable the end (non-trusted) user to get something like a "signed-url" to perform the query directly from their browser to BigQuery. The flow would then be like this:
User issues query to our backend.
Authentication is done and a signed url is generated and passed back into javascript.
The client then sends the signed url and the data is loaded directly into the browser.
Only that exact query that has been authorized may be performed, and no other queries could be done (for example, if the client copied any tokens from the javascript)
I would never, ever want the end user to know the ProjectId or Table Name(s) that they are querying.
Is something like this possible to do in BigQuery? Here is an example of a similar need in Cloud Storage. Here is an example of an authenticated/trusted user doing this in browser: https://github.com/googleapis/nodejs-bigquery/blob/master/samples/browseRows.js or . https://stackoverflow.com/a/11509425/651174, but is there a way to do this in-browser for a non-trusted user?

Below is an option that involves two levels of authorized views. This allows to shield not only underlying data from end user - but also hides what exactly data is being used
Let's assume data is in DatasetA. Below steps explain the logic
Create InternalView in DatasetB - this one will target real data from DatasetA.
Make InternalView as Authorized View for DatasetA
Create PublicView in DatasetC - this one will target InternalView
Make PublicView as Authorized View for DatasetB
Give users read access to DatasetC
Users will be ale to run PublicView which will actually be running PrivateView against readl data.
Meantime, users will not be able to see the definition of PrivateView thus will never know ProjectId or Table Name(s) that they are querying, etc.
Note: this does not address how we'd prevent users from being able to issue queries that we haven't pre-authorized? part of your question but I am adding my answer as you asked me to do
Meantime - at least theoretically - you can embed some logic into your PrivateView, which will be querying some internal metatable with info which user and when allowed to get result. Assuming that such meta-table will be managed by your backend based on authentication/token or whatever else you have in mind
Below is simplified and brief outline of that approach
#standardSQL
WITH `projectA.datasetA.table` AS (
SELECT 'data1' col UNION ALL
SELECT 'data2' UNION ALL
SELECT 'data3'
), `projectA.datasetA.applicationPermissions` AS (
SELECT 'user1#gmail.com' user UNION ALL
SELECT 'user2#gmail.com'
), `projectA.datasetB.privateView` AS (
SELECT d.*
FROM `projectA.datasetA.table` d
CROSS JOIN `projectA.datasetA.applicationPermissions` p
WHERE LOWER(user) = LOWER(SESSION_USER())
), `projectA.datasetC.publicView` AS (
SELECT *
FROM `projectA.datasetB.privateView`
)
SELECT *
FROM `projectA.datasetC.publicView`
If user1#gmail.com or user2#gmail.com will run below query
SELECT *
FROM `projectA.datasetC.publicView`
they will get below result
Row col
1 data1
2 data2
3 data3
while if user3#gmail.com will run same very query - result will be
Row col
Query returned zero records.
Obviously, you can extend your meta-table (applicationPermissions) with for example timeframe during which user will be allowed to get result (respective lines to check time conditions will need to be added to projectA.datasetB.privateView )

Stored procedure azure Cosmos DB returns empty collection

I tried to create a stored procedure using the sample sp creation code from Azure docs, but i couldn't fetch the collection details. It always returns null.
Stored Procedure
// SAMPLE STORED PROCEDURE
function sample(prefix) {
var collection = getContext().getCollection();
console.log(JSON.stringify(collection));
// Query documents and take 1st item.
var isAccepted = collection.queryDocuments(
collection.getSelfLink(),
'SELECT * FROM root r',
function (err, feed, options) {
if (err) throw err;
// Check the feed and if empty, set the body to 'no docs found',
// else take 1st element from feed
if (!feed || !feed.length) {
var response = getContext().getResponse();
response.setBody('no docs found');
}
else {
var response = getContext().getResponse();
var body = { prefix: prefix, feed: feed[0] };
response.setBody(JSON.stringify(body));
}
});
if (!isAccepted) throw new Error('The query was not accepted by the server.');
}
The console shows only this.
the results shows no doc found because of not getting collection.I have passed the partition key at time of execution via explorer.

I had a similar issue. I think the Azure portal doesn't execute stored procedures properly when the partition key is not a string.
In my case I had a partitionKey that is a number. When I executed the stored procedure via the portal I always got an empty resultSet, even though I had documents in my database. When I changed the structure a little, and made my partitionKey a string, the stored procedure worked fine.

Did you create the ToDoList Database with the Items Collection? Yo can do this from the Quick start blade in the Azure portal.
And then create an SP to run against that collection. There is no partition key required, so no additional params are required (leave blank).
The Collection is created without any documents. You may choose to add documents via the Query Explorer blade or via the sample ToDoList App that is available via the Quick start blade.

You are debugging in a wrong way.
It is perfectly fine to see "{\"spatial\":{}}" in your console log, even if the collection has items. Why? well because that is a property of that object.
So regarding what you said:
the results shows no doc found because of not getting collection
is false. I have the same console log text, but I have items in my collection.
I have 2 scenarios for why your stored procedure return no items:
I had the same issue trying on azure portal UI(in browser) and for my surprise I had to insert an item without the KEY in order that my stored procedure to see it.
On code you specify the partition as a string ie. new PartitionKey("/UserId") instead of your object ie. new PartitionKey(stock.UserId)

How to store user input into SQLite3 table using javascript?

I have a basic web page with two text boxes, one for the user's first name and the other for the user's last name. Once their name is entered, I want to save the data into a sqlite3 table via a button. The script I currently have in place is as follows:
<script>
document.getElementById('btnSave').addEventListener('click', AddRecord);
function AddRecord()
{
const sqlite3 = require('sqlite3').verbose();
let db = new sqlite3.Database('C:/Users/RedCode/Documents/Programming Tutorials/Practice Site/test.db');
var FirstName = document.getElementById('txtUsername').value;
var LastName = document.getElementById('txtSurname').value;
db.run('INSERT INTO testtbl(First_Name, Last_Name) VALUES("FirstName", "LastName")');
db.close();
}
</script>
When I check the table, the data is not stored in it. What is the correct way to save that user's data into the table?

This is not something you can achieve in browser. The tool you are trying to use is designed to be used with node which has access to the file system and can make calls to read and write from the sqlite.db file at will. Browsers cannot access the file system in this way.
If you really really need sqlite for real SQL type queries in your web app you can use something like: https://github.com/kripken/sql.js/ which is designed to be used in browser.
For regular storing of data in browser however there are inbuilt datastore tools
Web Storage
Index DB

Use JS to execute MySQL queries and the security issues it involves

I've been searching around the internet for a way to define a query in JavaScript, pass that query to PHP. Let PHP set up a MySQL connection, execute the query and return the results json encoded.
However my concern is with the security of this method since users could tamper with the queries and do things you don't want them to do or request data you do not want them to see.
Question
In an application/plugin like this, what kind of security measures would you suggest to prevent users from requesting information I don't want them to?
Edit
The end result of my plugin will be something like
var data = Querier({
table: "mytable",
columns: {"column1", "column2", "column3"},
where: "column2='blablabla'",
limit: "10"
});
I'm going to let that function make an AJAX request and execute a query in PHP using the above data. I would like to know what security risks this throws up and how to prevent them.

It's unclear from your question whether you're allowing users to type queries that will be run against your database, or if your code running in the browser is doing it (e.g., not the user).
If it's the user: You'd have to really trust them, since they can (and probably will) destroy your database.
If it's your code running in the browser that's creating them: Don't do that. Instead, have client-side code send data to the server, and formulate the queries on the server using full precautions to prevent SQL Injection (parameterized queries, etc.).
Re your update:
I can see at least a couple issues:
Here's a risk right here:
where: "column2='blablabla'"
Now, suppose I decide to get my hands on that before it gets sent to the server and change it to:
where: "column2=');DROP TABLE Stuff; --"
You can't send a complete WHERE clause to the server, because you can't trust it. This is the point of parameterized queries:
Instead, specify the columns by name and on the PHP side, be sure you're doing correct handling of parameter values (more here).
var data = Querier({
table: "mytable",
columns: {"column1", "column2", "column3"},
where: {
column2: {
op: '=',
value: 'blablabla'
}
}
limit: "10"
});
Now you can build your query without blindly trusting the text from the client; you'll need to do thorough validation of column names, operators, etc.
Exposing information about your scheme to the entire world is giving up information for free. Security is an onion, and one of the outer layers of that onion is obscurity. It's not remotely sufficient unto itself, but it's a starting point. So don't let your client code (and therefore anyone reading it) know what your table names and column names are. Consider using server-side name mapping, etc.

Depending on how you intend to do, you might have a hole bigger than the one made in this economy or no hole at all.
If you are going to write the query on client-side, and send to php, I would create a user with only select, insert, delete and update, without permissions to access any other database.
Ignore this if you use SQlite.
I advise against this!
If you build the query on server-side, just stuff to the server the data you want!
I would change the code into something like this:
var link = QuerierLink('sql.php');//filename to use for the query
var data = Querier('users',link);//locks access to only this table
data.select({
columns: ['id','name','email'],
where: [
{id:{'>':5}},
{name:{'like':'%david%'}}
],
limit:10
});
Which, on server-side, would generate the query:
select `id`,`name`,`email` from `db.users` where `id`>5 and `name` like '%david%' limit 10
This would be a lot better to use.
With prepared statements, you use:
select `id`,`name`,`email` from `db.users` where `id`>:id and `name` like :name limit 10
Passing to PDO, pseudo-code:
$query='select `id`,`name`,`email` from `'.$database_name.'.users` where `id`>:id and `name` like :name limit 10';
$result=$PDO->exec($query,array(
'id'=>5,
'name'=>'%david%'
)
);
This is the prefered way, since you have more control over what is passed.
Also, set the exact database name along the name of the table, so you avoid users accessing stuff from other tables/databases.
Other databases include information_schema, which has every single piece of information from your entire databasem, including user list and restrictions.
Ignore this for SQlite.
If you are going to use MySQL/MariaDB/other you should disable all read/write permissions.
You really don't want anyone writting files into your server! Specially into any location they wish.
The risk: They have a new puppy for the attackers to do what they wish! This is a massive hole.
Solution: Disable FILE privileges or limit the access to a directory where you block external access using .htaccess, using the argument --secure_file_priv or the system variable ##secure_file_priv.
If you use SQlite, just create a .sqlite(3) file, based on a template file, for each client connecting. Then you delete the file when the user closes the connection or scrap every n minutes for files older than x time.
The risk: Filling your disk with .sqlite files.
Solution: Clear the files sooner or use a ramdisk with a cron job.
I've wanted to implement something like this a long ago and this was a good way to exercice my mind.
Maybe I'll implement it like this!

Introducing easy JavaScript data access
So you want to rapidly prototype a really cool Web 2.0 JavaScript application, but you don't want to spend all your time writing the wiring code to get to the database? Traditionally, to get data all the way from the database to the front end, you need to write a class for each table in the database with all the create, read, update, and delete (CRUD) methods. Then you need to put some marshalling code atop that to provide an access layer to the front end. Then you put JavaScript libraries on top of that to access the back end. What a pain!
This article presents an alternative method in which you use a single database class to wrap multiple database tables. A single driver script connects the front end to the back end, and another wrapper class on the front end gives you access to all the tables you need.
Example/Usage
// Sample functions to update authors
function updateAuthorsTable() {
dbw.getAll( function(data) {
$('#authors').html('<table id="authors"><tr><td>ID</td><td>Author</td></tr></table>');
$(data).each( function( ind, author ) {
$('#authors tr:last').after('<tr><td>'+author.id+'</td><td>'+author.name+'</td></tr>');
});
});
}
$(document).ready(function() {
dbw = new DbWrapper();
dbw.table = 'authors';
updateAuthorsTable();
$('#addbutton').click( function() {
dbw.insertObject( { name: $('#authorname').val() },
function(data) {
updateAuthorsTable();
});
});
});
I think this is exactly what you're looking for. This way you won't have to build it yourself.

The more important thing is to be careful about the rights you grant to your MySQL user for this kind of operations.
For instance, you don't want them to DROP a database, nor executing such request:
LOAD DATA LOCAL INFILE '/etc/passwd' INTO TABLE test FIELDS TERMINATED BY '\n';
You have to limit the operations enabled to this MySQL user, and the tables he has accessed.
Access to total database:
grant select on database_name.*
to 'user_name'#'localhost' identified by 'password';
Access to a table:
grant select on database_name.table_name
to 'user_name'#'localhost' identified by 'password';
Then... what else... This should avoid unwanted SQL injection for updating/modifying tables or accessing other tables/databases, at least, as long as SELECT to a specific table/database is the only privillege you grant to this user.
But it won't avoid an user to launch a silly bad-performance request which might require all your CPU.
var data = Querier({
table: "mytable, mytable9, mytable11, mytable12",
columns: {"mytable.column1", "count(distinct mytable11.column2)",
"SUM(mytable9.column3)"},
where: "column8 IN(SELECT column7 FROM mytable2
WHERE column4 IN(SELECT column5 FROM mytable3)) ",
limit: "500000"
});
You have to make some check on the data passed if you don't want your MySQL server possibly down.

delete row from result set in web sql with javascript

I understand that the result set from web sql isn't quite an array, more of an object? I'm cycling through a result set and to speed things up I'd like to remove a row once it's been found. I've tried "delete" and "splice", the former does nothing and the latter throws an error. Here's a piece of what I'm trying to do, notice the delete on line 18:
function selectFromReverse(reverseRay,suggRay){
var reverseString = reverseRay.toString();
db.transaction(function (tx) {
tx.executeSql('SELECT votecount, comboid FROM counterCombos WHERE comboid IN ('+reverseString+') AND votecount>0', [], function(tx, results){
processSelectFromReverse(results,suggRay);
});
}, function(){onError});
}
function processSelectFromReverse(results,suggRay){
var i = suggRay.length;
while(i--){
var j = results.rows.length;
while(j--){
console.log('searching');
var found = 0;
if(suggRay[i].reverse == results.rows.item(j).comboid){
delete results.rows.item(j);
console.log('found');
found++;
break;
}
}
if(found == 0){
console.log('lost');
}
}
}

Actually you use DELETE sql command to delete the record from the database. For example:
tx.executeSql('DELETE FROM counterCombos WHERE comboid = ?', [comboid], success, error);
processSelectFromReverse could modified to include tx as input like
processSelectFromReverse(results,suggRay,tx)

Web SQL has been dropped from the HTML5 workup for almost 2 years now and isn't supported. Web Storage and Indexed DB have taken its place.
Your syntax for deleting objects: delete object; is correct. It's not used often in JavaScript but I have used it and that is the correct usage.
Using web-storage (more commonly called local storage) you would instansiate a localStorage object which comes with the method removeItem(item key), already built in.
You can see more at the above link on webstorage or go here for a tutorial.

If your concern is IOS or Android, making an application that is wrapped natively would allow you to create a javascript interface. From this interface you can access (for android) sqlite and create a nice, query0able solution. The downside is this would require people downloading your application rather then accessing a site.
http://caniuse.com/indexeddb
indexeddb support is growing rather slowly. With a proper coding style you could probably use local storage (as mentioned in the above comments) with pretty respectable speeds.