converting Google Visualization Query result into javascript array - javascript

The url to the spreadsheet I am querying is
docs.google.com/spreadsheets/d/1EIBhBQY1zbdBEKXsJIY1uyvdQw0b1cIBSrBE_tZvA6Y/edit?usp=sharing
The query url being used is
https://spreadsheets.google.com/tq?tqx=out:&key=1EIBhBQY1zbdBEKXsJIY1uyvdQw0b1cIBSrBE_tZvA6Y&gid=0&headers=1&tq=select%20B%2CC%2CD%20where%20(A%20matches%20%22DIS%22)
Is there a way to convert or store this result in a JavaScript array?
var dis = ["The Walt Disney Company","Entertainment",.1]
I need to be able to manipulate the data at one point and add the new data to the visualization.
Data from one of multiple queries --> Convert to array --> Manipulate data ex: multiplying an input --> data.addRows(manipulated input);

Your query does return a string containing JSON wrapped in a function call:
var responseText = 'google.visualization.Query.setResponse({…});';
This is because you specified out: as an argument for tqx (see Google Developers guides).
If you want it all raw, you can extract and parse the JSON of multiple queries and push the data to an array, so you end up with an array of arrays of row data. For your single query, you could start from something like this:
responseJSON = JSON.parse(
responseText.replace(/(^google\.visualization\.Query\.setResponse\(|\);$)/g,'')
);
var rowsArray = [];
responseJSON.table.rows.forEach(function(row){
var rowArray = [];
row.c.forEach(function(prop){ rowArray.push(prop.v); });
rowsArray.push(rowArray);
});
console.log(rowsArray); // === [["The Walt Disney Company", "Entertainment", 0.1]]

There is a more straightforward solution to this. What you get in the response is a JSONP string whose data is hold within a callback function, just as #dakab has mentioned.
Besides this, recently Google has included some extra text in the response to help with some anti-content-sniffing protections to their API. You can read more about this in this Github thread. The response you get now is an unparseable string in this form:
/*O_o*/
google.visualization.Query.setResponse({…});
One way to deal with both issues (the "comment" string and the data hidden inside the callback function) is to evaluate the function. Whether this is risky or not is something intrinsic to the JSONP format, so you must be aware of where your response comes from and decide if it's worth the risk. But, considering it comes from a request to a Google server, and in terms of parsing, it works.
So in your case, what you could do is just declare the callback function (note that you can pass your own function name in the query string, as also mentioned in the Google Developers guides) and then evaluate it. I take inspiration on this thread:
//Declare your call back function
function callback(data){
return data;
}
//Evaluate and store the data in your callback function
var result = eval(UrlFetchApp.fetch(url + uri, options).getContentText());
In "result" you'll have an already parsed JSON that you can convert to whatever you wish.

According to Google's documentation on their Visualization API for response formats, you can add a header in your request that will return JSON without the function or comment.
If you add a header named X-DataSource-Auth in your request, the Visualization API will respond in JSON format rather than JSONP format, which is the default format of the response and includes the JSON wrapped in a function handler.
However, even with this header present, the API prepends a strange string to the response: )]}' which I think has to do with the anti-content-sniffing mentioned by #Diego. Okay, Google — even with an OAuth token do you really need to do that?
So, to get at the actual JSON in that response, you can use the following Javascript to get around it. Assume responseBody is what the API actually returns to you, and that data is storing the JSON you want.
var data = JSON.parse(responseBody.replace(/^\)]\}'\n/, ''));

Assuming str is the returned JSONP formatted response:
var str = `/*O_o*/
google.visualization.Query.setResponse({"version":"0.6","reqId":"0","status":"ok","sig":"403123069","table":{"cols":[{"id":"A","label":"Timestamp","type":"datetime","pattern":"dd/MM/yyyy HH:mm:ss"},{"id":"B","label":"AskGod Search Query","type":"string"}],"rows":[{"c":[{"v":"Date(2020,9,25,12,30,5)","f":"25/10/2020 12:30:05"},{"v":"لا أعرف لماذا"}]}],"parsedNumHeaders":1}});`
console.log(JSON.parse(str.match(/(?<=.*\().*(?=\);)/s)[0]))

Related

How to fetch a Parse object and include object references?

Can a Parse object be fetched with object.fetch and at the same time include its object references as in query.include?
Here is the query example:
let query = new Parse.Query("MyCollection");
query.include("MyObjectReference");
return query.find();
How to do it with a fetch command?
Parse JS SDK >= 2.0.2
It is possible to fetch one or multiple objects with include:
fetchWithInclude https://parseplatform.org/Parse-SDK-JS/api/master/Parse.Object.html#fetchWithInclude
fetchAllWithInclude https://parseplatform.org/Parse-SDK-JS/api/master/Parse.Object.html#.fetchAllWithInclude)
Parse JS SDK < 2.0.2
It's not possible as the docs say:
By default, when fetching an object, related Parse.Objects are not
fetched. These objects’ values cannot be retrieved until they have
been fetched like so:
var post = fetchedComment.get("parent");
post.fetch({
success: function(post) {
var title = post.get("title");
}
});
Stumbled onto this via a google search and wanted to correct the record. The accepted answer to this is not correct.
You very much CAN do what the OP is asking by using fetchWithInclude([key1,key2.subkey,key2.subkey2,etc]);
See: https://parseplatform.org/Parse-SDK-JS/api/master/Parse.Object.html
A fetch command in Parse is essentially a find with only a "where equal to" on a single object id. So you can simply make a query for a single object id and Parse will handle it like a fetch, e.g. you can restrict a table to only allow fetch and this single object id query will still pass. I haven't read into the code, but I believe that a fetch is essentially a single object id (find) query. You can then also use the include of your find query.

How to properly send URL parameters containing commas using AngularJS?

I have a feeling I am just looking at this wrong, but I want to get feedback on the proper way to pass URL query parameters through Angular's $http.get() method - specifically, parameters that contain commas.
Let's say I have the following data, to be used as URL parameters in a GET request:
var params = {
filter : [
"My filter",
"My other filter",
"A filter, that contains, some commas"
],
sort : [
"ascending"
]
};
Now, I convert this structure to some parameters that can be fed into $http.get:
var urlParams = {};
angular.forEach(params, function(value, name) {
urlParams[name] = "";
for (var i = 0; i < value.length; i++) {
urlParams[name] += value[i];
if (i < value.length - 1) {
urlParams[name] += ","
}
}
}
At this point, urlParams looks like this:
{
filter : "My filter,My other filter,A filter, that contains, some commas",
sort : "ascending"
}
Now, that isn't what I want, since the third filter parameter has now turned into three separate parameters. (The API I am working with does not allow multiple values for a parameter to be passed in any other way than: "?param=value1,value2,value3") So, what I need to do is URI encode these values first, right? So, I add a encodeURIComponent() to the above routine like this:
urlParams[name] += encodeURIComponent(value[i]);
This gives me a parameters object that looks like this:
{
filter : "My%20filter,My%20other%20filter,A%20filter%2C%20that%20contains%2C%20some%20commas",
sort : "ascending"
}
Now, I make a request:
var config = {
params : urlParams
};
$http.get("/foo", config).then(function(response){
console.log(response);
});
... and this doesn't work, since Angular encodes the URL parameters as well, so the request ends up looking like this:
GET "/foo?filter=My%2520filter,My%2520other%2520filter,A%2520filter%2C%20that%20contains%2C%20some%20commas&sort=ascending"
As you can see the parameters are being encoded twice (the % signs are being encoded as %25), which of course, won't work.
Obviously, I am doing it wrong. But what is the right way? Or, do I need to ask the API developer to accept URL parameters like this?
GET "/foo?filter=My+filter,+with+a+comma&filter=Another+filter"
where multiple parameter values are stated separately, instead of being comma delimited?
As you've described the API, there is no way to reliably pass values containing commas.
Suppose you want to pass the items ["one","two","three,four"] as a list.
If you pass the strings as-is, the API will see (after the normal server-side URL decoding)
one,two,three,four
which makes the three,four indistinguishable from two separate items.
If you pass the strings URL-encoded, the entire parameter will be double-encoded, and the API will see (again, after URL decoding)
one,two,three%2Cfour
Now the parameters are distinguishable, but this requires support from the API to URL-decode each item separately.
Suppose you pass the strings like one,two,"three,four", i.e. items containing commas are quoted. The API can decode the parameters correctly, but it needs to support a more complex syntax (quoted strings) instead of simply splitting by commas...
...and so on. The bottom line is that without additional support from the API, I don't think there is anything you can do client-side to trick it into decoding strings containing commas correctly. There are many tweaks that the API developer can make, e.g.
Accepting some escape sequence for commas within list items which is unescaped server-side.
Accepting each item in a separate URL parameter.
Accepting JSON-encoded body via POST.
You will need to ask the API developer to do something.
I think you shouldn't have used comma as delimiter of the array.
I would recommend to
send json data using POST (which requires API change)
or use another string as delimiter. For example, ###.
FYI, you can simply join your array into string like this.
array.join(',')
From $http docs
If you wish override the request/response transformations only for a single request then provide transformRequest and/or transformResponse properties on the configuration object passed into $http.
Note that if you provide these properties on the config object the default transformations will be overwritten. If you wish to augment the default transformations then you must include them in your local transformation array.
In short you can use your encodeURIComponent() functionality to replace the default one by including transformRequest property in the config object for each request or you can establish a global override
See "Transforming Requests and Responses" in $http docs for more details
Not sure why you want to send this as GET in the first place

Populate Javascript Object With Data Returned From Program Call

I have a javascript function that calls an external program and I need to put the result into an object, which will contain multiple rows with multiple values for each, example below:
$.get(programcall , function(data) {
var dealers = {};
data = {0:{'name':'name1','address':'address1','phone':'phone1','miles':1.2},1:{'name':'name2','address':'address2','phone':'phone2','miles':2.2}};
dealers = data;
});
This test works because "data" is not enclosed in quotes, however when the content of "data" is returned from the called program, it just becomes text content in "dealers".
How can I get the value stored as an object?
The called program is MINE, so I can change the format if necessary to make it work.
The data will be a list of customers with name, address etc, which I want to process using javascript and to populate a DIV.
If the string is valid JSON, use the native JSON.parse function to turn it into an object.
For example:
data = JSON.parse('{"mything": 3}')
One thing to look out for: JSON needs double quotes around key names, so {"mything": 3} works but {'mything': 3} will not validate.
Your external server call is returning string content as the data object. This is, hopefully, a valid JSON format but it is still just a string.
What you probably want to do is use jQuery's getJSON function instead of a simple $.get, since it will take care of converting the response to a JSON object similar to your example.
$.getJSON(programcall, function(data) {
// data is now a JSON object not a string, if it's valid json from your server response

Json AJAX not working , problem in Response?

I have a servlet application that takes the user input from HTML form , extracts the required data from backend and makes graphs/charts and shows them to user.
The problem I am seeing is that if user selects first option from dropdown, everything works fine, the data is extracted from backend - I can see it in the AJAX response in firebug and then its parsed by json and then maps are created.
The data that is received from backend is (what I see in AJAX response):
{"responseStr":"[47.636597,-122.189495,0,1,47.643647,-122.212038,0,26,47.505288,-122.339112,0,1,47.622741,-122.314592,0,60,47.541612,-122.129318,0,1,47.568435,-122.161237,0,166,47.682308,-122.196004,0,2,47.666673,-122.284099,0,1,47.612953,-122.316700,0,2,47.600605,-122.322286,0,30,47.589557,-122.315608,0,27,47.636351,-122.327213,0,1,47.630270,-122.177084,2,0,47.630432,-122.140126,17,0,47.621644,-122.132080,1,3,47.630808,-122.153539,86,75,47.622367,-122.337023,495,3466,47.630886,-122.306255,1423,45,47.720287,-122.090885,255,82,47.702376,-122.093340,47,4,47.676897,-122.318752,1,0,47.760994,-122.322550,1,2,47.588854,-122.221273,1,0,39.530179,-119.818395,1,1,47.631306,-122.342762,1,0,47.737242,-122.323710,1,0,47.747054,-122.305083,2,0,47.752018,-122.316452,1,0]"}
This is then parsed in json via
function Respond(REQ){
var res = JSON.parse(REQ.responseText);
var myArr = JSON.parse(res.responseStr);
//forward myArr for processing
}
Now when the same user selects option 2, all works fine, the data is extracted from backend and I can see the following in the response
{"responseStr":"[00:00:00-01:00:00,100,30,0,01:00:00-02:00:00,100,29,0,02:00:00-03:00:00,100,34,0,03:00:00-04:00:00,100,5,0,04:00:00-05:00:00,100,7,0,05:00:00-06:00:00,100,23,0,06:00:00-07:00:00,78,29,0,07:00:00-08:00:00,48,17,0,08:00:00-09:00:00,24,35,0,09:00:00-10:00:00,18,29,0,10:00:00-11:00:00,5,28,0,11:00:00-12:00:00,45,57,0,12:00:00-13:00:00,65,69,0,13:00:00-14:00:00,64,58,0,14:00:00-15:00:00,73,46,0,15:00:00-16:00:00,72,27,0,16:00:00-17:00:00,94,9,0,17:00:00-18:00:00,69,15,0,18:00:00-19:00:00,14,9,0,19:00:00-20:00:00,25,13,0,20:00:00-21:00:00,81,38,0,21:00:00-22:00:00,53,74,0,22:00:00-23:00:00,76,55,0,23:00:00-24:00:00,89,16,0]"}
but when it comes to parse this via
function Respond(REQ){
var res = JSON.parse(REQ.responseText);
var myArr = JSON.parse(res.responseStr);
//forward myArr for processing
}
something wrong happens at line 2 of the function and hence user does not see a chart.
If I were to put alerts in Respond function,
function Respond(REQ){
var res = JSON.parse(REQ.responseText);
alert('here');
var myArr = JSON.parse(res.responseStr);
alert('here2');
//forward myArr for processing
}
then I do see first alert, but not the second. However, for the first case, i see both the alerts. So there is definitely something wrong on line 2. Can someone identify this by looking at the AJAX reponse?
It's looking like this is a problem on the server side, not on the Javascript side. When JSON.parse() tries to crunch the value of responseStr, it looks at the first value in the array, which is 00:00:00-01:00:00. As this not in quotes, it's not a valid string, nor is it a valid number, and so JSON.parse is failing. (In the first example, every response value is a valid floating-point number, which is why it works.)
Also, you're currently parsing JSON twice, once as part of the jQuery ajax request, and again with the the string contained in that object. Though there's nothing inherently wrong with that, it is slower and can create bugs. You should be able to encode everything into a single JSON string on the server side, and then just use the object directly in your Javascript.
Allrit, so in order to make the above a valid string, each of the 00:00:00-01:00:00 (similar ) strings should be enclosed in double quotes. So at a place in my servlet that was fetching this data from database, earlier I was doing
resultSet.getString(1);
now I changed it to
"\"" + resultSet.getString(1)+ "\"";
and it works.

How do I remove the square brackets at the end of a JS variable name during AJAX calls?

I currently have the following javascript array:
var stuffs = ['a', 'b'];
I pass the above to the server code using jQuery's load:
var data = {
'stuffs': stuffs
};
$(".output").load("/my-server-code/", data, function() {
});
On the server side, if I print the content of request.POST(I'm currently using Django), I get:
'stuffs[]': [u'a', u'b']
Notice the [] at the prefix of the variable name stuffs. Is there a way to remove that [] before it reaches the server code?
This is default behavior in jQuery 1.4+...if you want the post to be &stuffs=a&stuffs=b instead of &stuffs[]=a&stuffs[]=b you should set the traditional option to true, like this:
$.ajaxSetup({traditional: true});
Note this affects all requests... which is usually what you want in this case. If you want it to be per-request you should use the longer $.ajax() call and set traditional: true there. You can find more info about traditional in the $.param() documentation.
When an array is submitted using a GET request, through a form or AJAX, each element is given the name of the array, followed by a pair of optionally empty square brackets. So the jQuery is generating the url http://example.com/get.php?stuff[]=a&stuff[]=b. This is the only way of submitting an array, and the javascript is following the standard.
POST requests work in exactly the same way (unless the json is sent as one long json string).
In PHP, this is parsed back into the original array, so although the query string can be a little strange, the data is recieved as it was sent. $_GET['stuff'][0] works correctly in PHP.
I'm not sure how Django parses query strings.
The [] indicates that the variable is an array. I imagine that the appending of the [] to your variable name is Python/Django's way of telling you it is an array. You could probably implement your own print function which does not show them.

Categories