This question already has answers here:
You do not have permission to call openById
(5 answers)
Dynamic cell position; No permission to call setValues
(1 answer)
Closed 9 months ago.
I have 2 tabs in my Google Sheet:
dashboard:
db:
On dashboard in column B I call function yahoofinance(). This function checks if for the given ticker there exists data in db. If so, this data is returned. If not, OR if so but the data is empty, Yahoo! Finance is contacted to retrieve the data. So far so good.
Take JPM as an example now. It is called in row 3 in dashboard. In db we do find JPM but there is no data for the ticker, so we retrieve it live from Yahoo! Finance. Subsequently, I want to update the JPM row in db with this data, so that next time we open the dashboard, we do not contact Yahoo! again for this information.
However, see line under // update existing row.. the code generates an error Exception: You do not have permission to call setValues and I do not know how to solve it. Do you? Any help is greatly appreciated!
function yahoofinance(ticker) {
// First check if we have this data stored already
var db = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('db');
var tickers = db.getRange('A2:A').getValues();
var stored = false;
var row = 2;
for (var r = 0; r <= tickers.length; r++) {
if (tickers[r] == ticker) { stored = true; row = row + r; }
}
if (stored == true) { // the ticker is known in db
var range = db.getRange(row, 2, 1, 4);
if (range.isBlank()) { // ticker is known but no data yet
var data = get_live_data(ticker);
// update existing row
db.getRange(row, 2).setValues(data);
// return data
return data;
}
else {
return range.getValues();
}
}
else {
var data = get_live_data(ticker);
// append row to db
// return data to sheet
return data;
}
}
function get_live_data(ticker) {
const url = 'https://query2.finance.yahoo.com/v10/finance/quoteSummary/' + encodeURI(ticker) + '?modules=price,assetProfile,summaryDetail';
let response = UrlFetchApp.fetch(url, { muteHttpExceptions: true });
if (response.getResponseCode() == 200) {
var object = JSON.parse(response.getContentText());
}
let fwdPE = object.quoteSummary.result[0]?.summaryDetail?.forwardPE?.fmt || '-';
let sector = object.quoteSummary.result[0]?.assetProfile?.sector || '-';
let mktCap = object.quoteSummary.result[0]?.price?.marketCap?.fmt || '-';
return [[fwdPE, sector, mktCap]];
}
So based on line under // update existing row, you want to add the data in the other sheet and based on documentation, custom functions return values but will only set values in the current cell where you're using the function, you can't modify data in other cell and that's why you're getting the error. You're calling the function in B4 and are trying to set values in other sheet at the same time which is not allowed.
A custom function cannot affect cells other than those it returns a
value to. In other words, a custom function cannot edit arbitrary
cells, only the cells it is called from and their adjacent cells.
Related
I have around 300 Spreadsheets that I need to copy all data from each spreadsheet and merge into a Master Spreadsheet. I have a spreadsheet that lists all 300 spreadsheet Ids. This script works however its Very slow!
I also tried to manually enter all document Ids as a variable and it did not seem to make a difference.
Is there a better way to handle?
function combineData() {
const masterID = "ID";
const masterSheet = SpreadsheetApp.openById(masterID).getSheets()[0];
let targetSheets = docIds();
for (let i = 0, len = targetSheets.length; i < len; i++) {
let sSheet = SpreadsheetApp.openById(targetSheets[i]).getActiveSheet();
let sData = sSheet.getDataRange().getValues();
sData.shift() //Remove header row
if (sData.length > 0) { //Needed to add to remove errors on Spreadsheets with no data
let fRow = masterSheet.getRange("A" + (masterSheet.getLastRow())).getRow() + 1;
let filter = sData.filter(function (row) {
return row.some(function (cell) {
return cell !== ""; //If sheets have blank rows in between doesnt grab
})
})
masterSheet.getRange(fRow, 1, filter.length, filter[0].length).setValues(filter)
}
}
}
function docIds() {
let listOfId = SpreadsheetApp.openById('ID').getSheets()[0]; //list of 300 Spreadsheet IDs
let values = listOfID.getDataRange().getValues()
let arrayId = []
for (let i = 1, len = values.length; i < len; i++) {
let data = values[i];
let ssID = data[1];
arrayId.push(ssID)
}
return arrayId
}
I believe your goal is as follows.
You have 300 Spreadsheets.
You want to retrieve the values from the 1st tab of all Spreadsheets and also, you want to put the retrieved values to the 1st tab of the master Spreadsheet.
You want to reduce the process cost of the script.
Issue and workaround:
In the current stage, unfortunately, there is no method for retrieving the values from multiple Spreadsheets, simultaneously. If the sample script is prepared, it is required to obtain the values from each spreadsheet in a loop. In this case, the process cost becomes high. I think that this might be the reason for your current issue.
In this answer, as another approach, I would like to propose the following flow.
Create the URL list for exporting the values from Spreadsheets.
In the current stage, when Sheets API is used in a loop, an error occurs. So, in this workaround, I use the URL for exporting Spreadsheet as CSV data. In this case, it seems that even when this URL is accessed with a loop, no error occurs.
Retrieve CSV values from the URLs using UrlFetchApp.fetchAll.
fetchAll method works with the asynchronous process. Ref (Author: me)
Merge the retrieved values by parsing CSV data as an array.
Put the values to the master Spreadsheet using Sheets API.
By this flow, I thought that the process cost can be reduced. When this flow is reflected in a sample script, how about the following sample script?
Sample script:
Please set masterID and ssId. And, please enable Sheets API at Advanced Google services. And, please run myFunction.
function myFunction() {
const masterID = "###"; // Please set the master Spreadsheet ID.
const ssId = "###"; // Please set the Spreadsheet ID including the Spreadsheet IDs you want to retrieve in column "B".
// Retrieve Spreadsheet IDs.
const sheet = SpreadsheetApp.openById(ssId).getSheets()[0];
const ssIds = sheet.getRange("B2:B" + sheet.getLastRow()).getDisplayValues().reduce((ar, [b]) => {
if (b) ar.push(b);
return ar;
}, []);
// Retrieve values from all Spreadsheets.
const workers = 50; // Please adjust this value.
const headers = { authorization: "Bearer " + ScriptApp.getOAuthToken() };
const reqs = [...Array(Math.ceil(ssIds.length / workers))].map(_ => ssIds.splice(0, workers).map(id => ({ url: `https://docs.google.com/spreadsheets/export?exportFormat=csv&id=${id}`, headers, muteHttpExceptions: true })));
const values = reqs.flatMap(r =>
UrlFetchApp.fetchAll(r).flatMap(rr => {
if (rr.getResponseCode() == 200) {
const [, ...val] = Utilities.parseCsv(rr.getContentText());
return val;
}
return [];
})
);
// Put values to the master sheet.
const masterSheet = SpreadsheetApp.openById(masterID).getSheets()[0];
Sheets.Spreadsheets.Values.update({ values }, masterID, `'${masterSheet.getSheetName()}'!A${masterSheet.getLastRow() + 1}`, { valueInputOption: "USER_ENTERED" });
// DriveApp.getFiles(); // This comment line is used for automatically detecting the scope for Drive API. So, please don't remove this line.
}
When this script is run,
Spreadsheet IDs are retrieved from column "B" of the 1st sheet in the Spreadsheet of ssId.
Values are retrieved from all Spreadsheets.
In this script, the values are retrieved from every 50 Spreadsheets with the asynchronous process. If you increase const workers = 50; to const workers = 100;, the values are retrieved from every 100 Spreadsheets. But, if an error occurs when this value is increased, please adjust the value.
Put values using Sheets API.
When I tested this script for 50 Spreadsheet, the processing time was about 20 seconds. But, I'm not sure about your actual situation. So, please test this script.
Note:
In your script, listOfID is not declared. Please be careful about this.
Unfortunately, I cannot know your all Spreadsheets. So, if all values are more than 10,000,000 cells, an error occurs because of the maximum number of cells in a Spreadsheet. Please be careful about this.
If the number of values is large, an error might occur. At that time, please check my report.
References:
fetchAll(requests)
Method: spreadsheets.values.update
The .setValues() and .getValues() function themselves already run quite heavily specially if you have large data in the sheet, and using it together with for loop will really cause it to be slow since it iterates over 1 by 1. How about changing the for loop to forEach()
Try:
function combineData() {
const masterID = "1aRQ7rW9tGF25xdmjAfOtT6HtyZKQq0_AIYOGSZMKOcA";
const masterSheet = SpreadsheetApp.openById(masterID).getSheetByName("Master");
let targetSheets = docIds();
targetSheets.forEach(function(x){
let sSheet = SpreadsheetApp.openById(x).getActiveSheet();
let sData = sSheet.getDataRange().getValues();
sData.shift() //Remove header row
if (sData.length > 0) { //Needed to add to remove errors on Spreadsheets with no data
let fRow = masterSheet.getRange("A" + (masterSheet.getLastRow())).getRow() + 1;
let filter = sData.filter(function (row) {
return row.some(function (cell) {
return cell !== ""; //If sheets have blank rows in between doesnt grab
})
})
masterSheet.getRange(fRow, 1, filter.length, filter[0].length).setValues(filter)
}
})
}
function docIds() {
let listOfId = SpreadsheetApp.openById('1aRQ7rW9tGF25xdmjAfOtT6HtyZKQq0_AIYOGSZMKOcA').getSheets()[0]; //list of 300 Spreadsheet IDs
let values = listOfId.getDataRange().getValues();
values.shift()
let arrayId = []
values.forEach(function(val){
let data = val;
let ssID = data[1];
arrayId.push(ssID)
})
return arrayId
}
Also here are some of the best practices to improve the performance of the script: Best Practices
More details on forEach:
forEach()
Let me know if this helps!
Use the Sheets API, depending on the data it is an order of magintude faster than the native SpreadsheetApp. Add the Google Sheets API under Services in the left pane of the Apps Script editor.
Here is a code snipped of how we use one or the other API:
if(gridData && gridHeight) {
let range = sheet.getRange(startRow, 1, gridHeight, gridData[0].length);
if(useSheetsAPI) {
try {
SpreadsheetApp.flush();
let valueRange = Sheets.newValueRange();
valueRange.values = gridData;
let idAndName = getSpreadsheetIdAndSheetNameByName_(sheetName);
let rangeA1 = idAndName.sheetName + '!' + range.getA1Notation();
let options = { valueInputOption: 'USER_ENTERED' };
let result = Sheets.Spreadsheets.Values.update(valueRange, idAndName.spreadsheetId, rangeA1, options);
debugLog_('sheetReplace(): Sheets.Spreadsheets.Values.update result: '+result);
} catch (err) {
Logger.log('sheetReplace() ERROR: %s', err.message);
return 'ERROR: sheetReplace() failed: ' + err.message;
}
} else {
range.setValues(gridData);
}
}
/**
* Get spreadsheet Id and sheet name by sheet name
*
* #param {string|null} name name of sheet, either "sheet_id:Tab Name", "Tab Name"
* #return {object} object object with spreadsheetId and sheetName
*/
function getSpreadsheetIdAndSheetNameByName_(name) {
let spreadsheetId = '';
if(name && name.length > 44 && name.indexOf(':') > 40) {
// assume format: "sheet_id:Tab Name"
spreadsheetId = name.replace(/:.*$/, '');
name = name.replace(/^.*?:/, '');
} else {
// assume format "Tab Name"
spreadsheetId = SpreadsheetApp.getActiveSpreadsheet().getId();
}
return { spreadsheetId: spreadsheetId, sheetName: name };
}
Also, I submitted an enhancement request for better performance, see https://issuetracker.google.com/issues/222337394 and vote for it.
While ive seen similar questions ive not seen this specific one if im not mistaken.
So, i need to read all the info in a certain column (in this case it would be AF, regardless i havent been using that spreadsheet to try what ive been doing out) and then check whether a cell in that column has the value(or string, sorry new to Google Script) "Finalized" then copy the entire row that corresponds to that cell to a secondary sheet which would store all the finished cases and stuff.
ive been trying to find something thru google but its always how to copy the entire sheet and thats not useful i think, also what ive done rn copies the cells regardless of what the value is on that cell+column
function copyIf() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var source = ss.getSheetByName("Stuff");
var destination = ss.getSheetByName("Terminados");
var condition = source.getRange('D2:D2000').getValue();
if (condition == "Terminado") {
source.getRange('A2').copyTo(destination.getRange('A2'));
source.getRange('A3').copyTo(destination.getRange('A3'));
}
}
i am thinking i should probably implement some sort of for or while loop cause i want this to run constantly so whenever someone changes the status of a certain cell to Terminado to copy that row to the secondary sheet.
Try this
function copyIf() {
var feuille = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Stuff");
var archive = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Terminados");
if (feuille.getLastRow()==1){
SpreadsheetApp.getActive().toast('No data!', 'End of script 🗃️')
return
}
var data = feuille.getRange(2,1,feuille.getLastRow()-1,feuille.getLastColumn()).getValues()
var archiveData = []
var lignes = []
var ligne = 1
var col = 31 // AF
try {
data.forEach(function (row) {
if (row[col]=="Terminado") {
archiveData.push(row)
lignes.push(ligne)
}
})
archive.getRange(archive.getLastRow() + 1, 1, archiveData.length, archiveData[0].length).setValues(archiveData)
// lignes.reverse().forEach(x => feuille.deleteRow(x));
SpreadsheetApp.getActive().toast('Rows '+lignes.flat()+' hab been archived !', 'End of script 🗃️')
} catch (e) {
SpreadsheetApp.getActive().toast('No data to be archived!', 'End of script 🗃️')
}
}
If you want to delete rows after process, remove // before lignes.reverse().forEach(x => feuille.deleteRow(x));
I'm writing a JavaScript function to allow users of an application built using Oracle Application Express 21.1 to paste data from Excel spreadsheets into an Interactive Grid and save the data. Using the APEX JavaScript API I can update the model of the Interactive Grid with the data; the pasted values display correctly and when I subsequently access the model the correct values are returned.
However when the Interactive Grid is saved, those values aren't saved to the underlying database table. What happens is:
Null columns updated by the JavaScript function remain null
Columns with existing data and then set by the JavaScript function become null
Null columns and columns with existing data that are then changed normally by the user are updated correctly
Columns initially set by the JavaScript function and then changed normally by the user are updated correctly
The grid is a simple Interactive Grid region based on the default EMP table, with a static ID of EmployeeGrid, and saves using the Interactive Grid process that is automatically generated when the grid was created.
I have entered the following code in the Execute when Page Loads section:
$("#EmployeesGrid_ig").on('paste', onPaste);
I have entered the following code in the Function and Global Variable Declaration section:
function onPaste(e) {
if (!e.originalEvent.clipboardData ||
!e.originalEvent.clipboardData.items) return;
let items = e.originalEvent.clipboardData.items;
let data;
for (let i = 0; i < items.length; i++) {
if (items[i].type == 'text/plain') {
data = items[i];
break;
}
}
if (!data) return;
data.getAsString(function(text) {
// Split the clipboard data into rows.
text = text.replace(/\r/g, '').trim('\n');
let rowsOfText = text.split('\n');
let rows = [];
// Iterate over each row of text and push the trimmed data into rows[]
rowsOfText.forEach(function(rowOfText) {
let row = rowOfText.split('\t').map(function(colAsText) {
return colAsText.trim().replace(/^"(.*)"$/, '$1');
});
rows.push(row);
});
// We get the focused element (i.e. where the user wants to paste).
let $focused = $('.is-focused');
// We get metadata from the Interactive Grid.
let rowId = $focused.closest('tr').data('id');
let columnIndex = $focused.index();
let headerIndex = $focused.closest('table').find('th').eq(columnIndex).data('idx');
let ig$ = apex.region("EmployeesGrid").widget();
let grid = ig$.interactiveGrid("getCurrentView");
let model = grid.model;
let columns = grid.getColumns();
let record = model.getRecord(rowId);
//Map visible columns
let visibleColumns = columns.filter(function (val) { return !val.hidden; });
visibleColumns.sort(function(a,b){return a.index - b.index;});
// Complete the Promise after the grid is out of editing mode.
rows.forEach(function(row) {
row.forEach(function(value, offset) {
if (record !== null) {
visibleColumns.forEach(function(column, visColIdx) {
if (visColIdx === (headerIndex + offset)) {
if (model.allowEdit(record)) {
model.setValue(record, column.property, Number(value));
}
}
});
}
});
// To change record, get current record index and then get next record.
let recordIndex = model.indexOf(record);
record = model.recordAt(recordIndex + 1);
});
});
}
I have created a sample application on apex.oracle.com to demonstrate the behaviour, please note that I have set the grid to allow updates to existing rows only and that only the Sal and Comm number columns can be updated.
I found a similar question raised on Oracle Communities where user Woodrow could visually see values that were automatically updated in an Interactive Grid column but those values weren't present when the page was submitted.
The answer they found was to set the value as a string:
model.setValue(record, column.property, value);
instead of a number:
model.setValue(record, column.property, Number(value));
This was necessary even if the column was declared as a 'Number' column in APEX.
Another approach is to use apex.locale JSAPI, is more APEX native way and won't cause issues in the future with APEX upgrades
var number = apex.locale.toNumber( "1,234.56" );
number = apex.locale.toNumber( "$1,234.56", "FML999G999G990D00" );
number = apex.locale.toNumber( "$1234.56", "FML999G999G990D00" );
Check this out
https://docs.oracle.com/en/database/oracle/application-express/21.2/aexjs/apex.locale.html#.toNumber
I'm creating a function in Google Apps Script. The purpose of this function is selecting the table from the document and move values to the created Spreadsheet. The problem is that I can't get the table from the document (debugging is OK, but logs show selected table as empty {}).
function addAnswersTable() {
var File = function(Path) { // File object constructor
this.Path = Path;
this.Doc = DocumentApp.openById(this.Path);
this.getTable = new function()
// This function defines
// a getTable method to get
// the first table in the Document
{
if (this.Doc != undefined) {
var range = this.Doc.getBody();
var tables = range.getTables();
var table = tables[0];
return table;
}
}
}
// Creating Excel Table, where first column
// of selected table should be moved
var Table = SpreadsheetApp.create("AnswersTable");
// Creating new File object
var TrueAnswersFile = new File
('1_ne9iBaK-Z36yUYrISr3gru3zw3Qdsneiu14sWnjn34');
// Calling getTable method to get the table placed in File
var TrueAnswersTable = TrueAnswersFile.getTable;
for (var i = 1; i <= TrueAnswersTable.getNumRows; i++) {
// Filling spreadsheet "A" column with cells'
// values from table stored in File
Table.getActiveSheet().getRange("A" + i).setValue(TrueAnswersTable.getCell(1, i).getValue());
};
}
I except the output in Spreadsheet column "A" like :
A1. Just
A2. Cells'
A3. List item with
A4. Values From Table
Actually spreadsheet is empty
You want to retrieve the values from the column "A" of Google Document and put the values to the column "A" of the created Spreadsheet.
The table of index 0 in the Document has 4 rows and 1 column.
The values of each row is Just, Cells', List item with, Values From Table.
I could understand like above. If my understanding is correct, how about this modification?
Modification points:
In your script, the method is not used as the function. By this, the method is not run.
For example, TrueAnswersFile.getTable and TrueAnswersTable.getNumRows.
No method is used.
For example, getValue() of TrueAnswersTable.getCell(1, i).getValue().
new of this.getTable = new function() is not required.
In your script, getCell(1, i) of TrueAnswersTable.getCell(1, i) retrieves the values at from column "B" of the row 2.
If you want to retrieve the values from the row 1 of the column "A", please modify to getCell(i - 1, 0). But in this modification, the start of index is 0. So you can use getCell(i, 0).
When setValue() is used in the for loop, the process cost becomes high. In your case, you can use setValues() instead of it.
When above points are reflected to your script, it becomes as follows.
Modified script:
function addAnswersTable() {
var File = function(Path) {
this.Path = Path;
this.Doc = DocumentApp.openById(this.Path);
this.getTable = function() { // Modified
if (this.Doc != undefined) {
var range = this.Doc.getBody();
var tables = range.getTables();
var table = tables[0];
return table;
}
}
}
var Table = SpreadsheetApp.create("AnswersTable");
var TrueAnswersFile = new File('1_ne9iBaK-Z36yUYrISr3gru3zw3Qdsneiu14sWnjn34');
var TrueAnswersTable = TrueAnswersFile.getTable();
var values = []; // Added
for (var i = 0; i < TrueAnswersTable.getNumRows(); i++) { // Modified
values.push([TrueAnswersTable.getCell(i, 0).getText()]) // Modified
};
Table.getRange("A1:A" + values.length).setValues(values); // Added
}
References:
getCell(rowIndex, cellIndex)
getText()
Benchmark: Reading and Writing Spreadsheet using Google Apps Script
I searched the internet and I can't find a response to this nor the documentation for it.
I need to dynamically generate Google forms questions with data from a Google spreadsheet using app script, but I don't know how to reference and read a spreadsheet.
In your spreadsheet select Tools > Script Editor and adapt this to your needs:
/**
After any change in the sheet, update the combobox options in the Form
*/
function onChange(e) {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheets()[0];
var range = sheet.getDataRange();
var values = range.getValues();
var comboValues = []; // <-- cheddar will go here
// in this example we are interested in column 0 and discarding row 1 (the titles)
for (var i = 1; i <= values.length; i++) {
var v = values[i] && values[i][0];
v && comboValues.push(v)
}
// Sort the values alphabetically, case-insensitive
comboValues.sort(
function(a, b) {
if (a.toLowerCase() < b.toLowerCase()) return -1;
if (a.toLowerCase() > b.toLowerCase()) return 1;
return 0;
}
);
Logger.log(comboValues);
// Use your form ID here. You can get it from the URL
var form = FormApp.openById('<my-form-id>');
/*
Uncomment this to display the item IDs
and pick the one that you want to modify
var items = form.getItems();
for (i = 0; i < items.length; i++) {
Logger.log("ID: " + items[i].getId(), ': ' + items[i].getType());
}
*/
form.getItemById(807137578).asListItem().setChoiceValues(comboValues);
};
To debug, select the script in the combobox and click either "play" or "debug". The first time you will have to give it permissions to interact with your spreadsheet and form.
Once you are satisfied with the result, in the editor select Resources > Triggers for the active project and add this method to be triggered with any modification on the spreadsheet (on change, not on edit).
After this, your form options will be changed in real time after any change in your spreadsheet.
It's pretty straightforward, see here: https://developers.google.com/apps-script/guides/sheets#reading
You just need to open the sheet by its doc key, select the data and read the cells as a JS object.
Here is an example which works for me, pls kindly check:
function getSpreadsheetData(sheetId) {
// This function gives you an array of objects modeling a worksheet's tabular data, where the first items — column headers — become the property names.
var arrayOfArrays = SpreadsheetApp.openById(sheetId).getDataRange().getValues();
var headers = arrayOfArrays.shift();
return arrayOfArrays.map(function (row) {
return row.reduce(function (memo, value, index) {
if (value) {
memo[headers[index]] = value;
}
return memo;
}, {});
});
}
function makeOurForm() {
var sheetId='input_your_sheet_id'
getSpreadsheetData(sheetId).forEach(function (row) {
// Set your form template as follows
var formName=row.Name
// Create your form programmatically, each row means one form
var form = FormApp.create(formName)
form.setDescription('xxx');
var capitalizedName = row.Name.charAt(0).toUpperCase() + row.Name.slice(1);
form.addSectionHeaderItem().setTitle(capitalizedName);
var item = form.addMultipleChoiceItem();
item.setTitle('xxx')
.setChoices([
item.createChoice('xxx'),
]);
form.addParagraphTextItem().setTitle('xxx');
});
}
You can get your sheet Id from url, for example:
https://docs.google.com/spreadsheets/d/YourSheetId/edit#gid=0
Let me know if you have any further questions.