We Keep Coding

Google app script setValues() runs perfectly until it reaches a data validation error

This works perfectly if none of the data-validated cells raise an error on the sheet being written to. However, once the script reaches a cell where the copied value doesn't follow the data validation rules, it returns an error and stops executing. How can I ignore this error, continue to write into the cell, and continue running the script?
function addNew() {
Week = "2022-01-03"
x = 4
y = 17
for (var i=0; i<21; i++)
{
var readRange = SpreadsheetApp.getActive().getRange(`Versionality!I${x}:K${y}`)
var readValues = readRange.getValues();
var writeRange = SpreadsheetApp.openById("------");
var writeValues = writeRange.getRange(`${Week}!H${x}:J${y}`)
writeValues.setValues(readValues);
x += 17
y += 17
}
}

Here is an example of how you can “disable/enable” data validation to copy data:
Sample code:
function myFunction(){
// IMPORTANT: Make sure source and target dimensions are the same.
var sourceRange = SpreadsheetApp.getActive().getRange("source!B11:D11"); //Get source range of cells to be copied
var sourceRangeDVs = sourceRange.getDataValidations(); // cache the data validations currently applied to the sourceRange
sourceRange.clearDataValidations(); //clear validations on the source range before getting source values
var sourceValues = rangeSource.getValues(); //getting source values
//Next, get target range where the data will be set.
//Note: if you have to open a new file and you are handling several ranges to be copied in a loop,
// you may want to cache the target file on a variable to avoid opening every iteration of the loop
var targetRange = SpreadsheetApp.getActive().getRange("target!E6:G6"); //Get target range of cells for the values to be set
targetRange.setValues(sourceValues); //Set values copied from source in the target range
sourceRange.setDataValidations(sourceRangeDVs); //Set Data Validations cached back to source range
}
Please note that the spreadsheet names and ranges set on the sample code above are merely to exemplify, please modify it accordingly.

Let me propose this. You are doing a lot of getValues()/setValues() which can cause a performance issue. What I like to do is use only one getValues() and then we can extract what we need from the full array and write our portions of it as needed.
Also you were repeatedly opening the same spreadsheet by id.
I didn't test it because your date structure would be too hard to set up. But I'm pretty confident it will work.
function addNew() {
let Week = "2022-01-03";
let x = 3; // array index is 1 less than row
let y = 17;
let spread = SpreadsheetApp.getActiveSpreadsheet();
let readSheet = spread.getSheetByName("Versionality");
let readValues = readSheet.getDataRange().getValues();
let writeSheet = SpreadsheetApp.openById("------").getSheetByName(Week);
for (let i=0; i<21; i++) {
let subArray = [];
let j=x;
while( j < y ) {
subArray.push(readValues[j].slice(7,10)); // columns I to K
j++;
}
writeSheet.getRange(x+1,8,14,3).setValues(subArray); // columns H to J
x += 17;
y += 17;
}
}

How to fix the Error : TypeError : Cannot read properly 'setValues' of undefined google app script. How to implement it to my the code?

I made files with table. On those table there is some datas that are changing according to each files. So I created a file where there is a table that sum all of the others tables that are in others files.
First i made my code that sum tables of two files :
function juneFunction() {
var sheet_origin1=SpreadsheetApp
.openById("ID1")
.getSheetByName("Calcul/Machine");
var sheet_origin2=SpreadsheetApp
.openById("ID2")
.getSheetByName("Calcul/Machine");
var sheet_destination=SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
//define range as a string in A1 notation
var range1="BU4:CF71"
var values1=sheet_origin1.getRange(range1).getValues();
var values2=sheet_origin2.getRange(range1).getValues();
var destinationrange=sheet_destination.getRange("C3:N70");
//Here you sum the values of equivalent cells from different sheets
for(var i=0; i<values1.length;i++)
{
for(var j=0; j<values1[0].length;j++)
{
sum=values1[i][j]+values2[i][j];
destinationrange.getCell(i+1,j+1).setValue(sum);
}
}
}
This code is working perfectly.
But my goal is to sum 26 tables of 26 files.
So I tried to do it with 3 files, here is the code :
function juneFunction() {
var sheet_origin1=SpreadsheetApp
.openById("ID1")
.getSheetByName("Calcul/Machine");
var sheet_origin2=SpreadsheetApp
.openById("ID2")
.getSheetByName("Calcul/Machine");
var sheet_origin3=SpreadsheetApp
.openById("ID3")
.getSheetByName("Calcul/Machine");
var sheet_destination=SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
//define range as a string in A1 notation
var range1="BU4:CF71"
var values1=sheet_origin1.getRange(range1).getValues();
var values2=sheet_origin2.getRange(range1).getValues();
var values3=sheet_origin3.getRange(range1).getValues();
var destinationrange=sheet_destination.getRange("C3:N70");
//Here you sum the values of equivalent cells from different sheets
for(var i=0; i<values1.length;i++)
{
for(var j=0; j<values1[0].length;j++)
{
for(var k=0; k<values1[0].length;k++){
sum=values1[i][j][k]+values2[i][j][k]+values3[i][j][k];
destinationrange.getCell[[i+1,j+1,k+1]].setValues(sum);
}
}
}
}
Here I have this error : TypeError : Cannot read properly 'setValues' of undefined. This error happen here :
destinationrange.getCell[[i+1,j+1,k+1]].setValues(sum);
I think that the error is coming from the .getCell but i dont know why ...

Range.getCell() takes two arguments: the row offset and the column offset. You are handing it just one argument: an array of three numbers.
You should not be using Range.getCell() in the first place. Instead, use Array.map() to get the sheets, Array.forEach() to iterate over them, and finally Range.setValues() to write all results in one go, like this:
function sumRangeAcrossManySpreadsheets() {
const spreadSheetIds = ['id1', 'id2', 'id3',];
const sheetName = 'Calcul/Machine';
const rangeA1 = 'BU4:CF71';
const targetRange = SpreadsheetApp.getActiveSheet().getRange('C3');
const sheets = spreadSheetIds.map(id => SpreadsheetApp.openById(id).getSheetByName(sheetName));
const result = [];
sheets.forEach((sheet, sheetIndex) => {
if (!sheet) {
throw new Error(`There is no sheet ${sheetName} in spreadsheet ID '${spreadSheetIds[sheetIndex]}'.`);
}
const values = sheet.getRange(rangeA1).getValues();
values.forEach((row, rowIndex) => row.forEach((value, columnIndex) => {
if (!result[rowIndex]) {
result[rowIndex] = new Array(row.length).fill(0);
}
result[rowIndex][columnIndex] += Number(value) || 0;
}));
});
targetRange.offset(0, 0, result.length, result[0].length).setValues(result);
}

Google Sheets Script Exceeded maximum execution time, even with flush option?

So I am a noob in coding but managed to adjust the code a bit to do what I need it to do and that is to list the files inside a folder and its subfolder files.
The issue is that the files total is about 50k or more and keeps increasing each day XD
so now most of the time I get a "Exceeded maximum execution time" and sometimes's I don't. Inside the script, there is a flush function so it should reset the timer if I am correct?
I run the script every day I do not know what to do to fix this?
I think the best would be to have a function that will check if it is already listed and if not updated to skip it to speed up the script but again I just do not know atm ware to start.
If someone could help me to fix the " Exceeded maximum execution time" I would be extremely grateful.
this is the script:
function ListarTodo() {
/* Adapted from Code written by #Andres Duarte in this link:
https://stackoverflow.com/questions/59045664/how-to-list-also-files-inside-subfolders-in-google-drive/63182864#63182864
*/
// List all files and sub-folders in a single folder on Google Drive
// declare the folder name
var foldername = 'Beelden';
// declare this sheet
var sheet = SpreadsheetApp.getActiveSheet();
// clear any existing contents
sheet.clear();
// append a header row
sheet.appendRow(["Folder","Name", "Last Updated", "Size MB", "URL"]);
// getFoldersByName = Gets a collection of all folders in the user's Drive that have the given name.
// folders is a "Folder Iterator" but there is only one unique folder name called, so it has only one value (next)
var folders = DriveApp.getFoldersByName(foldername);
var foldersnext = folders.next();
var lintotal = 2;
// Initiate recursive function
lintotal = SubCarpetas(foldersnext, foldername, lintotal);
}
function SubCarpetas(folder, path, cantlineas) {
cantlineas = ListarArchivos(folder, path, cantlineas);
var subfolders = folder.getFolders();
while (subfolders.hasNext()) {
var mysubfolders = subfolders.next();
var mysubfolderName = mysubfolders.getName();
var newpath = "";
newpath = path + "/" + mysubfolderName;
cantlineas = SubCarpetas(mysubfolders, newpath, cantlineas);
}
return(cantlineas)
}
// list files in this folder
// myfiles is a File Iterator
function ListarArchivos(mifoldersnext, mipath, milintotal) {
var datos = []; //temporary array that we are going to use to record on the sheet
var files = []; //array with all the files that we find in the folder that we are evaluating
var file = []; //array that we use to dump the data of each file before saving it
var total = 0;
var sheet = SpreadsheetApp.getActiveSheet();
var myfiles = mifoldersnext.getFiles();
// We create an array with the data of each file and save the total number of files
while (myfiles.hasNext()) {
files.push(myfiles.next());
total++;
}
// we sort the array by file names alphabetically // sorts the files array by file names alphabetically
files = files.sort(function(a, b){
var aName = a.getName().toUpperCase();
var bName = b.getName().toUpperCase();
return aName.localeCompare(bName);
});
////
var vuelta = 0;
var bulk = 500; // We define the number of lines to record each time, in the GoogleDoc spreadsheet
var linea = milintotal; // we define in which line we are going to save in the spreadsheet
for (var i = 0; i < files.length; i++) { // we go through the array of files and format the information we need for our spreadsheet
file = files[i];
var fname = file.getName(); //file name
var fdate = file.getLastUpdated(); // date and time last modified
var fsize = file.getSize()/1024/1024; // file size, we pass it from byte to Kbyte and then to Mb
fsize = +fsize.toFixed(2); // we format it to two decimal places
var furl = file.getUrl(); //File URL
datos[vuelta] = [mipath+" ("+total+")", fname, fdate, fsize, furl]; // we put everything inside a temporary array
vuelta++;
if (vuelta == bulk) {// when it reaches the defined quantity, save this array with 10 lines and empty it
linea = milintotal;
// Logger.log("linea = "+linea); //DEBUG
// Logger.log("vuelta = "+vuelta); //DEBUG
// Logger.log("total = "+total); //DEBUG
// Logger.log("lintotal = "+milintotal); //DEBUG
// Logger.log("registros en datos = "+datos.length); //DEBUG
// Logger.log("data = "+datos); //DEBUG
sheet.getRange(linea, 1, bulk,5).setValues(datos); // we save the data of the temporary array in the sheet
SpreadsheetApp.flush(); // we force the data to appear on the sheet - without this the data does not appear until finished (it generates a lot of impatience)
milintotal = milintotal + vuelta;
datos = []; // empty the temporary array
vuelta = 0;
}
}
if (datos.length>0) {// When exiting the loop we record what is left in the data array
linea = milintotal;
// Logger.log("linea = "+linea); //DEBUG
// Logger.log("vuelta = "+vuelta); //DEBUG
// Logger.log("total = "+total); //DEBUG
// Logger.log("lintotal = "+milintotal); //DEBUG
// Logger.log("records in data = "+ data.length); //DEBUG
// Logger.log("data = "+datos); //DEBUG
sheet.getRange(linea, 1, datos.length,5).setValues(datos);
SpreadsheetApp.flush(); //ansiolítico
milintotal = milintotal + datos.length;
datos = [];
vuelta = 0;
}
return (milintotal)
}
Thank you all
UPDATE!
So after looking at multiple forumes i found that the best thing atm i can do is make a break for this script by using the following script:
function runMe() {
var startTime= (new Date()).getTime();
//do some work here
var scriptProperties = PropertiesService.getScriptProperties();
var startRow= scriptProperties.getProperty('start_row');
for(var ii = startRow; ii <= size; ii++) {
var currTime = (new Date()).getTime();
if(currTime - startTime >= MAX_RUNNING_TIME) {
scriptProperties.setProperty("start_row", ii);
ScriptApp.newTrigger("runMe")
.timeBased()
.at(new Date(currTime+REASONABLE_TIME_TO_WAIT))
.create();
break;
} else {
doSomeWork();
}
}
//do some more work here
}
But after trying I do not understand where to divide this to make it work T-T any ideas?

I think the best would be to have a function that will check if it is already listed and if not updated to skip it to speed up the script but again I just do not know atm ware to start.
These links from the official documentation should help:
https://developers.google.com/apps-script/reference/drive/drive-app#searchFiles(String)
https://developers.google.com/apps-script/reference/drive/drive-app#searchFolders(String)
https://developers.google.com/drive/api/v2/search-files
https://developers.google.com/drive/api/v2/ref-search-terms
You can use the information in the documentation to create a script that only pulls the most recently added and/or updated files.
However, if you need to process a large volume of recently updated files you'll need to leverage some kind of batch processing solution that spans multiple sessions.

Google Apps Script - XML Parser - Regex

I am using a Google Apps Script that pulls the content from a feed in a sheet.
This is the code that I'm using:
function processXML(FeedURL,sheetsFileDestinationURL,rawPasteSheetName,OPT_childNamesArray,OPT_Namespace){
var OPT_childNamesArray = ["link"]; // get only item url from the feed
var GoogleSheetsFile = SpreadsheetApp.openByUrl(sheetsFileDestinationURL);
var GoogleSheetsPastePage = GoogleSheetsFile.getSheetByName(rawPasteSheetName);
if (OPT_childNamesArray){
GoogleSheetsPastePage.getDataRange().offset(1,0).clearContent(); // get all filled cells, omitting the header row, and clear content
}
else {
GoogleSheetsPastePage.getDataRange().offset(0,0).clearContent(); // get all filled cells, INCLUDING the header row, and clear content
}
// Generate 2d/md array / rows export based on requested columns and feed
var exportRows = []; // hold all the rows that are generated to be pasted into the sheet
var XMLFeedURL = FeedURL;
var feedContent = UrlFetchApp.fetch(XMLFeedURL).getContentText(); // get the full feed content
var feedItems = XmlService.parse(feedContent).getRootElement().getChild('channel').getChildren('item'); // get all items in the feed
for (var x=0; x<feedItems.length; x++){
// Iterate through items in the XML/RSS feed
var currentFeedItem = feedItems[x];
var singleItemArray = []; // use to hold all the values for this single item/row
// Parse for specific children (requires names and namespace)
if (OPT_childNamesArray){
for (var y=0; y<OPT_childNamesArray.length; y++){
// Iterate through requested children by name and fill rows
var currentChildName = OPT_childNamesArray[y];
if (OPT_Namespace){
if (currentFeedItem.getChild(OPT_childNamesArray[y],OPT_Namespace)){
singleItemArray.push(currentFeedItem.getChildText(OPT_childNamesArray[y],OPT_Namespace));
}
else {
singleItemArray.push("null");
}
}
else {
if (currentFeedItem.getChild(OPT_childNamesArray[y])){
singleItemArray.push(currentFeedItem.getChildText(OPT_childNamesArray[y]));
}
else {
singleItemArray.push("null");
}
}
}
exportRows.push(singleItemArray);
}
// Parse for ALL children, does not require knowing names or namespace
else if (!OPT_childNamesArray){
var allChildren = currentFeedItem.getChildren();
if (x == 0){
// if looking at first item, create a header row first with column headings
var headerRow = [];
for (var h=0; h<allChildren.length; h++){
headerRow.push(allChildren[h].getName());
}
exportRows.push(headerRow);
}
for (var c=0; c<allChildren.length; c++){
singleItemArray.push(allChildren[c].getText());
}
exportRows.push(singleItemArray);
}
}
// Paste the generated md array export into the spreadsheet
if (OPT_childNamesArray){
GoogleSheetsPastePage.getRange(2,1,exportRows.length,exportRows[1].length).setValues(exportRows);
}
else if (!OPT_childNamesArray){
var maxRangeLength = 0;
var currentRowIndex = 1;
for (var x = 0; x<exportRows.length; x++){
if (exportRows[x].length > maxRangeLength){
maxRangeLength = exportRows[x].length;
}
GoogleSheetsPastePage.getRange(currentRowIndex,1,1,exportRows[x].length).setValues([exportRows[x]]);
currentRowIndex++;
}
}
}
My problem is this:
When I run this code I get:
https://url/115-396/
https://url/115-396/
https://url/115-396/
I need to remove "115-396/".
So I tryed to add this code but didn't work:
...
// Paste the generated md array export into the spreadsheet
if (OPT_childNamesArray){
for (var k = 0; k < exportRows.length; k++) {
var re = '115-396/'
var replacingItem = '';
var URL = exportRows[0].toString().replace(re, replacingItem);
}
GoogleSheetsPastePage.getRange(2,1,exportRows.length,exportRows[1].length).setValue(URL);
}
else if (!OPT_childNamesArray){
...
Edit after #Yuri reply:
// Paste the generated md array export into the spreadsheet
if (OPT_childNamesArray){
for ( k=0; k < exportRows[0].length; k++) {
var re = '115-396/'
var replacingItem = '';
exportRows[0][k] = exportRows[0][k].toString().replace(re, replacingItem);
}
GoogleSheetsPastePage.getRange(2,1,exportRows.length,exportRows[1].length).setValues(exportRows);
}
result:
https://url/
https://url/115-396/
https://url/115-396/
Basically, the regex is applied only to the first url.
How I can make that the regex is applied to all the url's?
Any help?
Thanks

You are using a for to iterate thru the exportRow array, but later on, you're not using the k iterator inside the for.
Then, you are not accessing the exportRows array, only the first position:
var URL = exportRows[0].toString().replace(re, replacingItem);
Shouldn't be?
var URL = exportRows[k].toString().replace(re, replacingItem);
In that case, it won't work, because URL it's not an array, so by doing this you are only saving the last assignation produced on the for iterator on the URL, I believe you are trying to do the following:
for ( k=0; k < exportRows.length; k++) {
var re = '115-396/'
var replacingItem = '';
exportRows[k] = exportRows[k].toString().replace(re, replacingItem);
}
And you'll have exportRows as an array of the desired url's without the 115-396 extensions.
Now you can place this on the spreadsheet with setValue as you were doing, but setValue is for strings, integers, etc, and not for arrays. For arrays you have setValues()
GoogleSheetsPastePage.getRange(2,1,exportRows.length,exportRows[1].length).setValues(exportRows);
But, then, the range of exportRows should match the range of your getRange selection, which I'm not sure it's happening.
Just to clarify it, exportRows.length is the length of the array, and exportRows[1] is the length of the string/url stored on the position 1 of the array.
Hope this helps, the question is not really clear neither the intentions, provide more info if still not working.
How to know the size of the range you're getting?
var myrange = GoogleSheetsPastePage.getRange(2,1,exportRows.length,exportRows[1].length)
Logger.log(myrange.getNumRows());
Logger.log(myrange.getNumColumns());
You'll be able to know the range you have on getRange and make it match with the exportRows size.
Make sure to check the attached documentation, and in case you have more doubts please open a new question related to it.

How can I get a string literal (character array?) from a value in javascript?

I am using Google App Script to grab text from a Google Sheet, then upload it to Firebase. JSON encoding of any data from the sheet is adding extra escape characters that I don't need. I am trying to figure out a way to prevent this because it does not add unnecessary escape characters if I use a string literal.
This function below is what I call to sync data from the sheet to firebase. I initialize an object from the gathered data, and then I send it off to the Firebase database for updating.
If I keep it as it is, the string at data[i][1] eventually has escape characters automatically added in. If I replace data[i][1] with any literal string (i.e. "Testing\n\n-Caleb") the value of that key keeps its entire string un-modified.
// Gets the data from the sheet and puts it in an object.
// It is called by running the sync button in the sheet.
function sync() {
// get the spreadsheet data
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName("main");
var [rows, columns] = [sheet.getLastRow(), sheet.getLastColumn()];
var data = sheet.getSheetValues(1, 1, rows, columns);
// put the data in an object
testAlert(data[1][1]); // prints "Testing
//
// -Caleb" (OK)
var obj = {}
for (var i = 1; i < data.length; i++) {
obj[i - 1] = {
Happy: data[i][0],
Meh: data[i][1], // THE VALUE OF TOPIC
//Meh: "Testing\n\n-Caleb" // THE LITERAL STRING OPTION
Down: data[i][2],
Sad: data[i][3],
Angry: data[i][4]
}
}
testAlert(obj[0].Meh). // prints "Testing
//
// -Caleb" (OK)
// convert to unused JSON object to print to the window what is happening
var jsonOBJ = JSON.stringify(obj[0].Meh)
testAlert(jsonOBJ). // prints "\"Testing\\n\\n-Caleb\"" (ERROR)
updateFirebaseData(obj, "emotions");
}
This is the function called at the end:
// updates the firebase database with specified data and location
function updateFirebaseData(obj, loc) {
var firebaseUrl = "https://gem-0-2.firebaseio.com/";
var secret = "FirebaseSecret";
var base = FirebaseApp.getDatabaseByUrl(firebaseUrl, secret);
var result = base.updateData(loc, obj);
}
The data at data[i][1] is "Testing\n\n-Caleb" and by the time it reaches the database, it is "\"Testing\n\n-Caleb\"", and so escaped characters that I intentionally place, like end-lines, aren't translated properly. However, if I put "Testing\n\n-Caleb" as the value for "Meh", then it stays as-is.

sheet.getSheetValues() is returning a string that's already encoded as JSON. If you want to store the actual string, you need to decode it with JSON.parse().
for (var i = 1; i < data.length; i++) {
obj[i - 1] = {
Happy: data[i][0],
Meh: JSON.parse(data[i][1]), // THE VALUE OF TOPIC
Down: data[i][2],
Sad: data[i][3],
Angry: data[i][4]
}