Is there a Google Sheets script that would be able to transpose my data, then sort it, then transpose it back. I need to do this formula below - but as a script.
=transpose(sort(transpose(J1:N4),1,))
My intention is to be able to sort my header columns A->Z so I can have a consistent placement of header columns every time I put them in. For example, sometimes the data comes as column A:G but the next time, the headers that used to be A:G are now AGBFCDE (scrambled). Sorting the entire columns going down is pretty strait forward but I need to sort A1:G1 while keeping the data below it still associated with the proper column header. For example, it would be "move entire column left or right" until the column headers are sorted A->Z. I am hoping this method of transposing-sorting-transposing back will do the trick but other recommendations would be welcome.
column 1 column4 column5 column 2 column 3
data 1 data 4 data 5 data 2 data 3
data 1 data 4 data 5 data 2 data 3
data 1 data 4 data 5 data 2 data 3
So as you can see above, the column headers are not in order. Every time I put them in, they will not be but I would like to know if there is preferably a script or some way to then sort Column 1:Column 5 in order even though they will never be pasted that way. If there is a better way besides transposing/sorting then I would be open to any method.
Here I created a script with the same concept of Transpose -> Sort -> Transpose.
function myFunction() {
// get all the data in the sheet
var ss = SpreadsheetApp.getActiveSheet();
var range = ss.getDataRange();
var values = range.getValues();
//convert rows into columns
var data = transpose(values);
//sort data
data.sort(
function(a,b) {
return b[0]-a[0] || a[0].localeCompare(b[0]);
});
//convert columns into rows
var data1 = transpose(data);
//write data back to spreadsheet
ss.getRange(1,1,values.length,values[0].length)
.setValues(data1);
}
function transpose(matrix) {
return matrix[0].map((col, i) => matrix.map(row => row[i]));
}
Example
Before
After
References
Transposing a 2D-array in JavaScript
JavaScript Sorting Arrays
Related
I am trying to take a data range of X and Y values and place a third column's value into another spreadsheet using the X and Y values as a cell range. How can use .map() method to utilize the array for a series of tasks? Is my only real option to use a "for loop" which can be very slow to go through many rows of data?
I have tried and successfully utilized the "for loop" function to go one by one through one sheet find the values in each column and plot the third column of values in another sheet. However, the larger that list gets the slower the process
function testMap() {
//Open Active Spreadsheet App
var ss = SpreadsheetApp.getActive();
//Capture sheets by name
var sheetCoord = ss.getSheetByName('Coordinates');
var sheetPlots = ss.getSheetByName('Plots');
//Create last row and column variables
var lR = sheetCoord.getLastRow();
var lC = sheetCoord.getLastColumn();
lR = lR-1;
//Create array of all data
var data = sheetCoord.getRange(2, 1, lR, lC).getValues();
//Create variables for placement into proper sheet
var x = data.map(function(f){ return f[0] });
var y = data.map(function(f){ return f[1] });
var v = data.map(function(f){ return f[2] });
sheetPlots.getRange(y, x, lR, lC).setValues(v);
}
I expect this to map the values in the range than incrementally take each value of each range and plot the "v" cell value in the coordinates on the Plots sheet. It just says
"Cannot convert 6,2,1 to (class). (line 23, file "Code")".
You want to put values to the cells using the coordinates retrieved from the Spreadsheet.
Column "A", "B" and "C" of the sheet of Coordinates are the column number, the row number and the value, respectively.
If my understanding is correct, how about this answer?
Issues:
The issue of your script is to use row and column of getRange(row, column, numRows, numColumns) as the array. The official document of getRange(row, column, numRows, numColumns) says as follows.
row: Integer
The starting row index of the range; row indexing starts with 1.
column: Integer
The starting column index of the range; column indexing starts with 1.
numRows: Integer
The number of rows to return.
numColumns: Integer
The number of columns to return.
In your case, it seems that the values are required to be put to the cell of each coordinate. By this, when the data is large, the process cost will become high.
Unfortunately, when setValues of Spreadsheet service is used, values are put to the continuous coordinates. This cannot be used for the situation that the values are put to the discrete coordinates.
For example, if your goal can use the situation that the cells are overwritten by the values including the empty values, setValues() can be used.
Solution:
In order to resolve above issues, I would like to propose to use the method of batchUpdate of Sheets API. When Sheets API is used, the values can be put to the cells of the discrete coordinates by one API call. And from my experiment, for putting values, when the data is large, Sheets API is faster than Spreadsheet service. From this situation, I proposed to use Sheets API.
Modified script:
Before you use this script, please enable Sheets API at Advanced Google Services.
function testMap() {
var ss = SpreadsheetApp.getActive();
var sheetCoord = ss.getSheetByName('Coordinates');
var sheetPlots = ss.getSheetByName('Plots');
var lR = sheetCoord.getLastRow();
var lC = sheetCoord.getLastColumn();
lR = lR-1;
var data = sheetCoord.getRange(2, 1, lR, lC).getValues();
// I modified below script.
var sheetId = sheetPlots.getSheetId();
var requests = data.map(function(e) {
var obj = {};
if (typeof e[2] == "string") obj.stringValue = e[2];
if (typeof e[2] == "number") obj.numberValue = e[2];
return {updateCells: {
range: {sheetId: sheetId, startRowIndex: e[1] - 1, endRowIndex: e[1], startColumnIndex: e[0] - 1, endColumnIndex: e[0]},
rows: [{values: [{userEnteredValue: obj}]}],
fields: "userEnteredValue"
}};
});
Sheets.Spreadsheets.batchUpdate({requests: requests}, ss.getId());
}
References:
getRange(row, column, numRows, numColumns)
Benchmark: Reading and Writing Spreadsheet using Google Apps Script
Advanced Google services
Method: spreadsheets.batchUpdate
UpdateCellsRequest
If I misunderstood your question and this was not the result you want, I apologize. At that time, can you provide a sample Spreadsheet? By this, I would like to confirm your situation.
Regarding the error
y and x are Arrays but the getRange method that uses four arguments require that each one of them are numbers, more specifically, integers. In other words, your code is passing the wrong data type for the first two arguments Ref. https://developers.google.com/apps-script/reference/spreadsheet/sheet#getrangerow-column-numrows-numcolumns
It's worth to note that v also is an Array (1D Array) but setValues requires a 2D Array.
Regarding the use of Array.protoype.map
In a broad sense you are using it correctly. Ref. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map, but not for the result that you are looking to get.
There are several ways to achieve that result
Use "brute force" to set the cell values of each coordinate one at a time, perhaps by using a for statement, Array.prototype.forEach or other similar methods.
Use "pseudo brute force", use Class RangeList
Set all the values at once by creating a 2D Array and set all the values at once by using setValues
Some background: We have a shared Google Sheet to track our openings, screenings, and other events at a movie theater. We have a main tab ("Master") that contains all of our events and the details that go with them, and a tab for archiving ("Archive").
I would like to write a script within Google Sheets to detect events & screenings that are from yesterday and earlier based on the date (in column E), take the full row(s) (events) that meet that criteria, copy & paste them to the separate "Archive" tab, and then delete the row(s) from the "Master" tab.
Anything to point me in the right direction would be super helpful. I found a few similar responses to this but they're specific to Excel/VBA and I'm not familiar with that (or much Javascript, for that matter).
I suggested that you do some tutorials to familiarise yourself with how to write scripts.
In this answer, I will flesh out the steps that your code needs to address. You will find many existing topics on the same or similar question. This is merely in order to enable you to better search for the elements of code that you need. Consider that this may be just one way of achieving your outcome.
You have one spreadsheet with two sheets and you will refer to both sheets at different stages. getSheetByName(name) will enable you to create a re-usable variable for a sheet.
You will need to find the bottom row in each sheet. getLastRow() will help.
You want to find rows in "Master" for dates, so you need to get ALL values for "Master".
You'll start by defining the range - use getRange(row, column, numRows, numColumns), though this is just one of 5 ways to define a range.
Having defined the range you'll need the values in "Master" so that you can access the date field. Use getValues() in conjunction with the range that defined. FWIW, note how this is in plural because there are lots of values. If you just wanted a single cell, you'd use getValue().
You'll want to loop through the rows in "Master" and find those rows that have a date prior to today. The "Removing Duplicate Rows in a Spreadsheet" tutorial shows one way of looping, and you can read up on basic JavaScript "Loops and iteration".
In your scenario, there is a 'hitch' with looping. If one adopts the "usual" process, then one will loop from the first row to the last. However, you are deleting a row from "Master" and, as each row is deleted, the row numbers of the remaining rows will/may change; so the "usual" process won't do. What you need to do is two things: first) loop from the bottom of the range; this will ensure that the row numbers of remaining rows will never change; second) sort the data so that the oldest dates are at the bottom. So... now you will loop from the bottom to the top, and you will evaluate all the oldest dates without any risk that when you encounter a date greater than "today", there will be NO risk of further rows with a date less than "today". Of course, once the code is complete, you can always re-sort the data on "Master" back to any order that you might wish.
You need to compare the date in the row in "Master" with today's date and then build a if...else statement so that you can define what to do depending on the result. Comparing dates is sometimes easier said than done. This topic is relevant Checking if one date is greater than the other using Google Script and you can search on other topics for "Google Sheets Script date comparison".
When you find a date less than today, you want to copy the details of that row to "Archive". This is a two part process first) to gather there the data from the row on "Master", and second) to "copy" that data to "Archive". Gathering the data will have been covered in the tutorials. There are many options for copying the data to "Archive". You could append a row and use setValues to update the new values. An alternative is to accumulate the additional "Archive" data and add it to the "Archive" after the loops have been completed.
When you find a date less than today, you want to delete the row from "Master". There's a command for that: deleteRow(rowPosition).
You can process your function manually, on demand, or you may prefer it to be automated as a time-driven installable trigger. The option is yours.
There are many ways that you can combine these elements.
In preparing the summary above, I had to make sure that I was providing accurate and complete advice. So the following is but one approach to achieving your goal. It should be noted that my test data assumes that columns A and C are formatted for date and time respectively.
function so5710086103() {
// set up spreadsheet and sheets
var ss = SpreadsheetApp.getActiveSpreadsheet();
var master = ss.getSheetByName("Master");
var archive = ss.getSheetByName("Archive");
// get the last row and column of Master
var masterLR = master.getLastRow();
var masterLC = master.getLastColumn();
// get the last row and column of Archive
var archiveLR = archive.getLastRow();
var archiveLC = archive.getLastColumn();
//Logger.log("DEBUG: Last Row - Master = "+masterLR+", and Archive = "+archiveLR);
//Logger.log("DEBUG: Last Column - Master = "+masterLC+", and Archive = "+archiveLC);
// create a range, sort it and get the data from "Master"
var masterRange = master.getRange(2, 1, masterLR - 1, masterLC);
// sort master based on date
masterRange.sort({
column: 1,
ascending: false
});
// Logger.log("DEBUG: Master range = "+masterRange.getA1Notation());
var masterData = masterRange.getValues();
//Logger.log("DEBUG: Length of Master data = "+masterData.length);
// create a range and get the data from "Archive"
var archiveRange = archive.getRange(1, 1, archiveLR, archiveLC);
var archiveData = archiveRange.getValues();
// create a formatted date for today
var formattedToday = Utilities.formatDate(new(Date), 'GMT+10',
'dd MMMM yyyy');
// loop through the rows
// from bottom to top
for (var i = (+masterLR - 2); i > 0; i--) {
// convert cell dates to comparable format
var DBdate = Utilities.formatDate(masterData[i][0], 'GMT+10',
'dd MMMM yyyy');
var DBtime = Utilities.formatDate(masterData[i][2], 'GMT+10',
'hh:mm a');
//Logger.log("DEBUG: i = "+i+", DBdate = "+DBdate+", Today = "+formattedToday);
// clear the temporary row array
var archivecells = [];
if (DBdate < formattedToday) {
// the table date is less than today, so archive the data
// Logger.log("DEBUG: i = "+i+", DBdate = "+DBdate+", Today = "+formattedToday+" - DB value is less than Today. ACTION: Archive this row");
// copy the row cells to temporary row array
archivecells.push(DBdate);
archivecells.push(masterData[i][1]);
archivecells.push(DBtime);
archivecells.push(masterData[i][3]);
archivecells.push(masterData[i][4]);
// copy the temporary row array to archivedata
archiveData.push(archivecells);
// delete the Master Row
master.deleteRow(i + 2);
} else {
// the table date is NOT less than today, so do nothing
// Logger.log("DEBUG: i = "+i+", DBdate = "+DBdate+", Today = "+formattedToday+" - DB value is NOT less than Today. ACTION: Do nothing");
}
// update the accumulated data to Archive.
archive.getRange(1, 1, archiveData.length, archiveLC).setValues(
archiveData);
}
}
getSheetByName(name)
getLastRow()
getRange(row, column, numRows, numColumns)
getValues()
Loops and iteration
Javascript if...else
deleteRow(rowPosition)
Master - Before
Master - After
Archive - After
I have 10 rows of data on my input step, i transform them in a for-loop and i should get more than 10 rows, but in this case i get the last transform of each iteration that the loop have for each data
I tried to use appendToFile() but the result data is not useful and pentaho read it as a unique header
On my alert() method i can see that the for loop transform the data.
var PERIODO = 2
var i
var fecha_final
//var ruta_acess ="D:\TEST.accdb"
//var contenido
////var contenido2
//var arreglo_completo
for (i=0; i<=PERIODO; i++){
fecha_final = dateAdd(FECHA_INICIO,"d",i)
Alert(i)
}
As I show in the below photo i get only 10 records and in the other photo appears the result that i want that are the results data of each iteration of the for-loop
Modified JavaScript value photo:
Expected result:
Obtained result:
For loops are not really a thing in PDI. Transformations work on sets of rows that flow through the steps, so it's best for performance and stability to use that mindset.
In your scenario each incoming row should end up as three copies, but with different calculated values based on a single new field (with values 0,1,2).
The way to do this in PDI is with a Join rows (cartesian product) step. It takes two sets of input rows and outputs a row for every combination of input rows, possibly filtered by defining a key field that has to match. So if you have 10 rows in the main input and 3 rows in the second, it will output 30 rows.
You will first need to create a data grid as the second input. Define a single integer field, name it something clear and on the second tab fill three rows with 0, 1 and 2 respectively.
Connect both inputs to the Join rows step. You don't need to configure any matching key.
The output of the Join step will be three rows for each input row, one with each of the values 0, 1, 2. Connect that output to a Calculator step and use the calculation Date A + B days to replace the logic from your javascript step.
what i mean is that in the obtained result photo the "i" variable only shows the value of "3" and i would like to have "1", "2" and "3"
to solve this i used
var row = createRowCopy(getOutputRowMeta().size())
var idx = getInputRowMeta().size()
row[idx++] = DPROCESS
this add a row for each result of the iteration.
before the tranformation result showed to me only the last value of each loop.
I am trying to create a small invoicing organization system using google sheets/drive. I have one sheet I call "tasks", where I plan to control everything from. Some of my columns include, "Client", "Project", "Requirements", "Details", "subcontractor"... As I acquire new tasks/clients, i'd find and append information respective of the task ("Project", "Requirements") to other sheets or, if none exist, create the folders, sheets, and append the respective necessary information from the "tasks" sheet to the new sheets. Some of the sheets will be sent to subcontractors, dependent on whether or not their tasks were updated or new ones were assigned to them in the original "tasks" sheet.
Within the sheets I send to subcontractors, there will be fields for them to fill out (rate, eta..), once filled, I will send that info to a third sheet to apply some margins, extra fees, and then send the info back to the original "tasks" sheet where it will fill appropriate cells.. Once all of the necessary information in a row is filled, it will be prepared and organized into an invoice for the client specified in the "client" column...
Anyways, i've been trying to learn javascript to implement all of this. As I plan to create folders, sheets, and append information based on the values entered in the rows and columns of the "tasks" sheet... I've placed a for loop in an onEdit function that does the following:
function onEdit(e) {
var ss = e.range.getSheet().getParent();
var sheet = e.range.getSheet();
var row = e.range.getRow();
var columns = [1, 2, 3, 4, 5, 6, 7, 8, 9];
//assign titles as 'keys' to array
var titles = []
//assign values of edited row to array
var values = []
//create an object to associate the title to the new edited values
var task = {}
for(var i in columns){
titles.push(sheet.getRange(1, columns[i]).getValue()); //push titles
values.push(sheet.getRange(row, columns[i]).getValue()); //push values of updated row
task[titles[i]] = values[i]; //add the values to their property names in task object
}
This works, and I can reference task["Client"], but i'd like to put this loop in a function so that I can use it again. I suppose I could do without it, but array "columns" only represents the columns I will be inputting on the "first round" --- when im sending information out...I will be inputting new information to columns 10-15, then 16-20, as the tasks progress.. and i'd like to run the for loop for those columns without having to create separate loops. To do this i've created the GetInfo function below:
function GetInfo(row,column){
for(var i in column){
titles.push(sheet.getRange(1, column[i]).getValue()); //push titles
values.push(sheet.getRange(row, column[i]).getValue()); //push values of updated row
this.task[titles[i]] = values[i]; //add the values to their task
}
}
What I am trying to accomplish is similar to what is outlined here. However, the "for(var..in..") is not mentioned in the examples and I think im missing something. In attempt to use the function for the first array of columns ive done this:
var list = new GetInfo(row,columns);
i'd like to reference the task as follows
list.task["client"]
Or var.task["name"], but the above doesn't work. When I toast list.task["Client] or try to append it to another cell, nothing happens - its blank. What am I doing wrong? How do I accomplish this correctly? What should I do?
Any help or guidance would be greatly appreciated. Please.
(other toasts are working, and the respective cell is not blank, without the function the for var in works)
I have a 2d array representing rows in a database. I'm using officeJS to load and manipulate the data in Excel. I update, insert, and delete rows. The challenge I'm facing is that I need to figure out the changed rows (inserted, deleted or updated) so that I can update only those rows in the database. I'm sending one query for the updated and inserted rows and one query for the deleted rows. I'm able to do this using lodash for data with 5000 rows and 10 columns. I'd like to scale this to a much larger data set and I'm wondering if there are any alternatives to what I'm currently doing. Below is the code I'm using to find the difference.
insertedOrUpdatedRows = _.differenceWith(modifiedData, originalData, _.isEqual);
deletedRows = _.differenceWith(originalData, modifiedData, compareFunction);
function compareFunction(a, b) {
if(a[0] == b[0]) {
return true;
}
else
return false;
}
Sample data array
[ [1,data,data,data],
[2,data,data,data] ]
The first element is the primary key.
Because you have mentioned that your Javascript engine is crashing (which it should not, at 50,000 rows - so I would revisit the logic), I would recommend chunking out the data using Lodash's _.chunk function:
_.chunk(modifiedData, modifiedData.length/500).map({
...
...
});
Ok im using the following logic. Not sure why its crashing at 50K rows. OriginalData and ModifiedData are in the format of the sample 2D array mentioned above.
var originalDataStrings = [];
var modifiedDataStrings = [];
var insertedOrUpdatedRows;
originalData.forEach(function(row){
originalDataStrings.push(JSON.stringify(row));
});
modifiedData.forEach(function(row){
modifiedDataStrings.push(JSON.stringify(row));
})
insertedOrUpdatedRows = _.differenceWith(modifiedDataStrings, originalDataStrings, _.isEqual);
console.log(insertedOrUpdatedRows);