Synchronize independent spreadsheet rows, filled by IMPORTRANGE() - javascript

I need to synchronize the contents of 2 spreadsheets that reference each other, keeping their rows in sync if a new row is added in one of the sheets.
I've got 2 spreadsheets in Google Sheets (although if there is a cross spreadsheet solution, both Excel and GS that would be great):
Spreadsheet1 has data in A:F and party1 (a set of users) writes their data in it.
Spreadsheet2 is and import range of A:F from spreadsheet1 and then has further details written in G:M, the data is written in by party2.
The way it works is party1 writes in their data in rows A1-F10 then party2 writes their additional data in spreadsheet2 based on what party1 has written in.
For example if Spreadsheet1 A1:F10 was a name, price, est delivery time, qty etc. of an item, Spreadsheet2 G1:M10 might be a bunch of data on order date, delivered (yes / no) etc.
The issue I'm currently having is that when the spreadsheets are setup they read across fine i.e. 1-10 in spreadsheet1 lines up with 1-10 in spreadsheet2, but after a while some new rows get added into spreadsheet1 between the old rows 2-5. This throws out the order in spreadsheet2 (now row 4 in spreadsheet1 doesn't line up with the row 4 in spreadsheet2 and the data becomes out of line). Is there away around this so that even if someone adds additional rows in the middle of existing rows both spreadsheets will update?

This is a classic problem in database design; how to associate information in two tables. The usual solution is to use key data; one or more columns that exist in both tables and provide a unique identifier, or key, to associate rows.
We can adapt that idea to your situation, with a script that will adjust the location of rows in Spreadsheet 2 to synchronize with Spreadsheet 1. To do that, we need to identify a key - say the Name column - which must exist in both spreadsheets.
This entails a small change in spreadsheet 2, where a Name column will now appear in column G, following the imported range in columns A-F.
A B C D E F G H I J
| Name | Price | est delivery time | qty | etc. of | an item | Name | order date | delivered | blah blah |
< - - - - - - - - - - - - Imported - - - - - - - - - - - > *KEY* < - - - - - - sheet 2 - - - - - >
Demo
Here's how that would look in action! This example is using two sheets in the same spreadsheet, just for convenience. In the demo, a new "Item" row is added in the middle of sheet 1, which automatically appears on sheet 2 thanks to the =IMPORTRANGE() function. The synchronizing function is running on a 1-minute timed Trigger, and you'll see it move things around about 20 seconds in.
You can grab a copy of the spreadsheet + embedded script here.
Code
/**
* Call syncTables() with the name of a key column.
*/
function doSyncTables() {
syncTables( "Name" );
}
/*
* Sync "Orders" spreadsheet with imported rows from "Items" spreadsheet.
*
* From: http://stackoverflow.com/a/33172975/1677912
*
* #param {String} keyName Column header used as key colum, appears
* at start of "Orders" data, following
* "Items" data.
*/
function syncTables( keyName ) {
var sheet2 = SpreadsheetApp.openById( sheetId2 ).getSheetByName('Orders');
// Get data
var lastCol = sheet2.getLastColumn();
var lastRow = sheet2.getLastRow(); // Includes all rows, even blank, because of =importRange()
var headers = sheet2.getRange(1, 1, 1, lastCol).getValues()[0];
var keyCol = headers.lastIndexOf( keyName ) + 1;
var itemKeys = sheet2.getSheetValues(1, 1, lastRow, 1).map(function(row) {return row[0]});
var itemData = sheet2.getSheetValues(1, 1, lastRow, keyCol-1);
var orderData = sheet2.getSheetValues(1, keyCol, lastRow, lastCol-keyCol+1);
var ordersByKey = []; // To keep track of orders by key
// Scan keys in orderData
for (var row=1; row<orderData.length; row++) {
// break loop if we've run out of data.
var orderKey = orderData[row][0];
if (orderKey === '') break;
ordersByKey[ orderKey ] = orderData.slice(row, row+1)[0];
var orderKey = orderData[row][0];
}
var newOrderData = []; // To store reordered rows
// Reconcile with Items, fill out array of matching orders
for (row = 1; row<itemData.length; row++) {
// break loop if we've run out of data.
var itemKey = itemData[row][0];
if (itemKey === '') break;
// With each item row, match existing order data, or add new
if (ordersByKey.hasOwnProperty(itemKey)) {
// There is a matching order row for this item
newOrderData.push(ordersByKey[itemKey]);
}
else {
// This is a new item, create a new order row with same key
var newRow = [itemKey];
// Pad out all columns for the new row
for (var col=1; col<orderData[0].length; col++) newRow.push('');
newOrderData.push(newRow);
}
}
// Update spreadsheet with reorganized order data
sheet2.getRange(2, keyCol, newOrderData.length, newOrderData[0].length).setValues(newOrderData);
}

the current answer by mogsdad is great as always. i just wanted to point out a less complex alternative:
if you can live with preventing spreadsheet1 from allowing insertions or deletion of rows, you will avoid the issue. instead of removing rows you could use a column to mark "deleted" for example (and use filters to remove from view).
to prevent row insertions and deletions in spreadsheet1, simply select an entire unused column to the right, and create a protected range so none of the editors have permission. that prevents modifying at the row level up to the last existing row (but new rows can still be inserted below the range)
it also doesnt prevent users from swapping two row's data. but its still good to know about this simpler alternative.

Related

How do you move logic from Google Sheet formula to Javascript?

I have a function yahoofinance which queries Yahoo!Finance for stock data based on a stock ticker. Before I can call the function, I need to do some modifications to the input.
Tickers in my sheet are of the form [venue].[company], for example XNAS.GOOG (Google at Nasdaq), XAMS.INGA (ING Bank at Amsterdam) and XSWX.CRSP (Crispr at Swiss Exchange).
The codes that I use for the venues are international, but most API's don't use these same codes. For instance, the Amsterdam stock exchange officially is XAMS, but AMS on Google API and AS on Yahoo. So I have a sheet called Exchanges with all these venues listed. When calling my custom yahoofinance function, it takes the standard 'XMAS.INGA' as input, then cuts off the venue (XAMS) and looks up the Yahoo name (AS), then passes INGA.AS to the Yahoo API:
=yahoofinance(index(split(A2, "."), 0, 2) & "." & vlookup(index(split(A2, "."), 0, 1), Exchanges!E:J, 6, false))
function yahoofinance(ticker) {
// send ticker to Yahoo API
}
The logic, which is now in cell B2 in the sheet, is getting complicated to follow. I would like to move it, if possible, into the function and pass only the ticker from A2:
=yahoofinance(A2)
function yahoofinance(ticker) {
const venue = ticker.split();
// etc
// end up with INGA.AS, and pass to Yahoo API
}
My question is: is it possible to move the logic that is now in my sheet, into the function? For example, if and how can I perform the same vlookup from within yahoofinance? Is it even possible?
I am planning to really extend the functionality of yahoofinance, perhaps even rework it to a generic finance function which also takes as argument to which API you want to pass a ticker: =yahoofinance(ticker, api), so any help to get me started with a solid basis here is greatly appreciated.
You can try this custom formula to get the modified value:
Script:
function yahoofinance(ticker) {
var [venue, company] = ticker.split("\.");
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName('Exchanges');
var lastRow = sheet.getLastRow();
// get Exchanges!E:J values
var mapping = sheet.getRange(1, 5, lastRow, 6).getValues();
// lookup equivalent, filter then get 6th column and append to company
var modifiedTicker = company + '.' + mapping.filter(row => row[0] == venue).flat()[5];
// you now have the modified ticker. Use it on your function.
// ...
return modifiedTicker;
}
B7 formula:
=index(split(A2, "."), 0, 2) & "." & vlookup(index(split(A2, "."), 0, 1), Exchanges!E:J, 6, false)
Output:

Cross reference names on spreadsheets to get specific data points (looping through an array)

I have two sheets. Test Data has 3-4k entries of many columns of data and Order Changes has no data at all. I would like to search two specific columns on Test Data, a column of names and a column of yes or no. If column two of Test Data contains a 'yes' in the cell then the name of that person would be placed into a cell on order changes.
This is what I have so far:
function isThreshold(){
var data = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Test Data");
var cdata = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Order Changes");
var lc = data.getLastColumn();
var lookUp = data.getRange(1,6,3,2).getValues();
lookUp.forEach(var info in lookUp){
}
Logger.log(lookUp);
}
I probably shouldn't loop through that many entries but I don't know of any other way. Should I combine the forEach loop with an if loop to get the desired result or use some other method?
I believe your goal as follows.
You want to retrieve the values from the cells "F1:G" of sheet "Test Data".
You want to search yes from the column "G" and when the column "G" is yes, you want to put the value of the column "F" to the sheet "Order Changes".
Modification points:
SpreadsheetApp.getActiveSpreadsheet() can be declared one time.
In this case, you can retrieve the values from the range of "F1:G" + data.getLastRow() of "Test Data", and create the array for putting to the sheet "Order Changes", and put it.
When above points are reflected to your script, it becomes as follows.
Modified script:
function isThreshold(){
var ss = SpreadsheetApp.getActiveSpreadsheet();
var data = ss.getSheetByName("Test Data");
var cdata = ss.getSheetByName("Order Changes");
var valuesOfTestData = data.getRange("F1:G" + data.getLastRow()).getValues();
var valuesForOrderChanges = valuesOfTestData.reduce((ar, [f, g]) => {
if (g.toLowerCase() == "yes") ar.push([f]);
return ar;
}, []);
if (valuesForOrderChanges.length > 0) {
cdata.getRange(1, 1, valuesForOrderChanges.length, valuesForOrderChanges[0].length).setValues(valuesForOrderChanges);
// or cdata.getRange(cdata.getLastRow() + 1, 1, valuesForOrderChanges.length, valuesForOrderChanges[0].length).setValues(valuesForOrderChanges);
}
}
In this modified script, from your question, it supposes that the columns "F" and "G" are the value of name and yes or no.
References:
getRange(a1Notation) of Class Sheet
reduce()

getRange/setValues does not always work

I help maintain a Google spreadsheet where new data is added via a HTML form.
When it comes to add new data the insertion point of the new data depends on one of the form fields (Application Received date).
The script finds where in the sheet the data should be inserted and does 3 things:
Inserts a blank row at the correct location
Copies the row above (so formulas and conditional formatting are copied)
Replaces the data in the cells from the copy with the values entered into the form
The issue is cells A to I are value based (populated from the form) and so are cells M to O, but cells J,K,L are calculations based on some cells in A to I.
This means I have to make 2 calls to getRange/setValues and sometimes the second call (the call to set cells M,N,O does not work. The result is a new row created with the correct data in cells A to I (and thus J,K,L) but cells M,N,O stay as whatever is in those cells in the row above.
Here is the relevant code.
// Assign object data for cells A to I
var newvalues = [
[ username, applyDate, maritalStatus, sponsorApprovalDate, processingOffice, inProcessDate, extraDocsRequestedDate, nonVisaExempt, decisionMadeDate ]
];
// Set cells A to I with data from form
sheet.getRange('A' + startingRowIndex + ':I' + startingRowIndex).setValues(newvalues);
// Now assign object data for cells M to O
newvalues = [
[ coprReceivedDate, location, notes ]
];
// Set cells M to O with data from form
sheet.getRange('M' + startingRowIndex + ':O' + startingRowIndex).setValues(newvalues);
As stated above the second sheet.getRange('...').SetValues() call fails to set the values.
Any ideas?
Instead of completely recalculating the locations of your output ranges, you could get an "anchor" point at the start of the row, then use the Range.offset() method to define additional ranges relative to the anchor.
// Assign object data for cells A to I
var newvalues = [
[ username, applyDate, maritalStatus, sponsorApprovalDate, processingOffice, inProcessDate, extraDocsRequestedDate, nonVisaExempt, decisionMadeDate ]
];
// Get range "anchor" for data from form
var newRow = sheet.getRange('A' + startingRowIndex );
// Set cells A to I with data from form
newRow.offset(0,0,newvalues.length,newvalues[0].length).setValues(newvalues);
// Now assign object data for cells M to O
newvalues = [
[ coprReceivedDate, location, notes ]
];
// Set cells M to O with data from form
newRow.offset(0,13,newvalues.length,newvalues[0].length).setValues(newvalues);

Order by sorted array

I have a page that contains a table like the following (automatically sorted by "Name" column)
Name Open Num Total Num
-----------------------------------
Doe, John 0 0
Smith, Sarah 4 3
Tyler, Rose 7 8
The second tr would look like this:
<tr id="1"><td class="N">Smith, Sarah</td><td class="O">4</td><td class="T">3</td></tr>
Where the row ID is a counter (first row = 0, second = 1, third = 2, etc.) and the cell class is to grab the column using jQuery ($(".O") gives Open Num column)
I'm trying to get the table to sort based off of the numerical columns (Open Num and Total Num). So output would look like this (sorted by Open Num or Total Num):
Name Open Num Total Num
-----------------------------------
Tyler, Rose 7 8
Smith, Sarah 4 3
Doe, John 0 0
So far, I store the numbers into an array arrQuick and I store the row number in a different array rowCount. I then use the Quick Sort method to sort the data, at the same time sorting the second array, which works perfectly. So now I have the sorted data and the order that my rows should be in.
The Problem
I cannot figure out how to get the table rows to update correctly.
So far I have this.
for(var i=0;i<rowCount.length;i++){
var tmpHolder=$("#"+i).html();
$("#"+i).html($("#"+rowCount[rowCount.length-(i+1)]).html());
$("#"+rowCount[rowCount.length-(i+1)]).html(tmpHolder);
}
When stepping through I can see that initially the updating is working. However, eventually it gets to some point rows are getting updated to places they shouldn't be and I'm not sure why.
You can sort the rows based on the values of table cells. The following method accepts a className of the cells and sorts the rows based on the text contents of that cells.
$.fn.sortTable = function(cls) {
this.find('tr').sort(function(a, b){
return $(a).find('td.'+cls).text() > $(b).find('td.' + cls).text();
}).appendTo(this.find('tbody'));
}
$('table').sortTable('O');
Updated method for supporting ascending and descending orders.
$.fn.sortTable = function (opt) {
var sort = typeof opt.sortType !== 'undefined' ? opt.sortType : 'asc',
cls = opt.cls;
if (!cls) return;
var $tr = this.find('tbody').children().sort(function(a, b){
return $(a).find('.' + cls).text() > $(b).find('.' + cls).text();
});
if (sort === 'des') {
$tr = $tr.toArray().reverse();
}
this.find('tbody').append($tr);
}
$('table').sortTable({
cls: 'N', // className of the cells
sortType: 'des' // order 'asc' || 'des'
});
http://jsfiddle.net/Af7mG/

How to access dictionary like structure in Protovis (Javascript)

I am trying to visualize a flickr dataset using protovis. I do understand the visualization part, but i have a question about accessing the data however. I was provided an example visualization and it accesses the data as folllowing:
var data = pv.range(250).map(function(row) {
return {
views: parseInt(Data.data(row, 2)), //refers to the 4 row and 2nd collumn in CSV
users: Data.data(row, 6),
date: Data.data(row, 8))), //more collumns excist but for now we only use these
};
});
As i understand a part of the data set is now stored in the variable data, namely views, users and date. Is this variable able to beaccessed like a dictionary?
What i am trying to do is checking whether there are date on which one user occurs more than 2 times. I thought of looping through the var data as follows:
dateUserDict {};
for (d=0; d < data.date.length; d++ ){
for (i=0; i < data.users.length; i++ ){
for (j=0; j < data.users.length; j++){
if (data.users[i] == data.users[j]){
userCounter++ //this should count the number of occurences of a specific user on a specific date
dateUserDict[data.date] = [data.user][userCounter]}
}
}
}
This does not seem to work. I am trying to store the events (the number of times a user occurs on a specific date) in a dictionary. If i get the dictionary as described i can easily visualise the whole thing. But it is this conversion from the first dict (data) to the second (dateUserDict) which bugs me!
Any help or a push is highly appreciated!
Thanks
jorrit
The function you provided will product a Javascript array of objects.
var data = pv.range(250).map(function(row) {
return {
views: parseInt(Data.data(row, 2)), //refers to the 4 row and 2nd collumn in CSV
users: Data.data(row, 6),
date: Data.data(row, 8))), //more collumns excist but for now we only use these
};
});
The result will look something like this:
var data = [ {views:10, users: 9, date: '09/13/1975'}, ... ]
So instead of using data.users.length, use data.length, and instead of data.users[i], you should be using data[i].users, etc.

Categories