I have js code which returns dates of layers (Using OpenLayers and Momentjs) that stored on my PC.
As we see that the function returns all the dates between two dates from the folder (folder) with a 60-second step. But I want to return just the dates which I have files (layers) on my PC because I don't have layers for all the dates.
So what I need is a function returns an array of dates for which I have tiles only, and then add from this layer to the map depends on the entered dates
function loopLayer() {
const FromDateTime = document.getElementById("fromdate").value;
const dateFrom = moment(FromDateTime, "YYYY-MM-DD HH:mm:ss", true);
if (!dateFrom.isValid()) {
log("something");
return;
}
const ToDateTime = document.getElementById("todate").value;
const dateTo = moment(ToDateTime, "YYYY-MM-DD HH:mm:ss", true);
if (!dateTo.isValid()) {
log("something");
return;
}
let loopDate = dateFrom;
for(let i=0; dateFrom.isSameOrBefore(dateTo) && i < 100; i++) {
// preventing from loading thousands of layers
loopLayerByDate(loopDate);
loopDate = loopDate.add(60, 'seconds');
}
}
function loopLayerByDate(dateObject) {
const folderDate = dateObject.format("YYYY-MM-DD_HHmmss");
const source = new ol.source.XYZ({
projection: 'EPSG:3854',
// adapt url and source tile type to your setup
url: "folder/" + folderDate + "/{z}/{x}/{-y}.png"
});
const layer = new ol.layer.Tile({
source: source,
title: "layer"
});
map.addLayer(layer)
}
An Website usually cannot read data from your local file system for security reasons. Otherwise any website could spy on your harddisk.
As you already found out, there is an exception to that rule: when you open a local HTML file, it can read file content from the harddisk. But you can't crawl through folders, so we cannot read a list of the available dates.
you have two options now:
add an <input type="file" multiple>, upload the files and use the FileAPI (example here).
your thing is a local html file which you open from your harddisk. You could use a trial-and-error guess approach
find a method to build a list of dates without the need to guess them, e.g. naming the folders with an increasing number instead of timestamps!
Use a server software which serves everything. The server can access the file-system and send the "front-end" the required list of dates. I won't provide an how-to here, there are tons of software-solutions and how-tos on stackoverflow and in search engines.
Option 2, local html:
Since you know roughly in what ranges the filenames is, you could use a brute-force approach and just query for all dates in a range and see which ones actually respond. Be aware that this approach is far from ideal and probably quite slow.
function guessValidDates(arrayOfDates){
const validDates = [];
arrayOfDates.forEach((date) => {
var xhttp = new XMLHttpRequest();
xhttp.open("GET", "time/" + date + "/1.png", true);
xhttp.send();
console.log('request returned', xhttp);
if (xhttp.response) {
validDates.push(date;
}
});
return validDates;
}
Example usage:
// example from loopLayer function
let loopDate = dateFromObject;
const allDates = [];
for(let i=0; dateFromObject.isSameOrBefore(dateToObject) && i < 100; i++) {
// the i counts as a failsafe, preventing us from loading billions of
// or whatever the pattern is for your file path
allDates.push(loopDate.format("YYYY-MM-DD_HHmmss"));
// step forward by X
loopDate = loopDate.add(1, 'minutes');
}
const validDates = guessValidDates(allDates);
// now you know the valid dates and can use them. Example:
validDates.forEach(function(someDate){
loopLayerByDate(someDate);
});
Or if you have a pattern where all layers for a single day have an increasing number, eg "time/" + yearMontDay + '_' + increasingNumber + "/{z}/{x}/{-y}.png", just keep adding layers until you get a non-valid response:
function isValidLayer(someDateString, someNumber) {
var xhttp = new XMLHttpRequest();
xhttp.open("GET", "time/" + someDateString + "_" + someNumber + "/1.png", true);
xhttp.send();
if (xhttp.response) {
return true;
}
return false;
}
// you can now count up until you don't get a valid response anymore:
let someDateString = dateObject.format("YYYY-MM-DD_HHmmss");
let increasingNumber = 0;
while(increasingNumber < 1000) {
// the condition is just a failsafe
if(isValidLayer(someDateString, increasingNumber) {
// add layer with that number to your map
const source = new ol.source.XYZ({
projection: 'EPSG:4326',
wrapX: false,
url: "time/" + folderDate + "_" + increasingNumber + "/{z}/{x}/{-y}.png"
});
// TODO add layer here and so on....
} else {
// no more layers, stop!
console.log('stopped after' + increasingNumber + 'layers on date ' + someDateString);
break;
}
}
Related
I am trying to use a script that creates a Parent folder and subfolders and also copies any templates within a drive
The sheet is here: https://docs.google.com/spreadsheets/d/1zc_ZWGXHcM5qM4uPvVcchPWObQLDJysxbL0x2j4Kr6E/copy
I got this from https://www.techupover.com/create-google-drive-folder-structure-files/
I am trying to change this script to copy an existing folder with files to the newly created folder. OR if possible change this script to copy multiple files within the newly created folders (This one restricts it to copy only 1 file). Could someone guide me here as I am new to this.
const CUSTOM_MENU_NAME = 'Neat Features'; // the name of the new menu in your google sheet
const CUSTOM_MENU_ITEM = 'Generate Folders Now'; // the menu item you'll click to run the script
const DATA_TAB_NAME = 'folder_data'; // name for the sheet tab that contains the folder info that will be created/processed
const LOG_TAB_NAME = 'log'; // name of the sheet tab that will store the log messages from this script
const DATE_FORMAT = 'yyyy-MM-dd HH:mm:ss'; // date format to use for the log entries. Probably dont change this unless you really really want to.
/**
*
* DO NOT CHANGE ANYTHING UNDER THIS LINE
*
* ONLY CHANGE THINGS IN THE CONFIG.GS FILE
*
*/
/**
* When the spreadsheet is open, add a custom menu
*/
function onOpen() {
var spreadsheet = SpreadsheetApp.getActive();
var customMenuItems = [
{name: CUSTOM_MENU_ITEM, functionName: 'processGoogleSheetData_'}
];
spreadsheet.addMenu(CUSTOM_MENU_NAME, customMenuItems);
}
/**
* Bulk create Google Folders from data within a Google Sheet.
* This is the function to call from the custom menu item.
* Others are referenced by this one.
*/
function processGoogleSheetData_() {
// get current spreadsheet
var ss = SpreadsheetApp.getActiveSpreadsheet();
// Log starting of the script
logEventToGoogleSheet('Script has started');
// display Toast notification
ss.toast('Starting!!!', 'Yep, we are starting...now.');
// get TimeZone
var timeZone = ss.getSpreadsheetTimeZone();
// get Data sheet
var dataSheet = ss.getSheetByName(DATA_TAB_NAME);
// get all data as a 2-D array
var data = dataSheet.getDataRange().getValues();
// create a name:value pair array to send the data to the next Function
var spreadsheetData = {ss:ss, timeZone:timeZone, dataSheet:dataSheet, data:data};
// run Function to create Google Folders
var doCreateFolders = createFolders(spreadsheetData);
// check success status
if (doCreateFolders) {
// display Toast notification
ss.toast('Script complete', 'Finished');
}
else {
// script completed with error
// display Toast notification
ss.toast('Got some errors. Check the logs', 'Finished');
}
// Log starting of the script
logEventToGoogleSheet('Script finished');
}
/**
* Loop through each row and create folders, set permissions
*/
function createFolders(spreadsheetData) {
// extract data from name:value pair array
var ss = spreadsheetData['ss'];
var timeZone = spreadsheetData['timeZone'];
var dataSheet = spreadsheetData['dataSheet'];
var data = spreadsheetData['data'];
// get last row number so we know when to end the loop
var lastRow = dataSheet.getLastRow();
var folderIdMap = new Object();
// start of loop to go through each row iteratively
for (var i=1; i<lastRow; i++) {
// extract values from row of data for easier reference below
var rootFolderId = data[i][0];
var templateDocId = data[i][1]
var newFolderName = data[i][2];
var parentFolderName = data[i][3];
var folderId = data[i][4];
var permissionEmailViewer = data[i][5];
var permissionEmailEditor = data[i][6];
// only perform this row if the folder ID is blank
if(folderId == '') {
// if the sheet doesn't have a specified parent folder name, then it goes in the root.
// if the parent folder name is supplied then it had to have been created by this script,
// so get it from script properties (where the ID is saved during an earlier loop)
if(parentFolderName == '') {
destinationFolderId = rootFolderId;
} else {
var thisMapKey = createMapString(rootFolderId + '___' + parentFolderName);
var destinationFolderId = folderIdMap[thisMapKey];
}
// display Toast notification
ss.toast(newFolderName, 'Creating New Folder');
// run Function to create Google Folder and return its URL/ID
var folderDetails = createFolder(newFolderName, destinationFolderId);
// check new Folder created successfully
if (folderDetails) {
// extract Url/Id for easier reference later
var newFolderUrl = folderDetails['newFolderUrl'];
var newFolderId = folderDetails['newFolderId'];
//push the key/folder id to the array map so we can use it later in the loop
var thisMapKey = createMapString(destinationFolderId + '___' + newFolderName);
folderIdMap[thisMapKey] = newFolderId;
// copy the template doc into the new directory (if specified)
if(templateDocId != '') {
makeCopy(templateDocId, newFolderId, newFolderName + ' Template Document');
}
// set the Folder ID value in the google sheet, inserting it as a link
var newFolderLink = '=HYPERLINK("' + newFolderUrl + '","' + newFolderName + '")';
dataSheet.getRange(i+1, 5).setFormula(newFolderLink);
// check if Viewer Permissions need adding - if there are emails in the column for this row
if (permissionEmailViewer != '') {
// run Function to add Folder permissions
var currentRow = i+1;
var addPermissionsFlag = addPermissions('VIEWER', timeZone, dataSheet, permissionEmailViewer,
newFolderId, currentRow, 8);
// if problem adding Permissions return for status message
if (addPermissionsFlag == false) {
// display Toast notification and return false flag
ss.toast('Error when adding Viewer Permissions to: ' + newFolderName, 'Error');
return false;
}
}
// check if Editor Permissions need adding - if there are emails in the column for this row
if (permissionEmailEditor != '') {
// run Function to add Folder permissions
var currentRow = i+1;
var addPermissionsFlag = addPermissions('EDITOR', timeZone, dataSheet, permissionEmailEditor,
newFolderId, currentRow, 9);
// if problem adding Permissions return for status message
if (addPermissionsFlag == false) {
// display Toast notification and return false flag
ss.toast('Error when adding EDITOR Permissions to: ' + newFolderName, 'Error');
return false;
}
}
// write all pending updates to the google sheet using flush() method
SpreadsheetApp.flush();
} else {
// write error into 'Permission Added?' cell and return false value
dataSheet.getRange(i+1, 4).setValue('Error creating folder. Please see Logs');
// new Folder not created successfully
return false;
}
} else {
ss.toast('Skipping Row - Folder ID already set', 'Moving onto the next row to process');
}
} // end of loop to go through each row in turn **********************************
// completed successfully
return true;
}
function makeCopy(sourceDocumentId, destinationFolderId, destinationFileName) {
var destinationFolder = DriveApp.getFolderById(destinationFolderId);
return DriveApp.getFileById(sourceDocumentId).makeCopy(destinationFileName,destinationFolder);
}
/**
* Function to create new Google Drive Folder and return details (url, id)
*/
function createFolder(folderName, destinationFolderId) {
try {
// get destination Folder
var destinationFolder = DriveApp.getFolderById(destinationFolderId);
}
catch(e) {
logEventToGoogleSheet('Error getting destination folder: ' + e + e.stack);
var destinationFolder = false;
}
// proceed if successfully got destination folder
if (destinationFolder) {
var documentProperties = PropertiesService.getDocumentProperties();
try {
// create new Folder in destination
var newFolder = destinationFolder.createFolder(folderName);
// get new Drive Folder Url/Id and return to Parent Function
var newFolderUrl = newFolder.getUrl();
var newFolderId = newFolder.getId();
var folderDetails = {newFolderUrl:newFolderUrl, newFolderId:newFolderId};
return folderDetails;
}
catch(e) {
logEventToGoogleSheet('Error creating new Folder: ' + e + e.stack);
return false;
}
}
else {
// return false as unable to get destination folder
return false;
}
}
/**
* Function to add permissions to each Folder using the provided email address(es).
*
* role var can be either VIEWER or EDITOR
*/
function addPermissions(role, timeZone, dataSheet, permissionEmail, newFolderId, currentRow, permAddedCol) {
// split up email address array to be able to loop through them separately
var emailAddresses = permissionEmail.split(',');
logEventToGoogleSheet(role + ' emailAddress array is: ' + emailAddresses);
// get length of array for loop
var emailAddressesLength = emailAddresses.length;
try {
// get Google Drive Folder
var newFolder = DriveApp.getFolderById(newFolderId);
}
catch(e) {
logEventToGoogleSheet('Error getting destination folder: ' + e + e.stack);
var newFolder = false;
}
// proceed if successfully got destination folder
if (newFolder) {
// loop through each email address and add as 'Editor' *******************
for (var i=0; i<emailAddressesLength; i++) {
var emailAddress = emailAddresses[i].trim();
logEventToGoogleSheet(role + ' emailAddress for adding permission is: ' + emailAddress);
try {
if(role == 'VIEWER') {
logEventToGoogleSheet('Adding ' + emailAddress + ' as ' + role);
newFolder.addViewer(emailAddress);
var success = true;
}
if(role == 'EDITOR') {
logEventToGoogleSheet('Adding ' + emailAddress + ' as ' + role);
newFolder.addEditor(emailAddress);
var success = true;
}
// add 'Edit' permission using email address
if (success) {
// write timestamp into 'Permission Added?' cell
var date = new Date;
var timeStamp = Utilities.formatDate(date, timeZone, DATE_FORMAT);
dataSheet.getRange(currentRow, permAddedCol).setValue(timeStamp);
}
else {
// write error into 'Permission Added?' cell and return false value
dataSheet.getRange(currentRow, permAddedCol).setValue('Error adding ' + role + ' permission. Please see Logs');
return false;
}
}
catch(e) {
logEventToGoogleSheet('Error adding ' + role + ' permission: ' + e + e.stack);
}
}
}
else {
// write error into cell and return false value
dataSheet.getRange(currentRow, permAddedCol).setValue('Error getting folder. Please see Logs');
// return false as unable to get Google Drive Folder
return false;
}
// return true as all permissions added successfully
return true;
}
/**
* Write log message to Google Sheet
*/
function logEventToGoogleSheet(text_to_log) {
// get the user running the script
var activeUserEmail = Session.getActiveUser().getEmail();
// get the relevant spreadsheet to output log details
var ss = SpreadsheetApp.getActiveSpreadsheet();
var googleSheet = ss.getSheetByName(LOG_TAB_NAME);
// create and format a timestamp
var now = new Date();
var timeZone = ss.getSpreadsheetTimeZone();
var niceDateTime = Utilities.formatDate(now, timeZone, DATE_FORMAT);
// create array of data for pasting into log sheet
var logData = [niceDateTime, activeUserEmail, text_to_log];
// append details into next row of log sheet
googleSheet.appendRow(logData);
}
/**
* Create a string that can easily be used as an object/array key for reference
* during the create folder loop. (Helps with identifying the parent folder dynamically)
*/
function createMapString(input_string) {
return input_string.replace(/\W/g, '');
}
I hope I figured out the solution. You need to make these three modifications:
Column B should contain Folder IDs (these folder contain files you want to copy) instead of Google Document IDs.
In line 136 you have to change the variable name templateDocId to templateFolderId this way:
var templateFolderId = data[i][1]
After the line 152 the code should look like this:
// copy the template doc into the new directory (if specified)
// if(templateDocId != '') {
// makeCopy(templateDocId, newFolderId, newFolderName + ' Template Document');
// }
if (templateFolderId != '') copy_files(newFolderId);
function copy_files(newFolderId) {
var src_folder = DriveApp.getFolderById(templateFolderId);
var dest_folder = DriveApp.getFolderById(newFolderId);
var files = src_folder.getFiles();
while (files.hasNext()) files.next().makeCopy(dest_folder);
}
After that the script will copy all the files from template folders into the new created folders (like 'Team A1', 'Team A2', etc).
The sheet with the changed script is here.
You can try this modification after the line 136:
// copy the template doc into the new directory (if specified)
// if(templateDocId != '') {
// makeCopy(templateDocId, newFolderId, newFolderName + ' Template Document');
// }
copy_files(folderId, newFolderId);
function copy_files(folderId, newFolderId) {
var src_folder = DriveApp.getFolderById(folderId);
var dest_folder = DriveApp.getFolderById(newFolderId);
var files = src_folder.getFiles();
while (files.hasNext()) files.next().makeCopy(dest_folder);
}
A haven't tried it. It's just a guess.
function start() {
var sourceFolder = 'source';
var targetFolder = 'target';
var source = DriveApp.getFoldersByName(sourceFolder);
var target = DriveApp.createFolder(targetFolder);
if (source.hasNext()) {
copyFolder(source.next(), target);
}
}
function copyFolder(source, target) {
var folders = source.getFolders();
var files = source.getFiles();
while (files.hasNext()) {
var file = files.next();
file.makeCopy(file.getName(), target);
}
while (folders.hasNext()) {
var subFolder = folders.next();
var folderName = subFolder.getName();
var targetFolder = target.createFolder(folderName);
copyFolder(subFolder, targetFolder);
}
}
So I'm needing to get the list of file names from a range of Google Drive URLs in a spreadsheet. Browsing around the net, I came across the code below. It works but only for the old style urls, which I heard Google changed in September 2021.
Note that links are not fully functional, please replace with real links to check!
The old style is:
https://drive.google.com/file/d/1GMUwYxZxsNpLiaYOiVMBwl41LpreQ-fc/view?usp=sharing
This works correctly from the code below.
What I'd like though is two things.
It should handle a range of a couple of columns, currently reading AE2:AE, and printing out on AM2:AM. What I'd like is to go through the range: AE2:AL and print out: AM2:AT
Secondly it should also handle the newer form urls:
https://drive.google.com/file/d/0B9EZQqsLDEqDUGlsdy1oVEtETGs/view?usp=sharing&resourcekey=0-h7HOcxayPaHJ5r6dAAslVQ
Current Code:
function getNames() {
var activeRange = SpreadsheetApp.getActiveSheet().getDataRange();
var height = activeRange.getHeight();
var links = SpreadsheetApp.getActiveSheet()
.getRange("AE2:AE" + height)
.getValues();
var nameValues = [];
links.forEach((row) => {
try {
var link = row[0];
var fileID = getIdFromLink(link);
var name = DriveApp.getFileById(fileID).getName();
nameValues.push([name]);
} catch (e) {
nameValues.push(["NO NAME FOUND"]);
}
});
var nameRange = SpreadsheetApp.getActiveSheet().getRange("AM2:AM" + height);
nameRange.setValues(nameValues);
}
function getIdFromLink(link) {
var regex = new RegExp(
/(?<=https:\/\/drive\.google\.com\/file\/d\/)(.+)(?=\/)/
);
return regex.exec(link)[0];
}
How should the code above be modified to enable what I'm wanting. Sorry, I tried a couple of if/else statements, but my Javascript knowledge is severely limited.
Any help would be greatly appreciated.
Current "screenshot" showing:
(1) - Old style url - correctly picking up file name (2)
(3) - New style url - not picking up file name (4)
Your getIdFromLink() function should work just fine as long as the files have not been shared in such a way that they require a resource key as well.
To work with resource keys, use DriveApp.getFileByIdAndResourceKey(), like this:
function getFileNamesByLink() {
const sheet = SpreadsheetApp.getActiveSheet();
const sourceRange = sheet.getRange('AE2:AL');
const targetRange = sheet.getRange('AM2');
const fileNames = sourceRange.getValues()
.map(row => row.map(link => getFileNameFromLink_(link)));
targetRange
.offset(0, 0, fileNames.length, fileNames[0].length)
.setValues(fileNames);
}
function getFileNameFromLink_(link) {
if (!link) {
return null;
}
const fileId = getIdFromLink_(link);
if (!fileId) {
return NaN;
}
let file;
try {
file = DriveApp.getFileById(fileId);
} catch (error) {
try {
file = DriveApp.getFileByIdAndResourceKey(fileId, getResourceKeyFromLink_(link));
} catch (error) {
return NaN;
}
}
return file.getName();
}
function getIdFromLink_(link) {
const match = String(link).match(/file\/d\/([-\w]+)/i);
return match ? match[1] : null;
}
function getResourceKeyFromLink_(link) {
const match = String(link).match(/resourcekey=([-\w]+)/i);
return match ? match[1] : null;
}
Note that the script may time out if you have thousands of links. If that happens, process the links in a piecemeal fashion, or see if the Advanced Drive Service works for you.
I'm referring to this question.
function renameEvents() {
var cal = CalendarApp.getCalendarById("Calendar Id");
var startTime = new Date(1850, 0, 1);
var endTime = new Date(2100, 0, 1);
var events = cal.getEvents(startTime, endTime);
var sp = PropertiesService.getScriptProperties();
if (!(sp.getProperty("count")) || sp.getProperty("count") == 0) {
var count = 0;
else if ((sp.getProperty("count") > 0) {
var count = sp.getProperty("count");
}
for (var i = count; i < events.length; i++) {
events[i].setTitle(events[i].getTitle() + " something");
events[i].setDescription(events[i].getDescription() + " something else");
sp.setProperty("count", i)
}
}
The initial function run is time-triggered. How to programmatically trigger the next function run to continue with the next chunk?
I believe your goal as follows.
You want to reduce the process cost of the following script. This script is from the URL in your question.
var cal = CalendarApp.getCalendarById("Calendar Id");
var startTime = new Date(1850, 0, 1);
var endTime = new Date(2100, 0, 1);
var events = cal.getEvents(startTime, endTime);
for (var i = 0; i < events.length; i++) {
events[i].setTitle(events[i].getTitle() + " something");
events[i].setDescription(events[i].getDescription() + " something else");
}
You want to modify the event title and event description of all events in the Google calendar.
Issue and workaround:
In your current script, the title and event description of all events try to modify in the for loop. And in your current script, the loop is run until an error occurs. And when the error occurs, the loop is started from countusing using the time-driven trigger. I understood like this.
In your above script, when the number of events are large, the maximum execution time is over. Because I think that the cost of setTitle and setDescription is high.
In your current script, in addition to setTitle and setDescription, setProperty is used in the for loop. In this case, the process cost will be more high.
In my answer, as a workaround, I would like to propose to use the batch request for your situation. The Calendar API can be run by the batch request. The batch request is run with the asynchronous process, and can run 100 requests by one API call. By this, the process cost can be reduced. So I thought that when the batch request is used for your situation, your all tasks might be able to be achieve by one running script.
Usage:
1. Install Google Apps Script library.
In order to use this sample script, please install a Google Apps Script library of BatchRequest. You can see the method for installing the library at https://github.com/tanaikech/BatchRequest#how-to-install.
2. Run sample script.
Please copy and paste the following script. And please enable Google Calendar API at Advanced Google services. And please set your calendar ID to calendarId.
function myFunction() {
const calendarId = "###";
var cal = CalendarApp.getCalendarById(calendarId);
var startTime = new Date(1850, 0, 1);
var endTime = new Date(2100, 0, 1);
var events = cal.getEvents(startTime, endTime);
// Create requests for the batch request.
const reqs = events.map(e => ({
method: "PUT",
endpoint: `https://www.googleapis.com/calendar/v3/calendars/${calendarId}/events/${e.getId().replace("#google.com", "")}`,
requestBody: {
start: {dateTime: e.getStartTime().toISOString()},
end: {dateTime: e.getEndTime().toISOString()},
summary: e.getTitle() + " something",
description: e.getDescription() + " something else"
}
}));
// Run batch requests.
const limit = 100;
const split = Math.ceil(reqs.length / limit);
for (let i = 0; i < split; i++) {
BatchRequest.Do({batchPath: "batch/calendar/v3", requests: reqs.splice(0, limit)});
}
}
When you run the function of myFunction, the title and description of all events are modified by the method of Events: update in Calendar API.
Note:
This script modifies the title and description of all events. So please be careful this. So at first, as a test run, I would like to test for the small number of events.
In my environment, when I tested above script for 100 events, the process time was about 7 seconds. I'm not sure about the number of your total events.
In this answer, I proposed to use the method of Events: update instead of the method of Events: patch. Because when Events: patch is used, I confirmed that only a part of requested all events were modified. I think that this might be a bug. So I used the method of "Events: update". In this case, I could confirm that all events were modified.
Please use this script with V8.
References:
Sending Batch Requests
Events: update
BatchRequest
Advanced Google services
Added:
In this sample script, no GAS library is used.
Sample script:
function myFunction() {
const calendarId = "###";
var cal = CalendarApp.getCalendarById(calendarId);
var startTime = new Date(1850, 0, 1);
var endTime = new Date(2100, 0, 1);
var events = cal.getEvents(startTime, endTime);
// Create requests for the batch request.
const reqs = events.map(e => ({
method: "PUT",
endpoint: `https://www.googleapis.com/calendar/v3/calendars/${calendarId}/events/${e.getId().replace("#google.com", "")}`,
requestBody: {
start: {dateTime: e.getStartTime().toISOString()},
end: {dateTime: e.getEndTime().toISOString()},
summary: e.getTitle() + " something",
description: e.getDescription() + " something else"
}
}));
// Run batch requests.
const limit = 100;
const split = Math.ceil(reqs.length / limit);
const boundary = "xxxxxxxxxx";
for (let i = 0; i < split; i++) {
const object = {batchPath: "batch/calendar/v3", requests: reqs.splice(0, limit)};
const payload = object.requests.reduce((s, e, i) => s += "Content-Type: application/http\r\nContent-ID: " + i + "\r\n\r\n" + e.method + " " + e.endpoint + "\r\nContent-Type: application/json; charset=utf-8\r\n\r\n" + JSON.stringify(e.requestBody) + "\r\n--" + boundary + "\r\n", "--" + boundary + "\r\n");
const params = {method: "post", contentType: "multipart/mixed; boundary=" + boundary, payload: payload, headers: {Authorization: "Bearer " + ScriptApp.getOAuthToken()}, muteHttpExceptions: true};
UrlFetchApp.fetch("https://www.googleapis.com/" + object.batchPath, params);
}
}
Added:
You want to achieve above without using Calendar API.
For this, how about this modification?
Modification points:
In your script, there is the syntax error for the following part.
if (!(sp.getProperty("count")) || sp.getProperty("count") == 0) {
var count = 0;
else if ((sp.getProperty("count") > 0) {
var count = sp.getProperty("count");
}
Please modify else if ((sp.getProperty("count") > 0) { to } else if (sp.getProperty("count") > 0) {.
The value retrieved by getProperty is the string type. From nothing happes in your replying, I thought that this might be the reason of your issue.
So how about the following modification?
Modified script:
From:
if (!(sp.getProperty("count")) || sp.getProperty("count") == 0) {
var count = 0;
else if ((sp.getProperty("count") > 0) {
var count = sp.getProperty("count");
}
To:
var count = Number(sp.getProperty("count")) || 0;
Reference:
getProperty(key)
I found this website http://www.mess.be/inickgenwuname.php
It allows you to type in a name and it will generate a random rapper name. I wanted to have a button on my website that just generates the name for you so I decided to write some javascript that will send a request to this website and parse the response to get the random name.
Here is the node.js code I wrote.
function getRandomName() {
var http = require('http');
var data = {
realname:"something"
};
var querystring = require("querystring");
var qs = querystring.stringify(data);
var qslength = qs.length;
var options = {
hostname: "www.mess.be",
path: "/inickgenwuname.php",
method: 'POST',
headers:{
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': qslength
}
};
var str = "";
var req = http.request(options, function(res) {
res.on('data', function (chunk) {
str+=chunk;
});
res.on('end', function() {
var s = str.slice(str.indexOf("From this day forward, I will be known as... ") + "From this day forward, I will be known as... ".length,
str.indexOf("-And you"));
s = s.replace("\n", "").trim();
console.log(s);
});
});
req.write(qs);
req.end();
}
When I went to the website and pressed f12 on chrome and inspected the code, I found this little segment...
So this is what I used to formulate the request to the php. However, I only guessed through trial and error that the data that needed to be send was key-value pair object where the key is realname. My question is, how would I have known this otherwise? Is there no way to find out from the website, where the data being send with the POST is being received from?
Why by guessing? The form tells you everything that needs to be sent.
Also you could press F12 -> Network, and then send the request. After this you look at the sent requests and search for a POST request. When you click on the /inickgenwuname.php request you get more information about it. In there you can see Response Headers / Request Headers and as a last category "Form Data". There you can see all the data that is sent with this form.
I hope this is the answer you were looking for.
Stealing bandwidth without proper compensation (so called web-scraping) is quite commonly frowned upon. I couldn't find anything on that site that allows for it although I did not search thoroughly.
Why don't you roll your own? It's very simple, as can be seen in this Q&D hack:
function wu_names(input){
// some nice, fitting adjectives. Add more
var adjectives = ["annoying", "crazy", "expert", "insane", "lucky", "sardonic", "pestering"];
// some nice, fitting nouns. Add more
var nouns = ["assassin", "bastard", "conjurer", "destroyer", "ninja", "prophet", "wizard"];
var first = "";
var second = "";
var hash = 0;
var primitive_hash = function(s){
var h = 0;
for(var i = 0;i < s.length;i++){
var c = s.charCodeAt(i);
// standard hash = hash * 31 + c
h = ((h << 5) - h>>>0) + c;
}
return h;
};
first = input.split(" ")[0];
// no useful entry at all
if(first === undefined){
return null;
}
hash = primitive_hash(first);
first = adjectives[hash % adjectives.length];
second = input.split(" ")[1];
// no second entry
if(second === undefined){
return null;
}
hash = primitive_hash(second);
second = nouns[hash % nouns.length];
return first + " " + second;
}
The lists of adjectives and nouns is quite short, you might add to them, as the comments suggest.
I have a problem, I know what information I want to scrape of a website and I also know where the information is at. I know in what class it's in and also the xpath.
The problem I'm having is that no matter what I try, it seems like I can't scrape the content.
This is my scrape function:
function scrape(doc, url) {
var itemType = detectWeb(doc, doc.location.href);
var keywords = new Array();
var keywordText = doc.evaluate('//div[span="Index Terms:"]/div', doc, null, XPathResult.ANY_TYPE, null).iterateNext();
if (keywordText) keywords = (Zotero.Utilities.trimInternal(keywordText.textContent.toLowerCase())).split(",");
var attachments = new Array();
var notes = new Array();
attachments.push({
document: doc,
mimeType: "text/html",
title: "IEEE Computer Snapshot"
});
var htmls = doc.evaluate('//img[#src="/plugins/images/digitalLibrary/dl_html_icon.gif"]/ancestor::a', doc, null, XPathResult.ANY_TYPE, null);
var htmlDoc;
//TESTING
//var affiliation = doc.getElementsByTagName('meta')[property='citation_author_institution'].content;
//var affiliations = [];
var abstracts;
if (htmlDoc = htmls.iterateNext()) {
//var urlField = htmlDoc.attributes.getNamedItem("onclick").value;
var urlField = htmlDoc.href;
urlField = urlField.substr(urlField.indexOf('"') + 1);
urlField = urlField.substr(0, urlField.indexOf('"'));
if (urlField.indexOf("?") > -1) {
urlField += '&' + templte;
} else {
urlField += '?' + templte;
}
urlField = "http://www2.computer.org" + urlField;
var mimeTypeField = "text/html";
var titleField = "IEEE Computer Full Text Snapshot";
var attachment = {
url: urlField,
mimeType: mimeTypeField,
title: titleField
};
attachments.push(attachment);
}
var pdfurl = ZU.xpathText(doc, '//div[#class="abs-pdf"]/a/#href')
if (pdfurl) {
var mimeTypeField = "application/pdf";
var titleField = "IEEE Computer Full Text PDF";
var attachment = {
url: pdfurl,
mimeType: mimeTypeField,
title: titleField
};
attachments.push(attachment);
} else {
notes.push({
note: "Complete PDF document was either not available or accessible. Please make sure you're logged in to the digital library to retrieve the complete PDF document."
});
}
var bibtex = doc.evaluate('//div[#id="bibText-content"]', doc, null, XPathResult.ANY_TYPE, null).iterateNext();
var bibtexlink = ZU.xpathText(doc, '//li/a[contains(text(), "BibTex") and contains(#href, ".bib")]/#href')
if (bibtex) {
bibtex = bibtex.textContent;
//bibtex = bibtex.substring(bibtex.indexOf("document.write('")+16,bibtex.indexOf("');Popup.document.close();"));
//workaround as bibtex translator obviously needs a whitespace following the first curly brace
bibtex = Zotero.Utilities.cleanTags(bibtex);
bibtex = Zotero.Utilities.trimInternal(bibtex);
var translator = Zotero.loadTranslator("import");
translator.setTranslator("9cb70025-a888-4a29-a210-93ec52da40d4");
translator.setString(bibtex);
translator.setHandler("itemDone", function(obj, item) {
if (item.url) { // add http to url
item.url = "http://" + item.url;
}
if (itemType) item.itemType = itemType;
item.attachments = attachments;
if (keywords) item.tags = keywords;
if (notes) item.notes = notes;
if (item.DOI) item.DOI = item.DOI.replace(/^.*?10\./, "10.");
//Affiliations
/*if (affiliation)
{
for (i=0; i<affiliations.length; i++)
{
affiliation.push(affiliations[i].textContent)
}
item.extra = affiliation.join("; ");
}*/
if (abstracts) {
item.abstractNote = abstracts;
}
item.complete();
});
translator.translate();
} else if (bibtexlink) {
ZU.doGet(bibtexlink, function(text) {
var translator = Zotero.loadTranslator("import");
translator.setTranslator("9cb70025-a888-4a29-a210-93ec52da40d4");
translator.setString(text);
translator.setHandler("itemDone", function(obj, item) {
if (item.url) { // add http to url
item.url = "http://" + item.url;
}
if (itemType) item.itemType = itemType;
item.attachments = attachments;
if (keywords) item.tags = keywords;
if (notes) item.notes = notes;
if (item.DOI) item.DOI = item.DOI.replace(/^.*?10\./, "10.");
//Affiliations
/*if (affiliation)
{
for (i=0; i<affiliations.length; i++)
{
affiliation.push(affiliations[i].textContent)
}
item.extra = affiliation.join("; ");
}*/
//Abstract
if (abstracts) {
item.abstractNote = abstracts;
}
item.complete();
});
translator.translate();
})
} else {
throw "No BibTeX found!";
}
}
It's the variable called abstracts that I wanna fill with the abstract from this website.
ieee article
I used Firebug to locate where this information was stored and found it in the class="article" in the div="tabs-main".
It looks something like this:
<div id="tabs-main">
<!-- place holder -->
<div class="tab-content" id="articleDetails" role="main" data-section="articleDetails.ajax"
>
<div class="article-blk">
<div class="article">
(I want this)--> <p>Distributed database systems (DDBS) have received considerable attention in recent years. Being a relatively young research field, there are still many problems associated with DDB systems that need solution. Concurrency control is one of these problems and, probably, the most extensively studied. However, most of the work has concentrated on the development of alternative solutions and the field seems to be ready for some comparative analysis work. This paper reports the results of a performance evaluation study on distributed database concurrency control algorithms. The research has resulted in the development of a formalism, based on Petri nets, for modeling and analysis purposes. The formalism, called the Extended Place/Transition Nets (EPTN), is both descriptively powerful in that it can be used to model various algorithms precisely and succinctly and to communicate them in a clear manner, while at the same time lending itself to be used as a performance evaluation tool. An EPTN simulator is implemented and various algorithms are studied using this tool. This paper describes both the formalism and the performance results that have been obtained.</p>
</div>
And in Firebug I also get the XPath which is:
/html/body/div[2]/div[8]/div/div[2]/div/div[2]/div[1]/div/div[1]
But I don't know how I can get this content. I have tried with
var abstracts = doc.querySelector(".article").innerHTML;
I have tried with doc.getElementByClassName().
But I can never get the content, var is always null.
Someone out there have an idea?