Erratic Google Cloud Function when running a shell script - javascript

I am trying to run a shell script on a Google Cloud Function.
I am using Node as the framework, and have tried different ways to run the script: execSync, execFileSync, spawnSync, etc:
module.exports.main = function() {
try {
const output = require('child_process').execSync(__dirname + '/run.sh', [],
{
stdio: 'inherit',
shell: '/bin/bash'
});
}
catch (error) {
process.exit(error.status);
}
}
The behavior I get is very erratic. Overall I think it worked once, but the rest of the time I either get a connection error and nothing happens, or the function terminates "successfully", but nothing from the shell script gets outputted, so I don't know whether it even ran or not.
To be more precise, the log lines I see when the function "fails" are only the following:
Function execution started
Function execution took 122 ms, finished with status: 'connection error'
I couldn't detect a pattern in when it fails and when it runs.
Exact reproduction steps:
Create a Google Cloud Function
set its runtime to Node 8
set its trigger to Pub/Sub topic
set its function to call to main
create a zip containing
index.js with the content above (module.exports.main = ...)
run.sh containing only
#! /bin/bash
echo "hi"
upload zip as source for Cloud Function
go to Testing tab of Cloud function and hit "Test this functions"
view logs

The solution I found was to switch to IBM Cloud Functions. They enable you to run a docker image, thus executing any binary is possible.
My suspicion as to why Google Cloud Function failed though, is that it probably ran out of memory. But that is just a guess

Related

Defichain WhaleApiClient function call gives TypeError: fn is not a function

I have a file called defichain.js and am trying to communicate with the defichain blockchain via OCEAN REST API in my project. I am using Node.js version 16.14.2. I am running the .js script with the latest version of truffle.
I already managed to print out the WhaleApiClient object in the console to check its available functions, so at least it is recognizing the client.
Anyways, when I am trying to get the current stats of the DeFi-Chain Blockchain (e.g. number of masternodes), I get TypeError: fn is not a function.
Clearly I am not calling any function named fn here.
const { WhaleApiClient } = require('#defichain/whale-api-client')
const client = new WhaleApiClient({
url: 'https://ocean.defichain.com',
timeout: 60000,
version: 'v0',
network: 'mainnet'
});
console.log(client); // prints client object with all of its categories
console.log(client.stats.get()) // prints Promise { <pending> }
// it seems like something is wrong here :(
client.stats.get().then((data) => {
console.log(data)
})
I am expecting an output in the console something similar to:
{
"data":{
"count":{
"blocks":1831387,
"prices":97,
"tokens":146,
"masternodes":11044
},
"burned":{
"address":156009915.966001,
"fee":237127,
"auction":473292.86501403,
"payback":49335487.46135226,
"emission":82354933.12182175,
"total":288423830.22026604
},
"tvl":{
"dex":874830839.7451664,
"masternodes":956478286.6812595,
"loan":260332219.5450214,
"total":2091641345.9714475
},
"price":{
"usd":4.164032593301086,
"usdt":4.164032593301086
},
"masternodes":{
"locked":[
{
"weeks":0,
"count":7435,
"tvl":655918414.0967871
},
{
"weeks":520,
"count":2859,
"tvl":238099383.6849561
},
{
"weeks":260,
"count":750,
"tvl":62460488.89951629
}
]
},
"loan":{
"count":{
"collateralTokens":6,
"loanTokens":28,
"openAuctions":8,
"openVaults":9915,
"schemes":6
},
"value":{
"collateral":260332219.5450214,
"loan":136835351.84522375
}
},
"emission":{
"masternode":84.5329931,
"dex":64.54739497,
"community":12.45295518,
"anchor":0.05072487,
"burned":92.04027360305092,
"total":253.62434172305092
},
"net":{
"version":2070000,
"subversion":"/DeFiChain:2.7.0/",
"protocolversion":70028
},
"blockchain":{
"difficulty":22297887949.45695
}
}
}
For more information on Ocean REST API, Defichain and its Jellyfish library:
Ocean REST API is a quite unknown project and still in its early stages. Ocean REST API is a global infrastructure project hosted by DeFiChain to simplify building decentralized light applications. Powered by the Jellyfish Ecosystem, Ocean Nodes are globally distributed around the world, any API request is served by the node nearest to the requester with auto fail-over.
https://jellyfish.defichain.com/ocean/
I solved it by myself after a long headache. The error showed following:
TypeError: fn is not a function
at Object.exec (/usr/local/lib/node_modules/truffle/build/webpack:/packages/require/require.js:127:1)
at node:internal/util:360:7
at new Promise (<anonymous>)
at bound exec (node:internal/util:346:12)
at Object.run (/usr/local/lib/node_modules/truffle/build/webpack:/packages/core/lib/commands/exec.js:75:1)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at Command.run (/usr/local/lib/node_modules/truffle/build/webpack:/packages/core/lib/command.js:183:1)
Looking at the first line, I thought that the problem might be in node_modules/truffle/ and has nothing to do with my rest-api calls in the first place.
I found out that truffle exec <example_script> does not properly handle require statements for local node modules in example_script and it was a common bug in 2016 (see:
https://github.com/trufflesuite/truffle/issues/255)
However, deep in the discussion I found an answer from Tim Coulter
"Due to technical reasons, truffle exec in Truffle 2.0 requires your script to output a module that's passed a callback function."
In the original post, he is referring to a post which leads to the documentation site of truffle which doesn't work anymore today.
I found this link here:
https://trufflesuite.com/docs/truffle/getting-started/writing-external-scripts/
In order for external scripts to be run correctly, Truffle expects them to export a function that takes a single parameter as a callback:
module.exports = function(callback) {
// TODO: implement your actions
// invoke callback
callback();
}
You can do anything you'd like within this script, as long as the callback is invoked when the script finishes. The callback accepts an error as its first and only parameter. If an error is provided, execution will halt and the process will return a non-zero exit code.

Why does timer for waits start before the steps are actually executed in a protractor test? Eventloop wrangling

tl;dr: When I run my test case, steps executed seem to work, but the test bails out early on a failure to find an element that hasn't even loaded yet. It seems like the waits I have around locating certain elements are loaded/launched as soon as the test is launched, not when the lines should actually be executed in the test case. I think this is happening because the page is barely (correctly) loaded before the "search" for the element to verify the page has loaded bails out. How do I wrangle the event loop?
This is probably a promise question, which is fine, but I don't understand what's going on. How do I implement my below code to work as expected? I'm working on creating automated E2E test cases using Jasmine2 and Protractor 5.3.0 in an Angular2 web app.
describe('hardware sets', () => {
it('TC3780:My_Test', async function() {
const testLogger = new CustomLogger('TC3780');
const PROJECT_ID = '65';
// Test Setup
browser.waitForAngularEnabled(false); // due to nature of angular project, the app never leaves zones, leaving a macrotask constantly running, thus protractor's niceness with angular is not working on our web app
// Navigate via URL to planviewer page for PROJECT_ID
await planListingPage.navigateTo(PROJECT_ID); // go to listing page for particular project
await planListingPage.clickIntoFirstRowPlans(); // go to first plan on listing page
await planViewerPage.clickOnSetItem('100'); // click on item id 100 in the plan
});
});
planViewerPage.po.ts function:
clickOnSetItem(id: string) {
element(by.id(id)).click();
browser.wait(until.visibilityOf(element(by.css('app-side-bar .card .info-content'))), 30000); // verify element I want to verify is present and visible
return expect(element(by.css('app-side-bar .card .info-content')).getText).toEqual(id); //Verify values match, This line specifically is failing.
}
This is the test case so far. I need more verification, but it is mostly done. I switched to using async function and awaits instead of the typical (done) and '.then(()=>{' statement chaining because I prefer not having to do a bunch of nesting to get things to execute in the right order. I come from a java background, so this insanity of having to force things to run in the order you write them is a bit much for me sometimes. I've been pointed to information like Mozilla's on event loop, but this line just confuses me more:
whenever a function runs, it cannot be pre-empted and will run entirely before any other code
runs (and can modify data the function manipulates).
Thus, why does it seem like test case is pre-evaluated and the timer's set off before any of the pages have been clicked on/loaded? I've implemented the solution here: tell Protractor to wait for the page before executing expect pretty much verbatim and it still doesn't wait.
Bonus question: Is there a way to output the event-loop's expected event execution and timestamps? Maybe then I could understand what it's doing.
The behavior
The code in your function is running asynchronously
clickOnSetItem(id: string) {
element(by.id(id)).click().then(function(){
return browser.wait(until.visibilityOf(element(by.css('app-side-bar .card .info-content'))), 30000);
}).then(function(){
expect(element(by.css('app-side-bar .card .info-content')).getText).toEqual(id);
}).catch(function(err){
console.log('Error: ' + err);
})
}

Unhandled exceptions in Google Apps Script

I have created a public Web App with access to my private spreadsheet data. I can catch and log exceptions intry..catch, but:
is it possible to catch all unhandled exceptions, like browsers window.onerror?
can I view logs of unhandled exceptions somewhere?
by exceptions like "Service invoked too many times" my app is even not getting run, so here I definitely can`t handle the exceptions. Is there logs with such kind of exceptions?
These are so simple questions, so that I'm bit confused to ask them, but after hours of research I could not find the answers.
Thank you in advance.
These are issues that are being addressed currently. Right now in the Apps Script Early Access Program are two new additions that handle these cases. The first is native integration with stackdriver logging and the addition of google.script.run.withLogger().
First off for now you need to apply for the EAP:
https://developers.google.com/apps-script/guides/apps-script-eap
Stackdriver Logging:
To log to stackdriver the console object has been added to the server side.
code.gs
console.log('This will log to stackdriver')
Check out the docs for all the methods of console.
https://developers.google.com/apps-script/guides/logging#stackdriver_logging
Example from the docs:
function measuringExecutionTime() {
// A simple INFO log message, using sprintf() formatting.
console.info('Timing the %s function (%d arguments)', 'myFunction', 1);
// Log a JSON object at a DEBUG level. The log is labeled
// with the message string in the log viewer, and the JSON content
// is displayed in the expanded log structure under "structPayload".
var parameters = {
isValid: true,
content: 'some string',
timestamp: new Date()
};
console.log({message: 'Function Input', initialData: parameters});
var label = 'myFunction() time'; // Labels the timing log entry.
console.time(label); // Starts the timer.
try {
myFunction(parameters); // Function to time.
} catch (e) {
// Logs an ERROR message.
console.error('myFunction() yielded an error: ' + e);
}
console.timeEnd(label);
}
In addition you can also check Log Exceptions in the scripts properties. This will generate a stackdriver entry every time any error occurs in your script.
Error recovery in a web app
To recover in a web app from a failure you have access to the withFailureHandler() method found in the google.script.run object. With this you can register a callback in the event your script hits an exception.
Full documentation can be found at:
https://developers.google.com/apps-script/guides/html/reference/run
If you are doing server side checks with try...catch you may be getting an exception but gracefully handling it. In this case withFailureHandler() will not execute and onSuccessHandler() propably isnt the best place to handle errors. In the EAP there is now a withLogger method to google.script.run. For now there no documentation for google.script.run.withLogger(). I found it by digging through devtools. withLogger() allows you to register a function as a callback when ever a stackdriver entry is created. This is particularly helpful when you have log exceptions checked in your script properties. In this sense it is a bit like withFailureHandler() but it can be triggered by any stackdriver entry you add though the server-side console object.
index.html
<script>
google.script.run
.withSuccessHandler(function(){console.log('OK')})
.withFailureHandler(function(e){console.error(e)})
.withLogger(function(e){console.warn("The following log was generated:"+e)})
.serverFunctionCall();
</script>
code.gs
function serverFunctionCall(){
console.log("This log will generate a callback");
return true;
}
Try/catch at global scope will work, however any let/const container will not be globally exposed.
To fix this, you can use var within the try/catch at global scope

Does Cucumber with Protractor need the callback?

I'm trying to understand how The WebDriver Control Flow works exactly.
According to the linked documentation (https://github.com/angular/protractor/blob/master/docs/control-flow.md) no callback method / call is needed in jasmine:
Protractor adapts Jasmine so that each spec automatically waits until the control flow is empty before exiting.
However, I have to use cucumber. I'm using the library protractor-cucumber-framework as described here: https://github.com/angular/protractor/blob/master/docs/frameworks.md#using-cucumber
It works well, but for some reason it works better when I skip the callback variable then when I try using it. For instance, this code fails:
this.Given(/^the login page is active$/, function (callback) {
browser.get('/').then(callback);
});
With the error ...
TypeError: text.split is not a function
[launcher] Process exited with error code 1
On the other hand, this codes works as I want it to work and cucumber / protractor seems to be waiting until the page is loaded, before executing further functions:
me.Given(/^the login page is active$/, function () {
browser.get('/');
});
But I couldn't find any documentation confirming that I really can omit the callback function.
Currently the page I tried to test doesn't use Angular and therefore I have the following code in my config file:
onPrepare: function() {
browser.ignoreSynchronization = true;
}
Protractor uses WebDriverJS underneath. And WebDriverJS uses a promise manager where it queues its commands. Here is some excerpts from their wiki page here
Internally, the promise manager maintains a call stack. Upon each turn
of the manager's execution loop, it will pull a task to execute from
the queue of the top-most frame. Any commands scheduled within the
callback of a previous command will be scheduled in a new frame,
ensuring they run before any tasks previously scheduled. The end
result is that if your test is written-in line, with all callbacks
defined by function literals, commands should execute in the order
they are read vertically on the screen. For example, consider the
following WebDriverJS test case:
driver.get(MY_APP_URL);
driver.getTitle().then(function(title) {
if (title === 'Login page') {
driver.findElement(webdriver.By.id('user')).sendKeys('bugs');
driver.findElement(webdriver.By.id('pw')).sendKeys('bunny');
driver.findElement(webdriver.By.id('login')).click();
}
});
driver.findElement(webdriver.By.id('userPreferences')).click();
This test case could be rewritten using !WebDriver's Java API as follows:
driver.get(MY_APP_URL);
if ("Login Page".equals(driver.getTitle())) {
driver.findElement(By.id("user")).sendKeys("bugs");
driver.findElement(By.id("pw")).sendKeys("bunny");
driver.findElement(By.id("login")).click();
}
driver.findElement(By.id("userPreferences")).click();
Now going back to your question, since you are omitting callback from your steps, cucumber is treating your test code as synchronous. See documentation here. And because the way protractor/WebdriverJS handles promise manager the way described above, everything works as expected for you.
As far as the error you are getting when using callback, I'm not sure. I do it exactly the same way you are doing. See here. I'm using cucumber ^0.9.2. It could be that your cucumber version has issues.
On a side note, I found that you could return promises instead of using callbacks to let cucumber know that you are done executing. So something like this works as well (assuming you are using ^0.9.2). I tested it,
me.Given(/^the login page is active$/, function () {
return browser.get('/');
});

Execute code in a sandbox, with modules in same context

I'm automating running the ECMA-402 test suite against the Intl polyfill I wrote, and I've hit some problems. Currently, the tests are run against a fully-built version of the library, which means having to recompile every time a change is made before the tests can run. I'm trying to improve it by splitting the code up into separate modules and using require to run the tests.
The main problem comes into focus when I try and run the tests using the vm module. If I add the polyfill to the test's sandbox, some of the tests fail when checking native behaviour — the polyfill's objects don't inherit from the test context's Object.prototype, for example. Passing require to the tests will not work because the modules are still compiled and executed in the parent's context.
The easiest solution in my head was to spawn a new node process and write the code to the process's stdin, but the spawned node process doesn't execute the code written to it and just waits around forever. This is the code I tried:
function runTest(testPath, cb) {
var test,
err = '',
content = 'var IntlPolyfill = require("' + LIB_PATH + '");\n';
content += LIBS.fs.readFileSync(LIBS.path.resolve(TEST_DIR, testPath)).toString();
content += 'runner();';
test = LIBS.spawn(process.execPath, process.execArgv);
test.stdin.write(content, 'utf8');
// cb runs the next test
test.on('exit', cb);
}
Does anyone have any idea why Node.js doesn't execute the code written to its stdin stream, or if there's another way I can get the module to compile in the same context as the tests?
You must close the stdin for the child process to consume data and exit. Do this when you are done passing code.
test.stdin.end();
In the end, I chose to use the -e command line switch to pass the code directly to the new node instance. It only took a slight modification to the code:
function runTest(testPath, cb) {
var test,
err = '',
content = 'var IntlPolyfill = require("' + LIB_PATH + '");\n';
content += LIBS.fs.readFileSync(LIBS.path.resolve(TEST_DIR, testPath)).toString();
content += 'runner();';
test = LIBS.spawn(process.execPath, process.execArgv.concat('-e', content));
// cb runs the next test
test.on('exit', cb);
}

Categories