We Keep Coding

"Illegal group end indicator...(not a group)" when decoding gtfs-r data

I'm trying to use a node.js app to regularly decode some gtfs-realtime data. It's mostly running fine, but every few hours I run into an error that crashes my app. The error message in my log says that there is an "Illegal group end indicator for Message .transit_realtime.FeedMessage 7 (not a group)"
I found this question/answer on StackOverflow but it doesn't seem to solve my particular problem. Here is an outline of the code I am using to decode the gtfs-r feed:
//process the response
var processBuffers = function(response) {
var data = [];
response.on('data', function (chunk) {
data.push(chunk);
});
response.on('end', function () {
data = Buffer.concat(data);
var decodedFeedMessage = transit.FeedMessage.decode(data);
allData = decodedFeedMessage.entity;
//continues processing with allData...
});
}
Thanks!

NodeJs crashed issue basically happen every time, everydays that any kind of fatal error trigger. And since your received data from 3-rd party, It will very had to make sure the data always correct to prevent error as well.
The simple solution is using another system to deploy your NodeJS application. I recommend 2 tools that very popular today, PM2 and Passenger . (PM2 is very simple to use). Those tool will help to auto restart your NodeJS application once it crashed
http://pm2.keymetrics.io/
https://www.phusionpassenger.com/library/walkthroughs/deploy/nodejs/ownserver/nginx/oss/install_passenger_main.html

Error trying to access Sharepoint Online host web list data from provider hosted add-in part using JSOM CDL

I've looked all over the internet for the answer to this, and still cannot figure it out. There are many examples CLOSE to what I'm trying to do, but none that actually are exact. Here's what I'm trying to do:
I have a provider hosted Sharepoint Add-In that has an aspx page with C# codebehind. I also have a javascript file in the provider hosted side (web side, NOT the app-web) that I'm trying to use to access some List data from the host web.
Here's the code I'm using:
var customWP_NewsAndInfo_allArticles;
var hostweburl;
var addinweburl;
// This code runs when the DOM is ready and creates a context object which is needed to use the SharePoint object model
$(document).ready(function () {
//Get the URI decoded URLs.
hostweburl =
decodeURIComponent(
getQueryStringParameter("SPHostUrl")
);
addinweburl =
decodeURIComponent(
getQueryStringParameter("SPAppWebUrl")
);
// resources are in URLs in the form:
// web_url/_layouts/15/resource
var scriptbase = hostweburl + "/_layouts/15/";
// Load the js files and continue to the successHandler
$.getScript(scriptbase + "SP.Runtime.js",
function () {
$.getScript(scriptbase + "SP.js",
function () { $.getScript(scriptbase + "SP.RequestExecutor.js", customWP_NewsAndInfo_retrieveListItems); }
);
}
);
});
function customWP_NewsAndInfo_retrieveListItems() {
// get parameters from the query string for add-in part properties
var sourceList = 'News and Information';
// context: The ClientContext object provides access to
// the web and lists objects.
// factory: Initialize the factory object with the
// app web URL.
var clientContext = new SP.ClientContext(addinweburl);
var factory = new SP.ProxyWebRequestExecutorFactory(addinweburl);
clientContext.set_webRequestExecutorFactory(factory);
var appContextSite = new SP.AppContextSite(clientContext, hostweburl);
this.web = appContextSite.get_web();
clientContext.load(this.web);
var web = clientContext.get_web();
var oList = web.get_lists().getByTitle(sourceList);
var items = oList.getItems(SP.CamlQuery.createAllItemsQuery());
var includeExpr = 'Include(Title)';
clientContext.load(items, includeExpr);
clientContext.executeQueryAsync(
Function.createDelegate(this, this.customWP_NewsAndInfo_onQuerySucceeded),
Function.createDelegate(this, this.customWP_NewsAndInfo_onQueryFailed)
);
}
I don't need to show the success and failure functions since it never reaches those. When it runs executeQueryAsync, it always comes back with this error and stack trace:
MicrosoftAjax.js:5 Uncaught TypeError: Cannot read property 'apply' of undefined
at Array.<anonymous> (MicrosoftAjax.js:5)
at MicrosoftAjax.js:5
at SP.ClientRequest.$3I_0 (SP.Runtime.js?_=1493695027549:2)
at Array.<anonymous> (MicrosoftAjax.js:5)
at MicrosoftAjax.js:5
at Sys.Net.WebRequest.completed (MicrosoftAjax.js:5)
at SP.ProxyWebRequestExecutor.$1W_1 (SP.RequestExecutor.js?_=1493695027551:2)
at Function.SP.ProxyWebRequestExecutorInternal.processSuccessCallback (SP.RequestExecutor.js?_=1493695027551:2)
at SP.RequestInfo.success (SP.RequestExecutor.js?_=1493695027551:2)
at SP.RequestExecutor.internalOnMessage (SP.RequestExecutor.js?_=1493695027551:2)
I even tried troubleshooting by trying a Sharepoint Hosted add-in but I get a very similar error:
Uncaught TypeError: Cannot read property 'apply' of undefined
at Array.<anonymous> (ScriptResource.axd?d=NmWpgseF5Lm2ObZqsFB7X8Sc_FI92_ZNS_ucNmpXp96dB8TO5HMUZeqx3R5ocUw8b6hyk9AKxItxXJ…:5)
at ScriptResource.axd?d=NmWpgseF5Lm2ObZqsFB7X8Sc_FI92_ZNS_ucNmpXp96dB8TO5HMUZeqx3R5ocUw8b6hyk9AKxItxXJ…:5
at SP.ClientRequest.$x_0 (SP.Runtime.js?_=1493693899820:2)
at SP.ClientRequest.$3I_0 (SP.Runtime.js?_=1493693899820:2)
at Array.<anonymous> (ScriptResource.axd?d=NmWpgseF5Lm2ObZqsFB7X8Sc_FI92_ZNS_ucNmpXp96dB8TO5HMUZeqx3R5ocUw8b6hyk9AKxItxXJ…:5)
at ScriptResource.axd?d=NmWpgseF5Lm2ObZqsFB7X8Sc_FI92_ZNS_ucNmpXp96dB8TO5HMUZeqx3R5ocUw8b6hyk9AKxItxXJ…:5
at Sys.Net.WebRequest.completed (ScriptResource.axd?d=NmWpgseF5Lm2ObZqsFB7X8Sc_FI92_ZNS_ucNmpXp96dB8TO5HMUZeqx3R5ocUw8b6hyk9AKxItxXJ…:5)
at SP.ProxyWebRequestExecutor.$1W_1 (SP.RequestExecutor.js?_=1493693899822:2)
at Function.SP.ProxyWebRequestExecutorInternal.processSuccessCallback (SP.RequestExecutor.js?_=1493693899822:2)
at SP.RequestInfo.success (SP.RequestExecutor.js?_=1493693899822:2)
I tried creating a custom list called "testing" and just used 'Include(Title)' but I received the same error. I tried inserting debugger; and stepping through the code using Chrome Developer tools but it's hard to read the obfuscated Microsoft code. Any help on figuring this out would be hugely appreciated! Thanks a lot ahead of time!
Paul

I was finally able to solve this issue by using this code and working my way back from it, since it worked when I ran it. I finally found the problem to be the call:
clientContext.executeQueryAsync(
Function.createDelegate(this, this.onQuerySucceeded),
Function.createDelegate(this, this.onQueryFailed)
);
When I compared to the github example, their call was like this:
clientContext.executeQueryAsync(onQuerySucceeded, onQueryFailed);
When I took out the Function.createDelegate part and just used the syntax they used, the error went away and I'm now able to load the list items!
As far as why this would be, maybe someone can shed some more light on this for us than I can? All I know is that whatever I tried, even testing on a Sharepoint Hosted add-in instead of Provider Hosted, it never worked until I changed this code. Now I can use Sharepoint JSOM using the Cross Domain Library on my Provider Hosted add-in on the web side. This is very valuable to me, especially when working with Publishing Images because of the limitations in this area with REST. I knew I could use REST instead in my case, but unfortunately, REST calls don't work well for my situation, since I'm also getting a Publishing Image column (Type="Image") and using REST you have to make a separate call for each list item.
I really hope this helps someone else in my position and saves you the hours I spent tracking this down!
Cheers!
Paul

How do I get intellisense for Javascript module in Atom or any other IDE?

I'm new to javascript and node js.
I've following code in my authentication.js file
I'm trying to get the intellisense working when I press client. ( and CTRL + space), I do not see anything.
How do I be able to see functions that are within auth.OAuth2 modules.
I remember in VS you can use /// reference paths. Not sure if that is the standard approach in ATOM as well. I looked over the internet and could not find any satisfactory answer.
How do people know what methods to use and what is their required signature without intellisense?
I'm literally crawling to make things work right now because of this. Do I have to read documentation for every modules/packages before I start using it? That'd take a lot of time.
Please also note that I have added all the packages like autocomplete, autocomplete-plus and so on for the intellisense to work magically but it doesn't. Intellisense does work but it displays everything else but not the functions of the modules I'm referring to in the example.
Any help/suggestion is much appreciated?
'use strict';
var config = require("../../config/config");
exports.verifyUser = function(req, res, next) {
var GoogleAuth = require('google-auth-library');
var auth = new GoogleAuth;
var client = new auth.OAuth2(config.clientID, config.clientSecret,config.callbackURL);
**client. //no intellisense**
// check header or url parameters or post parameters for token
var token = req.body.id_token || req.query.id_token || req.headers['id_token'];
if (token) {
client.verifyIdToken(
token,
config.clientID,
function (err) {
if (err) {
res.send("Un authorized");
} else {
next();
}
});
}
}

I've had great success using Visual Studio Code.
Its a lightweight IDE similar to Atom, its actually also built using Electron.
You can check out a tutorial about how to get things set up here.
https://blog.tallan.com/2017/03/02/synthetic-type-inference-in-javascript/

You need to add an intellesense plugin for the language you're using. Atom isn't really suited to noobs though, you should try out netbeans if you want a fully featured editor.

How to determine function parameters in Javascript?

I am a Java developer learning Javascript (Node.js).
This is the first piece of code I tried running :
var sys = require("sys"),
my_http = require("http");
my_http.createServer(function(request,response){
response.writeHeader(200, {"Content-Type": "text/plain"});
response.write("Hello World");
response.end();
}).listen(8080);
IF there was no documentation, how would have I known that createServer takes a function which takes request and response as parameter ? I am asking this because I want to prepare myself for all the undocumented code I will start facing soon. Here is the source for createServer function :
function createServer(options) {
var bunyan = require('./bunyan_helper');
var InternalError = require('./errors').InternalError;
var Router = require('./router');
var Server = require('./server');
var opts = shallowCopy(options || {});
var server;
opts.name = opts.name || 'restify';
opts.log = opts.log || bunyan.createLogger(opts.name);
opts.router = opts.router || new Router(opts);
server = new Server(opts);
server.on('uncaughtException', function (req, res, route, e) {
if (this.listeners('uncaughtException').length > 1 ||
res._headerSent) {
return (false);
}
res.send(new InternalError(e, e.message || 'unexpected error'));
return (true);
});
return (server);
}
I understand Javascript is a dynamically typed language, but wondering how do people debug or understand each other's code without knowing types.

Well the nice thing about javascript is it's interpreted meaning you always have access to the actual source code itself. For node, you can look in node_modules/blah to read the source, but the vast majority of what is on npm is also open source on github and you can read the source there, too.
In the browser the developer tools has an auto-format button if you encounter minified code, but in node usually you don't need that as code is published unminified.
That said, some things are documented well, sometimes documentation is wrong or out of date, and sometimes reading the source code is neither quick nor straightforward. But if something is really problematic for you and is both undocumented and hard to read, you can and should switch to something else on npm because "ain't nobody got time for that".

you must be very familiar with the api when using JavaScript.for example, document.getElementById(id). There is no hint in what the id is in the code,but it is well understood.

Issuing a synchronous HTTP GET request or invoking shell script in JavaScript from iOS UIAutomation

I am trying to use Apple's UIAutomation to write unit tests for an iOS Application that has a server-side component. In order to setup the test server in various states (as well as simulate two clients communicating through my server), I would like to issue HTTP get requests from within my javascript-based test.
Can anyone provide an example of how to either issue HTTP GET requests directly from within UIAutomation javascript tests, or how to invoke a shell script from within my UIAutomation javascript tests?
FWIW, most of the core objects made available by all browsers are missing within the UIAutomation runtime. Try to use XMLHTTPRequest for example and you will get an exception reporting that it cannot find the variable.
Thanks!

Folks,
I was able to work around this by sending HTTP requests to the iOS client to process and return the results in a UIAlertView. Note that all iOS code modifications are wrapped in #if DEBUG conditional compilation directives.
First, setup your client to send out notifications in the event of a device shake. Read this post for more information.
Next, in your iOS main app delegate add this code:
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(deviceShakenShowDebug:)
name:#"DeviceShaken"
object:nil];
Then add a method that looks something like this:
- (void) deviceShakenShowDebug:(id)sender
{
if (!self.textFieldEnterDebugArgs)
{
self.textFieldEnterDebugArgs = [[[UITextField alloc] initWithFrame:CGRectMake(0, 0, 260.0, 25.0)] autorelease];
self.textFieldEnterDebugArgs.accessibilityLabel = #"AlertDebugArgsField";
self.textFieldEnterDebugArgs.isAccessibilityElement = YES;
[self.textFieldEnterDebugArgs setBackgroundColor:[UIColor whiteColor]];
[self.tabBarController.selectedViewController.view addSubview:self.textFieldEnterDebugArgs];
[self.tabBarController.selectedViewController.view bringSubviewToFront:self.textFieldEnterDebugArgs];
}
else
{
if ([self.textFieldEnterDebugArgs.text length] > 0)
{
if ([self.textFieldEnterDebugArgs.text hasPrefix:#"http://"])
{
[self doDebugHttpRequest:self.textFieldEnterDebugArgs.text];
}
}
}
}
- (void)requestDidFinishLoad:(TTURLRequest*)request
{
NSString *response = [[[NSString alloc] initWithData:((TTURLDataResponse*)request.response).data
encoding:NSUTF8StringEncoding] autorelease];
UIAlertView *resultAlert =
[[[UIAlertView alloc] initWithTitle:NSLocalizedString(#"Request Loaded",#"")
message:response
delegate:nil
cancelButtonTitle:NSLocalizedString(#"OK",#"")
otherButtonTitles:nil] autorelease];
resultAlert.accessibilityLabel = #"AlertDebugResult";
[resultAlert show];
}
This code will add a UITextField to the very top view controller after a shake, slapped right above any navigation bar or other UI element. UIAutomation, or you the user, can manually enter a URL into this UITextField. When you shake the device again, if the text begins with "http" it will issue an HTTP request in code (exercise for the reader to implement doDebugHttpRequest).
Then, in my UIAutomation JavaScript file, I have defined the following two functions:
function httpGet(url, delayInSec) {
if (!delayInSec) delay = 1;
var alertDebugResultSeen = false;
var httpResponseValue = null;
UIATarget.onAlert = function onAlert(alert) {
httpResponseValue = alert.staticTexts().toArray()[1].name();
alert.buttons()[0].tap();
alertDebugResultSeen = true;
}
var target = UIATarget.localTarget();
var application = target.frontMostApp();
target.shake(); // bring up the input field
application.mainWindow().textFields()["AlertDebugArgsField"].setValue(url);
target.shake(); // send back to be processed
target.delay(delayInSec);
assertTrue(alertDebugResultSeen);
return httpResponseValue;
}
function httpGetJSON(url, delayInSec) {
var response = httpGet(url, delayInSec);
return eval('(' + response + ')');
}
Now, in my javascript file, I can call
httpGet('http://localhost:3000/do_something')
and it will execute an HTTP request. If I want JSON data back from the server, I call
var jsonResponse = httpGetJSON('http://localhost:3000/do_something')
If I know it is going to be a long-running call, I call
var jsonResponse = httpGetJSON('http://localhost:3000/do_something', 10 /* timeout */)
I've been using this approach successfully now for several weeks.

Try performTaskWithPathArgumentsTimeout
UIATarget.host().performTaskWithPathArgumentsTimeout("/usr/bin/curl", "http://google.com", 30);

Just a small correction. The answer that suggests using UIATarget.host().performTaskWithPathArgumentsTimeout is an easy way to make a request on a URL in iOS 5.0+, but the syntax of the example is incorrect. Here is the correct way to make this call:
UIATarget.host().performTaskWithPathArgumentsTimeout("/usr/bin/curl", ["http://google.com"], 30);
The "[" around the "args" param is important, and the test will die with an exception similar to the following if you forget the brackets:
Error: -[__NSCFString count]: unrecognized selector sent to instance
Here is a fully working example that hits google.com and logs all the output:
var result = UIATarget.host().performTaskWithPathArgumentsTimeout("/usr/bin/curl", ["http://www.google.com"], 30);
UIALogger.logDebug("exitCode: " + result.exitCode);
UIALogger.logDebug("stdout: " + result.stdout);
UIALogger.logDebug("stderr: " + result.stderr);

+1 for creative use of "shake()". However, that's not an option for some projects, especially those that actually use the shake feature.
Think outside the box. Do the fetching with something else (Python, Ruby, node.js, bash+wget, etc). Then, you can use the pre-canned response and auto-generate the ui-test.js on the fly by including that dynamically generated json payload as the "sample data" into the test. Then you simply execute the test.
In my opinion, the test is the test, leave that alone. The test data you are using, if it's that dynamic, it ought to be separated from the test itself. By doing it this way of fetching / generating JSON, and referencing it from the test, you can update that JSON however often you like, either immediately right before every test, or on a set interval like when you know the server has been updated. I'm not sure you would want to generate it while the test is running, that seems like it would create problems. Taking it to the next level, you could get fancy and use functions that calculate what values ought to be based on other values, and expose them as "dynamic properties" of the data, rather than that math being inside the test, but at that point I think the discussion is more of an academic one rather than the practical one of how.

Apple has recently updated UIAutomation to include a new UIAHost element for performing a task on the Host that is running the instance of Instruments that is executing the tests.