R version: 4.1.0
Shiny Server v1.5.16.958
Node.js v12.20.0
I have a shiny-server script that I'm having issues with. When I call this script using a web browser, I immediately get: "Disconnected from the server. Reload". Upon inspecting the browser debug console, I received this debug information that I'm not quite sure what to do about. Does anyone have any ideas on how I can resolve this? :
Listening on http://127.0.0.1:46259
gdata: read.xls support for 'XLS' (Excel 97-2004) files ENABLED.
gdata: read.xls support for 'XLSX' (Excel 2007+) files ENABLED.
Attaching package: ‘gdata’
The following object is masked from ‘package:stats’:
nobs
The following object is masked from ‘package:utils’:
object.size
The following object is masked from ‘package:base’:
startsWith
Attaching package: ‘bizdays’
The following object is masked from ‘package:stats’:
offset
Warning: Error in Calendar: could not find function "Calendar"
[No stack trace available]
Error in Calendar(hdays, weekdays = c("saturday", "sunday")) :
could not find function "Calendar"
shiny-server-client.min.js:1 The application unexpectedly exited.
We have figured out the problem. Bizdays has deprecated the function 'Calendar'. In our Server.R file we had to update the code to use 'create.calender' instead.
I'm making a Cordova app with backbone and my goal was to achieve socket authentification with a JSON Web Token (JWT).
To sign my JWT, I used webcrypto-jwt and it worked well when using the app in the browser.
Then I tried the app on my mobile and BBAAMM...
webcrypto-jwt uses the browser's window.crypto.subtle module.
var cryptoSubtle = (window.crypto && crypto.subtle) ||
(window.crypto && crypto.webkitSubtle) ||
(window.msCrypto && window.msCrypto.Subtle);
But no subtle on android web view!
So I used webcrypto-shim to add the crypto.subtle. But it doesn't work.
That's a screenshot of my cordova's window object. It does have a crypto key but with no subtle in it!
So I can't sign my JWT.
https://github.com/PeculiarVentures/webcrypto-liner will provide you a working webcrypto on most platforms.
I have used it with https://github.com/square/js-jose just fine.
You can test your browsers support for WebCrypto with this - https://peculiarventures.github.io/pv-webcrypto-tests/
Ryan
WebCryptographyApi is not supported on Android WebView, and webcrypto-shim is not targeted to this component
The library is targeted to fix these browsers having prefixed and buggy webcrypto api implementations:
Internet Explorer 11, Mobile Internet Explorer 11,
Safari 8+, iOS Safari 8+.
So you are getting the expected behaviour. I think window.crypto that is showing Cordova is the old implementation.
If you need key storage I suggest use the Android native keystore (or iOS if you build for it). If you are looking for cryptographic function, include a pure javascript library
After more research and tests I have found a pure js library that works on cordova.
jsrsasign
I used it to authentificate my JWT. It doesn't use the the crypto.subtle module.
// Header
var oHeader = { alg: 'HS256', typ: 'JWT' };
// Payload
var oPayload = {};
var tNow = KJUR.jws.IntDate.get('now');
var tEnd = KJUR.jws.IntDate.get('now + 1day');
oPayload.iss = "http://foobar.com";
oPayload.sub = "mailto:someone#hello.com";
oPayload.iat = tNow;
oPayload.exp = tEnd;
oPayload.jti = "id123";
oPayload.aud = "http://someUrl";
oPayload.email = "userEmail";
oPayload.pwd = "userPassword";
oPayload.deviceId = "deviceId";
// Sign JWT.
var sHeader = JSON.stringify(oHeader);
var sPayload = JSON.stringify(oPayload);
//secret -> your secret that the server gave you.
var sJWT = KJUR.jws.JWS.sign("HS256", sHeader, sPayload, secret);
console.log(sJWT);
So that's it. It solved my problem.
I know that the undefined crypto.subtle error still exists. I did not find any solutions to that problem.
I supose that one day the developers in charge of cordova will make the effort to support the cryto module that we can find in all other browser but for now the only solution is to used third party libraries.
I am wondering how would I go abouts in detecting search crawlers? The reason I ask is because I want to suppress certain JavaScript calls if the user agent is a bot.
I have found an example of how to to detect a certain browser, but am unable to find examples of how to detect a search crawler:
/MSIE (\d+\.\d+);/.test(navigator.userAgent); //test for MSIE x.x
Example of search crawlers I want to block:
Google
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Googlebot/2.1 (+http://www.google.com/bot.html)
Baidu
Baiduspider+(+http://www.baidu.com/search/spider_jp.html)
Baiduspider+(+http://www.baidu.com/search/spider.htm)
BaiDuSpider
This is the regex the ruby UA agent_orange library uses to test if a userAgent looks to be a bot. You can narrow it down for specific bots by referencing the bot userAgent list here:
/bot|crawler|spider|crawling/i
For example you have some object, util.browser, you can store what type of device a user is on:
util.browser = {
bot: /bot|googlebot|crawler|spider|robot|crawling/i.test(navigator.userAgent),
mobile: ...,
desktop: ...
}
Try this. It's based on the crawlers list on available on https://github.com/monperrus/crawler-user-agents
var botPattern = "(googlebot\/|bot|Googlebot-Mobile|Googlebot-Image|Google favicon|Mediapartners-Google|bingbot|slurp|java|wget|curl|Commons-HttpClient|Python-urllib|libwww|httpunit|nutch|phpcrawl|msnbot|jyxobot|FAST-WebCrawler|FAST Enterprise Crawler|biglotron|teoma|convera|seekbot|gigablast|exabot|ngbot|ia_archiver|GingerCrawler|webmon |httrack|webcrawler|grub.org|UsineNouvelleCrawler|antibot|netresearchserver|speedy|fluffy|bibnum.bnf|findlink|msrbot|panscient|yacybot|AISearchBot|IOI|ips-agent|tagoobot|MJ12bot|dotbot|woriobot|yanga|buzzbot|mlbot|yandexbot|purebot|Linguee Bot|Voyager|CyberPatrol|voilabot|baiduspider|citeseerxbot|spbot|twengabot|postrank|turnitinbot|scribdbot|page2rss|sitebot|linkdex|Adidxbot|blekkobot|ezooms|dotbot|Mail.RU_Bot|discobot|heritrix|findthatfile|europarchive.org|NerdByNature.Bot|sistrix crawler|ahrefsbot|Aboundex|domaincrawler|wbsearchbot|summify|ccbot|edisterbot|seznambot|ec2linkfinder|gslfbot|aihitbot|intelium_bot|facebookexternalhit|yeti|RetrevoPageAnalyzer|lb-spider|sogou|lssbot|careerbot|wotbox|wocbot|ichiro|DuckDuckBot|lssrocketcrawler|drupact|webcompanycrawler|acoonbot|openindexspider|gnam gnam spider|web-archive-net.com.bot|backlinkcrawler|coccoc|integromedb|content crawler spider|toplistbot|seokicks-robot|it2media-domain-crawler|ip-web-crawler.com|siteexplorer.info|elisabot|proximic|changedetection|blexbot|arabot|WeSEE:Search|niki-bot|CrystalSemanticsBot|rogerbot|360Spider|psbot|InterfaxScanBot|Lipperhey SEO Service|CC Metadata Scaper|g00g1e.net|GrapeshotCrawler|urlappendbot|brainobot|fr-crawler|binlar|SimpleCrawler|Livelapbot|Twitterbot|cXensebot|smtbot|bnf.fr_bot|A6-Indexer|ADmantX|Facebot|Twitterbot|OrangeBot|memorybot|AdvBot|MegaIndex|SemanticScholarBot|ltx71|nerdybot|xovibot|BUbiNG|Qwantify|archive.org_bot|Applebot|TweetmemeBot|crawler4j|findxbot|SemrushBot|yoozBot|lipperhey|y!j-asr|Domain Re-Animator Bot|AddThis)";
var re = new RegExp(botPattern, 'i');
var userAgent = navigator.userAgent;
if (re.test(userAgent)) {
console.log('the user agent is a crawler!');
}
The following regex will match the biggest search engines according to this post.
/bot|google|baidu|bing|msn|teoma|slurp|yandex/i
.test(navigator.userAgent)
The matches search engines are:
Baidu
Bingbot/MSN
DuckDuckGo (duckduckbot)
Google
Teoma
Yahoo!
Yandex
Additionally, I've added bot as a catchall for smaller crawlers/bots.
This might help to detect the robots user agents while also keeping things more organized:
Javascript
const detectRobot = (userAgent) => {
const robots = new RegExp([
/bot/,/spider/,/crawl/, // GENERAL TERMS
/APIs-Google/,/AdsBot/,/Googlebot/, // GOOGLE ROBOTS
/mediapartners/,/Google Favicon/,
/FeedFetcher/,/Google-Read-Aloud/,
/DuplexWeb-Google/,/googleweblight/,
/bing/,/yandex/,/baidu/,/duckduck/,/yahoo/, // OTHER ENGINES
/ecosia/,/ia_archiver/,
/facebook/,/instagram/,/pinterest/,/reddit/, // SOCIAL MEDIA
/slack/,/twitter/,/whatsapp/,/youtube/,
/semrush/, // OTHER
].map((r) => r.source).join("|"),"i"); // BUILD REGEXP + "i" FLAG
return robots.test(userAgent);
};
Typescript
const detectRobot = (userAgent: string): boolean => {
const robots = new RegExp(([
/bot/,/spider/,/crawl/, // GENERAL TERMS
/APIs-Google/,/AdsBot/,/Googlebot/, // GOOGLE ROBOTS
/mediapartners/,/Google Favicon/,
/FeedFetcher/,/Google-Read-Aloud/,
/DuplexWeb-Google/,/googleweblight/,
/bing/,/yandex/,/baidu/,/duckduck/,/yahoo/, // OTHER ENGINES
/ecosia/,/ia_archiver/,
/facebook/,/instagram/,/pinterest/,/reddit/, // SOCIAL MEDIA
/slack/,/twitter/,/whatsapp/,/youtube/,
/semrush/, // OTHER
] as RegExp[]).map((r) => r.source).join("|"),"i"); // BUILD REGEXP + "i" FLAG
return robots.test(userAgent);
};
Use on server:
const userAgent = req.get('user-agent');
const isRobot = detectRobot(userAgent);
Use on "client" / some phantom browser a bot might be using:
const userAgent = navigator.userAgent;
const isRobot = detectRobot(userAgent);
Overview of Google crawlers:
https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers
isTrusted property could help you.
The isTrusted read-only property of the Event interface is a Boolean
that is true when the event was generated by a user action, and false
when the event was created or modified by a script or dispatched via
EventTarget.dispatchEvent().
eg:
isCrawler() {
return event.isTrusted;
}
⚠ Note that IE isn't compatible.
Read more from doc: https://developer.mozilla.org/en-US/docs/Web/API/Event/isTrusted
People might light to check out the new navigator.webdriver property, which allows bots to inform you that they are bots:
https://developer.mozilla.org/en-US/docs/Web/API/Navigator/webdriver
The webdriver read-only property of the navigator interface indicates whether the user agent is controlled by automation.
It defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, for example, so that alternate code paths can be triggered during automation.
It is supported by all major browsers and respected by major browser automation software like Puppeteer. Users of automation software can of course disable it, and so it should only be used to detect "good" bots.
I combined some of the above and removed some redundancy. I use this in .htaccess on a semi-private site:
(google|bot|crawl|spider|slurp|baidu|bing|msn|teoma|yandex|java|wget|curl|Commons-HttpClient|Python-urllib|libwww|httpunit|nutch|biglotron|convera|gigablast|archive|webmon|httrack|grub|netresearchserver|speedy|fluffy|bibnum|findlink|panscient|IOI|ips-agent|yanga|Voyager|CyberPatrol|postrank|page2rss|linkdex|ezooms|heritrix|findthatfile|Aboundex|summify|ec2linkfinder|facebook|slack|instagram|pinterest|reddit|twitter|whatsapp|yeti|RetrevoPageAnalyzer|sogou|wotbox|ichiro|drupact|coccoc|integromedb|siteexplorer|proximic|changedetection|WeSEE|scrape|scaper|g00g1e|binlar|indexer|MegaIndex|ltx71|BUbiNG|Qwantify|lipperhey|y!j-asr|AddThis)
The "test for MSIE x.x" example is just code for testing the userAgent against a Regular Expression. In your example the Regexp is the
/MSIE (\d+\.\d+);/
part. Just replace it with your own Regexp you want to test the user agent against. It would be something like
/Google|Baidu|Baiduspider/.test(navigator.userAgent)
where the vertical bar is the "or" operator to match the user agent against all of your mentioned robots. For more information about Regular Expression you can refer to this site since javascript uses perl-style RegExp.
I found this isbot package that has the built-in isbot() function. It seams to me that the package is properly maintained and that they keep everything up-to-date.
USAGE:
const isBot = require('isbot');
...
isBot(req.get('user-agent'));
Package: https://www.npmjs.com/package/isbot
I am using a JavaScript error logging tool to log any JavaScript errors on my mobile site and I am seeing a large number (22,000 in under a month) of this error:
Uncaught TypeError: Cannot read property '_pmh' of null
I can see from the addthis_widget.js code that it is originating from that script.
I can also tell that it only affects mobile Android and Google Chrome browsers, these in particular:
Android 4, Android 4.1, Android 4.2, Chrome 11, Chrome 18, Chrome 25, Chrome 26, Chrome 27, Chrome Mobile 18, Chrome Mobile 25, Chrome Mobile 26
I am using the following external JavaScript include:
http://s7.addthis.com/js/250/addthis_widget.js#username=xa-4b6adff375a64db9
and have the following configuration object set:
<script type="text/javascript">
var addthis_config = {
data_ga_property: 'UA-18721873-1', // real GA profile hidden for privacy purposes
data_ga_social: true
};
</script>
Is there anything I can do to prevent this error from occuring so many times on these browsers?
I have located the cause of this issue and it is down the following code.
if (window.addthis) {
window.addthis = null;
window._adr = null;
window._atc = null;
window._atd = null;
window._ate = null;
window._atr = null;
window._atw = null;
}
This was an attempt to cleanup the objects lying around when moving between pages dynamically to prevent memory issues.
addThis now provide support for this problem, their support department sent me the following link:
http://www.addthis.com/blog/2013/05/07/a-brief-history-of-using-addthis-dynamically/#.Uklx4RCKyas
and the following explanation/information:
Along with other information from my peers and such, I've come up with
the proper code which should be executed when moving from virtual page
to virtual page such as in your mobile application. Rather than
nulling any of the AddThis related objects, they must be left alone.
When the page URL or Title (or any other AddThis configuration option)
that you want to be used by our buttons changes, you should execute
our method:
addthis.update(type, key, value)
It takes the three parameters: type, key, value.
Type is either "share" or "config". The key is the same key you would
set according to our API documentation. What you likely need to know
are just the two keys: "url" and "title". The value is of course the
value you want those options to have.
After you have updated all of the options you need to update using
that method, you need to invoke this method:
addthis.toolbox(cssSelector)
The cssSelector is usually going to be ".addthis_toolbox" but if for
some reason you don't want to refresh all of your toolboxes, you can
be more specific about it.
Once you have made those changes to your code, I believe that our
tools will work properly for you and that there will be no more "null
object" related javascript errors.
You can look at this JSFiddle to see an example of dynamically
changing content: http://jsfiddle.net/j5addthis/r6Utv/
I'm having the same issue, but it appears that there was, at one point, a fix to your specific case above by simply upgrading the script that you're using from addthis to version 3.0 by changing this:
http://s7.addthis.com/js/250/addthis_widget.js#username=xa-4b6adff375a64db9
To this:
http://s7.addthis.com/js/300/addthis_widget.js#username=xa-4b6adff375a64db9
Read more here: http://support.addthis.com/customer/portal/questions/1021465-addthis-widget-js-throwing-cannot-read-property-pmh-of-null-error-on-android-and-chrome
Note though, as I mentioned above, my issue continues to persist even with version 3.0 despite the AddThis help desk stating that should work.
I'm using the Firefox Addon SDK to build something that monitors and displays the HTTP traffic in the browser. Similar to HTTPFox or Live HTTP Headers. I am interested in identifying which tab in the browser (if any) generated the request
Using the observer-service I am monitoring for "http-on-examine-response" events. I have code like the following to identify the nsIDomWindow that generated the request:
const observer = require("observer-service"),
{Ci} = require("chrome");
function getTabFromChannel(channel) {
try {
var noteCB= channel.notificationCallbacks ? channel.notificationCallbacks : channel.loadGroup.notificationCallbacks;
if (!noteCB) { return null; }
var domWin = noteCB.getInterface(Ci.nsIDOMWindow);
return domWin.top;
} catch (e) {
dump(e + "\n");
return null;
}
}
function logHTTPTraffic(sub, data) {
sub.QueryInterface(Ci.nsIHttpChannel);
var ab = getTabFromChannel(sub);
console.log(tab);
}
observer.add("http-on-examine-response", logHTTPTraffic);
Mostly cribbed from the documentation for how to identify the browser that generated the request. Some is also taken from the Google PageSpeed Firefox addon.
Is there a recommended or preferred way to go from the nsIDOMWindow object domWin to a tab element in the SDK tabs module?
I've considered something hacky like scanning the tabs list for one with a URL that matches the URL for domWin, but then I have to worry about multiple tabs having the same URL.
You have to keep using the internal packages. From what I can tell, getTabForWindow() function in api-utils/lib/tabs/tab.js package does exactly what you want. Untested code:
var tabsLib = require("sdk/tabs/tab.js");
return tabsLib.getTabForWindow(domWin.top);
The API has changed since this was originally asked/answered...
It should now (as of 1.15) be:
return require("sdk/tabs/utils").getTabForWindow(domWin.top);
As of Addon SDK version 1.13 change:
var tabsLib = require("tabs/tab.js");
to
var tabsLib = require("sdk/tabs/helpers.js");
If anyone still cares about this:
Although the Addon SDK is being deprecated in support of the newer WebExtensions API, I want to point out that
var a_tab = require("sdk/tabs/utils").getTabForContentWindow(window)
returns a different 'tab' object than the one you would typically get by using
worker.tab in a PageMod.
For example, a_tab will not have the 'id' attribute, but would have linkedPanel property that's similar to the 'id' attribute.