Code reference: https://github.com/aredfox/screencapturer
Problem description:
Here is an electron app with a "MainWindow" that holds a button "Start capture". Once clicked it fires an event to the main process, the main process then launches a new, seperate, "BrowserWindow" object called 'captureWindow' with it's own capture.html and capture.js associated. In capture.js, every three seconds a screenshot is made and saved to c:\temp\screencap (this is a demo app to illustrate a problem, thus I did not make this configurable and hard coded the path in for now). Every time a capture is made in the 'craptureWindow' it freezes, which I expected it to. But, the 'mainWindow' object freezes as well, which I did not expect it to do. How should I handle this, so thaty the mainWindow does not freeze when a process is being run in another "BrowserWindow" object? I assumed electron BrowserWindows (or "tabs") had a seperate thread?
EDIT 20/12/2016
Possible culprit is desktopCapturer.getSources().
ADDENDUM: Found that the issue must be inside the codeblock of getMainSource, because when I cache that "source" result it doesn't freeze the whole of electron. Thus it must be that the filter method or getting the screen itself is causing the issue of the freeze.
function getMainSource(desktopCapturer, screen, done) {
const options = {
types: ['screen'], thumbnailSize: screen.getPrimaryDisplay().workAreaSize
}
desktopCapturer.getSources(options, (err, sources) => {
if (err) return console.log('Cannot capture screen: ', err)
const isMainSource = source => source.name === 'Entire screen' || source.name === 'Screen 1'
done(sources.filter(isMainSource)[0])
})
}
The solution though is not caching the result of getMainSource (aka the "source"), as it will result in the same image data each time of course. I verified that by writing to file as png, and indeed each screenshot then was the exact same, even though enough had changed on the desktop. TODO: Possible option is to setup a video stream and save an image from the stream?
If you're wanting to capture screenshots cross platform, i'd advice using the approach below in stead of relying on the built in electron-api's. Not that they're not good, but they're not suited for taking screenshots every three seconds for example.
The solution for me was the npm-module desktop-screenshot - and a npm package called hazardous, as this was needed on Windows & asar execution.
The code I ended up implementing was this - it might be a source of inspiration/example for your problem.
/* ******************************************************************** */
/* MODULE IMPORTS */
import { remote, nativeImage } from 'electron';
import path from 'path';
import os from 'os';
import { exec } from 'child_process';
import moment from 'moment';
import screenshot from 'desktop-screenshot';
/* */
/*/********************************************************************///
/* ******************************************************************** */
/* CLASS */
export default class ScreenshotTaker {
constructor() {
this.name = "ScreenshotTaker";
}
start(cb) {
const fileName = `cap_${moment().format('YYYYMMDD_HHmmss')}.png`;
const destFolder = global.config.app('capture.screenshots');
const outputPath = path.join(destFolder, fileName);
const platform = os.platform();
if(platform === 'win32') {
this.performWindowsCapture(cb, outputPath);
}
if(platform === 'darwin') {
this.performMacOSCapture(cb, outputPath);
}
if(platform === 'linux') {
this.performLinuxCapture(cb, outputPath);
}
}
performLinuxCapture(cb, outputPath) {
// debian
exec(`import -window root "${outputPath}"`, (error, stdout, stderr) => {
if(error) {
cb(error, null, outputPath);
} else {
cb(null, stdout, outputPath);
}
});
}
performMacOSCapture(cb, outputPath) {
this.performWindowsCapture(cb, outputPath);
}
performWindowsCapture(cb, outputPath) {
require('hazardous');
screenshot(outputPath, (err, complete) => {
if(err) {
cb(err, null, outputPath);
} else {
cb(null, complete, outputPath);
}
});
}
}
/*/********************************************************************///
Related
I am using https://github.com/lxieyang/chrome-extension-boilerplate-react as the basis to build a chrome extension. It all works fine, and everything does hot-reloading (popup, background, options, newtab) except for the content-script. Reloading the matching pages, does not reload the underlying .js. It takes to reload/turn-off-on the whole extension in order for the changes to go into effect.
So, in webpack.config.js i commented out 'contentScript' hoping for it to fix that, but it makes no difference.
...
chromeExtensionBoilerplate: {
notHotReload: [
//'contentScript'
],
},
...
In src/pages/Content/index.js it actually states
console.log('Must reload extension for modifications to take effect.');
When developing another extension in plain vanilla js, i dropped a hot-reload.js from https://github.com/xpl/crx-hotreload which worked perfectly. From what i understand it is the 'chrome.runtime.reload()' call that makes chrome completely reload the extension.
So my question(s) actually is:
When changing src/pages/Content/index.js, webpack does re-build the build/contentScript.bundle.js. But why doesn't manually reloading the tab/page recognize these changes, when for popup, background, etc. it does?
And if there is no way to let the above boilerplate reload the extension (i don't mind the hard reload) how would i be able to integrate the hot-reload.js (or its effect actually) into this boilerplate? That is, how do i reload the extension when build/contentScript.bundle.js is changed?
Thanks in advance!
For who is interested. I ended up placing mentioned hot-reload.js in my extension, and loading it from within the background script. That breaks webpack's hot-reloading, by reloading the entire extension on any file-change. But as long as i only work on the content script, thats fine. I can remove it once im done, or if i work on other scripts.
Use server-sent-events:
start.js
const SSEStream = require('ssestream').default;
let sseStream;
...
setupMiddlewares: (middlewares, _devServer) => {
if (!_devServer) {
throw new Error('webpack-dev-server is not defined');
}
/** 改动:/reload path SSE */
middlewares.unshift({
name: 'handle_content_change',
// `path` is optional
path: '/reload',
middleware: (req, res) => {
console.log('sse reload');
sseStream = new SSEStream(req);
sseStream.pipe(res);
res.on('close', () => {
sseStream.unpipe(res);
});
},
});
return middlewares;
}
webpack.compiler.hook
let contentOrBackgroundIsChange = false;
compiler.hooks.watchRun.tap('WatchRun', (comp) => {
if (comp.modifiedFiles) {
const changedFiles = Array.from(comp.modifiedFiles, (file) => `\n ${file}`).join('');
console.log('FILES CHANGED:', changedFiles);
if(watchRunDir.some(p => changedFiles.includes(p))) {
contentOrBackgroundIsChange = true;
}
}
});
compiler.hooks.done.tap('contentOrBackgroundChangedDone', () => {
if(contentOrBackgroundIsChange) {
contentOrBackgroundIsChange = false;
console.log('--------- 发起 chrome reload 更新 ---------');
sseStream?.writeMessage(
{
event: 'content_changed_reload',
data: {
action: 'reload extension and refresh current page'
}
},
'utf-8',
(err) => {
sseStream?.unpipe();
if (err) {
console.error(err);
}
},
);
}
});
crx background
if(process.env.NODE_ENV === 'development') {
const eventSource = new EventSource(`http://${process.env.REACT_APP__HOST__}:${process.env.REACT_APP__PORT__}/reload/`);
console.log('--- start listen ---');
eventSource.addEventListener('content_changed_reload', async ({ data }) => {
const [tab] = await chrome.tabs.query({ active: true, lastFocusedWindow: true });
const tabId = tab.id || 0;
console.log(`tabId is ${tabId}`);
await chrome.tabs.sendMessage(tabId, { type: 'window.location.reload' });
console.log('chrome extension will reload', data);
chrome.runtime.reload();
});
}
I'm trying to make a web worker to prevent stalling the React main thread. The worker is supposed to read an image and do various things.
The app was created using create-react-app.
Currently I have
WebWorker.js
export default class WebWorker {
constructor(worker) {
const code = worker.toString();
const blob = new Blob(['('+code+')()'], {type: "text/javascript"});
return new Worker(URL.createObjectURL(blob), {type: 'module'});
}
}
readimage.worker.js
import Jimp from "jimp";
export default () => {
self.addEventListener('message', e => { // eslint-disable-line no-restricted-globals
if (!e) return;
console.log('Worker reading pixels for url', e.data);
let data = {};
Jimp.read(e.data).then(image => {
// jimp does stuff
console.log('Worker Finished processing image');
})
postMessage(data);
})
};
And then in my React component AppContent.js I have
import WebWorker from "./workers/WebWorker";
import readImageWorker from './workers/readimage.worker.js';
export default function AppContent() {
const readWorker = new ReadImageWorker(readImageWorker);
readWorker.addEventListener('message', event => {
console.log('returned data', event.data);
setState(data);
});
// callback that is executed onClick from a button component
const readImageContents = (url) => {
readWorker.postMessage(url);
console.log('finished reading pixels');
};
}
But when I run it, I get the error
Uncaught ReferenceError: jimp__WEBPACK_IMPORTED_MODULE_0___default is not defined
How can I properly import a module into a web worker?
EDIT:
As per suggestions from Kaiido, I have tried installing worker-loader, and edited my webpack.config.js to the following:
module.exports = {
module: {
rules: [
{
test: /\.worker\.js$/,
use: { loader: 'worker-loader' }
}
]
}
};
But when I run it, I still get the error
Uncaught ReferenceError: jimp__WEBPACK_IMPORTED_MODULE_0__ is not defined
I'm not too much into React, so I can't tell if the module-Worker is the best way to go (maybe worker-loader would be a better solution), but regarding the last error you got, it's because you didn't set the type of your Blob when you built it.
In this case, it does matter, because it will determine the Content-Type the browser sets when serving it to the APIs that fetch it.
Here Firefox is a bit more lenient and somehow allows it, but Chrome is picky and requires you set this type option to one of the many javascript MIME-types.
const script_content = `postMessage('running');`;
// this one will fail in Chrome
const blob1 = new Blob([script_content]); // no type option
const worker1 = new Worker(URL.createObjectURL(blob1), { type: 'module'});
worker1.onerror = (evt) => console.log( 'worker-1 failed' );
worker1.onmessage = (evt) => console.log( 'worker-1', evt.data );
// this one works in Chrome
const blob2 = new Blob([script_content], { type: "text/javascript" });
const worker2 = new Worker(URL.createObjectURL(blob2), { type: 'module'});
worker2.onerror = (evt) => console.log( 'worker-2 failed' );
worker2.onmessage = (evt) => console.log( 'worker-2', evt.data );
But now that this error is fixed, you'll face an other error, because the format import lib from "libraryname" is still not supported in browsers, so you'd have to change "libraryname" to the path to your actual script file, keeping in mind that it will be relative to your Worker's base URI, i.e probably your main-page's origin.
I experienced the same problem. Firefox could not show me where exactly the error was (in fact it was plain misleading...) but Chrome did.
I fixed my problem by not relying on an import statement (importing one of my other files) which would only have worked within a React context. When you load a Worker script (via the blob()
/ URL() hack), it has no React context (as it is loaded at runtime and not at transpile time). So all the React paraphernalia __WEBPACK__blah_blah is not going to exist / be visible.
So... within react... import statements in worker files will not work.
I haven't thought of a workaround yet.
I am new to testing using protractor so for testing I have to take screenshots in an angular application for all the different routes in my app. I tried to do it on a small dummy angular app, so I cloned the Tour of heroes repo it has dashboard and Heroes route. I wrote the following code in app.po.ts :
import { browser, element, by } from 'protractor';
export class BlankPage {
navigateTo() {
return browser.get('/heroes');
}
getParagraphText() {
return element(by.tagName('h2')).getText();
}
}
and in app.e2e-spec.ts
import { BlankPage } from './app.po';
import {browser,by,element} from 'protractor';
import { protractor } from 'protractor';
import {createWriteStream} from 'fs' ;
describe('blank App', () => {
let page: BlankPage;
beforeEach(() => {
page = new BlankPage();
});
it('should display message saying app works', () => {
page.navigateTo();
expect(page.getParagraphText()).toEqual('My Heroes');
browser.takeScreenshot().then((png) =>{
var stream = createWriteStream("heroes.png"); /** change the png file name */
stream.write(new Buffer(png, 'base64'));
stream.end;
});
});
});
The idea was to navigate to heroes route and capture the screenshot. I got the screenshot but
Is there a way I can automate the task of going to all the routes and take screenshots ? In my actual website there are a lot of routes
I think the better solution for you is to add some reporter that will do everything for you, like taking screenshots after each test or after each failed tests and e.t.c.
Take a look at some reporters:
allure-jasmine - Highly recommended.
protractor-jasmine2-screenshot-reporter
protractor-jasmine2-html-reporter
protractor-html-reporter-2
protractor-html-screenshot-reporter
protractor-beautiful-reporter
But If you don't want to add any extra libraries to your project you can just put the browser.takeScreenshot() function to the afterEach function to take a screenshot after each test (it).
For instance:
describe('blank App', () => {
let page: BlankPage;
beforeEach(() => {
page = new BlankPage();
});
afterEach(() =>
browser.takeScreenshot().then((png) =>{
var stream = createWriteStream("heroes.png"); /** change the png file name */
stream.write(new Buffer(png, 'base64'));
stream.end;
});
});
it('should display message saying app works', () => {
page.navigateTo();
expect(page.getParagraphText()).toEqual('My Heroes');
});
});
I think the best approach for you would be the have a list of all the routes in your application and create a datadriven test to iterate over each one.
You would need a generic navigation function which could get to each page e.g navigateTo(routeName). That code would look something like this.
var routes = [
'homepage',
'myheroes',
'mainpage',
'heroprofile'
]
describe('blank App', () => {
for (let i = 0; i < routes.length; i++) {
it('should display message saying app works', () => {
navigateTo(routes[i]);
browser.takeScreenshot().then((png) => {
var stream = createWriteStream(routes[i] + ".png"); /** change the png file name */
stream.write(new Buffer(png, 'base64'));
stream.end;
});
});
}
});
protractor-image-compare
Really though I would recommend you use the npm package protractor-image-comparison. I've worked with this package and it does make visual validation very straightforward. It allows you to save new baseline images (golden images as you call them) if they are absent and compares them if they are present. The comparison are very sensitive to change but you can set how much of a difference you want to allow.
There would be no database required with this approach.
Note
Be aware also that different browsers take screenshots differently based. Chrome considers the "viewport" to be the visible portion of the browser but I believe in firefox you can screenshot the entire webpage at once.
Problem
I'd like to be able to track a users location even when the app is no longer in the foreground (e.g. The user has switch to another app or switched to the home screen and locked their phone).
The use case would be a user tracking a run. They could open the app and press 'start' at the beginning of their run, then switch or minimise the app (press the home button) and lock the screen. At the end of the run they could bring the app into the foreground and press 'stop' and the app would tell them distance travelled on the run.
Question
Is tracking background geolocation possible on both iOS and Android using pure react native?
The react native docs on geolocation (https://facebook.github.io/react-native/docs/geolocation) are not very clear or detailed. The documented linked above eludes to background geolocation on iOS (without being fully clear) but does not mention Android.
Would it be best that I use Expo?
UPDATE 2019 EXPO 33.0.0:
Expo first deprecated it for their SDK 32.0.0 to meet app store guidelines but then reopened it in SDK 33.0.0.
Since, they have made it super easy to be able to implement background location. Use this code snippet that I used to make background geolocation work.
import React from 'react';
import { Text, TouchableOpacity } from 'react-native';
import * as TaskManager from 'expo-task-manager';
import * as Location from 'expo-location';
const LOCATION_TASK_NAME = 'background-location-task';
export default class Component extends React.Component {
onPress = async () => {
await Location.startLocationUpdatesAsync(LOCATION_TASK_NAME, {
accuracy: Location.Accuracy.Balanced,
timeInterval: 5000,
});
};
render() {
return (
<TouchableOpacity onPress={this.onPress} style={{marginTop: 100}}>
<Text>Enable background location</Text>
</TouchableOpacity>
);
}
}
TaskManager.defineTask(LOCATION_TASK_NAME, ({ data, error }) => {
if (error) {
alert(error)
// Error occurred - check `error.message` for more details.
return;
}
if (data) {
const { locations } = data;
alert(JSON.stringify(locations); //will show you the location object
//lat is locations[0].coords.latitude & long is locations[0].coords.longitude
// do something with the locations captured in the background, possibly post to your server with axios or fetch API
}
});
The code works like a charm. One thing to note is that you cannot use geolocation in the Expo App. However, you can use it in your standalone build. Consequently, if you want to use background geolocation you have to use this code and then do expo build:ios and upload to the appstore in order to be able to get a users background location.
Additionally, note that you must include
"UIBackgroundModes":[
"location",
"fetch"
]
In the info.plist section of your app.json file.
The Expo Team release a new feature in SDK 32 that allow you tracking in background the location.
https://expo.canny.io/feature-requests/p/background-location-tracking
Yes is possible, but not using Expo, there are two modules that I've seen:
This is a comercial one, you have to buy a license https://github.com/transistorsoft/react-native-background-geolocation
And this https://github.com/mauron85/react-native-background-geolocation
Webkit is currently evaluating a Javascript-only solution. You can add your voice here
For a fully documented proof-of-concept example please see Brotkrumen.
The most popular RN geolocation library is https://github.com/react-native-geolocation/react-native-geolocation, and it supports this quite easily. I prefer this library over others because it automatically handles asking for permissions and such, and seems to have the simplest API.
Just do this:
Geolocation.watchPosition((position)=>{
const {latitude, longitude} = position.coords;
// Do something.
})
This requires no additional setup other than including the background modes fetch and location, and also the appropriate usage descriptions.
I find this more usable than Expo's API because it doesn't require any weird top level code and also doesn't require me to do anything other than create a watch position handler, which is really nice.
EDIT 2023!:
These days I would highly recommend using Expo's library instead of any of the other community libraries (mainly because our app started crashing when android got an OS update b/c of the lib I was using).
In fact, if you have to choose between expo and non expo library, always choose the expo library if only for the stability. Setting up expo's background location watching isn't super well documented but here's what I did to get it working in our app:
import { useEffect, useRef } from "react";
import * as Location from "expo-location";
import { LatLng } from "react-native-maps";
import * as TaskManager from "expo-task-manager";
import { LocationObject } from "expo-location";
import { v4 } from "uuid";
type Callback = (coords: LatLng) => void;
const BACKGROUND_TASK_NAME = "background";
const executor: (body: TaskManager.TaskManagerTaskBody<object>) => void = (
body
) => {
const data = body.data as unknown as { locations: LocationObject[] };
const l = data?.locations[0];
if (!l) return;
for (const callback of Object.values(locationCallbacks)) {
callback({
latitude: l.coords.latitude,
longitude: l.coords.longitude,
});
}
};
TaskManager.defineTask(BACKGROUND_TASK_NAME, executor);
const locationCallbacks: { [key: string]: Callback } = {};
const hasStartedBackgroundTaskRef = {
hasStarted: false,
};
function startBackgroundTaskIfNecessary() {
if (hasStartedBackgroundTaskRef.hasStarted) return;
Location.startLocationUpdatesAsync(BACKGROUND_TASK_NAME, {
accuracy: Location.Accuracy.Balanced,
}).catch((e) => {
hasStartedBackgroundTaskRef.hasStarted = false;
});
hasStartedBackgroundTaskRef.hasStarted = true;
}
function addLocationCallback(callback: Callback) {
const id = v4() as string;
locationCallbacks[id] = callback;
return {
remove: () => {
delete locationCallbacks[id];
},
};
}
export default function useLocationChangeListener(
callback: Callback | null,
active: boolean = true
) {
const callbackRef = useRef<null | Callback>(callback);
callbackRef.current = callback;
useEffect(() => {
if (!active) return;
if (!callback) return;
Location.getLastKnownPositionAsync().then((l) => {
if (l)
callback({
latitude: l.coords.latitude,
longitude: l.coords.longitude,
});
});
startBackgroundTaskIfNecessary();
const watch = Location.watchPositionAsync({}, (location) => {
callback({
latitude: location.coords.latitude,
longitude: location.coords.longitude,
});
});
const subscription = addLocationCallback(callback);
return () => {
subscription.remove();
watch.then((e) => {
e.remove();
});
};
}, [callback, active]);
useEffect(() => {
if (__DEV__) {
addLocationCallback((coords) => {
console.log("Location changed to ");
console.log(coords);
});
}
}, []);
}
You need to ask for background location permissions before this, BTW. Follow expos guide.
It's pretty risky trusting community libraries for stuff like this because of the fact that breaking android OS updates can happen at any moment and with open source maintainers they may or may not stay on top of it (you can more or less trust expo too, though)
I am trying to save the base64 string to the gallery. When I invoke this plugin my code get's crashed. Here is the link I used to check.
code I use
let options:Base64ToGalleryOptions = { prefix: '_img',mediaScanner: true }
//after the below line my gets close automatically any idea
this.base64ToGallery.base64ToGallery(base64Image[1],options)
.then(
res => {
debugger
console.log('Saved image to gallery ', res)
},
err => {
debugger
console.log('Error saving image to gallery ', err)
});
I am not able to debug
I am not able to understand why my app closes automatically after hitting this code
Update:
After installing this particular version of the plugin
ionic cordova plugin add cordova-base64-to-gallery#2.0.2
and moving my code to platform
this.platform.ready().then(() => {
this.base64ToGallery.base64ToGallery(base64Image,options)
.then(
res => {
console.log('Saved image to gallery ', res);
this.navCtrl.pop();
},
err => { //For ios i am getting as `plugin_not_installed`
console.log('Error saving image to gallery ', err);
this.navCtrl.pop()
});
})
But this same code is not working for ios according to the doc i have installed the same version which supports ios also (2.0.2) but it looks something is missing if any please let me know
Since you are unable to debug here are three problems I ran across until I got it to work, most likely the second problem if on Android or the third problem if on iOS.
1) Error saving image to gallery cordova_not_available
Fix for this was to create a project that had cordova baked in with the command ionic start blank --cordova
2) Error saving image to gallary Error while saving image I got this error message on an Android device. I looked at their code implementation here https://github.com/Nexxa/cordova-base64-to-gallery/blob/2f531aaa0bf17b900cf6bd9704082e72f183d325/src/android/Base64ToGallery.java
Saw that they have not done anything regarding WRITE_EXTERNAL_STORAGE permissions.
My solution was to add AndroidPermissions and check for WRITE_EXTERNAL_STORAGE permissions at runtime.
hasWriteAccess: boolean = false;
constructor(private base64ToGallery: Base64ToGallery,
private androidPermissions: AndroidPermissions) {
}
ionViewWillEnter() {
this.checkPermissions();
}
checkPermissions() {
this.androidPermissions
.checkPermission(this.androidPermissions
.PERMISSION.WRITE_EXTERNAL_STORAGE)
.then((result) => {
console.log('Has permission?',result.hasPermission);
this.hasWriteAccess = result.hasPermission;
},(err) => {
this.androidPermissions
.requestPermission(this.androidPermissions
.PERMISSION.WRITE_EXTERNAL_STORAGE);
});
if (!this.hasWriteAccess) {
this.androidPermissions
.requestPermissions([this.androidPermissions
.PERMISSION.WRITE_EXTERNAL_STORAGE]);
}
}
saveImage() {
if (!this.hasWriteAccess) {
this.checkPermissions();
}
let options: Base64ToGalleryOptions = {
prefix: '_img',
mediaScanner: true
};
this.base64ToGallery
.base64ToGallery(this.base64Data, options).then(
res => console.log('Saved image to gallery:', res),
err => console.log('Error saving image to gallery:', err)
);
}
3) This app has crashed because it attempted to access privacy-sensitive data without a usage description. The app's Info.plist must contain an NSPhotoLibraryAddUsageDescription key with a string value explaining to the user how the app uses this data.
Solution is to add NSPhotoLibraryAddUsageDescription to project_name/config.xml nested between <platform name="ios"> and </platform>
<config-file parent="NSPhotoLibraryAddUsageDescription" target="*-Info.plist">
<string>Saves images from base64 to your Photo Library</string>
</config-file>