Here I'm giving an example
class TestRealm {
constructor(realmDB) {
this.realmDB = realmDB;
this.allRealm = this.realmDB.objects('Person');
}
query1(primary_key) {
try {
const response = this.allRealm.filtered(`_id == "${primary_key}"`);
if (response.length === 0) {
console.log('unable to find');
return;
}
return response.toJSON()[0];
}
catch (e) {
console.log(e);
}
}
query2(primary_key) {
try {
const response = this.realmDB.objectForPrimaryKey('Person', `"${primary_key}"`);
if (!response) {
console.log('unable to find');
return;
}
return response.toJSON();
}
catch (e) {
console.log(e);
}
}
}
Assume after opening the realm, I'm passing the realm object to the TestRealm constructor. In this case which query is efficient for large data?
Let's start with getting the terminology lined up
This is a query (aka a filter) and will return a Realm Results object
const response = this.allRealm.filtered(`_id == "${primary_key}"`);
This is NOT a query and will return that specific object
const response = this.realmDB.objectForPrimaryKey('Person', `"${primary_key}"`);
So you're returning two different things - also response in the first example is a live updating Realm Results object so it has more overhead than returning the object itself.
Also note that for 'large data' the second option will be essentially unaffected by the dataset size (generally speaking)
The second example is much faster.
hey so I'm trying to const some JSON
const cidc = require('./cid/cid.json')
const dc = require('./details/cid.json')
const lc = require('./lik/cid.json')
if(!cidc){
return
}else{
fs.unlinkSync('./src/main/setting/cid/cid.json')
} if(!dc) {
return
}else {
fs.unlinkSync('./src/main/setting/details/cid.json')
} if (!lc){
return
}else {
fs.unlinkSync('./src/main/setting/lik/cid.json')
}
so I'm trying to delete the dc file and it error
how can I make if there is no such file named that it will do nothing (aka return nothing)
and if there is a file named that it will const it to a variable
Since require throws an error of Error: Cannot find module ... and you don't catch those errors, your script will fail.
You could define a new require-function where you catch the error and return undefined:
function safeRequire(path) {
try {
return require(path);
} catch(err) {
return undefined;
}
}
Then use this function in your script:
const cidc = safeRequire('./cid/cid.json')
const dc = safeRequire('./details/cid.json')
const lc = safeRequire('./lik/cid.json')
// rest of your code
Also you can simplify your if/else conditions by inverting the condition:
if (cidc) {
fs.unlinkSync('./src/main/setting/cid/cid.json')
}
if (dc) {
fs.unlinkSync('./src/main/setting/details/cid.json')
}
if (lc){
fs.unlinkSync('./src/main/setting/lik/cid.json')
}
Alternatively you don't even need to use require at all, just check if the files exist using e.g. fs.access(...).
You could directly use unlink with try catch without any requires
function unlink(filePath) {
try {
fs.unlinkSync(filePath);
} catch (e) {
//ignore
}
}
unlink('./src/main/setting/cid/cid.json')
unlink('./src/main/setting/details/cid.json')
unlink('./src/main/setting/lik/cid.json')
For reference: https://github.com/MarkPieszak/aspnetcore-angular2-universal/blob/master/Client/app/platform-modules/app.browser.module.ts#L51
Universal cache object is getting added globoally with initial state like this in the html that's sent to client:
<script>
window.UNIVERSAL_CACHE = { } // stuff gets loaded here
</script>
In my browser.module.ts I'm trying to load that initial state:
// imports
export const UNIVERSAL_KEY = 'UNIVERSAL_CACHE';
#ngModule({
// bootstrap, imports, providers, etc.
})
export class AppBrowserModule {
constructor(public cache: CacheService) {
this.doRehydrate();
}
// Universal Cache "hook"
doRehydrate() {
let defaultValue = {};
let serverCache = this._getCacheValue(CacheService.KEY, defaultValue);
this.cache.rehydrate(serverCache);
}
// Universal Cache "hook
_getCacheValue(key: string, defaultValue: any): any {
// Get cache that came from the server
const win: any = window;
/* I can console.log(win) to see the window object with .UNIVERSAL_CACHE, however if I console.log(win[UNIVERSAL_KEY]) it is undefined. */
if (win[UNIVERSAL_KEY] && win[UNIVERSAL_KEY][key]) {
let serverCache = defaultValue;
try {
serverCache = JSON.parse(win[UNIVERSAL_KEY][key]);
if (typeof serverCache !== typeof defaultValue) {
console.log('Angular Universal: The type of data from the server is different from the default value type');
serverCache = defaultValue;
}
} catch (e) {
console.log('Angular Universal: There was a problem parsing the server data during rehydrate');
serverCache = defaultValue;
}
return serverCache;
} else {
console.log('Angular Universal: UNIVERSAL_CACHE is missing');
}
return defaultValue;
}
}
Unfortunately, win[UNIVERSAL_KEY] is always undefined even though I can console.log(win) and see it, or in the dev tools I can type console.log(window.UNIVERSAL_CACHE) and see it. Any idea why that may be happening?
(I don't have enough reputation to just comment)
The problem come from this function which seems unfinished : https://github.com/angular/universal/blob/master/modules/platform-node/node-platform.ts#L418
When I want to make the message disappear in production, I remove manually the injectCacheInDocument function from my generate js file.
For example, I want to eliminate a specific variable name like $abc or '$abc', if it exist anywhere, we will throw a linting error. Its specifically for es6 code or just javascript code.
How can I do that in eslint? is it possble?
If its not what is the alternative I can do to check that without pollute my code base?
You can create your own eslint rule as mentioned in the comments. Here is a small example that reports all identifiers (excluding property names) with name foo:
export default function(context) {
return {
Identifier(node) {
if (
node.name === 'foo' &&
(
node.parent.type !== 'MemberExpression' ||
node.parent.computed ||
node.parent.object === node
)
) {
context.report(node, 'Do not use the variable name "foo"');
}
}
};
};
Live example: http://astexplorer.net/#/Lmzgbm2iRq
You could traverse the codebase and check all the files for the presence of the variable you wish to eliminate. It's perhaps a bit heavyweight for what you need but it might be an option.
Something like this should do the trick. You will have to replace $bad_variable_name with whatever your actual
variable is. You will also have to do something to make your build fail (if desired)
var fs = require('fs');
var checkDir = (dir) => {
var files = fs.readdirSync(dir);
files.forEach((file) => {
var path = dir + '/' + file;
var stat = fs.statSync(path);
if (stat && stat.isDirectory()) {
checkDir(path);
} else {
if(path.endsWith('this-file.js')){ //the file where this code is
return;
}
var fileContents = fs.readFileSync(path);
if(fileContents.indexOf('$bad_variable_name') > -1){
console.log('$bad_variable_name found in ' + path);
//do something here to fail your build
}
}
});
};
I have a Node.js application that, upon initialisation, reads two tables from an SQL database and reconstructs their relationship in memory. They're used for synchronously looking up data that changes (very) infrequently.
Problem: Sometimes I can't access the data, even though the application reports successfully loading it.
Code:
constants.js
module.exports = {
ready: function () { return false; }
};
var log = sysLog('core', 'constants')
, Geo = require('../models/geo.js');
var _ready = false
, _countries = []
, _carriers = [];
function reload() {
_ready = false;
var index = Object.create(null);
return Geo.Country.find().map(function (country) {
var obj = country.toPlainObject()
, id = obj.id;
delete obj.id;
index[id] = obj;
return Object.freeze(obj);
}).then(function (countries) {
log.debug('Loaded ' + countries.length + ' countries');
_countries = countries;
return Geo.Carrier.Descriptor.find().map(function (carrier) {
var obj = carrier.toPlainObject();
if (obj.country) {
obj.country = index[obj.country];
}
return Object.freeze(obj);
}).then(function (carriers) {
log.debug('Loaded ' + carriers.length + ' carriers');
_carriers = carriers;
});
}).finally(function () {
_ready = true;
});
}
reload().catch(function (err) {
log.crit({ message: 'Could not load constants', reason: err });
process.exit(-42);
}).done();
module.exports = {
reload : reload,
ready : function () { return _ready; },
countries : function () { return _countries; },
carriers : function () { return _carriers; }
};
utils.js
var log = sysLog('core', 'utils')
, constants = require('./constants');
module.exports = {
getCountryByISO: function(iso) {
if (!iso) {
return;
}
if ('string' != typeof iso) {
throw new Error('getCountryByISO requires a string');
}
if (!constants.ready()) {
throw new UnavailableError('Try again in a few seconds');
}
switch (iso.length) {
case 2:
return _.findWhere(constants.countries(), { 'iso2' : iso.toUpperCase() });
case 3:
return _.findWhere(constants.countries(), { 'iso3' : iso.toUpperCase() });
default:
throw new Error('getCountryByISO requires a 2 or 3 letter ISO code');
}
},
getCarrierByCode: function(code) {
if (!code) {
return;
}
if ('string' != typeof code) {
throw new Error('getCarrierByCode requires a string');
}
if (!constants.ready()) {
throw new UnavailableError('Try again in a few seconds');
}
return _.findWhere(constants.carriers(), { 'code' : code });
},
getCarrierByHandle: function(handle) {
if (!handle) {
return;
}
if ('string' != typeof handle) {
throw new Error('getCarrierByHandle requires a string');
}
if (!constants.ready()) {
throw new UnavailableError('Try again in a few seconds');
}
return _.findWhere(constants.carriers(), { 'handle' : handle });
}
};
Use case
if (data.handle) {
carrier = utils.getCarrierByHandle(data.handle);
if (_.isEmpty(carrier)) {
throw new InternalError('Unknown carrier', { handle: data.handle });
}
}
What's going on: All errors are logged; as soon as I see an error (i.e. "Unknown carrier") in the logs, I check the SQL database to see if it should've been recognised. That has always been the case so far, so I check the debug log to see if data was loaded. I always see "Loaded X countries" and "Loaded Y carriers" with correct values and no sign of "Could not load constants" or any other kind of trouble.
This happens around 10% of the time I start the application and the problem persists (i.e. didn't seem to go away after 12+ hours) and seems to occur regardless of input, leading me to think that the data isn't referenced correctly.
Questions:
Is there something wrong in constants.js or am I doing something very obviously wrong? I've tried setting it up for cyclical loading (even though I am not aware of that happening in this case).
Why can't I (sometimes) access my data?
What can I do to figure out what's wrong?
Is there any way I can work around this? Is there anything else I could to achieve the desired behaviour? Hard-coding the data in constants.js is excluded.
Additional information:
constants.reload() is never actually called from outside of constants.js.
constants.js is required only in utils.js.
utils.js is required in app.js (application entry); all files required before it do not require it.
SQL access is done through an in-house library built on top of knex.js and bluebird; so far it's been very stable.
Versions:
Node.js v0.10.33
underscore 1.7.0
bluebird 2.3.11
knex 0.6.22
}).finally(function () {
_ready = true;
});
Code in a finally will always get called, regardless of if an error was thrown up the promise chain. Additionally, your reload().catch(/* ... */) clause will never be reached, because finally swallows the error.
Geo.Country.find() or Geo.Carrier.Descriptor.find() could throw an error, and _ready would still be set to true, and the problem of your countries and carriers not being set would persist.
This problem would not have occurred if you had designed your system without a ready call, as I described in my previous post. Hopefully this informs you that the issue here is really beyond finally swallowing a catch. The real issue is relying on side-effects; the modification of free variables results in brittle systems, especially when asynchrony is involved. I highly recommend against it.
Try this
var log = sysLog('core', 'constants');
var Geo = require('../models/geo.js');
var index;
var _countries;
var _carriers;
function reload() {
index = Object.create(null);
_countries = Geo.Country.find().map(function (country) {
var obj = country.toPlainObject();
var id = obj.id;
delete obj.id;
index[id] = obj;
return Object.freeze(obj);
});
_carriers = _countries.then(function(countries) {
return Geo.Carrier.Descriptor.find().map(function (carrier) {
var obj = carrier.toPlainObject();
if (obj.country) {
obj.country = index[obj.country];
}
return Object.freeze(obj);
});
});
return _carriers;
}
reload().done();
module.exports = {
reload : reload,
countries : function () { return _countries; },
carriers : function () { return _carriers; }
};
constants.reload() is never actually called from outside of
constants.js.
That's your issue. constants.reload() reads from a database, which is an aysnchronous process. Node's require() is a synchronous process. At the time constants.js is required in utils.js and the module.exports value is returned, your database query is still running. And at whatever point in time that app.js reaches the point where it calls a method from the utils module, that query could still be running, resulting in the error.
You could say that requiring utils.js has the side-effect of requiring constants.js, which has the side-effect of executing a database query, which has the side-effect of concurrently modifying the free variables _countries and _carriers.
Initialize _countries and _carriers as unresolved promises. Have reload() resolve them. Make the utils.js api async.
promises.js:
// ...
var Promise = require('bluebird');
var countriesResolve
, carriersResolve;
var _ready = false
, _countries = new Promise(function (resolve) {
countriesResolve = resolve;
})
, _carriers = new Promise(function (resolve) {
carriersResolve = resolve;
});
function reload() {
_ready = false;
var index = Object.create(null);
return Geo.Country.find().map(function (country) {
// ...
}).then(function (countries) {
log.debug('Loaded ' + countries.length + ' countries');
countriesResolve(countries);
return Geo.Carrier.Descriptor.find().map(function (carrier) {
// ...
}).then(function (carriers) {
log.debug('Loaded ' + carriers.length + ' carriers');
carriersResolve(carriers);
});
}).finally(function () {
_ready = true;
});
}
reload().catch(function (err) {
log.crit({ message: 'Could not load constants', reason: err });
process.exit(-42);
}).done();
module.exports = {
reload : reload,
ready : function () { return _ready; },
countries : function () { return _countries; },
carriers : function () { return _carriers; }
};
utils.js
getCarrierByHandle: function(handle) {
// ...
return constants.carriers().then(function (carriers) {
return _.findWhere(carriers, { 'handle' : handle });
});
}
Use case:
utils.getCarrierByHandle(data.handle).then(function (carrier) {
if (_.isEmpty(carrier)) {
throw new InternalError('Unknown carrier', { handle: data.handle });
}
}).then(function () {
// ... next step in application logic
});
This design will also eliminate the need for a ready method.
Alternatively, you could call constants.reload() on initialization and hang all possibly-dependent operations until it completes. This approach would also obsolete the ready method.
What can I do to figure out what's wrong?
You could have analyzed your logs and observed that "Loaded X countries" and "Loaded Y carriers" were sometimes written after "Unknown carrier", helping you realize that the success of utils.getCarrierByHandle() was a race condition.