We Keep Coding

addPreSearch filter not applying

I am trying to use the addPreSearch function to add a custom filter to a lookup field, but the function does not seem to execute fully before the results of the lookup are displayed. The code for this looks something like this:
function onFieldChange(executionContext) {
var formContext = executionContext.getFormContext();
formContext.getControl("test_code").removePreSearch(testFunctionFilter);
formContext.getControl("test_code").addPreSearch(testFunctionFilter);
}
function testFunctionFilter(executionContext) {
var formContext = executionContext.getFormContext();
var record1 = formContext.getAttribute("test_record1_link").getValue(); //get linked record
var record1FullId, record1Id, stringRecordId, idLength, record1Guid = "0";
if (record1 != null) {
record1Id = record1[0].id;
record1Id = record1FullId.slice(1, -1);
stringRecordId = record1FullId.toString();
idLength = stringRecordId.length;
//Guid when retrieved from tablet does not have parenthesis on each end
if (idLength == 36) {
record1Guid = record1FullId;
} else {
record1Guid = recordId;
}
}
var fieldValue;
Xrm.WebApi.retrieveRecord("test_record1", record1Guid, "?$select=test_field1")
.then(function(result1) {
fieldValue = result1.test_field;
var options = generateOptions(executionContext, fieldValue); //creates option string using retrieved fieldValue
Xrm.WebApi.retrieveMultipleRecords("test_record2", options)
.then(function(result) {
var codes = getCodes(result2, fieldValue);
filter = generateFilter(codes, record1Guid); //creates custom filter using provided parameters
console.log(filter); //displays filter correctly
formContext.getControl("test_codelookup").addCustomFilter(filter, "test_coderecord"); //not working?
});
});
}
The filter is generated correctly using the functions used above whose definitions aren't shown. That isn't the issue. I've tried creating a separate test function where I hard coded one of the filters that the function above generated, and the lookup displayed the correct results. The testFunctionFilter should run to completion before the results of the lookup are displayed, correct? Because the filter is logged to the console after the results of the lookup appear. Are the nested asynchronous Xrm.WebApi calls somehow causing the issue? I'm not quite sure what is wrong. Please advise.

You are right. Xrm.WebApi calls are always Asynchronous, which is unusable in this case of adding dynamic filter using addCustomFilter.
You have to use XMLHttpRequest and make that call as Synchronous by setting third parameter as false like below:
var req = new XMLHttpRequest();
req.open("GET", Xrm.Utility.getGlobalContext().getClientUrl() +
"/api/data/v9.0/test_record1?$select=test_field1", false);

In order to work around the async delay, I think you're going to have to reorganise your code:
Add a form OnLoad event and execute the query to retrieve test_field1 and cache the results in a parameter
In the OnChange event, remove the presearch filter, re-execute the query to retrieve test_field1 and update the same parameter (from onload)
In testFunctionFilter use the cached results rather than building the presearch filter from scratch

node.js - mutual variable

I'm new to node.js, so before releasing my node.js app, I need to be sure it will work as it should.
Let's say I have an array variable and I intialize it on beginning of my script
myArray = [];
then I pull some data from an external API, store it inside myArray, and use setInterval() method to pull this data again each 30 minutes:
pullData();
setInterval(pullData, 30*60*1000);
pullData() function takes about 2-3 seconds to finish.
Clients will be able to get myArray using this function:
http.createServer(function(request, response){
var path = url.parse(request.url).pathname;
if(path=="/getdata"){
var string = JSON.stringify(myArray);
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end(string);
}
}).listen(8001);
So what I'm asking is, can next situation happen?:
An client tries to get data from this node.js server, and in that same moment, data is being written into myArray by pullData() function, resulting in invalid data being sent to client?
I read some documentation, and what I realized is that when pullData() is running, createServer() will not respond to clients until pullData() finishes its job?
I'm really not good at understanding concurrent programming, so I need your confirmation on this, or if you have some better solution?
EDIT: here is the code of my pullData() function:
var now = new Date();
Date.prototype.addDays = function(days){
var dat = new Date(this.valueOf());
dat.setDate(dat.getDate() + days);
return dat;
}
var endDateTime = now.addDays(noOfDays);
var formattedEnd = endDateTime.toISOString();
var url = "https://api.mindbodyonline.com/0_5/ClassService.asmx?wsdl";
soap.createClient(url, function (err, client) {
if (err) {
throw err;
}
client.setEndpoint('https://api.mindbodyonline.com/0_5/ClassService.asmx');
var params = {
"Request": {
"SourceCredentials": {
"SourceName": sourceName,
"Password": password,
"SiteIDs": {
"int": [siteIDs]
}
},
"EndDateTime" : formattedEnd
}
};
client.Class_x0020_Service.Class_x0020_ServiceSoap.GetClasses(params, function (errs, result) {
if (errs) {
console.log(errs);
} else {
var classes = result.GetClassesResult.Classes.Class;
myArray = [];
for (var i = 0; i < classes.length; i++) {
var name = classes[i].ClassDescription.Name;
var staff = classes[i].Staff.Name;
var locationName = classes[i].Location.Name;
var start = classes[i].StartDateTime.toISOString();
var end = classes[i].EndDateTime.toISOString();
var klasa = new Klasa(name,staff,locationName,start,end);
myArray.push(klasa);
}
myArray.sort(function(a,b){
var c = new Date(a.start);
var d = new Date(b.start);
return c-d;
});
string = JSON.stringify(myArray);
}
})
});

No, NodeJs is not multi-threaded and everything run on a single thread, this means except non-blocking calls (ie. IO) everything else will engage CPU until it returns, and NodeJS absolutely doesn't return half-way populated array to the end user, as long as you only do one HTTP call to populate your array.
Update:
As pointed out by #RyanWilcox any asynchronous (non-blocking syscall) call may hint NodeJS interpreter to leave your function execution half way and return to it later.

In general: No.
JavaScript is single threaded. While one function is running, no other function can be.
The exception is if you have delays between functions that access the value of an array.
e.g.
var index = i;
function getNext() {
async.get(myArray[i], function () {
i++;
if (i < myArray.length) {
getNext()
}
});
}
… in which case the array could be updated between the calls to the asynchronous function.
You can mitigate that by creating a deep copy of the array when you start the first async operation.

Javascript is single threaded language so you don't have to be worried about this kind of concurrency. That means no two parts of code are executed at the same time. Unlike many other programming languages, javascript has different concurrency model based on event loop. To achieve best performance, you should use non-blocking operations handled by callback functions, promises or events. I suppose that your external API provides some asynchronous i/o functions what is well suited for node.js.

If your pullData call doesn't take too long, another solution is to cache the data.
Fetch the data only when needed (so when the client accesses /getdata). If it is fetched you can cache the data with a timestamp. If the /getdata is called again, check if the cached data is older than 30 minutes, if so fetch again.
Also parsing the array to json..
var string = JSON.stringify(myArray);
..might be done outside the /getdata call, so this does not have to be done for each client visiting /getdata. Might make it slightly quicker.

Javascript memory consumption with map() over a large set and callbacks

I don't even know how to properly ask this question but I have concerns about the performance (mostly memory consumption) of the following code. I am anticipating that this code will consume a lot of memory because of map on a large set and a lot of 'hanging' functions that wait for external service. Are my concerns justified here? What would be a better approach?
var list = fs.readFileSync('./mailinglist.txt') // say 1.000.000 records
.split("\n")
.map( processEntry );
var processEntry = function _processEntry(i){
i = i.split('\t');
getEmailBody( function(emailBody, name){
var msg = {
"message" : emailBody,
"name" : i[0]
}
request(msg, function reqCb(err, result){
...
});
}); // getEmailBody
}
var getEmailBody = function _getEmailBody(obj, cb){
// read email template from file;
// v() returns the correct form for person's name with web-based service
v(obj.name, function(v){
cb(obj, v)
});
}

If you're worried about submitting a million http requests in a very short time span (which you probably should be), you'll have to set up a buffer of some kind.
one simple way to do it:
var lines = fs.readFileSync('./mailinglist.txt').split("\n");
var entryIdx = 0;
var done = false;
var processNextEntry = function () {
if (entryIdx < lines.length) {
processEntry(lines[entryIdx++]);
} else {
done = true;
}
};
var processEntry = function _processEntry(i){
i = i.split('\t');
getEmailBody( function(emailBody, name){
var msg = {
"message" : emailBody,
"name" : name
}
request(msg, function reqCb(err, result){
// ...
!done && processNextEntry();
});
}); // getEmailBody
}
// getEmailBody didn't change
// you set the ball rolling by calling processNextEntry n times,
// where n is a sensible number of http requests to have pending at once.
for (var i=0; i<10; i++) processNextEntry();
Edit: according to this blog post node has an internal queue system, it will only allow 5 simultaneous requests. But you can still use this method to avoid filling up that internal queue with a million items if you're worried about memory consumption.

Firstly I would advise against using readFileSync, and instead favour the async equivalent. Blocking on IO operations should be avoided as reading from a disk is very expensive, and whilst that's the sole purpose of your code now, I would consider how that might change in the future - and arbitrarily wasting clock cycles is never a good idea.
For large data files I would read them in in defined chunks and process them. If you can come up with some schema, either sentinels to distinguish data blocks within the file, or padding to boundaries, then process the file piece by piece.
This is just rough, untested off the top of my head, but something like:
var fs = require("fs");
function doMyCoolWork(startByteIndex, endByteIndex){
fs.open("path to your text file", 'r', function(status, fd) {
var chunkSize = endByteIndex - startByteIndex;
var buffer = new Buffer(chunkSize);
fs.read(fd, buffer, 0, chunkSize, 0, function(err, byteCount) {
var data = buffer.toString('utf-8', 0, byteCount);
// process your data here
if(stillWorkToDo){
//recurse
doMyCoolWork(endByteIndex, endByteIndex + 100);
}
});
});
}
Or look into one of the stream library functions for similar functionality.
H2H
ps. Javascript and Node works extremely well with async and eventing.. using sync is an antipattern in my opinion, and likely to cause code to be a headache in future

How to return data from CRM 2011 javascript odata function?

This seems silly that I haven't been able to accomplish this. I'm using the common code found on the web to do an odata query. The problem is the results stay in getFieldData(retrieveReq) routine. I don't want to immediately set a field on the current form. How can I get my values out of it so the data can be used in other javascript functions? Global variable would be good but nothing I've tried has worked. The below code displays "x".
var var1 = "x"; odataquery(); alert(var1);
The example given here has two alerts that display the data. How can Id and Name get outside of that function to be useful?
Edit1: Below is the main part of the routine that calls getFieldData(this). I want to use OwnerBUID and OwnerBUName in other javascript functions.
var retrieveReq = new XMLHttpRequest();
retrieveReq.open("GET", odataSelect, false);
retrieveReq.setRequestHeader("Accept", "application/json");
retrieveReq.setRequestHeader("Content-Type", "application/json; charset=utf-8");
retrieveReq.onreadystatechange = function () {
getFieldData(this);
};
retrieveReq.send();
function getFieldData(retrieveReq) {
if (retrieveReq.readyState == 4 && retrieveReq.status == 200) {
// 4=request complete, 200=OK
var retrieved = this.parent.JSON.parse(retrieveReq.responseText).d;
var retrievedValue = retrieved.results[0].BusinessUnitId;
OwnerBUID = retrievedValue.Id;
OwnerBUName = retrievedValue.Name;
}
}

I guess you want to put the data as the return value of a javascript function. You could do this:
var returnedData = function getFieldData(retrieveReq)
{
...
return data;
}
BTW, you could consider to use JayData, Breeze and Datajs sources code packages in your client application. They implement the low level APIs for consuming an odata service using javascript.

Dojo fetch, how to wait on two simultaneous async fetches?

Hey guys, im not well versed in dealing with asynchronous design patterns, and im having a problem writing a script that does two async data fetches.
Im using Dojo.data.api.Read.Fetch() to make two fetch() calls from seperate databases. The reulsts come back asynchronously. However, I have to cross reference the results, so i want my script to continue once BOTH async fetches are complete. I dont know how to do this, and therein lies the problem.
I am aware of the fetch's onComplete field and how to use it, BUT the best case solution i see there is to call the second fetch in the onComplete of the first fetch. I would like to do these fetches at the same time. Is there a way to do this?
Here's the current structure of my program for illustration purposes:
this.dict1.fetch({query:"blahblahblah", onComplete: function(items) { something here? }});
this.dict2.fetch({query:"blahblahbleh", onComplete: function(items) { or maybe something here? }});
this.orMaybeDoSomethingAfterBothFetches()
Any help would be greatly appreciated!

You could create dojo.Deferreds for each of the fetches and then use dojo.DeferredList and add the deferreds to it - see here. This solution allows you to take advantage of adding 'n' functions to the list of functions you want to call. It also takes advantage of all the dojo.Deferred's callback and errBack functionality.
var fetch1 = new dojo.Deferred();
fetch1.addCallback(this.dict1.fetch...);
var fetch2 = new dojo.Deferred();
fetch2.addCallback(this.dict2.fetch...);
var allFuncs = new dojo.DeferredList([fetch1, fetch2]);
var doStuffWhenAllFuncsReturn = function() {...};
allFuncs.addCallback(doStuffWhenAllFuncsReturn);

// this is a variation of a function I have answered quite a few similar questions on SO with
function collected(count, fn){
var loaded = 0;
var collectedItems = [];
return function(items){
collectedItems = collectedItems.concat(items);
if (++loaded === count){
fn(collectedItems);
}
}
}
var collectedFn = collected(2, function(items){
//do stuff
});
this.dict1.fetch({query:"blahblahblah", onComplete: collectedFn);
this.dict2.fetch({query:"blahblahbleh", onComplete: collectedFn);
An alternative solution is
var store = {
exec: function(){
if (this.items1 && this.items2) {
// do stuff with this.items1 and this.items2
}
}
};
this.dict1.fetch({query:"blahblahblah", onComplete: function(items) {
store.items1 = items;
store.exec();
});
this.dict2.fetch({query:"blahblahbleh", onComplete: function(items) {
store.items2 = items;
store.exec();
});