I am currently working on a "crowdsourced" average value calculator. The idea is to show a picture to people and ask them to guess the age of the person. Once they entered the value, I want to show them the average age the person was given.
Here is what I want to do exactly :
Put a form online and ask people to put on a value
Store the data entered
Return the mean value people put there
Calculate the standard deviation so that people cannot put a value too high or too low compared to what others put. That means the average value shown will be more accurate this way.
I am looking for the fastest way to do it, and here is what I thought about :
Store the data in an SQL table and return the mean value through the AVG() function..but then, how would I calculate the Std. Dev ?
Store the data in a txt file and use javascript to convert it to an array do the calculations.
But if I get like 20,000 different values, it might be slow to do either way. ?
I am quite a beginner in programming and what I propose might seem ridiculous...feel free to tell it to me !
Thank you all.
SQL Server has STDEV (from 2005 onwards) so SQL sounds good for you.
Returns the statistical standard deviation of all values in the
specified expression. May be followed by the OVER clause.
Syntax
STDEV ( [ ALL | DISTINCT ] expression )
Arguments
ALL
Applies the function to all values. ALL is the default.
DISTINCT
Specifies that each unique value is considered.
expression
Is a numeric expression. Aggregate functions and subqueries are not permitted. expression is an expression of the exact numeric or approximate numeric data type category, except for the bit data type.
Related
We have a data model where each entity has 600 boolean values. All of this data needs to travel over the wire from a node.js backend to an Angular frontend, via JSON.
I was thinking about various ways to optimize it (this is an internal API and is not public, so adherence to best practices is less important than performance and saving bandwidth).
I am not a native Javascript speaker, so was hoping to get some feedback on some of the options I was considering, which are:
Turning it into a bitfield and using a huge (600-bit) BigInt.
Is this a feasible approach? I can imagine it would probably be pretty horrific in terms of performance
Splitting the 600 bits into 10 integers (since JS integers are 64 bit), and putting those into an array in the JSON
Base64 encoding a binary blob (will be decoded to a UInt8Array I'm assuming?)
Using something like Protobuf? It might be overkill because I don't want more than 1-2 hours spent on this optimization; definitely don't want to make major changes to the architecture either
Side note: We don't have compression on the server end due to infrastructure reasons, which makes this more complicated and is the reason for us implementing this on the data level.
Thanks!
as Evan points out, transforming your boolean for example into a single character for true="t" and false="f", the 600 boolean will become a joined string of 600 chars which can very well be split into 3 strings of 200 of the sizes, then once received on the front just concatenate the transit and if you want to recover your Bollean values from the string, with a simple reg it becomes possible.
I don't know how the data is set and then obtained, just changing this parameter to which I think needs to be automated.
Once the final string is obtained on the front here is an example of reg ex which can convert your string to an array with your 600 boolean. It is also possible to define indexes by defining an object instead of the array.
function convert_myBool(str)
{
/*var reg = new RegExp('.{1}', 'g');
var tmpTab = str.replace(reg, function(matched){
return matched == "t"?true:false;
});*/
//map is best
tmpTab = str.split('').map((value) =>{
return value == "t"?true:false;
});
return tmpTab;
};
I wrote this dynamically so of course it can be pondered, improved replaced etc. Hoping to have helped :)
Can it be sorted in any way? If there are boolean values that always occur in conjunction with a related value you may be able to group them and simplify.
Depending on what your use for that data is, you may be able to cache some of the it or memoize based on usage frequency. There would be a space tradeoff with caching, however.
I have a project, where user can put in drop-down values that can be selected. One can select multiple values at a time. So, we have to store the selection and get it on edit mode.
First thought
Let's store them as comma separated in DB.
f.e.
If suggestions are A , B , C and user selects A and B, I was going to store A,B in DB and while getting back the value split it with comma.
Problem arises when user has genuine "comma" in the field, for an instance first,option & second,option. At that time joining with comma won't work.
Second thought
I can think of another option to store it in a stringified array format and parse it while getting back.
For the above instance, it would store the data as ["first,option","second,option"]. It seems to be a good (and only) option for me.
Even though I have a bit of hesitation doing so (which lead me questioning here!) because my users can access the api/DB value directly and for them it doesn't look good.
So, Is there any other way to address this issue to benefit both parties, developers and users? Thanks in advance!!
I'd suggest using a standardized format such as JSON, XML etc.
Serialize and parse and with a widely used library so all escaping of reserved / special characters is done for you. Rolling your own here will cause you problems!
Better yet, use different fields for each suggestion, this is a better design in general. As long as the number of potential fields is finite this will work, e.g. 1-10 suggestions.
If you're going down the JSON route, we can do this in JavaScript like this:
let suggestions = ['Choice A, commas are not, a problem, though punctuation is.', 'Choice B', 'Choice C'];
let json = JSON.stringify(suggestions);
// Save to DB
saveToDB(json);
let jsonFromDB = loadFromDB();
let deserializedSuggestions = JSON.parse(jsonFromDB);
console.log(deserializedSuggestions);
we use semicolon (;) for this exact use case in our current project.
So, as per your question, they will be stored in the DB as option1;option2;option3
and when we get it back from the DB we can use the split() method on it to convert it into an array of substrings.
var str = "option1;option2;option3";
var res = str.split(";");
console.log(res);
which would result in (3) ["option1", "option2", "option3"] in the console.
hope this helps.
Updated version.
I had figured out my previous issue with it but am still unsure how to do the last part. I do not know how to get the number that goes inside of the bracket instead of getting the value that is stored inside of it. The array consists of 12 numbers and for example I am looking for the highest value which might be 20 and was entered in the 4th spot. I can only figure out how to get it to display the 20 but I want to display the 4 instead. It has user input so the number that should be displayed may vary and I am unsure how to accomplish this.
I only need the arrays number to be displayed for the highest and lowest values. I don't know if I am using the wrong kind of array for this not.
So you have a couple issues but easy fixes. This only goes to fix image 3 which is highestValue.
1.) When you are setting highestValue you are saying highest value is equal to [January, amountOfRain]. You need to have it say highestValue <- data[1,2]. This will point it to amountOfRain.
2.) You need to change the name of either your Call or your variable highestValue because it's conflicting.
Give those a try and let me know what you get.
I have used:
SpreadsheetApp.getActiveSheet().getRange(... some Range ...).getValues();
to store strings from a spreadsheet's cells into an array. The strings are then compared with other strings, but the comparison (using ==) always fails even though the values are the same.
Browser.msgBox("is '"+topPlayerNames[j]+"' == '"+name+"'? "+(topPlayerNames[j] == name));
// displays:
is 'Data' == 'Data'? false
Why is the javascript comparison failing? Is there hidden formatting in cell values disrupting the comparison?
Since the getRange() will give you a two dimensional array, By interpreting them something like
var values = SpreadsheetApp.getActiveSheet().getRange(... some Range ...).getValues();
Browser.msgBox(values);
is just output the data in your expected way and exactly not matching with each other. The reason for not matching is your variable container keeps
values = [['a','b',...],['d','f',....]]. So it will output the data when you ask to out put the data. But when comparing,
Does [['a','b',...]] == 'a','b',... ?? The answer is it is not.
So you need to get out your data from the array to compare with others. In that case use values[0][0] to get 'a' (in my example) or to get 'd', values[1][0] .... and so on. You know element numbers are begin with 0 in javascript array. :)
Now you can compare them. So values[0][0] == 'a' will output TRUE (in my example)
However You didn't mentioned what is topPlayerNames[j], If it is also like values in above, then you need to consider it too in the explained way.
(Your question has limited variables. So I was need to write more when explaining. Next time please give some explained question with respective variable. Use short variables for long statements. That will help to answer in an easy way.)
See What is the correct way to check for string equality in JavaScript?
I would use
topPlayerNames[j].equals(name)
I never trust == with strings, as it so often causes a bug.
I need to implement a simple way to handle localization about weekdays' names, and I came up with the following structure:
var weekdaysLegend=new Array(
{'it-it':'Lunedì', 'en-us':'Monday'},
{'it-it':'Martedì', 'en-us':'Tuesday'},
{'it-it':'Mercoledì', 'en-us':'Wednesday'},
{'it-it':'Giovedì', 'en-us':'Thursday'},
{'it-it':'Venerdì', 'en-us':'Friday'},
{'it-it':'Sabato', 'en-us':'Saturday'},
{'it-it':'Domenica', 'en-us':'Sunday'}
);
I know I could implement something like an associative array (given the fact that I know that javascript does not provide associative arrays but objects with similar structure), but i need to iterate through the array using numeric indexes instead of labels.
So, I would like to handle this in a for cycle with particular values (like j-1 or indexes like that).
Is my structure correct? Provided a variable "lang" as one of the value between "it-it" or "en-us", I tried to print weekdaysLegend[j-1][lang] (or weekdaysLegend[j-1].lang, I think I tried everything!) but the results is [object Object]. Obviously I'm missing something..
Any idea?
The structure looks fine. You should be able to access values by:
weekdaysLegend[0]["en-us"]; // returns Monday
Of course this will also work for values in variables such as:
weekdaysLegend[i][lang];
for (var i = 0; i < weekdaysLegend.length; i++) {
alert(weekdaysLegend[i]["en-us"]);
}
This will alert the days of the week.
Sounds like you're doing everything correctly and the structure works for me as well.
Just a small note (I see the answer is already marked) as I am currently designing on a large application where I want to put locals into a javascript array.
Assumption: 1000 words x4 languages generates 'xx-xx' + the word itself...
Thats 1000 rows pr. language + the same 7 chars used for language alone = wasted bandwitdh...
the client/browser will have to PARSE THEM ALL before it can do any lookup in the arrays at all.
here is my approach:
Why not generate the javascript for one language at a time, if the user selects another language, just respond(send) the right javascript to the browser to include?
Either store a separate javascript with large array for each language OR use the language as parametre to the server-side script aka:
If the language file changes a lot or you need to minimize it per user/module, then its quite archivable with this approach as you can just add an extra parametre for "which part/module" to generate or a timestamp so the cache of the javascript file will work until changes occures.
if the dynamic approach is too performance heavy for the webserver, then publish/generate the files everytime there is a change/added a new locale - all you'll need is the "language linker" check in the top of the page, to check which language file to server the browser.
Conclusion
This approach will remove the overhead of a LOT of repeating "language" ID's if the locales list grows large.
You have to access an index from the array, and then a value by specifying a key from the object.
This works just fine for me: http://jsfiddle.net/98Sda/.
var day = 2;
var lang = 'en-us';
var weekdaysLegend = [
{'it-it':'Lunedì', 'en-us':'Monday'},
{'it-it':'Martedì', 'en-us':'Tuesday'},
{'it-it':'Mercoledì', 'en-us':'Wednesday'},
{'it-it':'Giovedì', 'en-us':'Thursday'},
{'it-it':'Venerdì', 'en-us':'Friday'},
{'it-it':'Sabato', 'en-us':'Saturday'},
{'it-it':'Domenica', 'en-us':'Sunday'}
];
alert(weekdaysLegend[day][lang]);