We Keep Coding

How to use fs and JSONStream to write a huge file without trigger RangeError: Maximum call stack size exceeded

What I did here is trying to replace JSON.stringify() with something else, to avoid RangeError. And I found the method below.
This is my code: (I use big-json module and this module implement json-stream-stringify for stringify)
const data = generateData();
const stringifyStream = json.createStringifyStream({
body: data
});
stringifyStream.on('data', function(str) {
console.log(str.toString());
})
I tried to print the str, it will print the string type data line by line, my question is: how to write the complete str to a file? I tried to add fs.writefileSync() after the console.log() and it will only write in the last line of str. Sorry I am new to stream and node.js. Hvae no idea how to fix it.
The other question is, can I find a way to re-format the output string just like JSON.stringify(data, null, 2) did?
FYI: The data is a really deep object. No circular, but may have repeated part. And I prefer to keep everything unchanged and not pruned.
Really appreciate any help here! Thank you for your time.

Have you tried separating JSON.stringify from fs.writeFileSync ? just to know wich one is actually having problems.
If it is JSON.stringify I suggest you to fragment the object in multiple ones or filter what you use. Whatever you can do to reduce the actual data. We don't know what de data really is so I cannot make assumptions on that but you never have those problems generally, if it's caused by the data being too big, you are doing things wrong!
If the problem is fs.writeFileSync and you have your JSON stringified then try using fs.createWriteStream to write the data.

How to read Python list in Javascript [in a Django template]

I'm programming in oTree (which is a Django based environment for social experiments) and I have the following problem. I defined some lists in Python and I'd like to import them and use them in an HTML template. If I print them in HTML I manage to see them without any problem, however, once I need to use them in Javascript, the program fails to read them and the single quotes of the elements of the list are converted in '.
The list is imported like this var filtered_elements = {{ array }};.
I think the problem is exactly here, as JS cannot work with them. Do you have any suggestion on how to do that? I considered using JSON, but since I'm quite new to programming, I cannot understand if it's just a waste of time or there is a simpler way out.
Thanks for your answers!

It sounds like your data is already JSON, otherwise you would be getting single quotes and u prefixes. So the only issue is Django autoescaping; you can disable it with the safe filter:
var filtered_elements = {{ array|safe }};

Your data should be JSON, instead of putting the Python list into the contact directly, put "array": json.dumps(array) in the context dictionary.
The JSON string doesn't need HTML escaping inside a tag, but it does need JS escaping! Otherwise some string may include something like </script><script>absolutely anything goes here... to run arbitrary JavaScript, if the JSON contains user data.
So use |escapejs:
var filtered_elements = {{ array|escapejs}};

How can I remove dynamic data in string before a value in that string? / using Lodash's _.trimStart with dynamic data?

I'm working urls returned from a server that I have no control over where and sometimes the urls return with extra data at the front.
For instance
sometimes it returns this
https://example.com/image/5119b3905.jpg
and this I can use, but sometimes it will return something like this
https://d1yww.cloudfront.net/9MA=/670x670/example.com/image/5119b3905.jpg
where I'd like to use remove everything before the example.com and to do that I could use something like lodash's _.trimStart method something like
_.trimStart('https://d1yww.cloudfront.net/9MA=/670x670/example.com/image/5119b3905.jpg',
'd1yww.cloudfront.net/9MA=/670x670');
but the d1yww.cloudfront.net/9MA=/670x670' is never static for me to do this and I don't know how to grab the dynamic data to use _.trimStart and I don't see any other useful lodash's methods and I don't know of any vanilla javascript ones.
TLDR: How can I remove dynamic data in string before a value in that string (in this example everything before the example.com)

You don't need lodash to do that
var str = 'https://d1yww.cloudfront.net/9MA=/670x670/example.com/image/5119b3905.jpg'
str.substr(str.indexOf('example.com'))

You could search for a Regular Expression
For Example:
/\/([-a-zA-Z0-9#:%._\+~#=]{2,256}\.[a-z]{2,6}\b)\//g
and look for the second match

How can I get Javascript to execute as a table is being built?

I have a table being built in HTML (using ASP), and it's stepping through a recordset. As it steps through the recordset, it creates a new row for the html table and fills it with data.
The problem I'm having is that it's using numbers that can be 10 or 11 digits long, and I want to format it with commas. I have a formatNumbers function that works excellently. However, basically what I need to do is this:
<td><script>formatNumber(<% = RS("total_rolled_lineal_ft")%>,0,0,true);</script></td>
I'm getting an Object Expected error. If we take a line from the executed HTML, here's what it looks like:
<td><script>formatNumber(10843537,0,0,true);</script></td>
Any clue what's causing my error, or, if I'm doing it completely wrong, how to fix it?
Also, formatNumber returns a string, in this case 10,843,537.

Thanks to #nnnnnn, I ended up using VB's FormatNumber() and came up with this
<% = FormatNumber(RS("total_rolled_lineal_ft"),0,true,true,true)%>, which works excellently.

I have never used straight ASP so maybe I am missing something in this answer.
Technically you can not execute Javascript while the ui is rendering, browsers tend to be a single threaded affair and will do one thing or the other.
But I would suggest that instead of binding the table directly to a record set you transform the record set into a ViewModel type class in the code behind.
You would then perform this conversion as you are building your ViewModel.

Django : View returns JSON content_dictionary, how to decode in Javascript

Let me explain what I'm trying to do, and if someone could point the correct way to do it & a solution to where I'm stuck that would be great !
Someone types url
www.ABC.com/showItem/Blackberry
I lookup "Blackberry" in my database and find data for it, now I want to show its details one a page.
Hence in the View I do this
return_data=simplejson.dumps(response_dict)
return render_to_response('workmodule/show_item_details.html', {"item_complete_data": return_data}, context_instance=RequestContext(request))
In myHTML I do this
data_from_django = {{ farm_complete_data }}
Question 1 : Is this the correct method to access the JSON data in the HTML ? Somehow I think there should be a better/cleaner way.
Question 2 : Another problem is all quotes are replaced with """ hence the javscript breaks. If above is the correct way, how to I "decode" the string correctly.
Note : I have used jquery's .ajax function earlier and it works great if you are on a page already and making a call to backend. The views in that case have returned the data in the same fashion as above & the data wasn't escaped. Or so it seemed by the time my ajax success: or error: functions handled it.
Thanks for taking time to look at this.

Question 1: that's about right, actually.
Question 2: Don't decode it, pipe it to safe: {{farm_complete_data|safe}} so it doesn't try to html-escape it for you.

Why pass it to a template at all? You just want the JSON, so in the view, do this:
return simplejson.dumps(response_dict)
Then there's no need to worry about encoding/quoting.