Okay so...I'm in the process of creating a python client for a coding game I play. This game is originally created in javascript and employs a json object of game data found at https://adventure.land/data.js. Within this data, each map has a list of doors, and each door within this list is a list of attributes. For example:
"doors":[[-965,-176,24,30,"woffice",0,1],[536,1665,64,32,"tunnel",0,2],[168,-149,32,40,"bank",0,3],[160,1370,24,32,"cave",0,4],[232,384,24,30,"arena",0,6],[-472,131,24,30,"tavern",0,8],[616,610,32,40,"mansion",0,10],[1936,-23,24,24,"level1",1,11],[169,-404,24,40,"hut",0,14],[1600,-547,60,40,"halloween",4,15],[312,-335,32,32,"mtunnel",0,16],[967,-584,32,32,"mtunnel",1,17],[1472,-434,32,32,"mtunnel",2,18]]
However, when sent from python to julia, it suddenly becomes:
"doors":[-965,-176,24,30,"woffice",0,1,536,1665,64,32,"tunnel",0,2,168,-149,32,40,"bank",0,3,160,1370,24,32,"cave",0,4,232,384,24,30,"arena",0,6,-472,131,24,30,"tavern",0,8,616,610,32,40,"mansion",0,10,1936,-23,24,24,"level1",1,11,169,-404,24,40,"hut",0,14,1600,-547,60,40,"halloween",4,15,312,-335,32,32,"mtunnel",0,16,967,-584,32,32,"mtunnel",1,17,1472,-434,32,32,"mtunnel",2,18]
Is there some way to prevent this from happening? If not, is there a possible workaround? Because my for door in doors code in python won't exactly have the same effect when given a direct translation to julia considering the random list flattening.
Edit: I've been asked to provide "minimum reproducible code" but I'm not too sure what the minimum is so I'll just say...
create a some function in julia that prints a dictionary
function printDict(d)
println(d)
end
create a dictionary in python that contains a nested list
nestedlistdict = {"data": [[0,1,2,3],[4,5,6,7]]}
call the printDict function from python while sending the nestedlistdict variable as an argument
from julia import Main
Main.include("path/to/your/julia/file")
Main.file.printDict(nestedlistdict)
result should be:
Dict{Any, Any}("data" => [[0, 1, 2, 3], [4, 5, 6, 7]])but instead my result is Dict{Any, Any}("data" => [0 1 2 3; 4 5 6 7])
Edit2: upon further testing, it seems to be having the issue because it converts a nested list into a Matrix object instead of a nested Vector object.
Julia uses the same interpretation of list of lists as numpy:
>>> np.array([[1,2,3],[4,5,6]])
array([[1, 2, 3],
[4, 5, 6]])
or in Julia:
julia> using PyCall
julia> py"[[1,2,3],[4,5,6]]"
2×3 Matrix{Int64}:
1 2 3
4 5 6
If you want to have an actual vector of vectors you need to do the same way as you need to do a vector of numpy arrays.
Python version:
>>> [np.array([1,2,3]),np.array([1,2,3])]
[np.array([1, 2, 3]), np.array([1, 2, 3])]
Julia version (this is what you are actually asking for):
julia> np = pyimport("numpy");
julia> py"[$np.array([1, 2, 3]), $np.array([1, 2, 3])]"
2-element Vector{Vector{Int32}}:
[1, 2, 3]
[1, 2, 3]
Another, perhaps simpler option could be packing that all to a tuple:
julia> py"tuple([[1,2,3],[4,5,6]])"
([1, 2, 3], [4, 5, 6])
I'm looking for a way get a console output from Google Chrome into my python program. I have a script coded in JS that finishes in around 1 second, but my python implementation (exactly same logic, etc, the only diff is that it's in Python and not JS) takes about 15 seconds to run. Therefore I'm looking for a way to get the printout in Chrome console to my python program.
This is the current way I'm doing it:
Python program uses pyautogui to click and does what it needs to do inside to trigger the function running in JS.
JS completes the function in 1s and prints to console, something like:
(22) [6, 4, 4, 6, 0, 0, 2, 4, 4, 6, 4, 2, 4, 4, 6, 0, 0, 2, 4, 4, 6, 0]
I need to get that into my python program so it can does the rest of the stuff..
I am attempting to implement a DraftJS editor that highlights words in a transcription while its recorded audio is playing (kind of like karaoke).
I receive the data in this format:
[
{
transcript: "This is the first block",
timestamps: [0, 1, 2.5, 3.2, 4.1, 5],
},
{
transcript: "This is the second block. Let's sync the audio with the words",
timestamps: [6, 7, 8.2, 9, 10, 11.3, 12, 13, 14, 15, 16, 17.2],
},
...
]
I then map this received data to ContentBlocks and initialize the editor's ContentState with them by using ContentState.createFromBlockArray(blocks)
It seems like the "DraftJS" way of storing the timestamp metadata would be to create an Entity for each word with its respective timestamp, and then scan through the currentContent as the audio plays and highlight entities up until the current elapsed time. But I am not sure if this is the right way to do this, as it doesn't seem performant for large transcriptions.
Note: the transcript needs to remain editable while maintaining this karaoke functionality
Any help or discussion is appreciated!
I ended up doing exactly what I described in the question: store timestamps in DraftJS entities. After a few more weeks with DraftJS it seems this is the correct way to do this.
In attempting to build a WebGL 3D library for myself (learning purposes mostly) I followed documentation that I found from various sources that stated that the TypedArray function set() (specifically for Float32Array), is supposed to be "as fast as" memcpy in C (obviously tongue in cheek), literally the fastest according to html5rocks. On appearances that seemed to be correct (no loop setup in javascript, disappearing into some uberfast typed array pure C nonsense, etc).
I took a gander at glMatrix (good job on it btw!), and noticed that he (author) stated that he unrolled all of the loops for speed. This is obviously something a javascript guru would do normally for as much speed as possible, but, based on my previous reading, I thought I had 1-up on this library, specifically, he created his lib to be functional with both arrays and typed arrays, thus I thought that I would get more speed by using set() since I was only interested in staying in TypedArray types.
To test my theory I set up this jsperf. Not only does set() comparatively lack speed, every other technique I tried (in the jsperf), beats it. It is the slowest by far.
Finally, my question: Why? I can theoretically understand a loop unrolling becoming highly optimized in spidermonkey or chrome V8 js-engines, but losing out to a for loop seems ridiculous (copy2 in jsperf), especially if its intent is theoretically to speed up copies due to the raw contiguous in memory data types (TypedArray). Either way it feels like the set() function is broken.
Is it my code? my browser? (I am using Firefox 24) or am I missing some other theory of optimization? Any help in understanding this contrary result to my thoughts and understandings would be incredibly helpful.
This is an old question, but there is a reason to use TypedArrays if you have a specific need to optimize some poorly performing code. The important thing to understand about TypedArray objects in JavaScript is that they are views which represent a range of bytes inside of an ArrayBuffer. The underlying ArrayBuffer actually represents the contiguous block of binary data to operate on, but we need a view in order to access and manipulate a window of that binary data.
Separate (or even overlapping) ranges in the same ArrayBuffer can be viewed by multiple different TypedArray objects. When you have two TypedArray objects that share the same ArrayBuffer, the set operation is extremely fast. This is because the machine is working with a contiguous block of memory.
Here's an example. We'll create an ArrayBuffer of 32 bytes, one length-16 Uint8Array to represent the first 16 bytes of the buffer, and another length-16 Uint8Array to represent the last 16 bytes:
var buffer = new ArrayBuffer(32);
var array1 = new Uint8Array(buffer, 0, 16);
var array2 = new Uint8Array(buffer, 16, 16);
Now we can initialize some values in the first half of the buffer:
for (var i = 0; i < 16; i++) array1[i] = i;
console.log(array1); // [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
console.log(array2); // [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
And then very efficiently copy those 8 bytes into the second half of the buffer:
array2.set(array1);
console.log(array1); // [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
console.log(array2); // [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
We can confirm that the two arrays actually share the same buffer by looking at the buffer with another view. For example, we could use a length-8 Uint32Array that spans the entire 32 bytes of the buffer:
var array3 = new Uint32Array(buffer)
console.log(array3); // [50462976, 117835012, 185207048, 252579084,
// 50462976, 117835012, 185207048, 252579084]
I modified a JSPerf test I found to demonstrate the huge performance boost of a copy on the same buffer:
http://jsperf.com/typedarray-set-vs-loop/3
We get an order of magnitude better performance on Chrome and Firefox, and it's even much faster than taking a normal array of double length and copying the first half to the second half. But we have to consider the cycles/memory tradeoff here. As long as we have a reference to any single view of an ArrayBuffer, the rest of the buffer's data can not be garbage collected. An ArrayBuffer.transfer function is proposed for ES7 Harmony which would solve this problem by giving us the ability explicitly release memory without waiting for the garbage collector, as well as the ability to dynamically grow ArrayBuffers without necessarily copying.
Well set doesn't exactly have simple semantics like that, in V8 after doing some figuring out of what should be done it will essentially arrive at exactly the same loop that the other methods are directly doing in the first place.
Note that JavaScript is compiled into highly optimized machine code if you play your cards right (all the tests do that) so there should be no "worshipping" of some methods just because they are "native".
I've also been exploring how set() performs and I have to say that for smaller blocks (such as the 16 indices used by the original poster), set() is still around 5x slower than the comparable unrolled loop, even when operating on a contiguous block of memory.
I've adapted the original jsperf test here. I think its fair to say that for small block transfers such as this, set() simply can't compete with unrolled index assignment performance. For larger block transfers (as seen in sbking's test), set() does perform better but then it is competing with literally 1 million array index operations so it would seem bonkers to not be able to overcome this with a single instruction.
The contiguous buffer set() in my test does perform slightly better than the separate buffer set(), but again, at this size of transfer the performance benefit is marginal