How to get the x position of the Facebook AR hand tracker? - javascript

For hand trackers, their patch has info X, Y, and Z and when a hand is tracked there are values that appear. How do i get those value via script? is it possible? There aren't any hand tracking script examples.
I have the following so far to determine if a hand is detected, still not sure about getting the position:
//im assuming this is the correct module
const HandTracking = require('HandTracking');
const Scene = require('Scene');
//this is to check if a hand is detected
HandTracking.count.monitor().subscribe(function(e){
if(e.newValue){
Debug.log("hand found");
}
})
UPDATE
So im assuming it has something like this :
var thehand = HandTracking.hand(0);
var posx = thehand.cameraTransform.x.lastValue;
Debug.log(posx);
But this way is deprecated? need to use subscribeWithSnapshot but I'm not sure how to implement this.
Also it needs to be dynamic(?) because the value changes with every movement.

You are on the right track, what you need is simply:
var posx = thehand.cameraTransform.x;
This will give you a signal for the hand x position. To debug this value you can use:
Diagnostics.watch("hand x",posx);
A signal is a value that changes over time, and it can be bound to another object property, for example:
mySceneObject.transform.x = thehand.cameraTransform.x;
This will bind the hand x position signal to the object x position, so that the object will move with the hand.
Read more about signals here: https://developers.facebook.com/docs/ar-studio/scripting/reactive
They are a very powerful tool and essential knowledge for scripting in AR Studio.
Hope this helps!

Related

draw function not looping as expected

I'm a beginner on here, so apologies in advance for naivety. I've made a simple image on Brackets using Javascript, trying to generate circles with random x and y values, and random colours. There are no issues showing when I open the browser console in Developer Tools, and when I save and refresh, it works. But I was expecting the refresh to happen on a loop through the draw function. Any clues as to where I've gone wrong?
Thanks so much
var r_x
var r_y
var r_width
var r_height
var x
var y
var z
function setup()
{
r_x = random()*500;
r_y = random()*500;
r_width = random()*200;
r_height = r_width;
x = random(1,255);
y= random(1,255);
z= random(1,255);
createCanvas(512,512);
background(255);
}
function draw()
{
ellipse(r_x, r_y, r_width, r_height);
fill(x, y, z);
}
Brackets.io is just your text editor (or IDE if you want to be technical) - so we can remove that from the equation. The next thing that baffles me is that something has to explicitly call your draw() method as well as the setup() method -
I'm thinking that you're working in some sort of library created to simplify working with the Canvas API because in the setup() method you're calling createCanvas(xcord,ycord) and that doesn't exist on it's own. If you want to rabbit hole on that task check out this medium article, it walks you thru all the requirements for creating a canvas element and then drawing on that canvas
Your also confirming that you're drawing at least 1 circle on browser refresh so i think all you need to focus on is 1)initiating your code on load and 2)a loop, and we'll just accept there is magic running in the background that will handle everything else.
At the bottom of the file you're working in add this:
// when the page loads call drawCircles(),
// i changed the name to be more descriptive and i'm passing in the number of circles i want to draw,
// the Boolean pertains to event bubbling
window.addEventListener("load", drawCircles(73), false);
In your drawCircles() method you're going to need to add the loop:
// im using a basic for loop that requires 3 things:
// initialization, condition, evaluation
// also adding a parameter that will let you determine how many circles you want to draw
function drawCircles(numCircles) {
for (let i = 0; i < numCircles; i++) {
ellipse(r_x, r_y, r_width, r_height);
fill(x, y, z);
}
}
here's a link to a codepen that i was tinkering with a while back that does a lot of the same things you are
I hope that helps - good luck on your new learning venture, it's well worth the climb!
Thank you so much for your help! What you say makes sense - I basically deleted the equivalent amount of code from a little training exercise downloaded through coursera, thinking that I could then essentially use it as an empty sandpit to play in. But there's clearly far more going on under the hood!
Thanks again!

Get the position of 3d objects in Facebook AR and changing them via script

I have a 3d object which i want to "move" from A to B via script. I am not too sure how to go about it; I don't understand the Facebook documents. Just a short example as a start would be great.
I assume something along the lines:
var object = Scene.root.find("object");
var lastPosX = object.transform.positionX.lastValue;
object.transform.positionX = //NOT SURE HOW TO PUT THE NEW POSITION
What you need to do is use the AnimationModule - here is a simple example of how to do that:
const Animation = require('Animation');
var obj = Scene.root.find("object");
//set up the length of the animations, 1000 = 1 second
var driver = Animation.timeDriver({durationMilliseconds: 1000});
//define the starting and ending values (start at 0, go to 100)
var sampler = Animation.samplers.linear(0, 100);
//create an animation signal to control the object x position
obj.transform.x = Animation.animate(driver, sampler);
//start the animation
driver.start();
Animation in ARS, like many other things, is based around the concept of "Reactive Programming" and working with "Signals" which are values that change over time. It is essential to get a good grasp of what a signal is and how it works to write useful code in ARS. Read through this for an introductory overview: https://developers.facebook.com/docs/ar-studio/scripting/basics
The above is a very basic example, but there is much more interesting, advanced, and complex effects that you can achieve using the AnimationModule, take a look at the documentation here for more information: https://developers.facebook.com/docs/ar-studio/reference/classes/animationmodule/
Hope this helps!

Is object facing another

I'd like to derive a value for how much one object is facing another. For example if it is directly facing the value would be 1 and facing directly away -1.
I have the object's global position and a quaternion for orientation. I'm using three.js
This is what I have so far, but its not giving me numbers I would expect.
Storing the object orientation as 3 part vector:
const lookVector = new THREE.Vector3(0,0,1);
const direction = lookVector.clone().applyMatrix4(torsoMesh.matrix);
this.lookVector = direction.sub(this.worldPosition);
Then using this to derive an angle to another point:
angleTo(pointOfInterest){
const vectorToPOI = pointOfInterest.sub(this.worldPosition);
const angle = this.lookVector.angleTo(vectorToPOI);
return angle;
}
one way to achieve this is using dot product. It is very simple, you just normalize both vectors and multiply each component respectively, then sum them up. I won't go into more detail here, because there is loads of resources explaining this on the internet. But here is an example of how to do it in three.js:
this.lookVector.normalize();
pointOfInterest.normalize();
var angleValue = this.lookVector.dot(pointOfInterest);
this will give exactly what you asked for (just remember that by calling normalize() on a vector, you are modifying it).
angleValue is actually just the cosine of the angle between the two vectors, so another way would just be to do:
.
.
return Math.cos(angle);
hope I understood you question correctly.
I know the mesh objects keep rotation values in radians. Maybe you can have a "parent mesh" so to speak and its its rotation values to determine this.

cesium - moving billboards

I am testing Cesiumjs to see if it can reflect a near-real-time expreience - for example: position of airplanes.
For that, I need to draw billboards and make them move - which I know is possible with cesium, just not sure how.
The code looks like this:
var billboards = scene.primitives.add(new Cesium.BillboardCollection());
var billboard = {
image : '/path/to/logo.png',
position : Cesium.Cartesian3.fromDegrees(-75.59777, 40.03883)
};
billboards.add(billboard);
My question is how do I change the position of the billboard. I couldn't find ant documentation that would explain.
I thought doing:
billboard.position = ... //new position
but how will cesium know that I've changed the position attribute unless it somehow turns that reference into a observable object.
So how do I update the location?
Thanks.
Cesium does indeed listen for changes to billboard.position
(source code here), so it is correct behavior for apps to simply write a new position.
Note that you must write the whole position at once, meaning you may not write to billboard.position.x. Instead, keep a "scratch" Cartesian3 around (don't create a new one every animation frame at 60fps), write to the x,y,z properties of your scratch variable, and then assign your scratch variable to billboard.position. You can see in the source that the assigned value will be cloned into another pre-existing Cartesian3, so you may immediately reuse the scratch variable.
Here's an example:
// Just once at app startup. Don't call "new" at 60fps.
var scratchCartesian3 = new Cesium.Cartesian3();
var ellipsoid = viewer.scene.mapProjection.ellipsoid;
function onTick() {
// This is safe to call at 60fps.
billboard.position = Cesium.Cartesian3.fromDegrees(
lon, lat, alt, ellipsoid, scratchCartesian3);
}
Also note that your question and the above answer are focused on the "Graphics Primitive" layer of the Cesium API. Cesium has one higher layer, called the "Entity" API, that you can use if you want Cesium to handle the concept of user-selectable objects with pop-up descriptions etc. Here's a Sandcastle demo showing how to add a billboard as a property of an entity, instead of as a primitive. This allows you to add other properties to the same entity, for example a name, description, label, 3D model, etc, and have them all be controlled from the same position property, and have Cesium take care of popup descriptions. The position property is more complex for entities than for primitives, for example it can be constant or sampled. This allows entities to change position over time when the timeline is shown.

Box2dWeb call function when objects collide

I'm fairly new to javascript and box2d, i was wondering if someone knows how i can call a custom function when two objects collide. I tried using some examples that uses the b2ContactListener without any succes. I've placed an object above another and let the standard Box2d physics do it's thing.
I recieve two console outputs, the first is null and the second is Ball with the following code:
var listener = new Box2D.Dynamics.b2ContactListener;
listener.BeginContact = function(contact) {
console.log(contact.GetFixtureA().GetBody().GetUserData());
console.log(contact.GetFixtureB().GetBody().GetUserData());
};.
The two objects that need to collide are a b2_dynamicbody (ball) and a b2PolygonShape. (rectangle). Using bodyDef.userData = "Ball"; in my Ball.js and bodyDef.userData = "Mouse"; in my Mouse.js i try to identify if they are hit. Instead only the ball is displayed.
Next to that i'm sure this is not the correct way for detecting collision :P I hope i've explained it well enough, could somebody steer me in the right direction?
Ok I solved it myself, apparently I had to add my custom event to the world I create with box2d. So, the issue was solved by me reading big chunks of box2d documentation/manual which can be found here:
I started with adding a String as UserData() to every object which can collide and has to do something else next to just colliding. Using the following code:
bodyDef.userData = "Car";
note: every object has to have it's own unique string.
Then I created a new contact listener (formulated in my question above) and listened for fixtures colliding with each other. When that happens, I 'save' the fixtures UserData() in variables which I can then use to look what objects collide with each other.
var contactListener = new Box2D.Dynamics.b2ContactListener;
contactListener.BeginContact = function(contact) {
var fixA = contact.GetFixtureA().GetBody().GetUserData();
var fixB = contact.GetFixtureB().GetBody().GetUserData();
// if else statement here...
};
world.SetContactListener(contactListener);
Finally, I added the last statement world.SetContactListener(contactListener); to add the event to the world, making it possible for it to listen to collisions, which I forgot to add and thus was my problem.
Hope someone finds this usefull!

Categories