WebSpeech Speech Synthesis: Pausing utterance1, playing another utterance2, and resuming utterance1 - possible? - javascript

I am using WebSpeech's speechSynthesis module to have a web application speak. However, it seems that you can only add utterances to a queue and then pause(), resume(), and cancel() the entire queue.
I have a situation where I want to have two utterances:
utterance1 = new SpeechSynthesisUtterance(text1);
utterance2 = new SpeechSynthesisUtterance(text2);
I would like to have utterance1 play, then pause it in the middle, have utterance2 play, and then resume utterance1. In code, it would look like this:
speechSynthesis.speak(utterance1);
// ... after a while
speechSyntehsis.pause(utterance1);
speechSynthesis.speak(utterance2);
// ... after a long while
speechSynthesis.resume(utterance1);
Unfortunately, speechSynthesis' methods pause(), resume(), and cancel() do not take any argument and act on the entire speech utterance queue. Is there any way to achieve this behavior?
If I could have multiple speechSynthesis objects, then I could create one for each utterance, but I believe I can only have one.
If I could keep track of where in the string the utterance has "been uttered to" then I could cancel it and then create a new utterance with the remainder of the text, but I don't know if that is possible.
Any suggestions?

I have already work in the speechSynthesis for a couple of months with my library Artyom.js , and according to the documentation (and all the tests that i've made ) pause a single synthesis instance and reanudate another is not possible because all the instances are related to the window.speechSynthesis (if someday the API changes, that will be another great step in the speechSynthesis). When you call the pause method of the speechSynthesis "instance", it will apply for all the queue and there's no other way.
According to the documentation :
// the only solution would be if the speechSynthesis official API had a constructor like
// and a real NEW instance be created
// var synthRealInstance = new speechSynthesis();
// but till the date ... nope :(
var synthA = window.speechSynthesis;
var synthB = window.speechSynthesis;
var utterance1 = new SpeechSynthesisUtterance('How about we say this now? This is quite a long sentence to say.');
var utterance2 = new SpeechSynthesisUtterance('We should say another sentence too, just to be on the safe side.');
synthA.speak(utterance1);
synthB.speak(utterance2);
synthA.pause();
// or synthB will anyway stop the synthesis of the queue
There is a property on the utterance (onmark) however is not well documented and probably will not work as this api still experimental.
The mark event is fired when a ‘mark’ tag is reached in a Speech Synthesis Markup Language (SSML) file. Just know that it’s possible to pass your speech data to an utterance using an XML-based SSML document. The main advantage of this being that it makes it easier to manage speech content when building applications that have large amount of text that need to be synthesised.
Read more about here.

Related

How to create a generic "joystick/gamepad event" in Javascript?

Issue:
In the current implementation of modern browsers, (like Firefox or Chrome), there are only two joystick/gamepad events:
gameadConnected
gamepadDisconnected
Since it appears that the original idea behind implementing joystick/gamepad support in the browser was to allow for in-browser games, the joystick was made dependents on the requestAnimationFrame() call to create a game-loop sync'd with v_sync.
However, in other use cases, for example where the joystick is being used to control something remotely over a network or wireless connection, the best case is to only send data when there is something useful to say - did something happen?  Using requestAnimationFrame() floods the interface with potentially useless data.
Unfortunately, there is currently no established interface for triggering gamepad events.&nbsp: (Note, there is some discussion of this very issue over on the Mozilla and W3C forums, so this may, eventually, change.)
Since flooding an industrial device or remote controlled system with useless messages isn't a "best practice" - the question becomes how to generate the equivalent of a gamepad event without flooding the network or stalling the browser in a wait-loop.
Webworkers was a thought, but they cannot be used because they don't have access to the window.event context and cannot interface with the joystick/gamepad.  At least not directly.
In order to handle this efficiently, some method of triggering an "event" that allows data to be sent, only when something of interest happens.
For the benefit of those who may be confronting this same issue, here is the solution I eventually implemented:
=======================
My solution:
This solution is based on the fact that the gamepad's time_stamp attribute only changes when something happens.  (i.e. A button was pressed or a joystick axis was moved,)
Keep track of the gamepad's time_stamp attribute and capture it on the initial gamepad connected event.
Provide a "gateway" condition that surrounds the routine that actually sends the data to the receiving device.
I implemented this in two steps as noted above:
First:
When the gamepad connects, I immediately capture the time_stamp attribute and store it in a variable (old_time).
window.addEventListener("gamepadconnected", (event) => {
js = event.gamepad;
gamepad_connected(); // Gamepad is now connected
old_time = gopigo3_joystick.time_stamp // Capture the initial value of the time_stamp
send_data(gopigo3_joystick) // send it to the robot
Then I do whatever looping and processing of data I need to do.
As a part of that loop, I periodically attempt to send data to the server device with the following code:
function is_something_happening(old_time, gopigo3_joystick) {
if (gopigo3_joystick.trigger_1 == 1 || gopigo3_joystick.head_enable == 1) {
if (old_time != Number.parseFloat(jsdata.timestamp).toFixed()) {
send_data(gopigo3_joystick)
old_time = gopigo3_joystick.time_stamp
}
}
return;
}
function send_data(gpg_data) {
// this sends a gamepad data frame to the robot for interpreting.
[code goes here];
return;
}
The first function, is_something_happening, tests for two qualifying conditions:
A specific joystick button press.  The robot is not allowed to move without a trigger being pressed so no data is sent.&nbsp "head_enable" is another condition that allows messages for head pan-and-tilt commands to be sent.
A change in the time_stamp value.  If the time_stamp value has not changed, nothing of interest has happened.
Both conditions must be satisfied, otherwise the test falls through and immediately returns.
Only if both conditions are met does the send_data() function get called.
This results in a stable interface that always gets called if something of interest has happened, but only if something of interest has happened.
Note:  There are keyboard commands that can be sent, but since they have active events, they can call send_data() by themselves as they only fire when a key is pressed.

Using Nightwatch for demo - slow down assertions by config

my team and me are using Nightwatch to write end-to-end-acceptance tests for a microservice oriented architecture with a total of five systems.
After putting in some work to set it up and wiring together our services with docker-compose, it works great now and all tests are clicked through on the UI in a browser (not headless).
We got the idea to use this for demos, too (initial sprint demo etc) and wondered if there is some kind of setting (which we didn't found until now) or other possibility to simple add some artificial delay between the clicks/tests/assertions and everything.
Does someone have an idea?
You can add pauses in your suite wherever you want by using:
.pause(5000) // a pause for 5 seconds
//or alternately
.pause(this.timeout)
this.timeout can be set in your base-test-case.js
var timeout = 5000; // in your variable declarations
and then in that same file, on your base Class prototype you want:
before: function (client) {
this.timeout = timeout;
browser.pause between clicks or setValue to have nice delay, anything between 100-300 miliseconds is good
http://nightwatchjs.org/api#pause

Cocos2d-html5 - Restarting/quitting engine possible?

With Cocos2d-html5, is there any way to “quit” the engine and re-initialize its state so you can later restart the engine. We are using a Cocos2d game in a single page web app. If the user navigates to another page, we want to programmatically remove the canvas div and attempt to exit the Cocos2d engine. Unfortunately, it seems there’s no good way to do this and attempting to load a new game causes errors.
How can you cleanly unload a scene and quit the engine?
We are using V3.1 and tried several approaches, none of them seem to work. For example:
Trying to reset many variables that are held in the cc object (detailed below). This approach forces you to reset variables that look private as they start with _ and are not in the documentation.
picking, choosing and rewriting some of CCBoot.js but this appeared complicated and not sustainable for engine updates as so much of it is depended upon throughout the library.
Other approaches I thought of but all sound like a hack:
3. Null the whole cc object and somehow run the script again but that might mean stripping out a script tag or module and adding and running it again.
4. Wrap the cc object in another object so it is easier to reset. CCBoot.js looks like it might attach some things to the window object.
I have got furthest with the first approach but am stuck with context issues. When leaving the canvas, before I remove it from the DOM I call these:
// Remove anything that might possibly keep a reference to the old context
cc.textureCache.removeAllTextures();
cc._drawingUtil = null;
cc.stencilBits = null;
// Purge the cc.director, schedules, event listeners, running scene, animations, cached data.
cc.director.purgeDirector();
// Remove references to the context
cc._renderContext = null;
cc.webglContext = null;
window.gl = null;
cc._mainRenderContextBackup = null;
// Remove reference to DOM elements
cc._canvas = null;
cc.container = null;
cc._gameDiv = null;
// Reset CCBoot variables that might stop us from re-initialising
cc._rendererInitialized = false;
cc._setupCalled = false;
cc._renderType = -1;
Then when we restart on the second or subsequent time call
// Reset all system and flag variables
cc._initSys(cc.game.config, cc.game.CONFIG_KEY);
cc.game.run();
And here are the kind of errors I get. It looks like it's not properly resetting the context:
WebGL: INVALID_OPERATION: bindTexture: object not from this context
WebGL: INVALID_OPERATION: texImage2D: no texture
WebGL: INVALID_OPERATION: uniformMatrix4fv: location is not from current program
WebGL: INVALID_OPERATION: vertexAttribPointer: no bound ARRAY_BUFFER
I've managed to get it stopping/resuming enough for a specific use case now, here is a demo:
http://plnkr.co/edit/BQmmHi?p=preview
There is now a public way to pause a game in v3.13, so I can call:
cc.game.pause();
followed by removing the cc.container from the DOM:
cc.container.parentNode.removeChild(cc.container);
Then I can place it back in the DOM again anywhere I would like:
targetContainerEl.appendChild(cc.container);
Then resume the game by calling:
cc.game.resume();
Note: The next line was was needed after replacing in the DOM in a complex app in cocos2d v3.5, but not sure if it's needed in v3.13 any more:
cc.EGLView._instance._frame = cc.container.parentNode;
The demo from the link above is being removed and placed in 2 different DOM elements and pausing/resuming.

mediaElementjs: how to get instance of the player

I'm stuck with a little problem with MediaElement.js player.
To get the instance of the player, I do this (works with html5 compatible browser):
// Get player
this.playerId = $('div#shotlist-player video').attr('id');
this.player = window[this.playerId];
But it's not working as soon as it fallback in flash. In fact, it's not working because I'm not calling an instance of MediaElement itself. But I don't see how I can call it.
The player is created with
$('video').mediaelementplayer({....});
How can I get the mediaelement object?
------------EDIT----------------
Ok I finally found how to make it works:
// Get player
mePlayer = $('div#shotlist-player video.video-js')[0];
this.player = new MediaElementPlayer(mePlayer);
Now I can user mediaElement instance correctly.
This post is a lot of speculation, but may be correct. Docs are lacking (;
The answer by sidonaldson is perfectly acceptable if you wish to create a new MediaElement instance and get a handle on it. If there's one already present, it seems to try to reinitialize another instance on that element and freaks out.
I am pretty sure mediaelement.js augments the builtin HTML5 controls by providing a JavaScript API to manipulate Flash/Silverlight players via those elements. I may be wrong, but other advice I've seen on this issue in multiple places is to do something like:
$playButton.click(function() {
$('video, audio').each(function() {
$(this)[0].player.play();
});
});
To create a play button as an external DOM element which will fire off all players on the page. This indicates to me that the implementation is something like I've described.
Try:
var player = $('video').mediaelementplayer({
success: function (me) {
me.play();
}
});
// then you can use player.id to return the id
// or player.play();

Start and stop Server-sent events notification

I have 3 Server-sent Events available to a page. Only one viewable at any time. I would like to stop the listener on 2 of the 3 event streams when 1 of them is active.
I have a switch statement testing for which is visible but can not pass the source.close() to my event directly as it is buried in a function:
var firstEventSource = function() {
var eventSrc = new EventSource('firstSTREAM.php');
eventSrc.addEventListener('message', onMessageHandler);
};
I was hoping to have fewer open connections to the server, especially with non-viewed data.
If you have a better suggestion I'm all ears!
Best,
T
function onMessageHandler(event) {
if ("your want to close that EventSource") {
event.target.close();
}
}
This question is hard to answer without more context, but I'll do my best.
You could think of the event resource as a pipe where you push all of your messages, and have the client listen for specific events, effectively multiplexing:
var handler = console.log.bind(console)
, events = new EventSource("/events")
events.addEventListener("new-friend", handler)
events.addEventListener("new-message", handler)
events.addEventListener("new-notification", handler)
This would reduce your connection count to exactly one, and would save you from doing costly reconnects whenever you switch between views. However, it has the drawback of your server pushing (possibly) unnecessary data down the pipe. After all, you're only viewing one message type at a time. You should consider whether this is an actual problem though. If your UI should update, perhaps with some kind of badge notification (like facebook's message or notification icons) then you will need to know about those messages even though the user may not be actively on that particular view. In any event, you should try to keep messages lean for performance sake.
If you can't or won't push all messages down the same pipe, you probably should go with your initial thought of having multiple resources or the ability to query the resource in question, and then opening and closing the connections. Bear in mind though that this could potentially be very costly, as the client could end up hammering the server with requests. Each view change would cause connections to be set up and tore down. It'd look something like this:
/* Assuming jquery is available and with the following html:
* <a class="stream" href="/friends>Friends</a>
* <a class="stream" href="/messages>Messages</a>
* <a class="stream" href="/notifications>Notifications</a>
*/
var currentEvents
, handler = console.log.bind(console)
$("a.stream").on("click", function() {
$el = $(this)
currentEvents && currentEvents.close()
currentEvents = new EventSource($el.attr("href"))
currentEvents.addEventListener("message", handler)
return false
})
In the end, it depends on context. If users aren't going to switch views very often, or the messages are really big, then you might want to go for the second approach. It'll feed less data down the pipe, but create and tear down connections as the user navigates. If the user often switches views however, or you can keep the message size reasonable, then I'd advocate multiplexing, like in the first solution. It'll keep one long-running connection where small messages of different types may be pushed to the client.

Categories