We Keep Coding

Hide REST and Websockets inside the same function using a listener callback for both

I am working on a project with 2 clients, 1 in React Native and 1 in React. There is a common API module, which is auto-generated by an OpenAPI specification, and a Services module that is responsible for calling the API methods and translating the DTOs to a more suitable format.
The app has real-time requirements and fetching is carried out in the following manner:
Let R be a resource on the server with a structure like { ...fields..., timestamp: 12345678 }.
Let N be the timestamp of now. The client apps need to fetch all the R resources that have a timestamp higher than N - 10 hours and also subscribe to get new R items, when and if any are created (through Websockets). If the user of the app is currently on a screen that only concerns old data (N - 10 hours <= timestamp <= N), then the Websocket subscription should be omitted.
My question is, how bad would it be if I implemented methods like the following where I completely disregard returning Promises, like when someone uses fetch, and I use callbacks to inform the user of the method that the "Promise resolved"?
type R = {
timestamp: Date
}
type API = {
getR: (start: Date, end: Date) => R[];
subscribeToR: (onNewR: (r: R) => void) => void;
}
class NewServices {
constructor(private readonly api: API) {}
// notice that end is optional and onNewR receives an array
getR(onNewR: (r: R[]) => void, start: Date, end?: Date): void {
if (end !== undefined) {
this.api.getR(start, end).then((resp) => {
onNewR(resp.data);
});
return;
}
// end is undefined here
const lEnd = new Date(); // now
this.api.getR(start, lEnd).then((resp) => {
onNewR(resp.data);
this.api.subscribeToR((r) => onNewR([r]));
});
}
}
// The current state of the services
class OldServices {
constructor(private readonly api: API) {}
// This class is an example, IRL it does DTO conversions, too
async getR(start: Date, end: Date): Promise<R[]> {
return (await this.api.getR(start, end)).data;
}
subscribeToR(onNewR: (r: R) => void): void {
this.api.subscribeToR(onNewR);
}
}
The apparent drawback is that I can no longer use React Query, which I liked.
However, I got tired of writing the if (end !== undefined) logic in the (thankfully reusable) business layer of my app and, also, I believe that this "either REST or Websocket" behavior should be abstracted out from the rest of the app and be hidden inside the Services layer, even though it feels like an anti-pattern. What do you suggest/think?

RxJs - What is the proper pattern/way to create an Observable array of Observables?

Am fairly new to RxJs, and trying to wrap my head around what the proper pattern is to simply create an Observable array of Observables.
I want to retrieve a list of User's Posts. The Posts themselves should be Observables, and I want to keep them all in an Observable array, so that when the array changes the calling code should be notified and update anything subscribed to the post "list". This is simple enough, but I also would like each of the Posts to be Observables, so if I retrieve a specific posts[i] from it, I should also be able to subscribe to these individual objects.
What is the proper way to do this?
Am using Angular 9, I have:
public getPosts(): Observable<Array<Post>> {
return new Promise((resolve, reject) = {
let posts: Observable<Array<Post>> = new Observable<Array<Post>>();
this.get<Array<Post>>('posts').subscribe(r => {
posts = from(r);
return resolve(posts);
});
});
}
This gives me an Observable<Array<Post>>, but how should I create an Observable<Array<Observable<Post>>>?
Is this an anti-pattern?

It all comes to convenience, if your server serves you differential data of what changed in post, then go ahead and create Observable<Observable<Post>[]>.
In your post, however, there are multiple problems. You cannot mix Observables with Promises. The method getPosts will return only the first post you get from API.
This is the solution ask for, but I am not sure, it is what you actually wanted...
public getPosts(): Observable<Array<Observable<Post>>> {
return this.get('posts').pipe(
switchMap(posts => combineLatest(
posts.map(post => this.get('post', post.id))
)),
);
}

it's unclear what you're trying to accomplish here, but you might want something more like this:
#Injectable({providedIn:'root'})
export class PostService {
// private replay subject will cache one value
private postSource = new ReplaySubject<Post[]>(1)
// public list of posts observable
posts$ = this.postSource.asObservable();
// function to select item by id out of list
post$ = (id) => this.posts$.pipe(map(posts => posts.find(p => p.id === id)))
getPosts() {
// function to get remote posts
return this.get<Post[]>('posts');
}
loadPosts() {
// function to load posts and set the subject value
this.getPosts().subscribe(posts => this.postSource.next(posts));
}
}
you'll have to define that get function and call loadPosts everytime you want to update the list.

Given informations:
!If any of this statements is wrong, please tell me and I will update the answer!
get function that returns an observable with one array that filled with posts
the get observable emits always when the posts are changing
the value inside the observable (Array>) is no observable and does not change over time
this.get<Array<Post>>('posts')
Possible functions
() => getPostById$
// This function returns you an observable with the post related to your id.
// If the id is not found the observable will not emit
// If the id is found the observable will only emit if the interface values have been changed
function getPostById$(id: string): Observable<Post> {
// Returns you either the post or undefined if not found
const findId = (id: string) => (posts: Array<Post>): Post | undefined =>
posts.find(post => post.id === id);
// Allows you only to emit, if id has been found
const existingPost = (post: Post | undefined): boolean => post != null;
// Allows you only to emit if your id has been changed
const postComparator = (prevPost: Post, currPost: Post): boolean =>
prevPost.value === currPost.value && prevPost.name === currPost.name;
return this.get('posts').pipe(
map(findId(id)),
filter(existingPost),
distinctUntilChanged(postComparator)
);
}
() => getPosts$
function getPosts$(): Observable<Array<Post>> {
return this.get('posts');
}
() => getStatePosts$
// This function allows to manage your own state
// 1. posts$: overwrites all posts
// 2. clear$: empties your posts$ observable
// 3. add$: adds one observable to the end of your posts
function statePosts$(posts$: Observable<Array<Posts>>, clear$: Observable<void>, add$: Observable<Post>): Observable<Array<Post>> {
const updatePosts = (newPosts: Array<Posts>) => (oldPosts: Array<Posts>) => newPosts;
const clearPosts = () => (oldPosts: Array<Posts>) => [];
const addPost = (post: Post) => (oldPosts: Array<Posts>) => [...oldPosts, post];
return merge(
// You can add as much update functions as you need/want (eg: deleteId, addPostAtStart, sortPosts, ...)
posts$.pipe(map(updatePosts)),
clear$.pipe(map(clearPosts)),
add$.pipe(map(addPost))
).pipe(
// The fn in you scan is the (oldPosts: Array<Posts>) function from one of your three update functions (updatePosts, clearPosts and addPosts).
// Whenever one of those three observables emits it first calls the left side of the function inside the map (post: Post) and returns a new function
// When this function reaches the scan it gets the oldPosts and is able to update it
scan((oldPosts, fn) => fn(oldPosts), [])
)
}
// Usage
private posts$: Observable<Array<Post>> = this.get('posts');
private clear$: Subject<void> = new Subject();
private add$: Subject<Post> = new Subject();
public statePosts$ = getStatePosts(posts$, clear$, add$);
Hint: Try to read the functions from the return statement first. And then check what is happening in the mapping/filtering or other operations. Hopefully I did not confuse you too much. If you have questions, feel free to ask.

Flush a RxJs Subject

I want to create a subject with a bufferTime pipe.
e.g.
subject.pipe(bufferTime(1000, null, this._bufferSize),
filter((v, i) => {
return v.length !== 0;
})
)
After using this subject and finishing the work I'd like for the user to call the onComplete / new method that will flush the remaining contents of the stream.
Since this is time based I could wait for the stream to flush itself, but as I'm using AWS Lambda runtime is money.
Is there a simple way to implement a flush?

I think you are looking for takeUntil operator:
const subject = new Subject();
const complete = new Subject();
const BUFFER_SIZE = 10;
subject
.pipe(
takeUntil(complete),
bufferTime(1000, null, BUFFER_SIZE),
)
.subscribe(buffer => {
console.log(Date.now(), buffer);
});
I use another Subject called complete that is used for completing the Observable and consequently flushing the buffer in bufferTime.
See working example here: https://stackblitz.com/edit/typescript-ihjbxb

Track user time in completing a particular action in a website

I want to track how much time user is taking in completing a particular action (including server response time and render time(DOM related changes )) in website.
I have tried it in Angular framework. To do it, I am thinking of recording the time when user started the action and I want to note the time when the action is completed. As a developer, I will know when user started the activity and when user finish the action like search, filter, edit, add, delete etc. So, we can take the difference b/w them. But to note every action, we have to write code in every part of the app. Can we create a plugin so that we can use it everywhere instead of writing same code everywhere to track the time of user. Any approach to create it? Or is there any tool available to achieve this feature?

Would something like this help?
#Injectable({provideIn: 'root'})
export class TrackingService {
private cache: {[id: number]: {description: string, time: number}} = {};
private id: number = 0;
public startTracking(actionDescription: string): number{
const id = ++this.id;
this.cache[id] = { description: actionDescription, time: new Date().getTime() };
return id;
}
public stopTracking(actionId: number){
const data = this.cache[actionId];
if(data){
const elapsed = new Date().getTime() - data.time;
// ...
// Do something with your 'elapsed' and 'data.description'
// ...
delete this.cache[id];
return {...data, elapsed: elapsed};
}
throw `No action with id [${actionId}] running! `;
}
}
Ad then anywhere you need to track an action:
private actionId: number;
constructor(private trackingService: TrackingService){}
startAction(){
this.actionId = this.trackingService.startTracking('Description');
}
stopAction(){
const trackingResult = this.trackingService.stopTracking(this.actionId);
}
You can automate the tracking in some places, for example for routing:
// app.module.ts
private routeChangeSubscription: Subscription;
private configLoadActionId: number;
private navigationActionId: number;
constructor(private router: Router, private trackingService: TrackingService){
this.routeChangeSubscription = router.events.subscribe((event: Event) => {
if (event instanceof RouteConfigLoadStart) {
this.configLoadActionId = this.trackingService.startTracking('configLoad');
}
else if (event instanceof RouteConfigLoadEnd) {
const result = this.trackingService.stopTracking(this.configLoadActionId);
// ... process the result if you wish
}
else if (event instanceof NavigationStart) {
this.navigationActionId = this.trackingService.startTracking('navigation');
}
else if (event instanceof NavigationEnd) {
const result = this.trackingService.stopTracking(this.navigationActionId);
// ... process the result if you wish
}
});
}
Or for HTTP requests:
// http-tracking.interceptor
export class HttpTrackingInterceptor implements HttpInterceptor {
constructor(private trackingService: TrackingService) {}
intercept(req: HttpRequest<any>, next: HttpHandler): Observable<HttpEvent<any>> {
const actionId = this.trackingService.startTracking('HTTP request');
return next.handle(req.clone()).pipe(
tap(r => this.trackingService.stopTracking(actionId))
);
}
}
// app.module.ts
#NgModule({
// ... other module stuff
providers: [
// ... other providers
{
provide: HTTP_INTERCEPTORS,
useClass: HttpTrackingInterceptor,
multi: true,
deps: [TrackingService]
}
]
})
export class AppModule { ... }
You can easily extend the TrackingService to return Promises or Observables or whatever else, in case you prefer that...
Hope this helps a little :-)

Can we create a plugin so that we can use it everywhere instead of
writing same code everywhere to track the time of user. Any approach
to create it? Or is there any tool available to achieve this feature?
It's a very important Feature Request by many. So, I write a detailed, working and simple solution on the subject here.
#himanshu-garg You are requesting a feature already created for this workflow. It's a plugin you can include in any website. It's none other than activity tracking in timeonsite.js
Look at the following code,
<head>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/timeonsite/1.2.0/timeonsitetracker.js"></script>
<script>
var config = {
// track page by seconds. Default tracking is by milliseconds
trackBy: 'seconds',
callback: function(data) { /* callback denotes your data tracking is real-time */
console.log(data);
var endPointUrl = 'http://example.com' //Replace with your actual backend API URL http://localhost/tos
if (data && data.trackingType) {
if (data.trackingType == 'tos') {
if (Tos.verifyData(data) != 'valid') {
console.log('Data abolished!');
return;
}
}
// make use of sendBeacon if this API is supported by your browser.
if (navigator && typeof navigator.sendBeacon === 'function') {
data.trasferredWith = 'sendBeacon';
var blob = new Blob([JSON.stringify(data)], {type : 'application/json'});
navigator.sendBeacon(endPointUrl, blob);
}
}
}
};
var Tos;
if (TimeOnSiteTracker) {
Tos = new TimeOnSiteTracker(config);
}
</script>
</head>
Then, when the user clicks on a specific action in the site, for example "edit the post" or "click on the create post",
You just initiate the Tos.startActivity() API like,
Tos.startActivity({actionPerfomed: 'Edit a post'});
Then when the user completes the edit or create post actions and when he finally clicks the "save/submit" button, you trigger the Tos.endActivity() API like,
Tos.endActivity({customData: 'custom data if any here...'});
You'll see following object directly saved into your table,
{
TOSId: 585872449448,
TOSSessionKey: "14802525481391382263",
TOSUserId: "anonymous",
title: "Test application - TimeOnSiteTracker",
URL: "http://example.com/post/nature-is-beautiful/edit.php",
activityStart: "2021-11-27 13:20:46.707",
activityEnd: "2021-11-27 13:20:50.213",
timeTaken:4,
timeTakenByDuration: "0d 00h 00m 04s"
timeTakenTrackedBy: "second",
trackingType: "activity",
actionPerfomed: "Edit a post", //optional fields
customData: "custom data if any here..." //optional fields
}
As you can see, the actions
"Edit/Create post" is captured
"timeTaken" is captured in seconds/milliseconds depending upon configuration
"type:activity" is captured
"activityStart" is captured
"activityEnd" is captured
"TOSUserId" // who does the action along with TOSSessionKey to uniquely identify the session.
What else you need? Since it's stored in SQL DB table, you can do analysis/reporting queries yourself and take it to top-level management for decisions. The same is the case for NoSQL as well. Timeonsite.js is supporting both RDBMS and NoSql DB types.
On top of it, 1.Minimize tab, 2.Inactive tab and 3.Switch tab's idle time are all computed and ignored automatically by the tracker itself.
This tracker can be plugged-in in any library Angular, React, Jquery etc. since it's plain vanilla JS library.
Let me know if you need more input on the subject. I can assist you on this.

You have to write a simple Event Tracker in your client code. Since I don't know which events you want to track, I'll provide the solution for a general case.
Also, you'll have to manually trigger the start and stop tracking.
EventTracker = {
trackedEvents: {},
start: function(key) {
var startTime = new Date();
this.trackedEvents[key] = {
start: startTime
}
},
stop: function(key) {
var endTime = new Date();
this.trackedEvents[key]['duration'] = (endTime - this.trackedEvents[key]['start']) / 1000 + 's';
this.trackedEvents[key]['end'] = endTime;
},
}
// Use EventTracker everywhere to track performance
// Example:
EventTracker.start('search_track'); // User searches, start tracking.
setTimeout(function() {
EventTracker.stop('search_track'); // Records fetched after 5 seconds. Stop tracking.
console.log(EventTracker.trackedEvents);
}, 5000);
You can track all events according to your need. For server response, use: EventTracker.start('search_ajax_track') when you make the request and stop the tracking when you get the response.
You can modify above code to measure other parameters according to your requirements.

I am going to recommend you use custom Google Analytics events. In particular User Timings. This allows you to log specific timings on your webpage, you can log with your own labels and categories.
To quote the documentation:
User timings allow developers to measure periods of time using the
analytics.js library. This is particularly useful for developers to
measure the latency, or time spent, making AJAX requests and loading
web resources.
I have some sample code below, this just hooks into clicks, and will get a descriptor from attribute data-name - if not available will just log as 'Anonymous Click' - you can customise this to not track unmarked items. You can also hook into ajax calls and other notable events, without knowing your specific requirements it's hard to give further examples.
Example markup helper to lock click events.
<button data-name="Foo"/>
The below code does the logging, note that it logs using window.performance.now() - which will return the time from when the page was loaded in milliseconds. This will allow you to generate a timeline of user interactions as opposed to getting raw time spent on a single task, which by the way Google Analytics reports can calculate for you.
(function($, Analytics) {
init_hooks();
function init_hooks() {
$('body').on('click', track);
}
function track(e) {
// Get a name to record this against
var name = e.target.data(name) || "Anonymous Click";
// Time since page loaded
var time = window.performance.now()
Analytics('send', {
hitType: 'timing',
timingCategory: 'Front End Intereactions',
timingVar: name,
timingValue: time
});
}
})(jQuery, ga)
Find out more look at the docs.

You could instrument your code with OpenTracing for Js.
You will need to add a request in your transaction start and end.
Also a OpenTracing server to receive request from the browser.

How do I ensure that a call to subscribe on the observer initially receives the most recent value?

I am using rxjs together with Angular 2 and Typescript. I would like to share a common web-resource (a "project" in the context of my app, essentially a JSON document) between multiple components. To achieve this I introduced a service that exposes an observable, which will be shared by all clients:
/**
* Handed out to clients so they can subscribe to something.
*/
private _observable : Observable<Project>;
/**
* Used to emit events to clients.
*/
private _observer : Observer<Project>;
constructor(private _http: Http) {
// Create observable and observer once and for all. These instances
// are not allowed to changed as they are passed on to every subscriber.
this._observable = Observable.create( (obs : Observer<Project>) => {
this._observer = obs;
});
}
Clients now simply get a reference to that one _observable and subscribe to it.
/**
* Retrieves an observable that always points to the active
* project.
*/
get ActiveProject() : Observable<Project> {
return (this._observable);
}
When some component decides to actually load a project, it calls the following method:
/**
* #param id The id of the project to set for all subscribers
*/
setActiveProject(id : string) {
// Projects shouldn't change while other requests are in progress
if (this._httpRequest) {
throw { "err" : "HTTP request in progress" };
}
this._httpRequest = this._http.get('/api/project/' + id)
.catch(this.handleError)
.map(res => new Project(res.json()));
this._httpRequest.subscribe(res => {
// Cache the project
this._cachedProject = res;
// Show that there are no more requests
this._httpRequest = null;
// Inform subscribers
this._observer.next(this._cachedProject)
console.log("Got project");
});
}
It basically does a HTTP request, transforms the JSON document into a "proper" instance and calls this._observer.next() to inform all subscribers about the change.
But if something subscribes after the HTTP request has already taken place, the see nothing until a new HTTP request is issued. I have found out that there is some kind of caching (or replay?) mechanism in rxjs that seems to adress this, but I couldn't figure out how to use it.
tl;dr: How do I ensure that a call to subscribe on the observer initially receives the most recent value?
Extra question: By "pulling the observer out of the observable" (in the constructor), have I essentially created a subject?

That's what BehaviorSubject does
import { BehaviorSubject } from 'rxjs/subject/BehaviorSubject';
...
obs=new BehaviourSubject(4);
obs.subscribe(); //prints 4
obs.next(3); //prints 3
obs.subscribe(); //prints 3

I usually achieve this with shareReplay(1). Using this operator with 1 as parameter will ensure that the latest value emitted will be kept in a buffer, so when there is a new subscriber that value is immediately passed on to it. You can have a look at the documentation :
var interval = Rx.Observable.interval(1000);
var source = interval
.take(4)
.doAction(function (x) {
console.log('Side effect');
});
var published = source
.shareReplay(3);
published.subscribe(createObserver('SourceA'));
published.subscribe(createObserver('SourceB'));
// Creating a third subscription after the previous two subscriptions have
// completed. Notice that no side effects result from this subscription,
// because the notifications are cached and replayed.
Rx.Observable
.return(true)
.delay(6000)
.flatMap(published)
.subscribe(createObserver('SourceC'));
function createObserver(tag) {
return Rx.Observer.create(
function (x) {
console.log('Next: ' + tag + x);
},
function (err) {
console.log('Error: ' + err);
},
function () {
console.log('Completed');
});
}
// => Side effect
// => Next: SourceA0
// => Next: SourceB0
// => Side effect
// => Next: SourceA1
// => Next: SourceB1
// => Side effect
// => Next: SourceA2
// => Next: SourceB2
// => Side effect
// => Next: SourceA3
// => Next: SourceB3
// => Completed
// => Completed
// => Next: SourceC1
// => Next: SourceC2
// => Next: SourceC3
// => Completed
Extra question: By "pulling the observer out of the observable" (in
the constructor), have I essentially created a subject?
I am not sure what you mean by that, but no. A subject is both an observer and an observable and have specific semantics. It is not enough to 'pull the observer out of the observable' as you say. For subjects semantics, have a look here : What are the semantics of different RxJS subjects?