I am writing an API wrapper in TypeScript. I would like the code to be asynchronous in order to maximally meet the rate limit of the API in question. The API wants requests to be submitted at a maximum rate of 1/second.
I intend to implement an API wrapper which is instantiated once, and allows the use of objects to reach the different endpoints. For instance, within the greater API there is a post and pool endpoint. I would like to access them like post_object.post.submit_request(argument1, ...) or post_object.pool.submit_request(argument1, ...).
I have created an object called state_info which is passed between the various objects, within which is contained a user-agent header, login information if provided, and a rate-limiter object from the Bottleneck library.
The issue I'm running into while testing is that my program doesn't seem to actually be limiting the rate of requests; no matter what I change the limit to in the arguments for Bottleneck, the requests all happen in about .600 seconds every time.
I am thinking this has something to do with passing around the rate-limiter object, or in accessing it from multiple places, but I'm unsure.
First, here is the code for the Model object, which represents access into the API.
import axios, { AxiosRequestConfig } from "axios";
import { StateInfo, Method } from "./interfaces";
export class Model {
public stateInfo: StateInfo;
constructor(stateInfo: StateInfo) {
// Preserve rate limiter, user agent, etc.
this.stateInfo = stateInfo;
}
//Updated to funcName = () => {} syntax to bind "this" to this class context.
private submit_request = (query_url: string, method: Method) => {
if (this.stateInfo.username && this.stateInfo.api_key) {
const axiosConfig: AxiosRequestConfig = {
method: method,
url: query_url,
headers: { "User-Agent": this.stateInfo.userAgent },
auth: {
username: this.stateInfo.username,
password: this.stateInfo.api_key,
},
};
return axios(axiosConfig);
} else {
const axiosConfig: AxiosRequestConfig = {
method: "get",
url: query_url,
headers: { "User-Agent": this.stateInfo.userAgent },
};
return axios(axiosConfig);
}
};
public submit_throttled_request = (url: string, method: Method) => {
return this.stateInfo.rateLimiter.schedule(
this.submit_request,
url,
method
);
};
}
Then, the code from which I call this class:
import { Model } from "./models/model";
import Bottleneck from "bottleneck";
const limiter: Bottleneck = new Bottleneck({ mintime: 1000, maxconcurrent: 1 });
const stateInfo = {
rateLimiter: limiter,
userAgent: "email#website.com | API Dev",
};
let modelObj: Model = new Model(stateInfo);
async function makeRequest() {
try {
let response = await modelObj.submit_throttled_request(
"https://www.website.com/api",
"get"
);
console.log(response.data.id + "|" + Date.now());
} catch (err) {
console.log(err);
}
}
let start = new Date();
for (let i = 0; i < 20; i++) {
makeRequest();
}
My expectation is that the operation would take, at a minimum, 10 seconds if only one request can be submitted per second. Yet I'm averaging half that, no matter what I include for mintime.
I've learned the answer to my own question after much head-scratching.
It turns out, in the "gotchas" section of the bottleneck API reference they note:
If you're passing an object's method as a job, you'll probably need to bind() the object:
with the following code:
// instead of this:
limiter.schedule(object.doSomething);
// do this:
limiter.schedule(object.doSomething.bind(object));
// or, wrap it in an arrow function instead:
limiter.schedule(() => object.doSomething());
This is the issue into which I was running. I was handing off axios(axiosContext) without binding the scope, so nothing was being sent off to the bottleneck ratelimiter. By wrapping is like so: this.state_info.rateLimiter.schedule(() => axios(axiosContext)); I have managed to correctly bind the context as needed.
Related
I have some queries from an API-Server that returns a json object that will be static over a user session, but not static forever.
It's a one-pager with Vue router.
How can I achieve that I:
can access this.myGlobals (or similar eg window.myGlobals) in all components, where my prefetched json-data from API-Server is stored.
My approach that is already working is to embed help.js via a mixin.
Oddly enough, I get hundreds of calls to this query. At first I thought that it only happened in the frontend and is chached, but the requests are actually sent hundreds of times to the server. I think it is a mistake of my thinking, or a systematic mistake.
i think the problem is, that the helper.js is not static living on the vue instance
main.js:
import helpers from './helpers'
Vue.mixin(helpers)
helpers.js:
export default {
data: function () {
return {
globals: {},
}
}, methods: {
//some global helper funktions
},
}, mounted() {
let url1 = window.datahost + "/myDataToStore"
this.$http.get(url1).then(response => {
console.log("call")
this.globals.myData = response.data
});
}
}
log in console:
call
SomeOtherStuff
(31) call
SomeOtherStuff
(2) call
....
log on server:
call
call
call (pew pew)
My next idea would be to learn vuex, but since its a easy problem, im not sure if i really need that bomb ?
You can use plugin to achieve this.
// my-plugin.js
export default {
install (Vue, options) {
// start fetching data right after install
let url1 = window.datahost + "/myDataToStore"
let myData
Vue.$http.get(url1).then(response => {
console.log("call")
myData = response.data
})
// inject via global mixin
Vue.mixin({
computed: {
myData () {
return myData
}
}
})
// or inject via instance property
Vue.prototype.$myData = myData
// or if you want to wait until myData is available
Vue.prototype.$myData = Vue.$http.get(url1)
.then(response => {
console.log("call")
myData = response.data
})
}
}
and use it:
Vue.use(VueResource)
Vue.use(myPlugin)
This is the method I'm using, pretty simple.
DailyCountTest: function (){
this.$store.dispatch("DailyCountAction")
let NewPatientTest = this.$store.getters.NewPatientCountGET
console.log(NewPatientTest)
}
The getter gets that data from a simple action that calls a django backend API.
I'm attempting to do some charting with the data so I need to assign them to variables. The only problem is I can't access the variables.
This is what the console looks like
And this is what it looks like expanded.
You can see the contents, but I also see empty brackets. Would anyone know how I could access those values? I've tried a bunch of map.(Object) examples and couldn't get any success with them.
Would anyone have any recommendation on how I can manipulate this array to get the contents?
Thanks!
Here is the Vuex path for the API data
Action:
DailyCountAction ({ commit }) {
axios({
method: "get",
url: "http://127.0.0.1:8000/MonthlyCountByDay/",
auth: {
username: "test",
password: "test"
}
}).then(response => {
commit('DailyCountMutation', response.data)
})
},
Mutation:
DailyCountMutation(state, DailyCount) {
const NewPatientMap = new Map(Object.entries(DailyCount));
NewPatientMap.forEach((value, key) => {
var NewPatientCycle = value['Current_Cycle_Date']
state.DailyCount.push(NewPatientCycle)
});
}
Getter:
NewPatientCountGET : state => {
return state.DailyCount
}
State:
DailyCount: []
This particular description of your problem caught my eye:
The getter gets that data from a simple action that calls a django backend API
That, to me, implies an asynchronous action and you might be getting a race condition. Would you be able to post a sample of your getter function to confirm my suspicion?
If that getter does indeed rely on an action to populate its contents, perhaps something to the effect of the following might do?
DailyCountTest: async () => {
await this.$store.dispatch('DailyCountAction')
await this.$store.dispatch('ActionThatPopulatesNewPatientCount')
let NewPatientTest = this.$store.getters.NewPatientCountGET
// ... do whatever with resulting array
}
You can also try with a computer property. You can import mapGetters
import { mapGetters } from 'vuex'
and later in computed properties:
computed: {
...mapGetters(['NewPatientCountGET'])
}
then you can use your NewPatientCountGET and it will update whenever the value changes in the store. (for example when the api returns a new value)
Hope that makes sense
I have been struggling on how i can mock a function so that i can return a fake value from that function.
I have a simple script that can make a api call, however this api call has two parameters. One parameter is provided through the parameter of the parent function, the other one is provided by making a call to another function. The return value from this function is what i need to mock.
The full code is quite complex, that's why i made a small sample of what i mean. Firstable i have the function makeTheCall. Within that function i call a function called setParameters.
const setParams = require('setParams.js');
module.exports.makeTheCall = (event) => {
const params = setParams('GET', '/todos/1');
const postData = {
name: event.name,
location: event.location
}
console.log(params); //dynamic params 'method' and 'callpath' are both undefined here (should be 'GET' and '/todos/1')
return doARequest(params, postData).then((result) => {
return result;
}).catch((error) => {
return error;
})
}
The setParams function is not so difficult. It just returns an object that contains some static and some dynamic values.
module.exports.setParams = (method, callPath) => {
return {
host: 'jsonplaceholder.typicode.com',
port: 433,
method: method,
path: callPath
}
}
Now, here is where the interesting part comes into play. When write a simple test the call cannot go through. This is, of course, because it cannot resolve the dynamic values method and callPath.
const makeTheCall = require('makeTheCall.js');
it('runs a happy flow scenario', () => {
const event = {
name: 'John Doe',
location: 'Somewhere'
}
return makeTheCall(event)
.then(response => {
//Do some usefull testing here
});
});
My question is how i can mock the return value of the setParams method so that it will return something like:
{
host: 'jsonplaceholder.typicode.com',
port: 433,
method: 'GET',
path: '/todos/1'
}
This way i can invoke my API call in my test without it causing an error. I have been looking into mocking using sinon, particularly into sinon stubs, like:
const params = setParams('GET', '/todos/1');
sinon.stub(params).returns({
host: 'jsonplaceholder.typicode.com',
port: 433,
method: 'GET',
path: '/todos/1'
});
However i think i overlook something because this does not work. The documentation is nice but after a couple of hours struggling and trying things i start to feel kinda lost.
Who knows / can point me in the right direction on how to mock the return value of the setParams function? An example will be highly appreciated.
You're not calling sinon.stub quite right. stub() needs an object and a function that is a property of that object. If you import with:
const setParams = require('setParams.js');
Then setParams will be the modules.export object and setParams will be a property, so you can stub it with something like:
let fakeParam = {
host: 'jsonplaceholder.typicode.com',
port: 433,
method: 'GET',
path: '/todos/1'
}
let paramStub = sinon.stub(params, 'setParams').returns(fakeParam)
In the broader picture it's not really clear what you are trying to test. With unit tests you generally try to reduce everything down to one small thing you want to assert. So in this case maybe you want to assert that when you call makeTheCall doARequest is called with the params returned from setParams. In that case you might also stub doARequest. Then you can assert with sinon.calledWith(doARequestStubb, fakeParam). You can have doARequestStubb resolve with a promise to the code doesn't break.
I'm finding a solution to async computed method in Components:
Currently, my component is:
<div class="msg_content">
{{messages}}
</div>
<script>
export default {
computed: {
messages: {
get () {
return api.get(`/users/${this.value.username}/message/`, {'headers': { 'Authorization': 'JWT ...' }})
.then(response => response.data)
}
}
},
}
</script>
Result:
{}
How to rewrite it in Promise mode? Because I think we can async computed by writing into Promise mode.
Computed properties are basically functions that cache their results so that they don't have to be calculated every time they are needed. They updated automatically based on the reactive values they use.
Your computed does not use any reactive items, so there's no point in its being a computed. It returns a Promise now (assuming the usual behavior of then).
It's not entirely clear what you want to achieve, but my best guess is that you should create a data item to hold response.data, and make your api.get call in the created hook. Something like
export default {
data() {
return {
//...
messages: []
};
},
created() {
api.get(`/users/${this.value.username}/message/`, {
'headers': {
'Authorization': 'JWT ...'
}
})
.then(response => this.messages = response.data);
}
}
es7 makes doing this quite trivial by using async and await in conjunction with axios' returned promise. You'll need the vue-async-computed package.
export default {
asyncComputed: {
async myResolvedValue() {
return await api.get(`/users/${this.value.username}/message/`, {'headers': { 'Authorization': 'JWT ...' }})
.then(response => response.data)
}
}
}
I bumped on a similar case where I need to re-run the computed function to fetch from a server every time a data or a props changes.
Without installing any extra package (vue-async-computed or vue3-async-computed as pointed by other answers), you can force a data to reload by creating a "virtual" computed method.
Lets say you want to fetch data from the server every time the user types their username, and depending on what was typed, you want to show a given message from the server.
From the example below, username and messages in this case are both reactive data, but there is no direct connection between them, so lets create a computed that depends on username by returning its value, which will force it to be called every time username is changed. Now you just need to call a function that can be async and will update messages after fetching from the server.
In the example below, I use ":dummy" just to force a call to my computed function.
<template>
<input v-model="username">
<div class="msg_content" :dummy="force_react">
{{messages}}
</div>
</template>
<script>
export default {
data: function () {
return {
messages: "",
username: "",
};
},
computed: {
force_react: function() {
this.get(); // called every time that this.username updates
return this.username; // becase its result depends on username
}
},
methods: {
async get() { // get's called every time that this.username updates
console.log("Got called");
let response = await api.get(`/users/${this.username}/message/`, {'headers': { 'Authorization': 'JWT ...' }});
this.messages = response.data;
}
},
}
</script>
You can see a working example here in Vue SFC playground
Why not using a watcher? Vue Documentation
You can use async function in the callback of the watcher.
I'm trying to consume a REST webservice, responding with a JSON String containing a fairly "complex" schema.
I created a model that contains every fields sent by the webservice.
Here are the relevant codes that should be a problem :
public getUser(user_id: number): PlanDeCharge.Modeles.User {
var toto;
this.UserRest.get({ user_id: user_id }, function(){}, function(err){
this.$window.location.href = "http://localhost:8080/myapp_webapp/login.do";
}).$promise.then(function(data){
toto = data;
});
return toto;
}
-
this.userConnecte = this.gestionUserService.getUser(759);
-
export function userRest($resource: ng.resource.IResourceService, $cookies: ng.cookies.ICookiesService): PlanDeCharge.Modeles.IUserResource {
this.key = $cookies.get("encodedKey");
var urlService: string = "http://localhost:8080/teambox_webapp/resource-rest/V1_1/users/:user_id";
return <PlanDeCharge.Modeles.IUserResource> $resource(urlService, {user_id: "#user_id"}, {
get:{
headers:{"key" : this.key}
}
});
}
app.factory("UserRest", ["$resource", "$cookies", userRest]);
I did a lot of modifications, trying to fix the call without success... The request actually get a response containing the JSON string, but I can't put it inside an object to be use (like user['id'] = 2)
Thanks in advance
I deleted the last post and made this new one, the first one wasn't clear enough and people were confused
When working with promises you should let Angular handle the resolvement.
Am I right, if you are actually using AngularJS 1 and not ng2 as the question is tagged? The syntax is ng1 anyways.
Some notes on the getUser method. Return the reference created by $resource instead of creating one your self. Further more, use the fat-arrow syntax on the callbacks to bind this to the proper context. See this article for more on this.
To remove even more code use TypeScripts object initialization and init the user id object with just { user_id }. This creates a JavaScript object with a property user_id with the value of user_id.
public getUser(user_id: number): SomeModel {
return this.UserRest
.get({ user_id }, () => { }, () => {
this.$window.location.href = "http://localhost:8080/myapp_webapp/login.do";
});
}
In your component or controller access
this.userConnecte = this.gestionUserService.getUser(759);
Lastly, the factory/service.
Use the fact that $resource is generic and set your variables as constants when not changed.
export function userRest(
$resource: ng.resource.IResourceService,
$cookies: ng.cookies.ICookiesService
): ng.resource.IResourceClass<PlanDeCharge.Modeles.IUserResource> {
this.key = $cookies.get("encodedKey");
const urlService = "http://localhost:8080/teambox_webapp/resource-rest/V1_1/users/:user_id";
return $resource<PlanDeCharge.Modeles.IUserResource>(urlService, { user_id: "#user_id" }, {
get: {
headers: { "key": this.key }
}
});
}
This should fix your problems and make to code more readable. :)