What I have been trying to do is hit an endpoint for my blog posts and then with this data remove extra layout markup that came in from Wordpress. I am using Axios to make the request and then transform response option in order to modify the data to remove the extra markup from the "post_body" object inside my response. This works on a single blog post but when I try to do this all my blog blog posts it return an object of 20 or so blog posts. What I want to do is loop through the objects transform my data and then make a post request back to another API to publish my blog post. What I can't figure out if this will be possible once my promise is resolved. Would I be able to create another for loop within the .then and find my "post_body" object and make the post request. Not sure if I am thinking about this in the right way or not. Any help is much appreciated.
var fieldName = "et_pb";
var regExp = new RegExp("\\[\/?(" + fieldName + ".*?)\\]", "g");
function getBlogPosts() {
return axios.get(allPosts, {
transformResponse: axios.defaults.transformResponse.concat(function(data, headers) {
// use data I passed into the function and the objects from the API
// pass in data into the function using forEach this will return an array
data.objects.forEach(function(i) {
// use the returned array on Objects.key to find the name of the array
Object.keys(i).forEach(function(k) {
// if the key equals execute code
// console.log(k);
if (k === "post_body") {
// fire Regex
data[k] = i[k].replace(regExp, '');
// console.log(data[k])
}
})
})
return data;
})
})
}
axios.all([getBlogPosts()])
.then(axios.spread(function(blogResponse) {
console.log(blogResponse.data);
}));
#James you are correct . you can chain multiple requests as below or you can go for asyn and await options .
axios.get(...) // for allPosts
.then((response) => {
return axios.get(...); // using response.data
})
.then((response) => {
console.log('Response', response);
});
Related
Background: Been trying for the last 2 day to resolve this myself by looking at various examples from both this website and others and I'm still not getting it. Whenever I try adding callbacks or async/await I'm getting no where. I know this is where my problem is but I can't resolve it myself.
I'm not from a programming background :( Im sure its a quick fix for the average programmer, I am well below that level.
When I console.log(final) within the 'ready' block it works as it should, when I escape that block the output is 'undefined' if console.log(final) -or- Get req/server info, if I use console.log(ready)
const request = require('request');
const ready =
// I know 'request' is deprecated, but given my struggle with async/await (+ callbacks) in general, when I tried switching to axios I found it more confusing.
request({url: 'https://www.website.com', json: true}, function(err, res, returnedData) {
if (err) {
throw err;
}
var filter = returnedData.result.map(entry => entry.instrument_name);
var str = filter.toString();
var addToStr = str.split(",").map(function(a) { return `"trades.` + a + `.raw", `; }).join("");
var neater = addToStr.substr(0, addToStr.length-2);
var final = "[" + neater + "]";
// * * * Below works here but not outside this block* * *
// console.log(final);
});
// console.log(final);
// returns 'final is not defined'
console.log(ready);
// returns server info of GET req endpoint. This is as it is returning before actually returning the data. Not done as async.
module.exports = ready;
Below is an short example of the JSON that is returned by website.com. The actual call has 200+ 'result' objects.
What Im ultimately trying to achieve is
1) return all values of "instrument_name"
2) perform some manipulations (adding 'trades.' to the beginning of each value and '.raw' to the end of each value.
3) place these manipulations into an array.
["trades.BTC-26JUN20-8000-C.raw","trades.BTC-25SEP20-8000-C.raw"]
4) export/send this array to another file.
5) The array will be used as part of another request used in a websocket connection. The array cannot be hardcoded into this new request as the values of the array change daily.
{
"jsonrpc": "2.0",
"result": [
{
"kind": "option",
"is_active": true,
"instrument_name": "26JUN20-8000-C",
"expiration_timestamp": 1593158400000,
"creation_timestamp": 1575305837000,
"contract_size": 1,
},
{
"kind": "option",
"is_active": true,
"instrument_name": "25SEP20-8000-C",
"expiration_timestamp": 1601020800000,
"creation_timestamp": 1569484801000,
"contract_size": 1,
}
],
"usIn": 1591185090022084,
"usOut": 1591185090025382,
"usDiff": 3298,
"testnet": true
}
Looking your code we find two problems related to final and ready variables. The first one is that you're trying to console.log(final) out of its scope.
The second problem is that request doesn't immediately return the result of your API request. The reason is pretty simple, you're doing an asynchronous operation, and the result will only be returned by your callback. Your ready variable is just the reference to your request object.
I'm not sure about what is the context of your code and why you want to module.exports ready variable, but I suppose you want to export the result. If that's the case, I suggest you to return an async function which returns the response data instead of your request variable. This way you can control how to handle your response outside the module.
You can use the integrated fetch api instead of the deprecated request. I changed your code so that your component exports an asynchronous function called fetchData, which you can import somewhere and execute. It will return the result, updated with your logic:
module.exports = {
fetchData: async function fetchData() {
try {
const returnedData = await fetch({
url: "https://www.website.com/",
json: true
});
var ready = returnedData.result.map(entry => entry.instrument_name);
var str = filter.toString();
var addToStr = str
.split(",")
.map(function(a) {
return `"trades.` + a + `.raw", `;
})
.join("");
var neater = addToStr.substr(0, addToStr.length - 2);
return "[" + neater + "]";
} catch (error) {
console.error(error);
}
}
}
I hope this helps, otherwise please share more of your code. Much depends on where you want to display the fetched data. Also, how you take care of the loading and error states.
EDIT:
I can't get responses from this website, because you need an account as well as credentials for the api. Judging your code and your questions:
1) return all values of "instrument_name"
Your map function works:
var filter = returnedData.result.map(entry => entry.instrument_name);
2)perform some manipulations (adding 'trades.' to the beginning of each value and '.raw' to the end of each value.
3) place these manipulations into an array. ["trades.BTC-26JUN20-8000-C.raw","trades.BTC-25SEP20-8000-C.raw"]
This can be done using this function
const manipulatedData = filter.map(val => `trades.${val}.raw`);
You can now use manipulatedData in your next request. Being able to export this variable, depends on the component you use it in. To be honest, it sounds easier to me not to split this logic into two separate components - regarding the websocket -.
I'm performing multiple task and each task is dependent on previous task execution. So in my example what I want is after getting all the Id, i should get their respective blob value and then finish the execution by storing it in a variable. I'm very new to javascript and angular, please help me out. Here's what I'm trying
//this method will get the response from the rest api
async getIDFromAssets(){
this.blobDataArray=[];
this.service.getAssetsData().subscribe(async (res: JSON) => {
//after getting the response I'm filtering through it to get sepcific Id using this.getFileId() method
this.getFileId(res).then((data)=>{
console.log("blob "+data)
})
})
}
//below method will get one Id at a time and will call another method to get it's blob value
async getFileId(res){
this.fileId = [];
Object.keys(res).forEach(keys => {
if (keys == 'emb') {
let responseValue = res[keys];
Object.keys(responseValue).forEach(async (keys1) => {
if (keys1 === 'file') {
let responseArray = responseValue[keys1];
for (let file of responseArray) {
let temp: string = file.metadata.contentType;
if (temp.startsWith('image')) {
//Here I'm getting id value 'file._id' and using that I'm calling another method 'getBlobData()' to get its blob value
let data=await this.getBlobData(file._id);
this.blobDataArray.push(data);
}
}
return this.blobDataArray
}
});
}
});
}
// method to get the blob value
async getBlobData(fileId){
this.articleDetailService.getBlobDataFromAssets(fileId).subscribe(async (res)=>{
let imageObj={
'id':fileId,
'blob':res
}
return imageObj;
})
}
You need to use RxJs to avoid the nested subscription to chain your calls, possible methods to use are mergeMap and filter
Please take a look at this answer here.
Background
I'm trying to construct an observable stream of values from the Stash Rest Api of pull requests. Unfortunately, the information of whether or not a PR has merge conflicts is available at a different endpoint to the list of merges.
The list of open pull requests is visible at, say, http://my.stash.com/rest/api/1.0/projects/myproject/repos/myrepo/pull-requests
For each PR, the data on merge conflicts is visible at http://my.stash.com/rest/api/1.0/projects/myproject/repos/myrepo/pull-requests/[PR-ID]/merge
Using the atlas-stash package, I can create and subscribe to an observable stream of pull requests (updated every second):
let pullRequestsObs = Rx.Observable.create(function(o) {
stash.pullRequests(project, repo)
.on('error', function(error) {o.onError(error)})
.on('allPages', function(data) {
o.onNext(data);
o.onCompleted();
});
});
let pullRequestStream = pullRequestsObs
.take(1)
.merge(
Rx.Observable
.interval(1000)
.flatMapLatest(pullRequestsObs)
);
pullRequestsStream.subscribe(
(data) => {
console.log(data)
// do something with data
},
(error) => log.error(error),
() => log.info('done')
);
This works as I want and expect. In the end, the pullRequestsStream is an observable whose values are lists of JSON objects.
My Goal
I would like the pullRequestsStream values to be updated so every element of the list includes information from the [PR-ID]/merge api.
I assume that this can be achieved using a map on pullRequestsStream, but I'm not succeeding in doing this.
let pullRequestWithMergeStream = pullRequestStream.map(function(prlist) {
_.map(prlist, function(pr) {
let mergeObs = Rx.Observable.create(function(o) {
stash.pullRequestMerge(project, repo, pr['id'])
.on('error', function(error) {o.onError(error)})
.on('newPage', function(data) {
o.onNext(data);
o.onCompleted();
}).take(1);
});
mergeObs.subscribe(
(data) => {
pr['merge'] = data;
return pr; // this definitely isn't right
},
(error) => log.error(error),
() => log.info('done')
);
});
});
With a bit of logging, I can see that both the pull-request and the merge apis are being hit correctly, but when I subscribe to pullRequestWithMergeStream I
get undefined values.
Using return within the the subscribe step within a map doesn't work (and doesn't seem like it should) but I can't figure out what pattern/idiom would achieve what I want.
Is there a correct way of doing this? Have I gone completely down the wrong track?
tl;dr
Can I update values from an Rxjs.Observable with information from a different observable?
You could use flatMap or concatMap to have one task trigger another one. You could use forkJoin to request the merges in parallel and collect the result in one place. It is not tested, but it should go like this :
pullRequestStream.concatMap(function (prlist){
var arrayRequestMerge = prlist.map(function(pr){
return Rx.Observable.create(function(o) {...same as your code});
});
return Rx.Observable.forkJoin(arrayRequestMerge)
.do(function(arrayData){
prlist.map(function(pr, index){pr['merge']=arrayData[index]
})})
.map(function(){return prlist})
})
PS : I supposed prlist was an array.
UPDATE
Following your comment, here is a version that will run only maxConcurrent calls in parallels.
pullRequestStream.concatMap(function (prlist){
var arrayRequestMerge = prlist.map(function(pr, index){
return Rx.Observable.create(function(o) {
stash.pullRequestMerge(project, repo, pr['id'])
.on('error', function(error) {o.onError(error)})
.on('newPage', function(data) {
o.onNext({data: data, index : index});
o.onCompleted();
}).take(1);
});
});
var maxConcurrent = 2;
Rx.Observable.from(arrayRequestMerge)
.merge(maxConcurrent)
.do(function(obj){
prlist[obj.index]['merge'] = obj.data
})})
.map(function(){return prlist})
})
I'm using this Gumroad-API npm package in order to fetch data from an external service (Gumroad). Unfortunately, it seems to use a .then() construct which can get a little unwieldy as you will find out below:
This is my meteor method:
Meteor.methods({
fetchGumroadData: () => {
const Gumroad = Meteor.npmRequire('gumroad-api');
let gumroad = new Gumroad({ token: Meteor.settings.gumroadAccessKey });
let before = "2099-12-04";
let after = "2014-12-04";
let page = 1;
let sales = [];
// Recursively defined to continue fetching the next page if it exists
let doThisAfterResponse = (response) => {
sales.push(response.sales);
if (response.next_page_url) {
page = page + 1;
gumroad.listSales(after, before, page).then(doThisAfterResponse);
} else {
let finalArray = R.unnest(sales);
console.log('result array length: ' + finalArray.length);
Meteor.call('insertSales', finalArray);
console.log('FINISHED');
}
}
gumroad.listSales(after, before, page).then(doThisAfterResponse); // run
}
});
Since the NPM package exposes the Gumorad API using something like this:
gumroad.listSales(after, before, page).then(callback)
I decided to do it recursively in order to grab all pages of data.
Let me try to re-cap what is happening here:
The journey starts on the last line of the code shown above.
The initial page is fetched, and doThisAfterResponse() is run for the first time.
We first dump the returned data into our sales array, and then we check if the response has given us a link to the next page (as an indication as to whether or not we're on the final page).
If so, we increment our page count and we make the API call again with the same function to handle the response again.
If not, this means we're at our final page. Now it's time to format the data using R.unnest and finally insert the finalArray of data into our database.
But a funny thing happens here. The entire execution halts at the Meteor.call() and I don't even get an error output to the server logs.
I even tried switching out the Meteor.call() for a simple: Sales.insert({text: 'testing'}) but the exact same behaviour is observed.
What I really need to do is to fetch the information and then store it into the database on the server. How can I make that happen?
EDIT: Please also see this other (much more simplified) SO question I made:
Calling a Meteor Method inside a Promise Callback [Halting w/o Error]
I ended up ditching the NPM package and writing my own API call. I could never figure out how to make my call inside the .then(). Here's the code:
fetchGumroadData: () => {
let sales = [];
const fetchData = (page = 1) => {
let options = {
data: {
access_token: Meteor.settings.gumroadAccessKey,
before: '2099-12-04',
after: '2014-12-04',
page: page,
}
};
HTTP.call('GET', 'https://api.gumroad.com/v2/sales', options, (err,res) => {
if (err) { // API call failed
console.log(err);
throw err;
} else { // API call successful
sales.push(...res.data.sales);
res.data.next_page_url ? fetchData(page + 1) : Meteor.call('addSalesFromAPI', sales);
}
});
};
fetchData(); // run the function to fetch data recursively
}
At a loss on this one.
I'm using Ember and Ember data. I've got this extra implementation of ic-ajax to make GET, POST and PUT calls. Anyway, i'm trying to make a GET call then turn those results into model instances.
return this.GET('/editor')
.then((data) => {
return data.drafts.map((draftData) => {
let draft = this.store.find('draft',draftData.id);
console.log(draft.get('type'));
return draft;
});
});
My API returns proper data as data.drafts. This map is supposed to return an array of promises that resolve to draft models. It does not. It resolves to a draft model that has id, date, and title. But that's it. I have 25 others attributions.
In another part of the application i'm getting drafts using findAll on the model. And those models look fine. But when I try store.findRecord('draft',id) i get these fake objects.
-- edit
This is what my ReOpenClass method looks like for getting an array of objects from the server and turning them into ember objects
search(critera) {
let query = { search: critera };
let adapter = this.store.adapterFor('application');
let url = adapter.buildURL('article','search');
return adapter.ajax(url,'GET', { data: query }).then(response => {
let articleRecords = response.articles.map((article) => {
let record;
try {
record = this.store.createRecord('article', article);
} catch(e) {
record = this.store.peekRecord('article', article.id);
}
return record;
});
return articleRecords;
});
},
So far I can't find a better way to pull this off.