How to avoid data duplication while fetching? - javascript

I have a simple fetch function that gets data (messages from db) and putting it into an array to display it with simple vanilla JS. The thing is I am calling this function every 2 seconds in order to check for new messages. But when I do that I duplicate my messages and it keeps adding instead of replacing. I am struggling to understand what I should do to change, not add.
(a little dummy question, sorry)
const list = document.getElementById('message-list');
const getData = () => {
fetch('/messages')
.then((resp) => resp.json())
.then(function(data) {
console.log(data)
for (let i = 0; i < data.length; i++) {
const listItem = document.createElement('li');
listItem.innerText = data[i].message;
const delButton = document.createElement('button');
delButton.innerHTML = 'Delete';
delButton.addEventListener('click', ()=>{
const message_id = data[i].message_id;
deleteItem(message_id);
})
listItem.appendChild(delButton);
list.appendChild(listItem)
}
})
}
setInterval(getData,2000)

Make a Set of the message_ids processed so far, and on further calls, ignore messages matching that message_id:
const seenIds = new Set();
const getData = () => {
fetch('/messages')
.then((resp) => resp.json())
.then(function(data) {
data
.filter(({ message_id }) => !seenIds.has(seenIds))
.forEach(({ message, message_id }) => {
seenIds.add(message_id);
const listItem = document.createElement('li');
listItem.innerText = message;
const delButton = document.createElement('button');
delButton.textContent = 'Delete';
delButton.addEventListener('click', () => {
deleteItem(message_id);
});
listItem.appendChild(delButton);
list.appendChild(listItem)
});
});
};
That said, it would probably be better to change your backend so that it can filter the items for you, rather than sending objects over the net that proceed to get ignored.

Related

my html created html in my to do list keeps disapering when i reload the page

i know that the problem is that let todoList is an empty array, but i dont know how to solve it.
the id tags in my created html is so e can create a delete button later
heres my code:
const textArea = document.querySelector("textarea");
const button = document.querySelector("button");
const listContainer = document.querySelector(".list-container");
let id = 0;
let todoList = [];
button.onclick = function () {
const listItem = {
title: textArea.value,
};
todoList.push(listItem);
addToStorage(todoList);
const dataFromStorage = getFromStorage();
createHtml(dataFromStorage);
};
function addToStorage(items) {
const stringify = JSON.stringify(items);
localStorage.setItem("list", stringify);
}
function getFromStorage() {
const data = localStorage.getItem("list");
const unstrigified = JSON.parse(data);
return unstrigified;
}
const createHtml = (data) => {
id++;
listContainer.innerHTML = "";
data.forEach((item) => {
listContainer.innerHTML += `<div class="list-item" data-id=${id}><p>${item.title} </p><button class="remove" data-id=${id}>Delete</button></div>`;
});
};
The problem here is you just forgot to load the data from localStorage when the page loaded like this
window.onLoad = () => {
const dataFromStorage = getFromStorage();
if(dataFromStorage){
createHtml(dataFromStorage);
} else {
createHtml([]);
}
}
The problem in the code is as follows
Initially the todolist will be an empty array. so when you do the below
todoList.push(listItem);
// adding to local storage which will override the existing todos when page is refreshed
addToStorage(todoList);
// So when the below line is executed only the latest todo will be returned
const dataFromStorage = getFromStorage();
createHtml(dataFromStorage);
Fix:
Initialise the todos from localstorage instead of an empty array
let todoList = [];
// change it as below
let todoList = getFromStorage();
Now Modify the getFromStorage() as below
// If the data is present in the localStorage then return it, else return empty array
function getFromStorage() {
const data = localStorage.getItem("list");
if (!data) return [];
const unstrigified = JSON.parse(data);
return unstrigified;
}
Now when the page is loaded, we need to display the todos. Add the below lines of code
window.onload = function () {
createHtml(todoList);
};
That's it. This will fix the issue.
Few minor improvements can be made as well.
todoList.push(listItem);
addToStorage(todoList);
const dataFromStorage = getFromStorage(); // this line is not necessary, remove it
createHtml(dataFromStorage); // change this to createHtml(todoList)
Codepen
Thanks.

How to change the contents of a class construcor in a javascript class and access it later?

Edit: I have a demo here
I want to fetch data from an API, add it to the constructor of a class, and then on button Click display it to the page.
class Entry {
constructor() {
this.pages = [];
}
loadAllEntries(url) {
fetch(url)
.then((response) => response.json())
.then((data) => {
this.pages = data;
// this would do it autuomatically, it's not what I want
//this.displayAllEntries("#notes-list");
});
}
displayAllEntries(node) {
let ul = document.createElement("ul");
ul.id = "display-list-note";
this.pages.map((page) => {
let li = document.createElement("li");
li.innerText = page.title;
ul.appendChild(this.createClickableItem(page));
});
document.querySelector(node).appendChild(ul);
}
createClickableItem(item) {
let li = document.createElement("li");
li.innerText = item.title;
li.addEventListener("click", () => {
console.log(item.id);
});
return li;
}
}
const courses = new Entry();
courses.loadAllEntries("http://localhost:1338/courses?favorite=true");
// on some click I want to run
courses.displayAllEntries("#notes-list");
But, displayAllEntries acts as if loadAllEntries never ran!
later on I want to do something like courses.deleteEntry(213) and change the contents of the constructor, (a glorified todo list type of thing)
Edit: I have a demo here
The smallest touch that will fix this is as follows:
return the promise from the fetch
loadAllEntries(url) {
// just return, no further changes required
return fetch(url)
.then(...
At the top level, carry on after the fetch is done with then
const courses = new Entry();
const url = "http://localhost:1338/courses?favorite=true";
courses.loadAllEntries(url).then(() => {
courses.displayAllEntries("#notes-list");
});
If the underlying methods work, this will work.

Receiving data from Firestore with pure JavaScript

this question is probably really easy but I kind of got stuck with Firestore. It's my first project with firestore + js and I'm trying to receive and display data from my database. I have a feeling that I am doing something really stupid because it seems like I am looping through my data and overriding it that's why I can see only one article element even though there should be 2 fetched from the database.
document.addEventListener('DOMContentLoaded', event => {
const app = firebase.app();
const db = firebase.firestore();
const myProducts = db.collection('products');
myProducts.onSnapshot(products => {
const productsContainer = document.querySelector('#products__container');
products.forEach(doc => {
data = doc.data();
console.log(data);
let productMarkup = `<article id="${doc.id}">
<h4>${data.name}</h4>
<p>${data.price}</p>
</article>`;
productsContainer.innerHTML = productMarkup;
console.log(productMarkup);
});
});
});
After Alex suggestion, I decided to take the different approach to create DOM elements
const db = firebase.firestore();
const myProducts = db.collection('products');
const productsContainer = document.querySelector('#products__container');
function renderProduct(doc) {
const docFrag = document.createDocumentFragment();
let article = document.createElement('article');
let productName = document.createElement('h4');
let productPrice = document.createElement('p');
article.setAttribute('id', doc.id);
productName.textContent = doc.data().name;
productPrice.textContent = doc.data().price;
docFrag.appendChild(productName);
docFrag.appendChild(productPrice);
article.appendChild(docFrag);
productsContainer.appendChild(article);
}
myProducts.onSnapshot(products => {
products.forEach(doc => {
data = doc.data();
console.log(data);
renderProduct(doc);
});
});
});```

Write out api object from fetch to html

I'm trying to get my code to output an api object to the html file.
const container = document.createElement('div');
container.setAttribute('class', 'container');
obj = fetch('https://apis.is/concerts')
.then(function(response) {
return response.json();
})
.then(function(data) {
return obj = data;
})
.then(() => idk())
function idk() {
let count = 0;
for(key in obj.results) {
count++;
};
console.log(count);
for(let i = 0; i < count; i++) {
const card = document.createElement('div');
card.setAttribute('class', 'card');
const h1 = document.createElement('h1');
h1.textContent = obj.results[i].eventDateName;
const p = document.createElement('p');
p.textContent = obj.results[i].dateOfShow;
container.appendChild(card);
card.appendChild(h1);
card.appendChild(p);
};
};
I have been trying to use DOM to create elements for the html file but it's like some of the code is being ignored.
If you want to render all the DOM you are creating you have to somehow add it to the DOM tree that browser is displaying. The simples way would be to add it to body node. document.querySelector('body').appendChild(container); once you're done with data processing.
But I would suggest to refactor your code a bit. For instance in this step you are assigning results to the original object where you are saving the promise with the results. Also that is a global object so pretty quick you might end up with a race condition.
.then(function(data) {
return obj = data;
})
Also the idk() function is coupled to that very specific variable obj which would make it really hard to test.
obj = fetch('https://apis.is/concerts')
.then(function(response) {
return response.json(); //subscribe to response stream
})
.then((response) => {
const allEvents = eventsDomTree(response.results); // create the events DOM tree based on response
document.querySelector('body').appendChild(allEvents); //append the created list to document DOM tree
});
function eventsDomTree(events) {
const allEvents = document.createElement('div');
events.forEach((event) => {
const card = document.createElement('div');
card.setAttribute('class', 'card');
const eventName = document.createElement('h1');
eventName.textContent = event.eventDateName;
const dateOfShow = document.createElement('p');
dateOfShow.textContent = event.dateOfShow
card.appendChild(eventName);
card.appendChild(dateOfShow);
allEvents.appendChild(card);
});
return allEvents;
}

Web scraper iterating over pages with Rx.js

About a month ago I built this web scraper using Async / Await as a async way of collecting info for a web scraper. I'm trying to build that very same scraper again using Rx.js. I've read through the docs and it seems to make sense, starting off is the hardest bit, but after that hump I made some progress.
You can see here that I get the first page on the site (page 0) and I need to use that page to get the count of pages (which is around 6000). I have that count and using the getPageURI(page) I can create each page URL, however my issue is that I can't figure out how to trigger, or fire, or pipe information back to the original pageRequestStream. I have this page count number and I need a way to iterate over it pushing data back to the first original pageRequestStream stream.
import cheerio from 'cheerio'
import Rx from 'rx'
import fetch from 'isomorphic-fetch'
const DIGITAL_NYC_URI = 'http://www.digital.nyc'
let getPageURI = (page) => `${DIGITAL_NYC_URI}/startups?page=${page}`
let getProfileURI = (profile) => `${DIGITAL_NYC_URI}${profile}`
function fetchURL(stream, dataType = 'json') {
return stream.flatMap(requestURL => {
return Rx.Observable.fromPromise(fetch(requestURL).then(res => res[dataType]()))
})
}
function getNumberOfPages($) {
let summary = $('.result-summary').text()
let match = summary.match(/Showing 1 - 20 of (\d+) Startups/)
return parseInt(match[1], 10)
}
function getCompaniesOnPage ($) {
let companySelector = 'h3.node-title a'
let companies = $(companySelector).map(function (i, el) {
let name = $(this).text()
let profile = $(this).attr('href')
return {
'name': name,
'profile': profile
}
}).get()
return companies
}
let pageRequestStream = Rx.Observable.just(getPageURI(0))
let pageResponseStream = fetchURL(pageRequestStream, 'text')
let parsedPageHTMLStream = pageResponseStream.map(html => cheerio.load(html))
let numberOfPagesStream = parsedPageHTMLStream.map(html => getNumberOfPages(html))
// not sure how to get this to iterate over count and fire url's into pageRequestStream
numberOfPagesStream.subscribe(pageCount => console.log(pageCount))
let companiesOnPageStream = parsedPageHTMLStream.flatMap(html => getCompaniesOnPage(html))
// not sure how to build up the company object to include async value company.profileHTML
companiesOnPageStream.subscribe(companies => console.log(companies))
// let companyProfileStream = companiesOnPageStream.map((company) => {
// return fetch(getProfileURI(company.profile))
// .then(res => res.html())
// .then(html => {
// company.profileHTML = html
// return company
// })
// })
Have a look at subjects, they allow you to fire events as you go.
Maybe this can serve as some inspiration
import cheerio from 'cheerio';
import Rx from 'rx';
import fetch from 'isomorphic-fetch';
function getCheerio(url) {
var promise = fetch(url)
.then(response => response.text())
.then(body => cheerio.load(body));
return Rx.Observable.fromPromise(promise);
}
const DIGITAL_NYC_URI = 'http://www.digital.nyc';
var pageRequest = new Rx.Subject();
pageRequest
.flatMap(pageUrl => getCheerio(pageUrl))
.flatMap(page$ => {
// here we pipe back urls into our original observable.
var nextPageUrl = page$('ul.pagination li.arrow a').attr('href');
if(nextPageUrl) pageRequest.onNext(DIGITAL_NYC_URI + '/' + nextPageUrl);
var profileUrls = page$('h3.node-title a')
.map(function() {
var url = page$(this).attr('href');
return DIGITAL_NYC_URI + '/' + url;
});
return Rx.Observable.from(profileUrls);
})
.flatMap(url => getCheerio(url))
.map(profile$ => {
// build the company profile here
return profile$('title').text();
})
.subscribe(value => console.log('profile ', value));
pageRequest.onNext(DIGITAL_NYC_URI + '/startups');

Categories