We Keep Coding

Use timestamp in a key with useSWRInfinite()

I'm using useSWRInfinite() for data fetching in my React app, and on the first page I need to include the current timestamp as a cursor in the getKey function (as there's no previousPageData). I can't do it because on every millisecond the key is changing and invoking the fetcher function again. Any suggestion will sure be help!
Here is my code at the moment:
const useQuestions = () => {
const fetcher = async (url: string) => fetch(url).then((res) => res.json());
const timestamp = new Date().toISOString();
const getKey = (pageIndex: number, previousPageData: any) => {
if (previousPageData && !previousPageData.questions) return null;
const cursorQuery = `?cursor=${pageIndex ? previousPageData?.from : timestamp}`;
if (pageIndex && !previousPageData?.cursor) return null;
return `/api/questions${fromQuery}`;
};
const { data, size, setSize, error, isLoading, isValidating, mutate } = useSWRInfinite(getKey, fetcher, {
initialSize: 1,
revalidateAll: true,
revalidateFirstPage: false,
persistSize: true,
// I tried to set persistSize as true to prevent refetching when 1st page key is changing but no luck
});

The useMemeo hook might help you in this case. It should stop the key from changing at every millisecond.
const useQuestions = () => {
const fetcher = async (url: string) => fetch(url).then((res) => res.json());
const timestamp = useMemo(() => new Date().toISOString(), []);
const getKey = (pageIndex: number, previousPageData: any) => {
if (previousPageData && !previousPageData.questions) return null;
const cursorQuery = `?cursor=${pageIndex ? previousPageData?.from : timestamp}`;
if (pageIndex && !previousPageData?.cursor) return null;
return `/api/questions${fromQuery}`;
};
const { data, size, setSize, error, isLoading, isValidating, mutate } = useSWRInfinite(getKey, fetcher, {
initialSize: 1,
revalidateAll: true,
revalidateFirstPage: false,
persistSize: true,
});
};

How to .push() each worker's result into an array in node js?

I crushed onto a wall pretty violently. The thing is that I am calling a function in the main thread with the form:
const {StaticPool} = require("node-worker-threads-pool");
const friendsTableWorkers = (result) => {
const pool = new StaticPool({
size: 8,
task: "./worker.js",
workerData: result
})
const nums = [23, 25]
let buffer = []
const size = Int32Array.BYTES_PER_ELEMENT*nums.length
const sharedBuffer = new SharedArrayBuffer(size)
const sharedArray = new Int32Array(sharedBuffer)
nums.forEach((num, index) => {
Atomics.store(sharedArray, index, num);
})
pool.exec(sharedArray).then(res => {
buffer.push( res )
}).finally(()=>{
console.log( buffer )
})
return buffer
}
which, in turn, use a fixed pool of workers of the form:
const { parentPort, workerData } = require("worker_threads")
const _ = require('lodash')
const { idToEnsembl } = require('./supportFunctionsBackend.js')
function friendSeeker(n) {
console.log(n)
let res = workerData.map(x => x._doc)
return {
ensemblGeneId: idToEnsembl(n),
nodeidentifier: n,
geneSetFriends: _.countBy(res, (o) => {
return (o.nodes.includes(n))
}).true}
}
parentPort.on("message", (param) => {
param.forEach( friend => {
if (typeof friend !== "number") {
throw new Error("param must be a number.");
}
const result = friendSeeker(friend);
parentPort.postMessage(result);
})
})
But, when I run it, I get the following:
23
[
{
ensemblGeneId: 'ENSG00000000003',
nodeidentifier: 23,
geneSetFriends: 2249
}
]
25
When the real result should be:
[
{
ensemblGeneId: 'ENSG00000000003',
nodeidentifier: 23,
geneSetFriends: 2249
},
{
ensemblGeneId: 'ENSG00000000005',
nodeidentifier: 25,
geneSetFriends: 321
}
]
or
[
{
ensemblGeneId: 'ENSG00000000005',
nodeidentifier: 25,
geneSetFriends: 321
},
{
ensemblGeneId: 'ENSG00000000003',
nodeidentifier: 23,
geneSetFriends: 2249
}
]
Which means that just one worker is pushing his result into the buffer array. My question is, then, how can I make all the workers push their result into the buffer array? The order is not important.
Thank you very much in advance

TensorFlow - which model to build?

We are creating a meta-search for realEstate. I would like to predict the running time (time the listing will be online) based on certain metrics with tensorflow.js sequential model.
I have following data:
listing.buyingPrice,
listing.foreClosure ? 1 :0,
listing.leasehold ? 1 : 0,
listing.squareMeter,
listing.rooms,
listing.grossReturn,
listing.locationFactor.score
This is the code I'm using:
const createModel = () => {
// Create a sequential model
const model = tf.sequential();
// Add a single hidden layer
model.add(tf.layers.dense({inputShape: [shapeLength], units: 50, useBias: true}));
// Add an output layer
model.add(tf.layers.dense({units: 1, useBias: true}));
return model;
};
async function trainModel(model, inputs, labels) {
// Prepare the model for training.
const learningRate = 0.0001;
model.compile({
optimizer: tf.train.adam(learningRate),
loss: tf.losses.meanSquaredError,
metrics: ['mse'],
});
const batchSize = 32;
const epochs = 50;
return await model.fit(inputs, labels, {
batchSize,
epochs,
shuffle: true
});
}
const shapeLength=7;
const convertListing = (listing) => ([
listing.buyingPrice,
listing.foreClosure ? 1 :0,
listing.leasehold ? 1 : 0,
listing.squareMeter,
listing.rooms,
listing.grossReturn,
listing.locationFactor.score
]);
const convertToTensor =(listings) => {
// Wrapping these calculations in a tidy will dispose any
// intermediate tensors.
return tf.tidy(() => {
// Step 1. Shuffle the data
tf.util.shuffle(listings);
// Step 2. Convert data to Tensor
const inputs = listings.map(listing =>
convertListing(listing)
);
const labels = listings.map(listing => listing.runningTime);
const inputTensor = tf.tensor2d(inputs, [inputs.length, shapeLength]);
const labelTensor = tf.tensor2d(labels, [labels.length, 1]);
return {
inputs: inputTensor,
labels: labelTensor,
}
});
};
const modelCreated = createModel();
export const doTraining = async () => {
const listings = await findExactMatch({
rented:false,
active:false,
zip:'10243',
}, OBJECT_TYPES.APARTMENTBUY);
const filteredListings = listings.filter(listing =>
listing.runningTime>=0
&& listing.buyingPrice>=0
&& listing.runningTime <100
&& listing.rooms>0
&& listing.squareMeter >0
&& listing.grossReturn >0
&& listing.locationFactor.score>0
);
const tensorData = convertToTensor(filteredListings);
const {inputs, labels} = tensorData;
// Train the model
await trainModel(modelCreated, inputs, labels);
console.log('Done Training');
filteredListings.map(listing=>{
console.log(listing.runningTime)
});
for(let i=0;i<10;i++){
console.log('Real running time',filteredListings[i].runningTime)
console.log('Predicted runningTime',await testModel(filteredListings[i],tensorData));
}
};
export const testModel = async (listing) => {
const input = convertListing(listing);
const inputData = tf.tensor2d([input], [1, shapeLength]);
const result = modelCreated.predict(inputData);
return result.dataSync()[0];
}
I'm using roughly 3000 listings. Problem is, that the running time I get predicted is WAY off. What to do to improve that? Is my model even correct? :)

Remove a subscription

I'm using a library (Polidea-BLE || react-native-ble-plx) to connect to an external device and recover information.
Basically I read some informations from external device and then I should pass these informations to another page to wrote in the db.
My problem is that I don't understand how to stop the reading, because at the moment I read data and directly pass these to another page.
They said that:
monitorCharacteristicForService also returns subscription with
remove() function so setting transactionId is not necessary.
But I don't understand how to use.
This is my code:
async setupNotifications1(device) {
var timeagm = 0
var time = 0
const service = this.serviceGeneral();
await device.monitorCharacteristicForService(service,this.AccGyrMg, (error, characteristic) => {
if (error)
{
this.error(error.message);
return;
}
const buf = Buffer.from(characteristic.value, "base64");
const [...acc_dx] = [2, 4, 6].map(index => buf.readInt16LE(index));
this.setState(state => ({acc_dx,array_acc_dx: [...state.array_acc_dx,[timeagm, acc_dx]]
}));
timeagm += 20
}
);
await device.monitorCharacteristicForService(service,this.Pressure,(error, characteristic) => {
if (error)
{
this.error(error.message);
return;
}
const buf = Buffer.from(characteristic.value, "base64");
const [...pressure_dx] = [0, 2, 4, 6, 8].map(index => buf.readUInt16LE(index));
this.setState(state => ({pressure_dx,array_pressure_dx: [...state.array_pressure_dx,[time, pressure_dx] ]
}));
time += 20
}
);
and when the user click on a stopButton I pass the data without stop the reading ( at the moment and this is what I should correct)
stopConnection() {
console.log("Inizio stopConnection");
Actions.registerattivita(
{
array_acc_dx: this.state.array_acc_dx,
array_pressure_dx: this.state.array_pressure_dx,
}
)
}
In your opionion how can I do to use this remove() to stop the reading of data? Thank you.

this.subscriptionMonitor = device.monitorCharacteristicForService(...)
and then in stopConnection()
if(this.subscriptionMonitor) {
this.subscriptionMonitor.remove()
}

Take data from yesterday

So I need to take data only from the day before. Example: today is 2018/9/25, I need the data to be taken on 2018/9/24 only. But from my code below, it takes from 23 until 25. Which is more than one day, and it took also two days before from the date I need. I don't know which code that make a wrong result. Anyone can help me with this? I really appreciate it.
Api.js
const TO_DAYS = 4194304 * 1000 * 60 * 60 * 24; // this part that might be the cause
const ROOT_DATE = moment([2018, 3, 30]); // this part that might be the cause
const ROOT_DATE_ID = 440557948108800000; // this part that might be the cause
const DATE_ID = function(date) {
return ROOT_DATE_ID + date.diff(ROOT_DATE, "days") * TO_DAYS;
}; // this part that might be the cause
class discordApi {
constructor() {
this.lock = new AsyncLock();
}
get(momentDate, authorId, offset = 0) {
const url =
config.endpoint +
querystring.stringify({
min_id: DATE_ID(momentDate),
author_id: authorId
});
return fetch(url, {
headers: {
method: "GET",
Authorization: config.auth
}
}).then(res => {
// console.log(res.url);
return res.json();
});
}
async getAllData(momentDate) {
const allData = config.targets.map(author_id =>
this.get(momentDate, author_id)
);
return Promise.all(allData);
}
index.js
var yesterday = moment().subtract(1, "days"); // this part that might be the cause
async function sendEmail() {
const data = await discordApi.getAllData(yesterday);
const unfilteredMessages = data.reduce(
(prev, current) => [...prev, ...current.messages],
[]
);
const filteredMessages = unfilteredMessages.reduce((prev, current) => {
if (prev.length === 0) {
return [...prev, current];
}
const currentConversationIsDuplicated = isConversationDuplicated(
prev[prev.length - 1],
current
);
if (currentConversationIsDuplicated) {
return prev;
}
ret
urn [...prev, current];
}, []);
const convo = await discordApi.AllConvo(filteredMessages);
const mailOptions = {
from: "lala#gmail.com",
to: maillist,
subject: "Discord-Bot Daily Data",
html: convo
};
transporter.sendMail(mailOptions, function(err, info) {
if (err) console.log(err);
else console.log("Message Sent!");
});
}