We Keep Coding

Rxdb infinitely pulling in replicateRxCollection

I'm working with rxdb and I have pull and push handlers for the backend I have used supabase
I have setup the code for replication as follows:
replication.ts
import { RxDatabase } from "rxdb";
import { RxReplicationPullStreamItem } from "rxdb/dist/types/types";
import { replicateRxCollection } from "rxdb/plugins/replication";
import { Subject } from "rxjs";
import { supabaseClient, SUPABASE_URL } from "src/config/supabase";
import { DbTables } from "src/constants/db";
import {
blockPullHandler,
blockPushHandler,
} from "./repilicationhandlers/block";
import { CheckpointType, RxBlockDocument, RxBlocksCollections } from "./types";
export async function startReplication(
database: RxDatabase<RxBlocksCollections>
) {
const pullStream$ = new Subject<
RxReplicationPullStreamItem<RxBlockDocument, CheckpointType>
>();
supabaseClient
.from(DbTables.Block)
.on("*", (payload) => {
console.log("Change received!", payload);
const doc = payload.new;
pullStream$.next({
checkpoint: {
id: doc.id,
updated: doc.updated,
},
documents: [doc] as any,
});
})
.subscribe((status: string) => {
console.log("STATUS changed");
console.dir(status);
if (status === "SUBSCRIBED") {
pullStream$.next("RESYNC");
}
});
const replicationState = await replicateRxCollection({
collection: database.blocks,
replicationIdentifier: "supabase-replication-to-" + SUPABASE_URL,
deletedField: "archived",
pull: {
handler: blockPullHandler as any,
stream$: pullStream$.asObservable(),
batchSize: 10,
},
push: {
batchSize: 1,
handler: blockPushHandler as any,
},
});
replicationState.error$.subscribe((err) => {
console.error("## replicationState.error$:");
console.log(err);
});
return replicationState;
}
blockPullHandler:
export const blockPullHandler = async (
lastCheckpoint: any,
batchSize: number
) => {
const minTimestamp = lastCheckpoint ? lastCheckpoint.updated : 0;
console.log("Pulling data", batchSize, lastCheckpoint);
const { data, error } = await supabaseClient
.from(DbTables.Block)
.select()
.gt("updated", minTimestamp)
.order("updated", { ascending: true })
.limit(batchSize);
if (error) {
console.log(error);
throw error;
}
const docs: Array<Block> = data;
return {
documents: docs,
hasMoreDocuments: false,
checkpoint:
docs.length === 0
? lastCheckpoint
: {
id: lastOfArray(docs).id,
updated: lastOfArray(docs).updated,
},
};
};
blockPushHandler:
export const blockPushHandler = async (
rows: RxReplicationWriteToMasterRow<RxBlockDocumentType>[]
) => {
if (rows.length !== 1) {
throw new Error("# pushHandler(): too many push documents");
}
const row = rows[0];
const oldDoc: any = row.assumedMasterState;
const doc: Block = row.newDocumentState;
console.log(row, oldDoc, doc);
// insert
if (!row.assumedMasterState) {
const { error } = await supabaseClient.from(DbTables.Block).insert([doc]);
console.log("Error 1", error);
if (error) {
// we have an insert conflict
const conflictDocRes: any = await supabaseClient
.from(DbTables.Block)
.select()
.eq("id", doc.id)
.limit(1);
return [conflictDocRes.data[0]];
} else {
return [];
}
}
// update
console.log("pushHandler(): is update");
const { data, error } = await supabaseClient
.from(DbTables.Block)
.update(doc)
.match({
id: doc.id,
replicationRevision: oldDoc.replicationRevision,
});
console.log("Error 2", error);
if (error) {
console.log("pushHandler(): error:");
console.log(error);
console.log(data);
throw error;
}
console.log("update response:");
console.log(data);
if (data.length === 0) {
// we have an updated conflict
const conflictDocRes: any = await supabaseClient
.from(DbTables.Block)
.select()
.eq("id", doc.id)
.limit(1);
return [conflictDocRes.data[0]];
}
return [];
};
But the issue is when I start the application and the pull handler is called correctly but it doesn't stop calling the pull handler and it sends continuous request one after another even after it has fetched the documents even when I set hasMoreDocuments to false It keeps sending requests and running the replicator. Is there something wrong with my configuration?
database.ts:
export const createDatabase = async () => {
const database = await createRxDatabase({
name: "sundaedb",
storage: getRxStorageDexie(),
});
await database.addCollections({
blocks: {
schema: blockSchema as any,
conflictHandler: conflictHandler as any,
},
documents: {
schema: documentSchema as any,
conflictHandler: conflictHandler as any,
},
});
database.blocks.preInsert((docData) => {
docData.replicationRevision = createRevision(
database.hashFunction,
docData as any
);
return docData;
}, false);
database.blocks.preRemove((docData) => {
console.log(" PRE REMOVE !!");
console.log(JSON.stringify(docData, null, 4));
const oldRevHeight = parseRevision(docData.replicationRevision).height;
docData.replicationRevision =
oldRevHeight + 1 + "-" + database.hashFunction(JSON.stringify(docData));
console.log(JSON.stringify(docData, null, 4));
return docData;
}, false);
database.blocks.preSave((docData) => {
const oldRevHeight = parseRevision(docData.replicationRevision).height;
docData.replicationRevision =
oldRevHeight + 1 + "-" + database.hashFunction(JSON.stringify(docData));
return docData;
}, false);
return database;
};

Use Effect doesn't run on websocket ON

I am working on a websocket project in react. But when I send a message, the websocket does reload to display new messages. I have to reload my page manually to show the changes.
Here's my use Effect Hook
useEffect(() => {
if (socket.current) {
socket.current.on('msgSent', ({ newMsg }) => {
console.log('MESSAGE SENT', newMsg)
if (newMsg.receiver === openChatId.current) {
setMessages((prev) => [...prev, newMsg])
setChats((prev) => {
const previousChat = prev.find(
(chat) => chat.messagesWith === newMsg.receiver
)
previousChat.lastMessage = newMsg.msg
previousChat.date = newMsg.date
return [...prev]
})
}
})
}
}, [])
When I remove the useEffect dependency (i.e []), It works but it renders the message multiple times on the screen.
Here's the rest of my frontend client code
const openChatId = useRef('')
const auth = useContext(AuthContext)
const queryMessage = new URLSearchParams(search).get('message')
useEffect(() => {
if (!socket.current) {
socket.current = io(process.env.REACT_APP_API)
}
if (socket.current) {
socket.current.emit('join', { userId: auth.user._id })
socket.current.on('connectedUsers', ({ users }) => {
users.length > 0 && setConnectedUsers(users)
})
}
if (chats.length > 0 && !queryMessage) {
history.push(`/messages?message=${chats[0].messagesWith}`, undefined, {
shallow: true,
})
}
return () => {
if (socket.current) {
socket.current.emit('logout')
socket.current.off()
}
}
}, [chats])
useEffect(() => {
const getAllChats = async (token) => {
try {
setLoading(true)
const res = await getChats(token)
if (res) {
setChats(res)
setLoading(false)
}
} catch (err) {
console.log(err)
setLoading(false)
}
}
getAllChats(auth.token)
}, [])
useEffect(() => {
const loadMessages = () => {
socket.current.emit('loadMessages', {
userId: auth.user._id,
messagesWith: queryMessage,
})
socket.current.on('messagesLoaded', async ({ chat }) => {
setMessages(chat.messages)
setBannerData({
firstName: chat.messagesWith.firstName,
lastName: chat.messagesWith.lastName,
profilePicUrl: chat.messagesWith.profilePicUrl,
})
openChatId.current = chat.messagesWith._id
})
socket.current.on('noChatFound', async () => {
const { firstName, lastName, profilePicUrl } = await ChatGetUserInfo(
queryMessage,
auth.token
)
setBannerData({ firstName, lastName, profilePicUrl })
setMessages([])
openChatId.current = queryMessage
})
}
if (socket.current) {
loadMessages()
}
}, [queryMessage])
const sendMsg = (msg) => {
if (socket.current) {
socket.current.emit('sendNewMsg', {
userId: auth.user._id,
msgSendToUserId: openChatId.current,
msg,
})
}
}
The backend works very well. U think my problem is with the useEffect

I fixed it. I was missing the [chats] dependency

Getting a nested parameter return value in the wrapped function return

I have a scenario where I need to get the return value from a function that passed to another function as a parameter. I tried multiple ways. But couldn't get the returnValue to the CreateProfileComponent from ProfileAction.js file.
// ProfileAction.js
export default (database) => {
return {
createProfile: async (createdProfile) => {
const profileCollection = database.get("profiles");
const { name, email } = createdProfile;
try {
await database.action(async () => {
const returnValue = await profileCollection.create((profile) => {
profile.name = name;
profile.email = email;
});
});
} catch (error) {
console.log("createProfile", error);
}
},
};
};
// CreateProfileComponent.js
const CreateProfileComponent = () => {
const database = useDatabase();
const profileAction = ProfileAction(database);
const createdRecord = await profileAction.createProfile({
name: "John Doe",
email: "johndoe#gmail.com",
});
}
Finally what I want is the returnValue value in CreateProfileComponent. The functions database.actions() and profileCollection.create() are used from a third party library (WatermelonDB)

I am not sure what database.action does but you should return a value in this function. Like following: return await database.action(async () => {
And throw an error on catch
export default (database) => {
return {
createProfile: async (createdProfile) => {
const profileCollection = database.get("profiles");
const { name, email } = createdProfile;
try {
return await database.action(async () => {
const returnValue = await profileCollection.create((profile) => {
profile.name = name;
profile.email = email;
});
});
} catch (error) {
console.log("createProfile", error);
throw error;
}
},
};
};
// CreateProfileComponent.js
const CreateProfileComponent = () => {
const database = useDatabase();
const profileAction = ProfileAction(database);
try {
const createdRecord = await profileAction.createProfile({
name: "John Doe",
email: "johndoe#gmail.com",
});
} catch (e) {
}
}

Cannot read property 'conversationId' of undefined while using a reducer function

Just to make it clear router uses the code below and my messages.js are inside api folder....
router.use("/messages", require("./messages"));
so my api call is correct.
Backend for posting the message.... I know conversationId will be null if no conversation exists but... I am trying to send message where conversation exists already and still I am getting cannot read the conversationId of undefined....
// expects {recipientId, text, conversationId } in body
// (conversationId will be null if no conversation exists yet)
router.post("/", async (req, res, next) => {
try {
if (!req.user) {
return res.sendStatus(401);
}
const senderId = req.user.id;
const { recipientId, text, conversationId, sender } = req.body;
// if we already know conversation id, we can save time and just add it to message and return
if (conversationId) {
const message = await Message.create({ senderId, text, conversationId });
return res.json({ message, sender });
}
// if we don't have conversation id, find a conversation to make sure it doesn't already exist
let conversation = await Conversation.findConversation(
senderId,
recipientId
);
if (!conversation) {
// create conversation
conversation = await Conversation.create({
user1Id: senderId,
user2Id: recipientId,
});
if (onlineUsers.includes(sender.id)) {
sender.online = true;
}
}
const message = await Message.create({
senderId,
text,
conversationId: conversation.id,
});
res.json({ message, sender });
} catch (error) {
next(error);
}
});
module.exports = router;
This is the frontend that posts the data to the backend....
const saveMessage = async (body) => {
const { data } = await axios.post("/api/messages", body);
return data;
};
Okay so here is detail information on how I am dispatching it.
class Input extends Component {
constructor(props) {
super(props);
this.state = {
text: "",
};
}
handleChange = (event) => {
this.setState({
text: event.target.value,
});
};
handleSubmit = async (event) => {
event.preventDefault();
// add sender user info if posting to a brand new convo,
// so that the other user will have access to username, profile pic, etc.
const reqBody = {
text: event.target.text.value,
recipientId: this.props.otherUser.id,
conversationId: this.props.conversationId,
sender: this.props.conversationId ? null : this.props.user,
};
await this.props.postMessage(reqBody);
this.setState({
text: "",
});
};
render() {
const { classes } = this.props;
return (
<form className={classes.root} onSubmit={this.handleSubmit}>
<FormControl fullWidth hiddenLabel>
<FilledInput
classes={{ root: classes.input }}
disableUnderline
placeholder="Type something..."
value={this.state.text}
name="text"
onChange={this.handleChange}
/>
</FormControl>
</form>
);
}
}
export default connect(
mapStateToProps,
mapDispatchToProps
)(withStyles(styles)(Input));
const mapDispatchToProps = (dispatch) => {
return {
postMessage: (message) => {
dispatch(postMessage(message));
},
};
};
// message format to send: {recipientId, text, conversationId}
// conversationId will be set to null if its a brand new conversation
export const postMessage = (body) => (dispatch) => {
try {
const data = saveMessage(body);
if (!body.conversationId) {
dispatch(addConversation(body.recipientId, data.message));
} else {
dispatch(setNewMessage(data.message));
}
sendMessage(data, body);
} catch (error) {
console.error(error);
}
};
So I have attached what I want to do here now....
But I am still getting the problem....
// CONVERSATIONS THUNK CREATORS, this is how I am getting data from the backend
export const fetchConversations = () => async (dispatch) => {
try {
const { data } = await axios.get("/api/conversations");
dispatch(gotConversations(data));
} catch (error) {
console.error(error);
}
};
export const setNewMessage = (message, sender) => {
return {
type: SET_MESSAGE,
payload: { message, sender: sender || null },
};
};
// REDUCER
const reducer = (state = [], action) => {
switch (action.type) {
case GET_CONVERSATIONS:
return action.conversations;
case SET_MESSAGE:
return addMessageToStore(state, action.payload);
case ADD_CONVERSATION:
return addNewConvoToStore(
state,
action.payload.recipientId,
action.payload.newMessage
);
default:
return state;
}
};
I am getting an error saying Cannot read property 'conversationId' of undefined while using a reducer function... Should I give the setintial value of the message to empty?
export const addMessageToStore = (state, payload) => {
const { message, sender } = payload;
// if sender isn't null, that means the message needs to be put in a brand new convo
if (sender !== null) {
const newConvo = {
id: message.conversationId,
otherUser: sender,
messages: [message],
};
newConvo.latestMessageText = message.text;
return [newConvo, ...state];
}
return state.map((convo) => {
if (convo.id === message.conversationId) {
const convoCopy = { ...convo };
convoCopy.messages.push(message);
convoCopy.latestMessageText = message.text;
return convoCopy;
} else {
return convo;
}
});
};

Issue
The saveMessage function is declared async
const saveMessage = async (body) => {
const { data } = await axios.post("/api/messages", body);
return data;
};
but the postMessage action creator isn't async so it doesn't wait for the implicitly returned Promise to resolve before continuing on and dispatching to the store. This means that data.message is undefined since a Promise object doesn't have this as a property.
export const postMessage = (body) => (dispatch) => {
try {
const data = saveMessage(body); // <-- no waiting
if (!body.conversationId) {
dispatch(addConversation(body.recipientId, data.message));
} else {
dispatch(setNewMessage(data.message));
}
sendMessage(data, body);
} catch (error) {
console.error(error);
}
};
Solution
Declare postMessage async as well and await the data response value.
export const postMessage = (body) => async (dispatch) => {
try {
const data = await saveMessage(body); // <-- await response
if (!body.conversationId) {
dispatch(addConversation(body.recipientId, data.message));
} else {
dispatch(setNewMessage(data.message));
}
sendMessage(data, body);
} catch (error) {
console.error(error);
}
};

I want to get attachments as well that comes with the message Gmail API

Okay so with this code I am able to get the actual mail content which comes by Email but what I want is I want to get message and attachment too. what can be done in order to list out attachments and later on giving option to download the attachments gmail api.
var ifrm = document.getElementById("iframe").contentWindow.document;
ifrm.body.innerHTML = getMessageBody(message.payload);
};
let getMessageBody = (message) => {
var encodedBody = "";
if (typeof message.parts === "undefined") {
encodedBody = message.body.data;
} else {
encodedBody = getHTMLPart(message.parts);
}
return Base64.decode(encodedBody);
};
let getHTMLPart = (arr) => {
for (var x = 0; x <= arr.length; x++) {
if (typeof arr[x].parts === "undefined") {
if (arr[x].mimeType === "text/html") {
return arr[x].body.data;
}
} else {
return getHTMLPart(arr[x].parts);
}
}
return "";
};
Gmail API when I click on message.
getOneMessage = (messageId) => {
return window.gapi.client.gmail.users.messages
.get({
userId: "me",
id: messageId,
})
.then(
(response) => {
this.setState({
message: response.result,
});
},
(err) => {
console.error("getMessage error", err);
}
);
};
handleMessageClick = (e) => {
const messageId = e.currentTarget.getAttribute("id");
this.getOneMessage(messageId);

Solution
You are using the Users.messages: get endpoint. This is fine to retrieve the message body, but to retrieve attachments you will have to use the Users.messages.attachments: get. Here you can find a link to the documentation.
Proposed code editing:
getAttachments = (message, callback) => {
var parts = message.payload.parts;
for (var i = 0; i < parts.length; i++) {
var part = parts[i];
if (part.filename && part.filename.length > 0) {
var attachId = part.body.attachmentId;
var request = gapi.client.gmail.users.messages.attachments.get({
'id': attachId,
'messageId': message.id,
'userId': userId
});
request.execute(function(attachment) {
callback(part.filename, part.mimeType, attachment);
});
}
}
}
getOneMessage = (messageId) => {
return window.gapi.client.gmail.users.messages
.get({
userId: "me",
id: messageId,
})
.then(
(response) => {
this.setState({
message: response.result,
});
// Get the attachment and do something with it
getAttachments(response.result, callback);
},
(err) => {
console.error("getMessage error", err);
}
);
};
handleMessageClick = (e) => {
const messageId = e.currentTarget.getAttribute("id");
this.getOneMessage(messageId);
Reference
Users.messages.attachments