I'm trying to read and write from/to an Azure Cosmos DB with two different bots (js, v4, ms botframework).
Chatbot 1:
- Chat with user, save user data and use it later
Chatbot 2:
- Read and display some user data
I use the following client: https://github.com/Microsoft/BotFramework-WebChat
Scenario:
I fixate my userID in the client (which has a directline to bot 1) to let's say "123"
I use Bot 1 and enter my username in the dialog (prompted by bot)
I refresh the Website on which Bot 1 is running with the same id "123"
I see that the bot still has my data stored
I change the ID in my client to "124"
I use Bot 1 and see there is no stored data (which is expected since ID "124" has never chatted with Bot 1)
I change the ID back to "123"
I use bot 1 and see that data from step 2 is still there
I use bot 2 with the id "123"
I see that there is no data ("undefined")
I use bot with ID "123" again
I see that the data from step 2 is gone
Which means that whenever I access the database with my second bot it seems like the data is cleared / deleted.
This is how I access the DB in index.js:
//Add CosmosDB (info in .env file)
const memoryStorage = new CosmosDbStorage({
serviceEndpoint: process.env.ACTUAL_SERVICE_ENDPOINT,
authKey: process.env.ACTUAL_AUTH_KEY,
databaseId: process.env.DATABASE,
collectionId: process.env.COLLECTION
})
// ConversationState and UserState
const conversationState = new ConversationState(memoryStorage);
const userState = new UserState(memoryStorage);
// Use middleware to write/read from DB
adapter.use(new AutoSaveStateMiddleware(conversationState));
adapter.use(new AutoSaveStateMiddleware(userState));
This is how I use the DB in bot.js:
constructor(conversationState, userState, dialogSet, memoryStorage) {
// Creates a new state accessor property.
// See https://aka.ms/about-bot-state-accessors to learn more about the bot state and state accessors
this.conversationState = conversationState;
this.userState = userState;
// Memory storage
this.memoryStorage = memoryStorage;
// Conversation Data Property for ConversationState
this.conversationData = conversationState.createProperty(CONVERSATION_DATA_PROPERTY);
// Properties for UserState
this.userData = userState.createProperty(USER_DATA_PROPERTY);
this.investmentData = userState.createProperty(INVESTMENT_DATA_PROPERTY);
}
async displayPayout (step) {
console.log("Display Payout");
// Retrieve user object from UserState storage
const userInvestData = await this.investmentData.get(step.context, {});
const user = await this.userData.get(step.context, {});
await step.context.sendActivity(`Hallo ${user.name}. Am Ausgang kannst du dir deine Bezahlung von ${userInvestData.payout} abholen.` );
}
The code snipped is from bot 2. Bot 1 saves the data in the same way. You can find the repos here:
Bot 1: https://github.com/FRANZKAFKA13/roboadvisoryBot
Bot 2: https://github.com/FRANZKAFKA13/displayBot
Client for Bot 1: https://github.com/FRANZKAFKA13/ra-bot-website-c
Client for Bot 2: https://github.com/FRANZKAFKA13/ra-bot-website-display
I also tried to use the "readOnly" key from CosmosDB in bot 2, which throws an error:
[onTurnError]: [object Object]
(node:1640) UnhandledPromiseRejectionWarning: TypeError: Cannot perform 'set' on a proxy that has been revoked
at adapter.sendActivities.then (C:\Users\X\Implementierung\display_bot\node_modules\botbuilder-core\lib\turnContext.js:175:36)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:229:7)
(node:1640) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
Another behavior that I have noticed: When I trigger a "join event" through my client with a redux store, the userdata is not saved as well (every time I refresh the page, the data is gone, despite using the same id "123" all the time)
dispatch({
type: 'WEB_CHAT/SEND_EVENT',
payload: {
// Event starting bot's conversation
name: 'webchat/join',
value: {}
}
Any ideas? Thanks in advance
Since storage ids (see image) are created automatically based off of user ids (that may also be created automatically and varies by channel) and channel ids, this can be very difficult to do. It can make it very difficult to persist user and conversation data, particularly across bots and channels.
Example ID:
Here's more on how IDs work.
Personally, I would write my own, custom storage, instead of (or in addition to) saving it with UserState.
To write your data, do something like this:
const changes = {};
const userDataToWrite = {
name: step.result,
eTag: '*',
}
// Replace 'UserId' with however you want to set the UserId
changes['UserId'] = userDataToWrite;
this.memoryStorage.write(changes);
This will store a document that looks like this (I set 'UserId' to 'user123':
To read:
const userDataFromStorage = await this.memoryStorage.read(['UserId']);
userDataFromStorage will look like this:
{ UserId:
{ name: 'myName',
eTag: '"0000c700-0000-0000-0000-5c7879d30000"' } }
You'll have to manage userIds yourself, but this will ensure that the data can be read across bots, channels, and users.
Edit: Solved it by adding "[this.userID]" after each "user" call.
I tried your method and whenever I write the data, a new eTag is created which leads to the object being split apart:
"document": {
"25781dc4-805d-4e69-bf89-da1f4d72e7cb": {
"25781dc4-805d-4e69-bf89-da1f4d72e7cb": {
"25781dc4-805d-4e69-bf89-da1f4d72e7cb": {
"25781dc4-805d-4e69-bf89-da1f4d72e7cb": {
"25781dc4-805d-4e69-bf89-da1f4d72e7cb": {
"name": "",
"age": "",
"gender": "",
"education": "",
"major": "",
"eTag": "\"00003998-0000-0000-0000-5c797fff0000\""
},
"name": "Jane Doe",
"eTag": "\"00003e98-0000-0000-0000-5c7980080000\""
},
"age": 22,
"eTag": "\"00004898-0000-0000-0000-5c7980150000\""
},
"gender": "female",
"eTag": "\"00004d98-0000-0000-0000-5c79801b0000\""
},
"education": "Bachelor",
"eTag": "\"00005498-0000-0000-0000-5c7980200000\""
},
"major": "Business Administration",
"complete": true
How can I prevent this?
My Code:
In Constructor:
this.changes = {};
this.userID = "";
this.userDatax = {
name: "",
age: "",
gender: "",
education: "",
major: "",
eTag: '*',
}
In Dialogs:
async welcomeUser (step) {
console.log("Welcome User Dialog");
//step.context.sendActivity({ type: ActivityTypes.Typing});
// Initialize UserData Object and save it to DB
this.changes[this.userID] = this.userDatax;
await this.memoryStorage.write(this.changes);
}
async promptForAge (step) {
console.log("Age Prompt");
// Read UserData from DB
var user = await this.memoryStorage.read([this.userID]);
console.log(user);
// Before saving entry, check if it already exists
if(!user.name) {
user.name = step.result;
user.eTag = '*';
// Write userData to DB
this.changes[this.userID] = user;
await this.memoryStorage.write(this.changes);
}
}
Related
How can I get a notice when a tweet was deleted using the twitter API?
In version 1.0 of the API, I was able to get the notification in a stream using this:
var Twit = require("twit");
var T = new Twit({
consumer_key: "555",
consumer_secret: "555",
access_token: "555",
access_token_secret: "555",
timeout_ms: 60 * 1000,
strictSSL: true
});
var userIds = [ "123", "456" ];
var stream = T.stream("statuses/filter", { follow: userIds.join(",") });
stream.on("delete", (x) => console.log("Tweet was deleted", x));
However, without notice. The deleted events stopped being streamed.
So now I'm trying to do it with v2 of the twitter API like this:
const BEARER_TOKEN = "555";
const { ETwitterStreamEvent, TweetStream, TwitterApi, ETwitterApiError } = require("twitter-api-v2");
const appClient = new TwitterApi(BEARER_TOKEN);
const stream = await appClient.v2.getStream("tweets/compliance/stream", { partition: 1 });
stream.on(ETwitterStreamEvent.Data, (x) => console.log("Got data", x));
The call to getStream() throws the following error:
data: {
client_id: '555',
detail: 'When authenticating requests to the Twitter API v2 endpoints, you must use keys and tokens from a Twitter developer App that is attached to a Project. You can create a project via the developer portal.',
registration_url: 'https://developer.twitter.com/en/docs/projects/overview',
title: 'Client Forbidden',
required_enrollment: 'Standard Basic',
reason: 'client-not-enrolled',
type: 'https://api.twitter.com/2/problems/client-forbidden'
}
I also tried using an app only login such as this:
const TWITTER_CLIENT = new TwitterApi({
appKey: CONSUMER_KEY,
appSecret: CONSUMER_SECRET,
accessToken: ACCESS_TOKEN,
accessSecret: ACCESS_TOKEN_SECRET
});
var appClient = await TWITTER_CLIENT.appLogin();
That throws the same error as above.
Using 2.0's getStream() with /tweets/search/stream/ does return an event a tweet is created, but not when it is deleted. It also has a limited query with only 5 rules per stream and rules are only 512 characters in length. Which won't cover all the screen names I currently track in the 1.0 version of the API.
I also tried using compliance jobs, but it takes a very long time and ends up returning an empty array anyways instead of any info about the tweet ids I provided:
var job = await appClient.v2.sendComplianceJob({
type: "tweets",
ids:[
// the ids are not from my dev account or from
// a account that authed my app
"555", // id of tweet I deleted
"123", // id of tweet I deleted
"456", // id of tweet I didn't delete
]
});
// takes 5-10 minutes to say its complete
var jobResults = await appClient.v2.complianceJobResult(job.data);
// echos: jobResults: []
console.log("jobResults", jobResults);
How can I get a stream event of when a tweet is deleted (of any specific user I choose) using the v2 API of twitter?
Unfortunately, Twitter have deprecated User deletes a Tweet event type.
The Only other option is to save all tweets for the accounts you are tracking on your database then compare them to current tweets using lookup API.
but you can only check 100 tweets on every request, so you will need to make a job that loops to check every 100 tweet, then to inform you if a tweet was deleted.
ids: A comma separated list of Tweet IDs. Up to 100 are allowed in a single request. Make sure to not include a space between commas and fields.
source: https://developer.twitter.com/en/docs/twitter-api/tweets/lookup/api-reference/get-tweets#tab0
I have a collection named "adverts"
/adverts/{advert_id} <-- where advert_id is auto generated by firestore.
And I have my collection "users"
/user/{user_id} <--- where user_id is defined by a username
So inside the "adverts" docs I have the next map
user_data:{
avatar_url: "",
first_name: "example name",
last_name: "example last name",
rating: 5,
username: "exampleusername"
}
This info comes from the user document each time an advert is created. So I want to update this map in advert collection, every time the user updates his data.
Is it possible to update this fields with a batch assuming that more than one document in adverts could exists? (I'm trying to avoid reading all files and rewrite them, I just want to write)
I was trying to achieve this by (is an onUpdate cloud function):
const before = change.before.data(); // Data before the update
const after = change.after.data(); // Data after the update
const user_id = after.username;
let batch = db.batch()
let advertsRef = db.collection("adverts").where("user_data.username", "==", user_id)
batch.update(advertsRef, {
"user_data.avatar_url": after.avatar_url,
"user_data.first_name": after.first_name,
"user_data.last_name": after.last_name,
"user_data.overall_adverts_rating": after.overall_adverts_rating,
"user_data.username": after.username,
})
batch.commit().then(()=>{
console.log("done")
})
.catch(error =>{
console.log(error)
})
But I'm getting the next error:
Error: Value for argument "documentRef" is not a valid DocumentReference.
at Object.validateDocumentReference (/workspace/node_modules/#google-cloud/firestore/build/src/reference.js:2034:15)
at WriteBatch.update (/workspace/node_modules/#google-cloud/firestore/build/src/write-batch.js:312:21)
at /workspace/index.js:147:9
at cloudFunction (/workspace/node_modules/firebase-functions/lib/cloud-functions.js:134:23)
at /layers/google.nodejs.functions-framework/functions-framework/node_modules/#google-cloud/functions-framework/build/src/invoker.js:199:28
at processTicksAndRejections (internal/process/task_queues.js:97:5)
I guess is cause my .where() is not referring to a specific file.
In your cloud function, you'll need to iterate through each matching advert. I.e. you are rewriting all of the documents which match the query which means you'll need to read each one, and update each one. e.g.
let adverts = await db.collection("adverts").where("user_data.username", "==", user_id).get
for (doc of adverts.docs) {
doc.user_data = after
batch.update(doc)
}
await batch.commit()
I have the following code:
(function () {
var gameDataLocalStorageName = "myTest3";
var defaultUserSettings = {
volumes: {
musicVolume: 0.3,
sfxVolume: 0.5,
voicesVolume: 1
}
};
var savedGames = [
{
screenshot: "data uri here",
day: "1",
month: "1",
year: "1",
time: "1",
gameData: {
fonts: [{
id: 123,
name: "Arial"
}],
globalSpeeches: {
anotherVal: "something"
}
}
}
];
console.log(gameDataLocalStorageName);
console.log(defaultUserSettings);
console.log(savedGames);
/* Create db START */
var db = new Dexie(gameDataLocalStorageName);
db.version(1).stores({
usersData: ""
});
db.usersData.put(defaultUserSettings, 'userSettings');
db.usersData.put(savedGames, 'savedGames');
}());
/* Create db END */
/* Recall db START */
setTimeout(function(){
var db2 = new Dexie("myTest3");
db2.version(1).stores({
usersData: "userSettings,savedGames"
});
db2.usersData.toArray().then(function (results) {
console.log("User settings is: ", results[1]);
console.log("Saved games is: ", results[0]);
});
}, 3000);
Which runs great. However how can I obtain the data again without having to render out as an array toArray(). Currently to obtain them I have to hardcode results[0] and results[1] which is also not in the same order as I entered them into the db.
Ideally I want to do something like:
db2.get('usersData.userSettings');
db2.get('usersData.savedGames');
The sample show you are changing primary key which is not supported:
The first declaration specifies a table "usersData" with outbound primary keys:
db.version(1).stores({
usersData: ""
});
Then in the setTimout callback, you redeclare it with:
db2.version(1).stores({
usersData: "userSettings,savedGames"
});
...which means you want an inbound primary key from the property "userSettings" and and index on property "savedGames".
There are three errors here:
You cannot change declaration without incrementing version number which is not done here.
You cannot change primary key on an existing database.
Promises are not catched so you do not see the errors.
It seems what you really intend is so use Dexie as a key/value store, which is perfectly ok but much simpler to do than the sample shows.
If you put() (or add()) a value using a certain key, you retrieve the same using get().
If so, try the following:
db.version(1).stores({
usersData: "",
});
And don't forget to catch promises or await and do try/catch.
(async ()=>{
await db.usersData.put(defaultUserSettings, 'userSettings')
await db.usersData.put(savedGames, 'savedGames');
// Get using key:
const userSettings = await db.usersData.get('userSettings');
console.log("User settings is: ", userSettings);
const savedGames = await db.usersData.get('savedGames');
console.log("User settings is: ", savedGames);
})().catch(console.error);
However, putting entire arrays as values in a key/value store is not very optimal.
Maybe only have two tables "userSettings" and "savedGames" where each saved game would be its own row? Will you support multiple users or just one single user? If multiple, you could add an index "userId" to your tables.
If so, try the following:
db.version(2).stores({
userSettings: "userId" // userId is primary key
savedGames: "++gameId, userId" // incremented id and userId is foreign key
});
(async ()=>{
await db.userSettings.put({...defaultUserSettings, userId: "fooUser"});
await db.savedGames.bulkPut(savedGames.map(game =>
({...game, userId: "fooUser"}));
// Get user settings:
const userSettings = await db.usersData.get('fooUser');
console.log("User settings is: ", userSettings);
const savedGames = await db.usersData.where({userId: "fooUser"}).toArray();
console.log("Saved games for fooUser are: ", savedGames);
})().catch(console.error);
I plan to create a main tree named users which will include the name different users used as username. So, from each username will be included their data e.g. Full Name, Address, Phone No.
I want to know how to get each user's data when they log in on their profile.
First of all i suggest you spend some time getting familiar with firebase by reading the Firebase Guide (Link to old Firebase Guide). Everything you need to know to answer your own question is available there. But for simplicity i will put an example here:
Lets start with security, here are the basic firebase rules you need for this example: (source: Understanding Security) (old source: Understanding Security)
{
"rules": {
"users": {
"$user_id": {
".write": "$user_id === auth.uid"
}
}
}
}
I will skip the actual user creation and logging in and focus on the question about storing and retrieving user data.
Storing data: (source: Firebase Authentication) (old source: User Authentication)
// Get a reference to the database service
var database = firebase.database();
// save the user's profile into Firebase so we can list users,
// use them in Security and Firebase Rules, and show profiles
function writeUserData(userId, name, email, imageUrl) {
firebase.database().ref('users/' + userId).set({
username: name,
email: email
//some more user data
});
}
The resulting firebase data will look like this:
{
"users": {
"simplelogin:213": {
"username": "password",
"email": "bobtony"
},
"twitter:123": {
"username": "twitter",
"email": "Andrew Lee"
},
"facebook:456": {
"username": "facebook",
"email": "James Tamplin"
}
}
}
And last but not least the retreiving of the data, this can be done in several ways but for this example i'm gonna use a simple example from the firebase guide: (source: Read and Write data) (old source: Retreiving Data)
//Get the current userID
var userId = firebase.auth().currentUser.uid;
//Get the user data
return firebase.database().ref('/users/' + userId).once('value').then(function(snapshot) {
//Do something with your user data located in snapshot
});
EDIT: Added example of return data
So when you are logged in as user twitter:123 you will get a reference to the location based on your user id and will get this data:
"twitter:123": {
"username": "twitter",
"email": "Andrew Lee"
}
Though I agree with Andre about setting the rules for good security - I would handle the data a bit differently. Instead of generating the string I use the child() method. It's a matter of personal preference.
Get the UID and define a data object:
let user = firebase.auth().currentUser
let uid = user.uid
let yourdata = { foo: 'something', bar: 'other'}
Save the data:
firebase.database().ref('users').child(uid).set(yourdata)
.then((data) => {
console.log('Saved Data', data)
})
.catch((error) => {
console.log('Storing Error', error)
})
Fetch the data:
firebase.database().ref('users').child(uid).once('value')
.then((data) => {
let fetchedData = data.val()
console.log('Fetched Data', fetchedData)
})
.catch((error) => {
console.log('Fetching Error', error)
})
Please notice that set() will override your data so you might want to use push() later on.
I'm building a multiplayer, turn-based game using meteor.js. The application will handle multiple games, so I'd like to separate my users into rooms.
I've done it before using socket.io channels, but I'm struggling to understand how it should be done in Meteor.
The flow I'd like to achieve is:
User visits http://localhost:3000/join/userId
I make a server-side call to an external API using "sessionId" as parameter, getting user's userId, his assigned roomId and an array of allowed userId's for this room
I'd like to create a room with roomId for the user or join him to an existing one. I know I should create a 'Rooms' collection, but I don't know how to tie users to my rooms and publish messages only to those present in the given room.
I'd like to avoid using 'accounts' package, because I don't need authorisation on my side - it'll be handled by step #2 mentioned above - but if the easiest and cleanest way of doing it involves adding this package, I can change my mind.
Your Rooms collection could look like:
{
_id: "<auto-generated>",
roomId: "roomId",
users: [ "user1", "user2", "user3", ... ],
messages: [
{ message: "", userId: "" },
{ message: "", userId: "" },
{ message: "", userId: "" },
...
]
}
The server-side API call returns
userId and roomId among other information.
So you can do a
Rooms.update({ roomId: roomId }, { $push: { users: userId } }, { upsert: true });
This would push the user into the exiting room or create a new room and add the user.
Your publish function could look like:
Meteor.publish("room", function(roomId) {
// Since you are not using accounts package, you will have to get the userId using the sessionId that you've specified or some other way.
// Let us assume your function getUserId does just that.
userId: getUserId( sessionId );
return Rooms.find({ roomId: roomId, users: userId });
// Only the room's users will get the data now.
});
Hope this helps.