We Keep Coding

How to get a user's keypair from their public key only (Solana)?

I'm making a dApp and I want to add a button where a user (the one with their wallet connected) can send exactly 0.01 SOL to another user. I already wrote the function in my Rust program and after testing it with anchor test it seems to be working when I use my own personal wallet's Keypair to sign the transaction. However, now I am writing the event handler function in my web app's frontend and I'm not sure what to pass for the signers parameter if I want the user to sign the transaction. What do I pass if I don't know their secret key? Is there a way that I can generate a user's Keypair from their public key alone or would I need to use the Solana Wallet Adapter for this? Any help would be appreciated. This is my first time working with Solana!
This is the function:
const tipSol = async (receiverAddress) => {
try {
const provider = getProvider();
const program = new Program(idl, programID, provider);
const lamportsToSend = LAMPORTS_PER_SOL / 100;
const amount = new anchor.BN(lamportsToSend);
await program.rpc.sendSol(amount, {
accounts: {
from: walletAddress,
to: receiverAddress,
systemProgram: SystemProgram.programId,
},
signers: ?
})
console.log('Successfully sent 0.01 SOL!')
window.alert(`You successfully tipped ${receiverAddress} 0.01 SOL!`)
} catch (error) {
console.error('Failed to send SOL:', error);
window.alert('Failed to send SOL:', error);
}
}

Frontends never access private keys. Instead the flow is something like:
Frontend creates the transaction
Frontend sends the transaction to the wallet
Wallet signs the transaction
Wallet returns the signed transaction to the frontend
Frontend send the transaction
You can use the #solana/wallet-adapter to implement this on your frontend https://github.com/solana-labs/wallet-adapter
In practice it would be something like this in your frontend
export const Component = () => {
const { connection } = useConnection();
const { sendTransaction } = useWallet();
const handle = async () => {
const ix: TransactionInstruction = await tipSol(receiverKey);
const tx = new Transaction().add(ix);
const sig = await sendTransaction(tx, connection);
};
// ...
};

Hooks.js running the db connection and results twice in sveltekit

I'm using sveltekit and trying to understand all the new features added after retiring Sapper. One of those new features is hooks.js which runs on the server and not accessible to the frontend. It makes dealing with db safe. So I created a connection to my mongodb to retrieve user's data before I use the db results in my getSession function. It works but I noticed that it access my database TWICE. Here is my hooks.js code:
import * as cookie from 'cookie';
import { connectToDatabase } from '$lib/mongodb.js';
export const handle = async ({event, resolve})=>{
const dbConnection = await connectToDatabase();
const db = dbConnection.db;
const userinfo = await db.collection('users').findOne({ username: "a" });
console.log("db user is :" , userinfo) //username : John
const response = await resolve(event)
response.headers.set(
'set-cookie', cookie.serialize("cookiewithjwt", "sticksafterrefresh")
)
return response
}
export const getSession = (event)=>{
return {
user : {
name : "whatever"
}
}
}
The console.log you see here returns the user data twice. One as soon as I fire up my app at localhost:3000 with npm run dev and then less than a second, it prints another console log with the same information
db user is : John
a second later without clicking on anything a second console.log prints
db user is : John
So my understanding from the sveltekit doc is that hooks.js runs every time SvelteKit receives a request. I removed all prerender and prefetch from my code. I made sure I only have the index.svelte in my app but still it prints twice. My connection code I copied from an online post has the following:
/**
* Global is used here to maintain a cached connection across hot reloads
* in development. This prevents connections growing exponentially
* during API Route usage.
*/
Here is my connection code:
import { MongoClient } from 'mongodb';
const mongoURI ="mongodb+srv://xxx:xxx#cluster0.qjeag.mongodb.net/xxxxdb?retryWrites=true&w=majority";
const mongoDB = "xxxxdb"
export const MONGODB_URI = mongoURI;
export const MONGODB_DB = mongoDB;
if (!MONGODB_URI) {
throw new Error('Please define the mongoURI property inside config/default.json');
}
if (!MONGODB_DB) {
throw new Error('Please define the mongoDB property inside config/default.json');
}
/**
* Global is used here to maintain a cached connection across hot reloads
* in development. This prevents connections growing exponentially
* during API Route usage.
*/
let cached = global.mongo;
if (!cached) {
cached = global.mongo = { conn: null, promise: null };
}
export const connectToDatabase = async() => {
if (cached.conn) {
return cached.conn;
}
if (!cached.promise) {
const opts = {
useNewUrlParser: true,
useUnifiedTopology: true
};
cached.promise = MongoClient.connect(MONGODB_URI).then((client) => {
return {
client,
db: client.db(MONGODB_DB)
};
});
}
cached.conn = await cached.promise;
return cached.conn;
So my question is : is hooks.js runs twice all the time, one time on the server and one time on the front? If not, then why the hooks.js running/printing twice the db results in my case?
Anyone?

VM Exception while processing transaction: revert TransferHelper: TRANSFER_FROM_FAILED when running manipulatePrice.js test for bot.js

I am using the code from Dapp Unversity's trading bot masterclass to try to create a bot that will scan cryptocurrency decentralized exchanges for price differences in token pairs and then execute a smart contract to use a flash loan to profit off of this. When testing, I am able to see run a ganache-cli node and run my scanning bot to listen for swap opportunities on ganache. There is a script designed to create a swap opportunity by swapping a large amount of SHIB for WETH on the test net to see if the smart contract will deploy and execute when a swap opportunity is detected by the bot. However, running this script yields the error
UnhandledPromiseRejectionWarning: Error: Returned error: VM Exception while processing transaction: revert TransferHelper: TRANSFER_FROM_FAILED
Also, in the ganache-cli terminal, I get:
Runtime Error: revert
Revert reason: TransferHelper: TRANSFER_FROM_FAILED
Here are the commands I run to get to the points above:
First, I successfully run ganache-cli -f wss://eth-mainnet.alchemyapi.io/v2/<Your-App-Key> -u 0x0e5069514a3dd613350bab01b58fd850058e5ca4 -p 7545 with my app key.
Then, I successfully run node bot.js in another terminal to scan for swap opportunities on ganache. Finally, I run node scripts\manipulatePrice.JS which outputs "Beginnig Swap... Input token: SHIB Output token: WETH" before outputting the above error.
I have tried using node --trace-warnings to show where the warning was created, but was led nowhere helpful. I am wondering if it has something to do with the Runtime Error: revert message? Below is the code for the manipulatePrice.js script I am trying to run to test my bot. I can attach more code if need be, but don't want to post too much. If anyone has insight as to what or where the issue might be, I would greatly appreciate it!!
require("dotenv").config();
const Web3 = require('web3')
const {
ChainId,
Token,
WETH
} = require("#uniswap/sdk")
const IUniswapV2Router02 = require('#uniswap/v2-periphery/build/IUniswapV2Router02.json')
const IUniswapV2Factory = require("#uniswap/v2-core/build/IUniswapV2Factory.json")
const IERC20 = require('#openzeppelin/contracts/build/contracts/ERC20.json')
// -- SETUP NETWORK & WEB3 -- //
const chainId = ChainId.MAINNET
const web3 = new Web3('http://127.0.0.1:7545')
// -- IMPORT HELPER FUNCTIONS -- //
const { getPairContract, calculatePrice } = require('../helpers/helpers')
// -- IMPORT & SETUP UNISWAP/SUSHISWAP CONTRACTS -- //
const config = require('../config.json')
const uFactory = new web3.eth.Contract(IUniswapV2Factory.abi, config.UNISWAP.FACTORY_ADDRESS) // UNISWAP FACTORY CONTRACT
const sFactory = new web3.eth.Contract(IUniswapV2Factory.abi, config.SUSHISWAP.FACTORY_ADDRESS) // SUSHISWAP FACTORY CONTRACT
const uRouter = new web3.eth.Contract(IUniswapV2Router02.abi, config.UNISWAP.V2_ROUTER_02_ADDRESS) // UNISWAP ROUTER CONTRACT
const sRouter = new web3.eth.Contract(IUniswapV2Router02.abi, config.SUSHISWAP.V2_ROUTER_02_ADDRESS) // UNISWAP ROUTER CONTRACT
// -- CONFIGURE VALUES HERE -- //
const V2_FACTORY_TO_USE = uFactory
const V2_ROUTER_TO_USE = uRouter
const UNLOCKED_ACCOUNT = '0x0e5069514a3Dd613350BAB01B58FD850058E5ca4' // SHIB Unlocked Account
const ERC20_ADDRESS = process.env.ARB_AGAINST
const AMOUNT = '40500000000000' // 40,500,000,000,000 SHIB -- Tokens will automatically be converted to wei
const GAS = 450000
// -- SETUP ERC20 CONTRACT & TOKEN -- //
const ERC20_CONTRACT = new web3.eth.Contract(IERC20.abi, ERC20_ADDRESS)
const WETH_CONTRACT = new web3.eth.Contract(IERC20.abi, WETH[chainId].address)
// -- MAIN SCRIPT -- //
const main = async () => {
const accounts = await web3.eth.getAccounts()
const account = accounts[1] // This will be the account to recieve WETH after we perform the swap to manipulate price
const pairContract = await getPairContract(V2_FACTORY_TO_USE, ERC20_ADDRESS, WETH[chainId].address)
const token = new Token(
ChainId.MAINNET,
ERC20_ADDRESS,
18,
await ERC20_CONTRACT.methods.symbol().call(),
await ERC20_CONTRACT.methods.name().call()
)
// Fetch price of SHIB/WETH before we execute the swap
const priceBefore = await calculatePrice(pairContract)
await manipulatePrice(token, account)
// Fetch price of SHIB/WETH after the swap
const priceAfter = await calculatePrice(pairContract)
const data = {
'Price Before': `1 ${WETH[chainId].symbol} = ${Number(priceBefore).toFixed(0)} ${token.symbol}`,
'Price After': `1 ${WETH[chainId].symbol} = ${Number(priceAfter).toFixed(0)} ${token.symbol}`,
}
console.table(data)
let balance = await WETH_CONTRACT.methods.balanceOf(account).call()
balance = web3.utils.fromWei(balance.toString(), 'ether')
console.log(`\nBalance in reciever account: ${balance} WETH\n`)
}
main()
//
async function manipulatePrice(token, account) {
console.log(`\nBeginning Swap...\n`)
console.log(`Input Token: ${token.symbol}`)
console.log(`Output Token: ${WETH[chainId].symbol}\n`)
const amountIn = new web3.utils.BN(
web3.utils.toWei(AMOUNT, 'ether')
)
const path = [token.address, WETH[chainId].address]
const deadline = Math.floor(Date.now() / 1000) + 60 * 20 // 20 minutes
await ERC20_CONTRACT.methods.approve(V2_ROUTER_TO_USE._address, amountIn).send({ from: UNLOCKED_ACCOUNT })
const receipt = await V2_ROUTER_TO_USE.methods.swapExactTokensForTokens(amountIn, 0, path, account, deadline).send({ from: UNLOCKED_ACCOUNT, gas: GAS });
console.log(`Swap Complete!\n`)
return receipt
}

I ran into this issue for the same bot myself and I found the solution. Instead of using .send you should use .sendSignedTransaction and sign it yourself with the correct parameters. BUT if you are sending from an unlocked account, then use ethers.js to get a signer without knowing the private key!
So, you should replace
await ERC20_CONTRACT.methods.approve(V2_ROUTER_TO_USE._address, amountIn).send({ from: UNLOCKED_ACCOUNT })
const receipt = await V2_ROUTER_TO_USE.methods.swapExactTokensForTokens(amountIn, 0, path, account, deadline).send({ from: UNLOCKED_ACCOUNT, gas: GAS });
With this ethers.js alternative
const { ethers } = require("ethers")
const provider = new ethers.providers.JsonRpcProvider("http://127.0.0.1:8545")
const signer = provider.getSigner(UNLOCKED_ACCOUNT)
console.log(provider.chainId)
/*
Define the above code at the top,
Then put the rest of your code here,
Then use this for the ethers transaction
*/
const approvalReceiptEthers = await signer.sendTransaction({
from: UNLOCKED_ACCOUNT,
to: ERC20_CONTRACT._address,
data: ERC20_CONTRACT.methods.approve(V2_ROUTER_TO_USE._address, web3.utils.fromWei(amountIn).toString()).encodeABI(),
gasLimit: GAS
})
/*
* Verify that your unlocked account is allowed to use the funds
*/
var allowance = await ERC20_CONTRACT.methods.allowance(UNLOCKED_ACCOUNT, V2_ROUTER_TO_USE._address).call()
console.log("ALLOWANCE:\t\t", web3.utils.fromWei(allowance).toString(), 'ether')
console.log("ATTEMPTED AMOUNT:\t", web3.utils.fromWei(amountIn).toString(), 'ether')
const swapReceiptEthers = await signer.sendTransaction({
from: UNLOCKED_ACCOUNT,
to: V2_ROUTER_TO_USE._address,
data: V2_ROUTER_TO_USE.methods.swapExactTokensForTokens(web3.utils.fromWei(amountIn).toString(), 0, path, account, deadline).encodeABI(),
gasLimit: GAS
}).then(console.log)
Note: the following code, will only work provided that you signed the transaction with THE SENDER'S PRIVATE KEY. So, if you are sending a transaction on behalf of your own account, use this code (presumably for bot.js)
const approvalTransaction = {
'from' : 'your account address here',
'to' : ERC20_CONTRACT._address,
'data' : ERC20_CONTRACT.methods.approve(V2_ROUTER_TO_USE._address, web3.utils.fromWei(amountIn).toString()).encodeABI(),
'gas' : GAS
}
const transaction = {
'from' : 'your account address here',
'to' : V2_ROUTER_TO_USE._address,
'data' : V2_ROUTER_TO_USE.methods.swapExactTokensForTokens(web3.utils.fromWei(amountIn).toString(), 0, path, account, deadline).encodeABI(),
'gas' : GAS
}
const signedApprovalTx = await web3.eth.accounts.signTransaction(approvalTransaction, process.env.DEPLOYMENT_ACCOUNT_KEY, )
const signedTx = await web3.eth.accounts.signTransaction(transaction, process.env.DEPLOYMENT_ACCOUNT_KEY)
await web3.eth.sendSignedTransaction(signedApprovalTx.rawTransaction)
const receipt = await web3.eth.sendSignedTransaction(signedTx.rawTransaction)
NOTE: For those of you following the same Trading Bot masterclass, this same logic applies to the bot.js code as well when you execute your trade!.
SUMMARY: If you are sending a signed transaction using an unlocked account with ganache-cli, you will need to use ethers.js to sign the message without knowing the private key. Otherwise if you are sending a signed transaction on behalf of yourself, you can use Web3.js to sign the message with your own private

NestJS stops working after a few requests

I'm getting a serious issue. NestJS stops working after a few requests. I'm using postman to make a specific API call, and then, after less than 10 requests to the same route, it gets extremely slow and i'm getting timeout from Postman everytime.
main.ts:
import { NestFactory } from '#nestjs/core';
import { AppModule } from './app.module';
import {Logger} from '#nestjs/common'
import * as rateLimit from 'express-rate-limit';
import { WrapContentInterceptor } from './dashboard/dashboard.interceptors';
const PORT = 5000
const TAG = 'main'
async function bootstrap() {
const logger = new Logger(TAG)
const app = await NestFactory.create(AppModule);
// app.useGlobalInterceptors(new WrapContentInterceptor())
app.enableCors()
await app.listen(PORT);
app.use(
rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 1000, // limit each IP to 100 requests per windowMs
}),
);
logger.log(`listening port::${PORT}`)
}
bootstrap();
controller where I'm getting the issue:
// some regular imports
#Controller('dashboard')
#UseGuards(AuthGuard()) // protecting routes
export class DashboardController {
constructor(
private dashboardService:DashboardService,
){}
// other routes
#Get('/cockpit/general-numbers') // route that i'm getting the issue
async getNumbersForCockpit(
#Query('desiredDate',ParseStringToDatePipe) desiredDay?:Date
):Promise<GeneralNumbersForCockpit>{
this.logger.log(`getNumbersForCockpit::${desiredDay.toISOString()}`)
let installation = await this.dashboardService.getInstallationsOfDay(desiredDay?desiredDay:undefined)
let averageTicket = await this.dashboardService.getAverageTicketPlanFromInterval(
desiredDay?getFirstDayOfMonth(desiredDay):undefined,
desiredDay?desiredDay:undefined
)
return {
averageTicket:averageTicket,
installation:installation.result
}
}
}
Ps: I realized another thing:
This error is happening in the routes where I'm using a pure SQL raw at the service layer to a MariaDB database running at Docker container.
For example at the service layer:
```
async getData(dateField:Date=new Date()):Promise<SimpleNumericalData>{
// this.logger.debug(`f::dateField:${dateField}`)
this.logger.debug(`getData::dateField:${dateField.toISOString()}`)
const queryRunner = this.connection.createQueryRunner()
let res = await queryRunner.manager.query(`
select count(*) as result from my_table
where my_field ='I'
and DATE(date_field) = DATE('${dateField.toISOString()}')
`)
//await this.connection.close()
// this.logger.debug(`Result:${JSON.stringify(res)}`)
return {
result:Number(res[0]['result'])
}
}

I guess your issue comes from the way you use the db connection. You created ~ 6 or 10 connections to db, but you did not release them, and the default limit of mariadb's connection pool is 10 (I guess).
Finally, new request want to create new connection, but it reach the limit, new request waiting for another connection be release, but it waiting forever.
In this case, you can extend the limit (not root cause but I think it is good to know).
Add connectionLimit under extra in ormconfig.json (or where you create db config)
{
...,
"extra": { connectionLimit: 10, ... }
}
More information
And you have to release the new connection what you just created in getData function, right after the query finish (success or error) :
...
...
await queryRunner.release()
Note: Let's take care about when your query throw an error, you can use try/catch/finally
...
let res = await queryRunner.manager.query(`
select count(*) as result from my_table
where my_field ='I'
and DATE(date_field) = DATE('${dateField.toISOString()}')
`).finally(() => queryRunner.release())
...

Why is my data import to Algolia search using the API script timing out

I am trying to implement single index searching using Algoliasearch for my iOS mobile app. I have about 110 users on my application. However, when I upload their data to Algolia search's index the function times out before uploading all users. Instead it throws an Error message in the http browser and declares a timeout in the Firestore console.
Firestore console:
sendCollectionToAlgolia
Function execution took 60044 ms, finished with status: 'timeout'
I created the function using this tutorial:
https://medium.com/#soares.rfarias/how-to-set-up-firestore-and-algolia-319fcf2c0d37
Although i have ran into some complications, I highly recommend that tutorial if you have your app using swiftUI iOS platform and implement cloud functions using Typescript.
Heres my function:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
import algoliasearch from 'algoliasearch';
admin.initializeApp();
const db = admin.firestore();
const algoliaClient = algoliasearch(functions.config().algolia.appid, functions.config().algolia.apikey)
const collectionIndexName = functions.config().projectId === 'PROJECT-XXXX' ? 'prod_SEARCH' : 'dev_SEARCH';
const collectionIndex = algoliaClient.initIndex(collectionIndexName);
//rename to uploadUsersToAlgolia
export const sendCollectionToAlgolia = functions.https.onRequest(async (req, res) => {
const algoliaRecords: any[] = [];
const querySnapshot = await db.collection('users').get();
querySnapshot.docs.forEach(doc => {
const document = doc.data();
const record = {
objectID: doc.id,
fullname: document.fullname,
bio: document.bio,
username: document.username,
uid: document.uid,
profileImageURL: document.profileImageURL,
backgroundImageURL: document.backgroundImageURL,
fcmToken: document.fcmToken,
accountCreated: document.accountCreated,
inspirationCount: document.inspriationCount,
BucketListCount: document.BucketListCount,
CompletedBucketListCount: document.CompletedBucketListCount,
FriendsCount: document.FriendsCount
};
algoliaRecords.push(record);
});
// After all records are created, we save them to
collectionIndex.saveObjects(algoliaRecords, (_error: any, content: any) => {
res.status(200).send("users collection was indexed to Algolia successfully.");
});
});

If you just want to change the default 1 minute timeout, you can do that when you configure the function.
functions.runWith({timeoutSeconds: X}).https.onRequest(async (req, res)
Increasing the timeout won't help if your function doesn't end up sending a response, so you should also add some logging/debugging to figure out if the final call to res.send() is actually happening. If the function never sends a response, it will definitely time out no matter what happens.