I am using node v14.17.0,"apollo-server-express": "^2.25.0", "graphql-upload": "^12.0.0"
I am trying to upload an image but i dont get createReadStream from the image which i upload from graphiql. I am using Altair graphiql
Here is my index.js
const app = express();
app.use(
graphqlUploadExpress({
maxFileSize: 30000000,
maxFiles: 20,
})
);
const server = new ApolloServer({
typeDefs,
resolvers,
context: contextMiddleware,
uploads: false,
});
server.applyMiddleware({ app });
app.listen(PORT, () => {
console.log("Success");
});
In my typedefs i have mentioned
gql`
scalar Upload
and my resolvers have
Upload: GraphQLUpload,
I have tried following this issue apollographql/apollo-server#3508 but it doesn't seem to work for me. Downgrading to nodev12 did not help.
My uploaded image contains following
{"filename":"download123.jpg","mimetype":"image/jpeg","encoding":"7bit"}
Below is a single file upload example that uses the graphql-upload and apollo-server-express packages.
Package versions:
"apollo-server-express": "^2.15.1",
"graphql-upload": "^12.0.0",
"graphql": "^14.6.0",
Nodejs version: v14.16.0.
GraphQL server:
import { ApolloServer, gql } from 'apollo-server-express';
import express from 'express';
import path from 'path';
import { createWriteStream, unlink } from 'fs';
import { graphqlUploadExpress } from 'graphql-upload';
const app = express();
const PORT = 3000;
const UPLOAD_DIR = path.resolve(__dirname, './uploads');
const typeDefs = gql`
scalar Upload
type Query {
dummy: String
}
type Mutation {
upload(upload: Upload!): Boolean
}
`;
const resolvers = {
Query: {},
Mutation: {
async upload(_, { upload }) {
const { file } = upload;
const { filename, mimetype, createReadStream } = await file;
const stream = createReadStream();
const id = Date.now();
const uploadPath = `${UPLOAD_DIR}/${id}-${filename}`;
console.log(filename, mimetype, createReadStream, stream);
await new Promise((resolve, reject) => {
const writeStream = createWriteStream(uploadPath);
writeStream.on('finish', resolve);
writeStream.on('error', (error) => {
unlink(uploadPath, () => {
reject(error);
});
});
stream.on('error', (error) => writeStream.destroy(error));
stream.pipe(writeStream);
});
return true;
},
},
};
app.use(
graphqlUploadExpress({
maxFileSize: 30000000,
maxFiles: 20,
}),
);
const server = new ApolloServer({
typeDefs,
resolvers,
uploads: false,
});
server.applyMiddleware({ app });
app.listen(PORT, () => console.log(`GraphQL server is listening on http://localhost:${PORT}/graphql`));
Send client GraphQL request using Altair graphql.
The logs of the server side, get the createReadStream successfully.
GraphQL server is listening on http://localhost:3000/graphql
matej-sefcik-GCRbVZydPT4-unsplash.jpg image/jpeg [Function: createReadStream] <ref *1> ReadStream {
_readableState: ReadableState {
objectMode: false,
highWaterMark: 16384,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: [],
flowing: null,
ended: false,
endEmitted: false,
reading: false,
sync: true,
needReadable: false,
emittedReadable: false,
readableListening: false,
resumeScheduled: false,
errorEmitted: false,
emitClose: true,
autoDestroy: true,
destroyed: false,
errored: null,
closed: false,
closeEmitted: false,
defaultEncoding: 'utf8',
awaitDrainWriters: null,
multiAwaitDrain: false,
readingMore: false,
decoder: null,
encoding: null,
[Symbol(kPaused)]: null
},
_events: [Object: null prototype] { close: [Function: remove] },
_eventsCount: 1,
_maxListeners: undefined,
_pos: 0,
_writeStream: WriteStream {
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: true,
ending: false,
ended: false,
finished: false,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 63781,
writing: true,
corked: 0,
sync: false,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: [Function: nop],
writelen: 63781,
afterWriteTickInfo: null,
buffered: [],
bufferedIndex: 0,
allBuffers: true,
allNoop: true,
pendingcb: 1,
prefinished: false,
errorEmitted: false,
emitClose: true,
autoDestroy: false,
errored: null,
closed: false
},
_events: [Object: null prototype] {
error: [Array],
unpipe: [Function: onunpipe],
close: [Function],
finish: [Function],
ready: [Function],
drain: [Function: pipeOnDrainFunctionResult]
},
_eventsCount: 6,
_maxListeners: undefined,
_fd: null,
_path: null,
_pos: 0,
_readStreams: Set(1) { [Circular *1] },
_released: false,
_cleanupSync: [Function (anonymous)],
[Symbol(kCapture)]: false
},
[Symbol(kCapture)]: false
}
Upload success:
source code: https://github.com/mrdulin/apollo-graphql-tutorial/tree/master/src/file-upload
I was able to resolve the error after upgrading to apollo-server v3. I think the v2 was somewhat conflicting with graphql-upload lib
const { filename, mimetype, createReadStream } = await file.promise;
Related
I have currently implemented Auth0 authentication on my NextJS app by following their docs, and am trying to call an external ExpressJS application by following their docs: https://github.com/auth0/nextjs-auth0/blob/main/EXAMPLES.md#access-an-external-api-from-an-api-route.
I have NextJS running on port 3000 and Express running on port 3001.
My express server is set up as follows:
const jwtCheck = expressjwt({
secret: jwksRsa.expressJwtSecret({
cache: true,
rateLimit: true,
jwksRequestsPerMinute: 5,
jwksUri: process.env.AUTH0_JWKS_URI as string,
}) as GetVerificationKey,
audience: process.env.AUTH0_AUDIENCE as string,
issuer: process.env.AUTH0_ISSUER as string,
algorithms: ['RS256'],
})
app.use(jwtCheck)
When I try to make an unauthenticated request from my NextJS app to the external API, I get the UnauthorizedError: No authorization token was found error which ensures it is working. If I comment out app.use(jwtCheck) the requests are successful as there is no auth check.
I have my NextJS application set up as follows:
// [...auth0].ts
import { handleAuth, handleLogin } from '#auth0/nextjs-auth0'
export default handleAuth({
login: handleLogin({
authorizationParams: {
audience: process.env.AUTH0_AUDIENCE, // or AUTH0_AUDIENCE
// Add the `offline_access` scope to also get a Refresh Token
scope: 'openid profile email read:trips', // or AUTH0_SCOPE
},
}),
})
// /api/trips.ts
import { getAccessToken, withApiAuthRequired } from '#auth0/nextjs-auth0'
const apiURL = process.env.NEXT_PUBLIC_SERVER_URL
export default withApiAuthRequired(async function trips(req, res) {
// If your access token is expired and you have a refresh token
// `getAccessToken` will fetch you a new one using the `refresh_token` grant
const { accessToken } = await getAccessToken(req, res, {
scopes: ['read:trips'],
})
const response = await fetch(`${apiURL}/trips`, {
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*',
Accept: 'application/json',
Authorization: `Bearer ${accessToken}`,
},
})
const trips = await response.json()
res.status(200).json(trips)
})
// /pages/trips/index.tsx
const TripsPage = (props: InferGetServerSidePropsType<typeof getServerSideProps>) => {
if (!props.user) return <div>User error!</div>
return (
<div>
// components that use the fetched data
</div>
)
}
export const getServerSideProps = withPageAuthRequired({
async getServerSideProps() {
const { data } = await axios.get('/api/trips')
try {
return {
props: {
data: data,
},
}
} catch (e) {
return {
notFound: true,
}
}
},
})
export default TripsPage
However, when I try to go to the http://localhost:3000/trips page after logging in, I get a error:
GET http://localhost:3000/trips 500 (Internal Server Error)
Uncaught Error: connect ECONNREFUSED ::1:80
at <unknown> (Error: connect ECONNREFUSED ::1:80)
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1487:16)
It seems like the request doesn't even leave my NextJS application. Seems like the internal API call failed. However, if I directly go to http://localhost:3000/api/trips I do get a response with the JSON data and if I console.log the accessToken it is valid. So authentication is working, but the issue is in the actual page which calls the NextJS proxy API, which calls my ExpressJS API.
The full error log on the NextJS console is:
error - AxiosError: connect ECONNREFUSED ::1:80
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1487:16) {
port: 80,
address: '::1',
syscall: 'connect',
code: 'ECONNREFUSED',
errno: -4078,
config: {
transitional: {
silentJSONParsing: true,
forcedJSONParsing: true,
clarifyTimeoutError: false
},
adapter: [Function: httpAdapter],
transformRequest: [ [Function: transformRequest] ],
transformResponse: [ [Function: transformResponse] ],
timeout: 0,
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
maxBodyLength: -1,
env: { FormData: [Function] },
validateStatus: [Function: validateStatus],
headers: {
Accept: 'application/json, text/plain, */*',
'User-Agent': 'axios/0.27.2'
},
method: 'get',
url: '/api/trips',
data: undefined
},
request: <ref *1> Writable {
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
buffered: [],
bufferedIndex: 0,
allBuffers: true,
allNoop: true,
pendingcb: 0,
constructed: true,
prefinished: false,
errorEmitted: false,
emitClose: true,
autoDestroy: true,
errored: null,
closed: false,
closeEmitted: false,
[Symbol(kOnFinished)]: []
},
_events: [Object: null prototype] {
response: [Function: handleResponse],
error: [Function: handleRequestError],
socket: [Function: handleRequestSocket]
},
_eventsCount: 3,
_maxListeners: undefined,
_options: {
maxRedirects: 21,
maxBodyLength: 10485760,
protocol: 'http:',
path: '/api/trips',
method: 'GET',
headers: [Object],
agent: undefined,
agents: [Object],
auth: undefined,
hostname: null,
port: null,
nativeProtocols: [Object],
pathname: '/api/trips'
},
_ended: true,
_ending: true,
_redirectCount: 0,
_redirects: [],
_requestBodyLength: 0,
_requestBodyBuffers: [],
_onNativeResponse: [Function (anonymous)],
_currentRequest: ClientRequest {
_events: [Object: null prototype],
_eventsCount: 7,
_maxListeners: undefined,
outputData: [],
outputSize: 0,
writable: true,
destroyed: true,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
maxRequestsOnConnectionReached: false,
_defaultKeepAlive: true,
useChunkedEncodingByDefault: false,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
strictContentLength: false,
_contentLength: 0,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
_closed: true,
socket: [Socket],
_header: 'GET /api/trips HTTP/1.1\r\n' +
'Accept: application/json, text/plain, */*\r\n' +
'User-Agent: axios/0.27.2\r\n' +
'Host: localhost\r\n' +
'Connection: close\r\n' +
'\r\n',
_keepAliveTimeout: 0,
_onPendingData: [Function: nop],
agent: [Agent],
socketPath: undefined,
method: 'GET',
maxHeaderSize: undefined,
insecureHTTPParser: undefined,
path: '/api/trips',
_ended: false,
res: null,
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: false,
host: 'localhost',
protocol: 'http:',
_redirectable: [Circular *1],
[Symbol(kCapture)]: false,
[Symbol(kBytesWritten)]: 0,
[Symbol(kEndCalled)]: true,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype],
[Symbol(errored)]: null,
[Symbol(kUniqueHeaders)]: null
},
_currentUrl: 'http:/api/trips',
[Symbol(kCapture)]: false
},
page: '/trips'
}
Any idea what's going on here?
Try using 127.0.0.1 instead of localhost. I had the same issue.
The issue was that we not allowed to call an internal NextJS /api/ route in getServerSideProps. We need to call the external API directly, and not through the internal /api proxy.
More information: https://github.com/auth0/nextjs-auth0/issues/995
I have been trying to make a music system to play songs in vcs using ytdl-core and ytsearcher while using discord js v13 for the bot. But I encounter a long error here it is:
node:events:504
throw er; // Unhandled 'error' event
^
AudioPlayerError: write EPIPE
at afterWriteDispatched (node:internal/stream_base_commons:160:15)
at writevGeneric (node:internal/stream_base_commons:143:3)
at Socket._writeGeneric (node:net:793:11)
at Socket._writev (node:net:802:8)
at doWrite (node:internal/streams/writable:406:12)
at clearBuffer (node:internal/streams/writable:561:5)
at onwrite (node:internal/streams/writable:461:7)
at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10)
Emitted 'error' event on AudioPlayer instance at:
at OggDemuxer.onStreamError (E:\_Documents\iobotnew\node_modules\#discordjs\voice\dist\index.js:956:14)
at Object.onceWrapper (node:events:646:26)
at OggDemuxer.emit (node:events:538:35)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
resource: <ref *3> AudioResource {
playStream: OggDemuxer {
_readableState: ReadableState {
objectMode: true,
highWaterMark: 16,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: [],
flowing: false,
ended: false,
endEmitted: false,
reading: true,
constructed: true,
sync: false,
needReadable: true,
emittedReadable: false,
readableListening: true,
resumeScheduled: false,
errorEmitted: true,
emitClose: true,
autoDestroy: true,
destroyed: true,
errored: Error: write EPIPE
at afterWriteDispatched (node:internal/stream_base_commons:160:15)
at writevGeneric (node:internal/stream_base_commons:143:3)
at Socket._writeGeneric (node:net:793:11)
at Socket._writev (node:net:802:8)
at doWrite (node:internal/streams/writable:406:12)
at clearBuffer (node:internal/streams/writable:561:5)
at onwrite (node:internal/streams/writable:461:7)
at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10) {
errno: -4047,
code: 'EPIPE',
syscall: 'write'
},
closed: true,
closeEmitted: false,
defaultEncoding: 'utf8',
awaitDrainWriters: null,
multiAwaitDrain: false,
readingMore: false,
dataEmitted: false,
decoder: null,
encoding: null,
[Symbol(kPaused)]: null
},
_events: [Object: null prototype] {
prefinish: [Function: prefinish],
close: [
[Function (anonymous)],
[Function: onclose],
[Function: bound onceWrapper] {
listener: [Function: onFailureCallback]
}
],
end: [
[Function: onend],
[Function: bound onceWrapper] {
listener: [Function: onFailureCallback]
}
],
finish: [
[Function: onfinish],
[Function: bound onceWrapper] {
listener: [Function: onFailureCallback]
}
],
error: [Function: onerror],
readable: [
[Function: bound onceWrapper] {
listener: [Function (anonymous)]
},
[Function: bound onceWrapper] {
listener: [Function: onReadableCallback]
}
]
},
_eventsCount: 6,
_maxListeners: undefined,
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: true,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
buffered: [],
bufferedIndex: 0,
allBuffers: true,
allNoop: true,
pendingcb: 0,
constructed: true,
prefinished: false,
errorEmitted: true,
emitClose: true,
autoDestroy: true,
errored: Error: write EPIPE
at afterWriteDispatched (node:internal/stream_base_commons:160:15)
at writevGeneric (node:internal/stream_base_commons:143:3)
at Socket._writeGeneric (node:net:793:11)
at Socket._writev (node:net:802:8)
at doWrite (node:internal/streams/writable:406:12)
at clearBuffer (node:internal/streams/writable:561:5)
at onwrite (node:internal/streams/writable:461:7)
at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10) {
errno: -4047,
code: 'EPIPE',
syscall: 'write'
},
closed: true,
closeEmitted: false,
[Symbol(kOnFinished)]: []
},
allowHalfOpen: true,
_remainder: null,
_head: null,
_bitstream: null,
[Symbol(kCapture)]: false,
[Symbol(kCallback)]: null
},
edges: [
<ref *1> {
type: 'ffmpeg ogg',
to: Node {
edges: [ [Object], [Object], [Object] ],
type: 'ogg/opus'
},
cost: 2,
transformer: [Function: transformer],
from: Node { edges: [ [Object], [Circular *1] ], type: 'arbitrary' }
},
<ref *2> {
type: 'ogg/opus demuxer',
to: Node { edges: [ [Object] ], type: 'opus' },
cost: 1,
transformer: [Function: transformer],
from: Node {
edges: [ [Circular *2], [Object], [Object] ],
type: 'ogg/opus'
}
}
],
metadata: null,
volume: undefined,
encoder: undefined,
audioPlayer: <ref *4> AudioPlayer {
_events: [Object: null prototype] { idle: [Function (anonymous)] },
_eventsCount: 1,
_maxListeners: undefined,
_state: {
status: 'buffering',
resource: [Circular *3],
onReadableCallback: [Function: onReadableCallback],
onFailureCallback: [Function: onFailureCallback],
onStreamError: [Function: onStreamError]
},
subscribers: [
PlayerSubscription {
connection: VoiceConnection {
_events: [Object: null prototype] {},
_eventsCount: 0,
_maxListeners: undefined,
rejoinAttempts: 0,
_state: [Object],
joinConfig: [Object],
packets: [Object],
receiver: [VoiceReceiver],
debug: null,
onNetworkingClose: [Function: bound onNetworkingClose],
onNetworkingStateChange: [Function: bound onNetworkingStateChange],
onNetworkingError: [Function: bound onNetworkingError],
onNetworkingDebug: [Function: bound onNetworkingDebug],
[Symbol(kCapture)]: false
},
player: [Circular *4]
],
behaviors: { noSubscriber: 'pause', maxMissedFrames: 5 },
debug: [Function (anonymous)],
[Symbol(kCapture)]: false
},
playbackDuration: 0,
started: false,
silencePaddingFrames: 5,
silenceRemaining: -1
}
}
If anyone can help me here is my code:
const { SlashCommandBuilder } = require('#discordjs/builders');
const ytdl = require('ytdl-core');
const { joinVoiceChannel, createAudioResource, createAudioPlayer, AudioPlayerStatus } = require('#discordjs/voice');
const { YTSearcher } = require('ytsearcher');
const searcher = new YTSearcher({
key: "myyoutubeapikey",
revealKey: true
})
module.exports = {
data: new SlashCommandBuilder()
.setName('play')
.setDescription('Permet de jouer de la musique dans votre salon vocal !')
.addStringOption(option => option.setName('song').setDescription('Le nom de la musique / URL').setRequired(true)),
async execute(interaction) {
const song = interaction.options.getString('song');
let vc = interaction.member.voice.channel;
if(!vc){
return interaction.reply({ content: ":warning: **- Vous devez être dans un salon vocal pour pouvoir jouer de la musique !**", ephemeral: true });
}
let result = await searcher.search(song, { type: "video" })
console.log(result.first.url)
const stream = ytdl(result.first.url, {
filter: "audioonly",
});
const player = createAudioPlayer();
const resource = createAudioResource(stream);
const connections = joinVoiceChannel({
channelId: vc.id,
guildId: vc.guild.id,
adapterCreator: vc.guild.voiceAdapterCreator,
})
player.play(resource);
connections.subscribe(player);
player.on(AudioPlayerStatus.Idle, () => {
connections.destroy();
});
}
};
I tried a lot of things and I looked before making this post if someone already had the same problem (with the same context)
Try to replace
const stream = ytdl(result.first.url, {
filter: "audioonly",
});
on the
const stream = ytdl(result.first.url, {
filter: "audioonly",
quality: 'highestaudio',
highWaterMark: 1 << 25
});
I am getting back a response in node to fetch a PDF that I would like to save the binary data and upload it to S3 which I can then pull as I please for historical purposes. Right now I am having problems trying to extract that PDF raw data/binary data so that I can save that as a PDF and upload it, I think I am getting closer. this is the response.body that I am getting.
PassThrough {
_readableState: ReadableState {
objectMode: false,
highWaterMark: 16384,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: null,
pipesCount: 0,
flowing: null,
ended: false,
endEmitted: false,
reading: false,
sync: false,
needReadable: false,
emittedReadable: false,
readableListening: false,
resumeScheduled: false,
emitClose: true,
autoDestroy: false,
destroyed: false,
defaultEncoding: 'utf8',
awaitDrainWriters: null,
multiAwaitDrain: false,
readingMore: false,
decoder: null,
encoding: null,
[Symbol(kPaused)]: null
},
readable: true,
_events: [Object: null prototype] {
prefinish: [Function: prefinish],
unpipe: [Function: onunpipe],
error: [ [Function: onerror], [Function] ],
close: [Function: bound onceWrapper] { listener: [Function: onclose] },
finish: [Function: bound onceWrapper] { listener: [Function: onfinish] }
},
_eventsCount: 5,
_maxListeners: undefined,
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
bufferedRequest: null,
lastBufferedRequest: null,
pendingcb: 0,
prefinished: false,
errorEmitted: false,
emitClose: true,
autoDestroy: false,
bufferedRequestCount: 0,
corkedRequestsFree: {
next: null,
entry: null,
finish: [Function: bound onCorkedFinish]
}
},
writable: true,
allowHalfOpen: true,
_transformState: {
afterTransform: [Function: bound afterTransform],
needTransform: false,
transforming: false,
writecb: null,
writechunk: null,
writeencoding: null
},
[Symbol(kCapture)]: false
}
This is my code:
platform.on(platform.events.loginSuccess, async function(e) {
let resp = await platform.get('/restapi/v1.0/account/~/extension/'+accountId+'/message-store', {
messageType: ['Fax'],
dateFrom: ['2021-05-01'],
dateTo: ['2021-05-16']
})
let jsonObj = await resp.json()
let attachId = jsonObj.records[0].id
let getMessageContent = await platform.get('/restapi/v1.0/account/~/extension/'+accountId+'/message-store/'+attachId+'/content/'+attachId, {
contentDisposition: ['Inline']
})
I know the response I am getting is correct because if I run that URL in postman and use the preview option, I can see the pdf (see screenshot). What am I missing?
That looks about right to me. The response.body is a readable node.js stream containing the binary data, and my guess is that Postman reads this stream automatically and displays the result.
The good news is that this type of stream is exactly what S3 wants as a request body when uploading files, which means that you do not need to save it anywhere - you can just pass it through directly to S3. Depending on your implementation, your code could look something like this:
const result = await s3Client.upload({
Bucket: 'MY_S3_BUCKET',
Key: 'myfilename.pdf',
Body: response.body,
}).promise();
Where response.body is the stream you have logged above. This should put the pdf file right up in S3 ready for downloading.
But if you do insist on saving the file, you can also pass the stream to the fs.writeFileSync method like so:
fs.writeFileSync('my-pdf.pdf', response.body);
I am trying to Upload stream to AWS s3. now my File Manager can upload stream of objects on server without any load or bloating. But when it comes to AWS s3 it returns successful response and even AWS SDK is not reporting anything about misconfiugration it also returns Successful Upload so my File Manager is fine But. Problem is that when i take a look on AWS Console i only see path of given file not file itself..For better Understanding Please take a look at following code and Images.
Stream to Upload (File)
ReadStream {
_readableState: ReadableState {
objectMode: false,
highWaterMark: 65536,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: null,
pipesCount: 0,
flowing: null,
ended: false,
endEmitted: false,
reading: false,
sync: true,
needReadable: false,
emittedReadable: false,
readableListening: false,
resumeScheduled: false,
emitClose: false,
autoDestroy: false,
destroyed: false,
defaultEncoding: 'utf8',
awaitDrain: 0,
readingMore: false,
decoder: null,
encoding: null,
[Symbol(kPaused)]: null
},
readable: true,
_events: [Object: null prototype] {
end: [ [Function], [Function: remove] ],
error: [Function],
close: [Function: remove]
},
_eventsCount: 3,
_maxListeners: undefined,
path: '/tmp/capacitor-7957c2e3611868508c7ac8ec64c1a7c2.tmp',
fd: null,
flags: 'r',
mode: 438,
start: undefined,
end: Infinity,
autoClose: true,
pos: undefined,
bytesRead: 0,
closed: false,
name: undefined,
_writeStream: WriteStream {
_writableState: WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: true,
needDrain: false,
ending: true,
ended: true,
finished: true,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: false,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
afterWriteTickInfo: null,
bufferedRequest: null,
lastBufferedRequest: null,
pendingcb: 0,
prefinished: true,
errorEmitted: false,
emitClose: false,
autoDestroy: false,
bufferedRequestCount: 0,
corkedRequestsFree: [Object]
},
writable: false,
_events: [Object: null prototype] { error: [Function] },
_eventsCount: 1,
_maxListeners: undefined,
path: '/tmp/capacitor-7957c2e3611868508c7ac8ec64c1a7c2.tmp',
fd: 30,
flags: 'w',
mode: 438,
start: undefined,
autoClose: false,
pos: undefined,
bytesWritten: 46355,
closed: false,
_readStreams: Set { [Circular] },
error: null,
_cleanupSync: [Function],
[Symbol(kCapture)]: false,
[Symbol(kIsPerformingIO)]: false
},
error: null,
[Symbol(kCapture)]: false,
[Symbol(kIsPerformingIO)]: false
}
FileManager AWS S3 Code
if (this.configuration.Aws && this.configuration.Aws.s3 && this.configuration.Aws.s3.accessKey && this.configuration.Aws.s3.secretKey) {
// Local variable.
let _Aws, _StreamUpload
// Upload file to s3
_Aws = new Aws.S3({
'accessKeyId': this.configuration.Aws.s3.accessKey,
'secretAccessKey': this.configuration.Aws.s3.secretKey
})
/*
* Only successful upload
* reply to client else report
* failure.
*/
if (!_.isEmpty(_Aws) && !(_Aws instanceof Error)) {
console.log('erw---rer>>', file)
/*
* Steam upload to aws.
* upload file to aws.
*/
_StreamUpload = await new Promise((__resolve, __reject) =>
// Upload file stream to aws s3
_Aws.upload({
'Bucket': this.configuration.Aws.s3.bucketName,
'Key': path,
'Body': file
}, (error, __data) => error instanceof Error ? __reject(error) : __resolve(__data)))
/*
* Only return _StreamUpload if
* uploading didnt contain any error.
*/
if (_StreamUpload && !(_StreamUpload instanceof Error)) {
// Return upload.
return { 'storageType': 's3', 'path': _StreamUpload.Location }
}
// Report failure.
return _StreamUpload
}
// Report failure.
return _.isEmpty(_Aws) ? new Error('UPLOAD_FAILED(AWS)') : _Aws
}
AWS S3 Response
{
ETag: '"1ea966ff33714024a163d51e06b9bad8"',
Location: 'https://truckpe.s3.amazonaws.com//ck7ef9esk000k0758t9r3mngq/1585455100505/',
key: '/ck7ef9esk000k0758t9r3mngq/1585455100505/',
Key: '/ck7ef9esk000k0758t9r3mngq/1585455100505/',
Bucket: 'truckpe'
}
AWS S3 Console Screen Shot
Following the Axios NuxtJS config, I created a proxy config like this on my nuxt.config.js:
proxy: {
'/api/': {
target: 'https://myapidomain.com/',
pathRewrite: { '^/api/': '' },
changeOrigin: true
}
}
This config works perfectly on dev environment, both server side rendering and client side rendering. Here is the code we use to create an api wrapper api.js:
export default (context, inject) => {
inject('api', {
getPageForSlug: (slugRoute) => {
return context.$axios.$get(`/api/pageForSlug?routeName=${slugRoute}`)
},
})
}
and then from any vue class:
const response = await app.$api.getPageForSlug(params.slug_route)
the only problem occurs when this code is called from the zeit now deployment. The client side works perfectly, but the server side returns this error:
START RequestId: dd92dbad-135f-414b-bc1d-df9faffaa681 Version: $LATEST
2019-08-30T17:46:03.098Z dd92dbad-135f-414b-bc1d-df9faffaa681 INFO λ Cold start took: 5265.617811ms
END RequestId: dd92dbad-135f-414b-bc1d-df9faffaa681
REPORT RequestId: dd92dbad-135f-414b-bc1d-df9faffaa681 Duration: 438.53 ms Billed Duration: 500 ms Memory Size: 3008 MB Max Memory Used: 92 MB
N',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
validateStatus: [Function: validateStatus],
data: undefined },
request:
Writable {
_writableState:
WritableState {
objectMode: false,
highWaterMark: 16384,
finalCalled: false,
needDrain: false,
ending: false,
ended: false,
finished: false,
destroyed: false,
decodeStrings: true,
defaultEncoding: 'utf8',
length: 0,
writing: false,
corked: 0,
sync: true,
bufferProcessing: false,
onwrite: [Function: bound onwrite],
writecb: null,
writelen: 0,
bufferedRequest: null,
lastBufferedRequest: null,
pendingcb: 0,
prefinished: false,
errorEmitted: false,
emitClose: true,
autoDestroy: false,
bufferedRequestCount: 0,
corkedRequestsFree: [Object] },
writable: true,
domain: null,
_events:
[Object: null prototype] { response: [Function], error: [Function] },
_eventsCount: 2,
_maxListeners: undefined,
_options:
{ protocol: 'http:',
maxRedirects: 21,
maxBodyLength: 10485760,
path: '/api/pageForSlug?routeName=lud_form',
method: 'GET',
headers: [Object],
agent: undefined,
auth: undefined,
hostname: 'localhost',
port: '3000',
nativeProtocols: [Object],
pathname: '/api/pageForSlug',
search: '?routeName=lud_form' },
_redirectCount: 0,
_redirects: [],
_requestBodyLength: 0,
_requestBodyBuffers: [],
_onNativeResponse: [Function],
_currentRequest:
ClientRequest {
domain: null,
_events: [Object],
_eventsCount: 6,
_maxListeners: undefined,
output: [],
outputEncodings: [],
outputCallbacks: [],
outputSize: 0,
writable: true,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
useChunkedEncodingByDefault: false,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
_contentLength: 0,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
socket: [Socket],
connection: [Socket],
_header:
'GET /api/pageForSlug?routeName=lud_form HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nx-now-deployment-url: lps-5fxmi58fz.now.sh\r\nx-now-trace: staging-gru1\r\nx-real-ip: 177.45.65.235\r\nx-zeit-co-forwarded-for: 177.45.65.235\r\nupgrade-insecure-requests: 1\r\nx-forwarded-proto: https\r\nx-now-id: 7xllz-1567187175018-d0ce35475a9e\r\naccept-encoding: gzip, deflate\r\nuser-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36\r\nx-forwarded-for: 177.45.65.235\r\nx-forwarded-host: lps-danicuki.playax.now.sh\r\ndnt: 1\r\naccept-language: en-US,en;q=0.9\r\nconnection: close\r\nHost: localhost:3000\r\n\r\n',
_onPendingData: [Function: noopPendingOutput],
agent: [Agent],
socketPath: undefined,
timeout: undefined,
method: 'GET',
path: '/api/pageForSlug?routeName=lud_form',
_ended: false,
res: null,
aborted: undefined,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
_redirectable: [Circular],
[Symbol(isCorked)]: false,
[Symbol(outHeadersKey)]: [Object] },
_currentUrl: 'http://localhost:3000/api/pageForSlug?routeName=lud_form' },
response: undefined,
isAxiosError: true,
toJSON: [Function] }
END RequestId: 6e355938-32d5-459a-adf7-08fb97101e29
REPORT RequestId: 6e355938-32d5-459a-adf7-08fb97101e29 Duration: 281.53 ms Billed Duration: 300 ms Memory Size: 3008 MB Max Memory Used: 117 MB
RequestId: 6e355938-32d5-459a-adf7-08fb97101e29 Error: Runtime exited with error: exit status 1
Runtime.ExitError
How to make the server side API request work on all environments?
This can be achieved using a rewrite in your now.json Vercel file. No need for a hack.
{
"version": 2,
"builds": [
{
"src": "nuxt.config.js",
"use": "#nuxtjs/vercel-builder",
"config": {}
}
],
"rewrites": [
{
"source": "/api/:match*",
"destination": "https://myapidomain.com/:match*"
}
]
}
Are you ready for the ultimate hack?
This is actually a common issue. When calling zeit now, a server doesn't actually get propped up. As such, you'll need to do it yourself.
The only time I've encountered a need to do this, I used express to solve it. A quick and dirty solution is to create another script (proxy.js), and have something like so:
import express from 'express'
import proxy from 'express-http-proxy'
const app = express()
app.use('/', proxy(`https://myapidomain.com`))
export default app
This does nothing but set up a reverse proxy to your API.
From there, in your zeit config, find your builds key if you have one, and replace it with this:
"builds": [
{ "use": "#now/next", "src": "package.json" },
{ "use": "#now/node", "src": "proxy.js" }
]
This will prop up and execute proxy.js when you build using zeit now.
Do note that this does mean you'll need to get the host in proxy.js to match the "normal" host. If you know the format of your config file (which you should), it may be possible to import and parse that in order to have one source of truth.
Let me know if this worked.