Using Oak, how can I serve HTML without the extension? e.g.
host:port/home.html -> host:port/home
Here's my current code to render my public/views folder:
router.get('/:page', async (ctx: Context, next: () => Promise<unknown>) => {
await send(ctx, ctx.request.url.pathname, {
root: join(Deno.cwd(), 'public', 'views'),
extensions: ['htm', 'html']
});
await next();
});
The extensions option is not working or maybe I just use it the wrong way.
Edit
My fix is currently removing the .html extension (e.g. home.html -> home). Pretty sure there's a better way than this
You can use this to send the file:
router.get('/path', async (ctx:any) => {
const text = await Deno.readTextFile('./file.html');
ctx.response.headers.set("Content-Type", "text/html")
ctx.response.body = text;
});
Related
Since zlib has been added to node.js I'd like to ask a question about unzipping .gz with async/await style, w/o of using streams, one by one.
In the code below I am using fs-extra instead of standard fs & typescript (instead of js), but as for the answer, it doesn't matter will it have js or ts code.
import fs from 'fs-extra';
import path from "path";
import zlib from 'zlib';
(async () => {
try {
//folder which is full of .gz files.
const dir = path.join(__dirname, '..', '..', 'folder');
const files: string[] = await fs.readdir(dir);
for (const file of files) {
//read file one by one
const
file_content = fs.createReadStream(`${dir}/${file}`),
write_stream = fs.createWriteStream(`${dir}/${file.slice(0, -3)}`,),
unzip = zlib.createGunzip();
file_content.pipe(unzip).pipe(write_stream);
}
} catch (e) {
console.error(e)
}
})()
As for now, I have this code, based on streams, which is working, but in various StackOverflow answers, I haven't found any example with async/await, only this one, but it also uses streams I guess.
So does it even possible?
//inside async function
const read_file = await fs.readFile(`${dir}/${file}`)
const unzip = await zlib.unzip(read_file);
//write output of unzip to file or console
I understand that this task will block the main thread. It's ok for me since I write a simple day schedule script.
Seems I have figure it out, but I am still not hundred percent sure about it, here is example of full IIFE:
(async () => {
try {
//folder which is full of .gz files.
const dir = path.join(__dirname, '..', '..', 'folder');
const files: string[] = await fs.readdir(dir);
//parallel run
await Promise.all(files.map(async (file: string, i: number) => {
//let make sure, that we have only .gz files in our scope
if (file.match(/gz$/g)) {
const
buffer = await fs.readFile(`${dir}/${file}`),
//using .toString() is a must, if you want to receive readble data, instead of Buffer
data = await zlib.unzipSync(buffer , { finishFlush: zlib.constants.Z_SYNC_FLUSH }).toString(),
//from here, you can write data to a new file, or parse it.
json = JSON.parse(data);
console.log(json)
}
}))
} catch (e) {
console.error(e)
} finally {
process.exit(0)
}
})()
If you have many files in one directory, I guess you could use await Promise.all(files.map => fn()) to run this task in parallel. Also, in my case, I required to parse JSON, so remember about some nuances of JSON.parse.
I'm implementing a youtube video downloader using ytdl-core with Nodejs backend and Reactjs frontend. However, using ytdl-core library I'm able to send to youtube video file to frontend with this codeblock
app.get('/download', (req, res) => {
let { url, itag } = req.query;
let id = ytdl.getURLVideoID(url);
ytdl.getInfo(id, (err, info) => {
if (err) {
console.log(err);
throw err;
}
else{
let audioandvideo = ytdl.filterFormats(info.formats, 'audioandvideo');
let video = audioandvideo.filter(obj => obj.itag === itag);
video = video[0]
res.header('Content-Disposition', `attachment; filename="${info.title}.${video.container}"`);
res.header('Content-Type', 'video/webm');
ytdl(url, {
format: video
}).pipe(res);
}
})
});
However, the file downloads correctly if I redirect the webpage to the route like this
window.location.href = `http://localhost:5000/download?url=${this.state.url}&itag=${itag}`;
This works fine and the video downloads correctly. But as it's a redirection I can't do that in a hosted site. So, I need to use axios to do this.
I did some research and found out some solutions. I tried with js-file-download library following the accepted answer here. It downloads the file to the client directory but the file won't play. This is the codeblock I used for that
downloadVideo(itag){
axios.get(`http://localhost:5000/download`, {
params: { url: this.state.url, itag },
})
.then(response => {
fileDownload(response.data, `video.mp4`);
})
.catch(err => console.log(err));
}
As it's not working I tried another approach mentioned in the previously mentioned StackOverflow answer as well. It also downloads the file but doesn't work as well.
How can I get this fixed? What may be the reason my axios request doesn't work correctly?
EDIT :
downloadVideo(itag){
axios.get(`http://localhost:5000/download`, {
params: { url: this.state.url, itag },
responseType: Blob
})
.then(response => {
fileDownload(response.data, `video.mp4`);
})
.catch(err => console.log(err));
// window.location.href = `http://localhost:5000/download?url=${this.state.url}&itag=${itag}`;
}
This is the frontend code. If I use the commented code block (window.location.href) instead of axios.get the file gets downloaded and it works. But if I use axios.get a file gets downloaded but it seems to be a broken file as it's not playing.
I want to test scripts in an environment where we can not export modules. I have installed Jest version 23.1.0 and there aren't other packages in my package.json file.
Using jsdom 'old' api I have come up with a solution that works as expected:
script.js
var exVar = "test";
script.test.js
const jsdom = require('jsdom/lib/old-api.js');
test('old jsdom api config', function(done) {
jsdom.env({
html: "<html><body></body></html>",
scripts: [__dirname + "/script.js"],
done: function (err, window) {
expect(window.exVar).toBe("test");
done();
}
});
});
However with this implementation I have to re-write the config for every test, because it looks like the jsdom config gets re-written every time.
What I have tried
So far I have tried running this configuration:
const jsdom = require('jsdom/lib/old-api.js');
jsdom.env({
html: "<html><body></body></html>",
scripts: [__dirname + "/script.js"],
done: function (err, window) {
console.log('end');
}
});
with this test:
test('old jsdom api config', function(done) {
expect(window.exVar).toBe("test");
done();
});
in different ways: inside beforeAll, inside a script linked through setupFiles or through setupTestFrameworkScriptFile in the Jest configuration object, but still nothing works.
Maybe I could extend jest-environment as suggested in the docs, but I have no idea of the syntax I should be using, nor of how to link this file to the tests.
Thanks to my co-worker Andrea Talon I have found a way of using the same setup for different tests (at least inside the same file) using the 'Standard API' (not the 'old API').
Here is the complete test file.
const {JSDOM} = require("jsdom")
const fs = require("fs")
// file to test
const srcFile = fs.readFileSync("script.js", { encoding: "utf-8" })
// the window
let window
describe('script.js test', () => {
beforeAll((done) => {
window = new JSDOM(``, {
runScripts: "dangerously"
}).window
const scriptEl = window.document.createElement("script")
scriptEl.textContent = srcFile
window.document.body.appendChild(scriptEl)
done()
})
test('variable is correctly working', (done) => {
expect(window.exVar).toBe("test");
done()
})
})
Additional setup
In order to load multiple scripts I have created this function which accepts an array of scripts:
function loadExternalScripts (window, srcArray) {
srcArray.forEach(src => {
const scriptEl = window.document.createElement("script")
scriptEl.textContent = src
window.document.body.appendChild(scriptEl)
});
}
So instead of appending every single script to the window variable I can load them by declaring them at the top of the file like this:
// files to test
const jQueryFile = fs.readFileSync("jquery.js", { encoding: "utf-8" })
const srcFile = fs.readFileSync("lib.js", { encoding: "utf-8" })
and then inside the beforeAll function I can load them altogether like this:
loadExternalScripts(window, [jQueryFile, srcFile])
This question already has answers here:
Why don't self-closing script elements work?
(12 answers)
Closed 4 years ago.
I'm building an application using React an Apollo 2 and I'm trying to write a script to generate static html files for server-side rendering.
I'm currently using this logic to render my React component on the server and it's working fine:
export default (Page) => (req, res) => {
const App = (
<ApolloProvider client={client(fetch)}>
<Main>
<Page />
</Main>
</ApolloProvider>
);
getDataFromTree(App).then(() => {
const data = {
html: ReactDOMServer.renderToString(App),
seo: { title: "Home", description: "" }
};
res.render("index", { data });
});
}
But I wanted to generate the .html files so I could serve them statically, without running this method every time. I tried to compile the result of renderToString to an .ejs file and then write to a file using fs, but it didn't work. This is the code:
const component = ReactDOMServer.renderToString(App);
const template = ejs.compile(fs.readFileSync(path.join(__dirname, 'landing_template.ejs'), 'utf-8'));
const html = template({ component });
fs.writeFile(
path.join(__dirname, "../views/landing.ejs"),
html,
err => {
if(err) console.log(err);
else console.log("done.");
});
The file was written successfully but If I open my Page Source the part added by the compile method is grayed out and seems to be ignored:
I also tried just reading the file and replacing the string using .replace and inserting a dummy div, but it's always the same result.
// using compile
const template = ejs.compile(fs.readFileSync(path.join(__dirname, 'landing_template.ejs'), 'utf-8'));
const html = template({ component });
const template = ejs.compile(fs.readFileSync(path.join(__dirname, 'landing_template.ejs'), 'utf-8'));
const html = template({ component: "<div>teste</div>" });
// using readFile
const template = fs.readFileSync(path.join(__dirname, 'landing_template.ejs'), 'utf-8');
const html = template.replace("<%- component %>", component);
const template = fs.readFileSync(path.join(__dirname, 'landing_template.ejs'), 'utf-8');
const html = template.replace("<%- component %>", "<div>teste</div>");
// maybe this is the problem?
fs.writeFile(
path.join(__dirname, "../views/landing.ejs"),
html,
{ encoding: "utf8" },
err => {
if(err) console.log(err);
else console.log("done.");
});
I guess the browser is not recognizing this new text as HTML and failing to parse it. Does any one know a way I can make this work?
Thanks!
<script /> is wrong. You have to use this:
<script src="URL"></script>
What a cool idea! This is probably happening because, as you mentioned, the browser is not recognizing the fact that you are sending html. Try explicitly setting the Content-Type header to text/html before calling render:
export default (Page) => (req, res) => {
const App = (
<ApolloProvider client={client(fetch)}>
<Main>
<Page />
</Main>
</ApolloProvider>
);
getDataFromTree(App).then(() => {
const data = {
html: ReactDOMServer.renderToString(App),
seo: { title: "Home", description: "" }
};
res.setHeader("Content-Type", "text/html");
res.render("index", { data });
});
}
Edit:
Ah I see it now! You cannot self-close <script> tags in HTML unfortunately, and this is tripping the browser up. Close the script tag in your header like so:
<script async src="/build/app.bundle.js" type="text/javascript"></script>
That should fix the problem.
Does anyone have a proper example on how to configure BrowserMobProxy with WebDriverIO? This is so I can capture network traffic. I previously had it working with WebDriverJS, which is essentially a deprecated version of WebDriverIO.
You can use the below code to do that. Make sure your browsermob proxy and selenium server is running. Then copy paste below code in a test.js file and put it in webdriverio installed folder. From cmd go to that folder and run node test.js . stuff.har should be generated where test.js is located.
var Proxy = require('browsermob-proxy').Proxy
, webdriverio = require("./node_modules/webdriverio/")
, fs = require('fs')
, proxy = new Proxy()
;
proxy.cbHAR('search.yahoo.com', doSeleniumStuff, function(err, data) {
if (err) {
console.error('ERR: ' + err);
} else {
fs.writeFileSync('stuff.har', data, 'utf8');
}
});
function doSeleniumStuff(proxy, cb) {
var browser = webdriverio.remote({
host: 'localhost'
, port: 4444
, desiredCapabilities: { browserName: 'firefox', seleniumProtocol: 'WebDriver', proxy: { httpProxy: proxy } }
});
browser
.init()
.url("http://search.yahoo.com")
.setValue("#yschsp", "javascript")
.submitForm("#sf")
.end().then(cb);
}
If you just want to capture the network traffic, then there is one more way to do it.
Webdriverio allows you to use Chrome Dev Tools Protocol.
Please read webdriverio blog
This is one of the examples on how to use chrome dev tools along with webdriverio, do let me know in case you need more help.
const { remote } = require('webdriverio')
let browser;
(async () => {
browser = await remote({
automationProtocol: 'devtools',
capabilities: {
browserName: 'chrome'
}
})
await browser.url('https://webdriver.io')
await browser.call(async () => {
const puppeteerBrowser = browser.getPuppeteer()
const page = (await puppeteerBrowser.pages())[0]
await page.setRequestInterception(true)
page.on('request', interceptedRequest => {
if (interceptedRequest.url().endsWith('webdriverio.png')) {
return interceptedRequest.continue({
url: 'https://user-images.githubusercontent.com/10379601/29446482-04f7036a-841f-11e7-9872-91d1fc2ea683.png'
})
}
interceptedRequest.continue()
})
})
// continue with WebDriver commands
await browser.refresh()
await browser.pause(2000)
await browser.deleteSession()
})().catch(async (e) => {
console.error(e)
await browser.deleteSession()
})
Since I had no luck solving this problem using browsermob proxy (AFAIK it wasn't updated in a while)
I created a small npm module to capture selenium tests as HAR files - https://www.npmjs.com/package/har-recorder
I took #Raulster24 suggestion and implemented it using the Chrome Dev Tools Protocol - https://github.com/loadmill/har-recorder/blob/master/index.js