For years I have been building apps with Vue-CLI which bundles Webpack. I see this prefix (protocol?) webpack:// but I don't actually know what it means.
For instance, in the generated source map for an app.xxx.js.map, I see:
{"version":3,"sources":["webpack:///webpack/bootstrap","webpack:///./src/App.vue?797c",...
So what does that webpack:// mean, exactly? It's obviously not a real protocol that the browser can resolve. So resolves it? And how?
The particular problem I'm having that leads me to want to understand this is that my generated chunk-vendors.8c348425.js.map begins like this:
{"version":3,"sources":["webpack:///js/chunk-vendors.68249437.js"],...
In this case, there is no generated chunk-vendors.68249437.js file anywhere. Should there be? Or is that an abstract generated name that doesn't actually refer to a real file?
I'm seeing errors in FireFox that I'm having a bit of trouble interpreting:
I think it's saying that it's (somehow) running that non-existent chunk-vendors.68249437.js file, and that is failing to fetch some (unidentified) resource?
tl;dr - webpack:// is an arbitrary protocol name to play nicely with browsers' source map viewers. Nesting each file name under this prefix displays them in a separate dropdown (webpack-demo in this case):
Webpack (most JS bundlers, really) take some number of uncompressed source files and combine them into a single* compressed** bundle file that can be loaded by a browser in a single network request.
At a high level, source maps exist to allow developers to easily translate console messages and stack traces from their file line/column number in the bundle back to their original position in a source file. This may be something as simple as "line 1001 in the bundle was actually line 500 in foo.js", or something more tricky like "lines 1001-1021 were part of the compilation output from line 20 in MyComponent.vue". You can read the latest spec for source maps, but in essence they contain three things:
Name of the output bundle file
Name of the input file(s) used to generate the bundle
Encoded mapping data connecting each line in (1) back to a file in (2)
The simplest source map looks something like this (omitted all but the relevant parts):
{
"file": "bundle.js",
"sourceRoot": "/",
"sources": ["foo.js", "bar.js"],
"mappings": "..."
}
This mapping is generated when two source files, foo.js and bar.js, are combined into a single bundle file, bundle.js. When a browser renders this source map, it expects to find the original source files at /foo.js and /bar.js, respectively.
Instead of hosting the files individually, a source map may instead contain the files' contents itself:
{
"file": "bundle.js",
"sources": ["foo.js", "bar.js"],
"sourcesContent": ["/* foo.js contents */", "/* bar.js contents */"],
"mappings": "..."
}
When rendering this source map, the browser just reads the file contents directly, instead of making a request to another URL. File names in sources are expected to be in the same order as sourcesContent.
Alright, we're almost there.
So long as the files are inlined in the source map (sourcesContent is set), the file names are arbitrary. In other words, foo.js doesn't have to correspond to a real file named foo.js, you could just as easily call it foo/bar/baz/garply.xyz and any source map viewer would have no issue rendering it (remember, the file contents are stored by array index).
Most source map viewers (I've checked Chrome and Firefox specifically) will display sources in a separate dropdown based on their protocol prefix. The screenshot above shows a number of files nested under webpack-demo; all of their sources entries begin with webpack://webpack-demo/.... It doesn't matter what the protocol is; foobar:// works just as well, with the caveat that each unique protocol prefix will get its own dropdown in the source map filetree. In fact, you can customize this prefix by setting devtoolModuleFilenameTemplate (see here).
So, in short - webpack:// is an arbitrary, unique prefix used to nest source files in their own dropdown in the source map filetree. The filenames that follow webpack:// are usually 1:1 with actual files in the filesystem, but that's not always the case***.
* OK, code splitting is a thing, but single bundle file in the simple case.
** A bundle file doesn't have to be compressed, it can simply be every source file concatenated one after the other, but in practice bundles are usually run through one or more minficiation/uglification passes.
*** Webpack may perform one or more intermediate compression passes, combining sets of vendor files from node_modules into chunks. I'm not 100% on the specifics here, but one way to avoid this is to use a different devtool (see here).
Related
I have a javascript file that isn't part of my project but that needs to be bundled up with it when it is served. This file has contents basically in the form...
FT.manifest({
"foo": "bar",
"id": 3
})
...where the Object part is well formed JSON.
I have webpack set up and am using the copy-webpack-plugin to move this file to my ./dist folder along with all my bundled JS and other assets.
Nothing fancy, just a straight file to file copy...
new CopyWebpackPlugin({
patterns: [
{from:`./src/templates/${dirName}/manifest.js`, to: `${dirName}/manifest.js`},
]
})
This works fine when I run webpack in development mode and the copied file looks exactly the same as the original but when I run in production mode the content get inlined and loses the speech marks from around the keys in the Object portion. The output now looks like this...
FT.manifest({foo:"bar",id:3})
Notice the lack of speech marks on foo and id. There is obviously some optimisation happening when --mode=production that is removing the line breaks and other formatting (which is fine) but it is also stripping the speech marks out and this causes problems for some steps that happen down the line.
So, can you tell my why this might be happening what I can do stop this particular file from being optimised during the copying process.
I can't do anything about the format of the particular file as it is from a third party but I can use other plugins or change the process at my end if needed.
Thanks for your help
I think the question of how to exclude certain files from the 'production' optimisation processes is a valid one but I've come round to the idea that my specific problem was more to do with my clumsy solution than anything else. I have found a way to do this that is quicker and more elegant.
What I was doing was to use copy-webpack-plugin to move the files from my ./src to my ./dist and then read them and manipulate them in place after they were emitted which adds a whole set of file system actions as well as requiring the js file to be read as a string and the Object portion to be pulled out and converted to JSON (which fails after the file has been optimised and the speech marks stripped out).
What I have learned is that the copy-webpack-plugin itself allows you to perform transformations on the file data as it is being copied. This means you get the data as it is in the original and can do away with the whole process of then re-reading and transforming the final emitted file.
{
from:`./src/${dirName}/myFile.js`,
to: `${dirName}/myFile.js`,
transform:{
transformer(content, absolutePath){
let str = content.toString();
// Make changes to str here and return the amended data
return str;
}
}
}
This is all there in the plugin docs so probably isn't new to most people but I didn't know you could do this. Maybe someone else will find it useful.
I've a Webpack 4.1 configuration that use code splitting and output chunks names using a pattern like myproj-[name]-[contenthash].chunk.js.
I'm copying all of the production bundle files, for every version, in the same directory on the server, being sure (until now) that chunks are unique and I have no clashing.
Today I found an issue releasing a new version of the application: I've a file named myproj-modulex-0bb2f31cc0ca424a07d8.chunk.js that was also generated with the old version (that's the scope of contenthash, isn't it?). I'm expecting that the content of the file is identical but it isn't.
There's only one character changed (the array index). The chunk start with...
(window.webpackJsonp_XXXX=window.webpackJsonp_XXXX||[]).push([[7],{"2d0274e27fde9220edd9"...
...while the old version was using ...push([[6],....
One of the difference of the new version from the old ones is that I added new code splitting points.
So: it seems that new split points changed chunks order, but webpack still use the same generated filename (probably because contenthash is referred to the real module content?).
The issue is critical: when the new file is copied on the server it overwrite the old file and so client using old version are not working anymore because chunk is loaded in a wrong position on the push array (I guess).
Error is:
"Error: Loading chunk 6 failed.
(missing: https://.../myproj-xxx-0bb2f31cc0ca424a07d8.chunk.js)"
There's a way to fix this issue, maybe naming pushed chunks, or specifying the order, or generated different hashes? chunkhash ?
Webpack uses ids as a chunk references and those ids are not guaranteed to remain the same for the same chunks among different builds. contenthash is used for files extracted by ExtractTextWebpackPlugin. The same source content will get the same contenthash but the generated file may differ due to id changes.
Try using myproj-[name]-[chunkhash].chunk.js instead.
Also take a look at optimization.moduleIds and optimization.chunkIds settings.
What would I use to find which resources are required by a NodeJS file?
For example, if I had a file called "file.js" containing this:
import x from './x';
const y = require('./y');
// Some more code
How do I parse that file and extract './x' and './y'?
Why would you do this?
I'm playing with the idea of an architectural tool. To do this, I want to know which files are being required by the targeted source code.
I know that Webpack follows this information when it creates bundles, so that it can stack the required files in an appropriate order in a single concatenated (well, minified) file.
I don't need to do the concatenation, but I want to find which files would be used.
When I find out which files are being used by which files, I plan to assist a user in organising them in an orderly manner (e.g. by pointing out circular dependencies).
For trivial cases, you could try feeding the source to some JS parser and search the AST for calls to require(); as long as require() is called with a string constant as a parameter, it shouldn't be hard to determine the dependencies. More complex situations could cause problems, though.
I understand this might just be impossible but when you're making JS available for easier debugging in devtools via the helpful //# sourceURL comment, I'd also like to map it to its respective local file, for easy editing.
Clarification on #// sourceURL=dynamicScript.js:
Note: Notice the "//# sourceURL=dynamicScript.js" line at the end of dynamicScript.js file. This technique gives a name to a script created with eval, and will be discussed in more detail in the Source Maps section. Breakpoints can be set in dynamic JavaScript only if it has a user supplied name.
The sourced file now exists in Sources under "no-domain", and is unable to map to my workspace's dynamicScript.js file.
You can map your local web app directory to a server path, so that you can live edit the JS file that evaluates some code, but there is no way to map the dynamically generated named script to a file on the system as far as I'm aware.
If you use eval to execute code from a string, adding //# sourceURL=dynamicScript.js' simply tells Chrome to simulate that script being an actual file, so that you can debug etc. The file doesn't actually exist, it's in memory. The dynamic 'file' cannot appear as part of a local workspace because it simply doesn't exist on the system.
I want to create a Chrome extension that contains a text file with static data (a dictionary of English words) and I want the extension to be able to parse that file. I've only managed to find FileReader class, but it looks like it's made for reading user-selected files, while in my case I always want to read the same exact file included in extension's package. As a workaround, I can convert the file to a Javascript array of strings declared in some .js file included in the manifest, but in that case the whole contents would be loaded into memory at once, while what I need is to read the data line by line. Is there any way to do this?
You can go the FileReader route, since you can obtain the Entry of your package directory with chrome.runtime.getPackageDirectoryEntry().
However, an easier way is to just make a XHR to your file using chrome.runtime.getURL() with a relative path. The first way is useful when you want to list files, though.