We Keep Coding

Fetch error when the URL includes a variable that has URL encoding - but it works when copied into a browser / edit: and when preventDefault() is used

I'm not sure I really understand why this is being treated this way - is it because fetch tries to encode all variables? If so how do I fix it?
This fetch URL will work when the only variables I put into it are coordinates (numbers). In my overall code, I have a string that I encoded myself from an array that can have a varying bracket structure, so that is the reasoning.
When I copy and paste this into a browser and paste in my variable where it should go, it works. But the fetch doesn't work.
Is fetch trying to do something more to my variable that already has encoding? I don't understand, because the non-variable part of the string is already encoded, so I'm assuming the issue has to do with how fetch is treating the variable.
Anyway, see the structure where the variable is in my URL too, I guess (sorry it's long):
JS:
let testString = "%5B%5B%5B-89.0000%2C40.5555%5D%2C%5B-89.1111%2C40.5555%5D%5D%5D";
fetch('https://hazards.fema.gov/gis/nfhl/rest/services/FIRMette/NFHLREST_FIRMette/MapServer/1/query?where=&text=&objectIds=&time=&geometry=%7B%22paths%22%3A+' + testString + '%7D&geometryType=esriGeometryPolyline&inSR=4326&spatialRel=esriSpatialRelIntersects&distance=&units=esriSRUnit_Foot&relationParam=&outFields=*&returnGeometry=false&returnTrueCurves=false&maxAllowableOffset=&geometryPrecision=&outSR=&havingClause=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&historicMoment=&returnDistinctValues=false&resultOffset=&resultRecordCount=&returnExtentOnly=false&datumTransformation=&parameterValues=&rangeValues=&quantizationParameters=&featureEncoding=esriDefault&f=pjson')
.then(function (response) {
return response.json();
})
.then (function (data) {
appendData(data);
})
.catch(function (err) {
console.log('error: ' + err);
});
Edit: I even tried straight up running the fetch with the string from my variable, and it worked. So it has to do with the variable - it just doesn't make much sense.
My full messy testing code (add the coordinates to list then run)
Edit 2: See my below answer. In my actual code I had add/remove options functionality that was causing an issue in my function to fetch the data. I needed these options to build my string used in the fetch. I used preventDefault() to fix the problem - but I'm still not positive on why it works with that specifically.

I found my issue, so I'm answering my question - I welcome any explanation though.
I posted a jsfiddle of my code but it's a mess, but to sum up what was going on:
I had a list of options that the user could add to add coordinates, and then run those. In my problem code, these options were referenced under a GetData() function which ran my fetch. Outside of that GetData() function, these options were created, but I didn't think that was the problem because the brackets were closed around both the add and remove an option functionalities.
I had a button for add, and a button for remove. These functionalities were made, for example, with:
btnRemove.onclick = (e) => {
e.preventDefault();
// add/remove/getselected/etc functionalities here
}
I created a function just like this to replace my GetData() function, reading:
linearGPSsubmit.onclick = (e) => {
e.preventDefault();
}
The preventDefault() was originally to help aid the add/remove functionality.
That code above executes on submit button click. What I don't fully understand is why the fetch runs at all when preventDefault() is used? Shouldn't it stop me from submitting the form? Nevertheless... no more 404 and the fetch fetches. Cool. It feels wrong, but it works.

Testing multiple HTML pages with CasperJS?

I'm working with CasperJS on a webpage that has a search box. Using Casper, I'm successfully able to perform a search and see that the form is filled and execute the search, however I'm having trouble specifically getting the number of results to test that it's valid.
When I look at the source of the webpage, I have the XPath of the element and it's nested in a few divs.
But when I try to either do assertExists() on that path, or even return the result of getElementByXPath() to a var, it doesn't work. In the first case the test fails and in the second case it prints null.
This is the XPath:
//*[#id="total"]
Here is what the snippet of source looks like:
<div id="pgContent"><div id="results_pagination1_container">
<span style="float: right; font-size: .9em">
Found <span id="total">721</span> item(s)</span>
</div>
This is the CasperJS code, relevant to here.
casper.test.begin(
'Testing that we get the right amount of results for a search query',
2, function suite(test) {
casper.start(catapult, function() {
test.assertTitle("Search", "Search"); // Good
test.assertExists('#input-searchbox'); // Good
this.fillSelectors('form#inputForm', {
'input[name="queryStr"]' : 'airplane'
}, true);
//this.click('input#btnSearch');
});
casper.then(function() {
var resultsNum = __utils__.getElementByXPath('//*[#id="total"]');
this.echo(resultsNum);
test.assertExists('//*[#id="total"]');
});

Clientutils and DOM access only in page context
The CasperJS clientutils module is only defined in the page context. Whenever CasperJS starts, it injects those utils into the page for later convenience. The problem is that the page and its DOM are only accessible from inside of the evaluate() function.
Another thing to remember is that you cannot get DOM nodes out of the page context. Either you do your stuff completely in the page context (inside of evaluate()) or you get a representation of the node like the textContent property:
var resultsNum = this.evaluate(function(){
return __utils__.getElementByXPath('//*[#id="total"]').textContent;
});
Here is the bit from the documentation (since CasperJS is built on top of PhantomJS):
Note: The arguments and the return value to the evaluate function must be a simple primitive object. The rule of thumb: if it can be serialized via JSON, then it is fine.
Closures, functions, DOM nodes, etc. will not work!
Difference between CSS selectors and XPath expressions
The other thing is that if not otherwise noted through the name all functions assume that the selectors are CSS selectors. Looking at test.assertExists('//*[#id="total"]'); makes it clear that '//*[#id="total"]' is not a CSS selector, but an XPath expression. You can use XPath expressions in CasperJS through the XPath utility:
var x = require('casper').selectXPath;
test.assertExists(x('//*[#id="total"]'));
or you can simply use the equivalent CSS selector:
test.assertExists('#total');

window.onload to determine what code to run depending on page value

I use the window.onload function throughout my website, via a single external .js file, and I usually check to see what page i'm on, so that certain statements will run, e.g:
window.onload = function() {
var curPage = document.getElementById('page').value;
if (curPage === "index.html") {
// do something here
}
if (curPage === "about.html") {
// do something else here
}
}
with 'curPage' being a value from an input within each html document
<input id="page" type="hidden" value="my-page.html" />
I see people adding an id to the body element to achieve the same effect, like so:
<body id="pageisIndex">
However, I would like to know if there is an even better way to initialize/pull a variable within the HTML document without the use of an input or element id.
would something like ...
<script type="text/javascript">
var curPage === 'index.html';
</script>
... be alright/proper to use while also using an external .js file?
EDIT:
All of the answers provided are very good, I believe this is one of those times where it comes down to an individuals opinion; however, using location.pathname or document.URL are great methods that I will be testing out. Awesome!
EDIT 2:
I found something pretty cool I thought I would post here, the following gives us the last page within the path.
page = location.pathname.split('/').pop();
console.log(page);

You could possibly try RegExp with given page name itself without any hidden field:
var url = window.location.pathname;
if(url.match('index.html')) {
//to do
} else if(url.match('about.html')) {
//to do
} else {
//to do
}

You can use:
location.pathname
which returns everything in the url after the domain. For example, for this page it's:
/questions/27956897/window-onload-to-determine-what-code-to-run-depending-on-page-value
It does not include the query string if present.
If you want just what follows the last forward-slash, that's discussed in this post: Get value of a string after a slash in JavaScript, which will work if you don't care about the rest of the path but just want to extract the filename.

I used the last option you have shown which seemed to be a good solution. Setting a variable at the beginning of a page and checking its value is easy. I think instead of giving the exact page name I would go for something which is meaningful like "intropage" or "sellingpage" which will be easier to decipher at a later time.
<script type="text/javascript">
var curPage = 'intropage';
</script>
Another approach that could be to used if you don't want to set id or variable is to check the location.pathname or location.href to get the current location and take necessary steps.

Cleaning up location.pathname to load files and select elements with href attributes

I have a script where essentially I'm trying to find the location of a .php file using javascript/jquery (with location.pathname). So, my problem is basically that if the user inputs something weird like:
url.com/ or url.com//// or url.com////index.php//// or url.com////index.php.////, then I need a way of dealing with this so I can obtain /index.php so I can select that file and load some content from it (using ajax), as well as selecting an element that has href = "/index.php" so I can make it an active link.
There's also the additional problem of something like this:
url.com/projects/index.php, url.com////projects//index.php for which I'd like to have an output of /projects/index.php to properly select the file once again.
Is there a standard way for doing this? I'd like to avoid using some regex or string replace method because I'm not sure if it will be able to handle all cases, although if the proper way to go about his is to do that then I'll go ahead and implement it. The browser uses a parser to determine what file to load, so if there's a solution that uses something similar (maybe built in or something) then that would be great. I tried searching for jquery url parsers or cleanups but I'm actually not sure what the term is that I should be looking for so my searches came up short.
EDIT: Just as some background, I'm implementing this basically: http://css-tricks.com/rethinking-dynamic-page-replacing-content/, but I need a way to tweak it so it can find files in sub directories as well.
EDIT2: Here's an example of what I mean:
EDIT3: Here's the ajax call I'm using, which fires on popstate:
var file = location.pathname;
$("#content").load(file + " #content", function() {
$("#menu ul a").removeClass("current");
$("#menu ul a[href="+file+"]").addClass("current");
});
When I do this for weird browser entries, the load and href fail obviously, since the href attribute is set to href = "/index.php" in my website. The load function also fails for weird inputs, even though the page can be loaded.

Replace multiple forward slashes with a single forward slash, then remove the domain name if necessary. Example:
$(document).ready(function() {
var userStr = 'url.com////projects//index.php';
userStr = userStr.replace(/\/{2,}/g, '/').replace(/url.com/, '');
// userStr = /projects/index.php
});
This obviously won't work for every combination of URL that your user might provide, but then I doubt you'll find a regular expression that handles every possibility either. If a user sends you "url.com////index.php.////" then give them a 404 in return.

using JQuery to fetch an html document and parse it into a DOM tree

So essentially I'm trying to build my own version of GitHub's tree slider. The relevant Javascript/JQuery code is:
// handles clicking a link to move through the tree
$('#slider a').click(function() {
history.pushState({ path: this.path }, '', this.href) // change the URL in the browser using HTML5 history module
$.get(this.href, function(data) {
$('#slider').slideTo(data) // handle the page transition, preventing full page reloads
})
return false
})
// binds hitting the back button in the browser to prevent full page reloads
$(window).bind('popstate', function() {
$('#slider').slideTo(location.pathname)
}
Ok, hopefully that's understandable. Now here's my interpretation of what's going on here, followed by my problem/issue:
The callback function for the GET request when navigating through the tree is the slideTo method, and an HTML string is passed in as an argument to that function. I'm assuming that slideTo is a function defined elsewhere in the script or in a custom library, as I can't find it in the JQuery documentation. So, for my purposes, I'm trying to build my own version of this function. But the argument passed into this function, "data", is just the string of HTML returned from the GET request. However, this isn't just a snippet of HTML that I can append to a div in the document, because if I perform the same GET request (e.g. by typing the url into a web browser) I would expect to see a whole webpage and not just a piece of one.
So, within this callback function that I am defining, I would need to parse the "data" argument into a DOM so that I can extract the relevant nodes and then perform the animated transition. However, this doesn't make sense to me. It generally seems like a Bad Idea. It doesn't make sense that the client would have to parse a whole string of HTML just to access part of the DOM. GitHub claims this method is faster than a full page reload. But if my interpretation is correct, the client still has to parse a full string of HTML whether navigating through the tree by clicking (and running the callback function) or by doing full page loads such as by typing the new URL in the browser. So I'm stuck with either parsing the returned HTML string into a DOM, or ideally only fetching part of an HTML document.
Is there a way to simply load the fetched document into a Javascript or JQuery DOM object so I can easily manipulate it? or even better, is there a way to fetch only an element with an arbitrary id without doing some crazy server-side stuff (which I already tried but ended up being too spaghetti code and difficult to maintain)?
I've also already tried simply parsing the data argument into a JQuery object, but that involved a roundabout solution that only seems to work half the time, using javascript methods to strip the HTML of unwanted things, like doctype declarations and head tags:
var d = document.createElement('html');
d.innerHTML = data;
body = div.getElementsByTagName("body")[0].innerHTML;
var newDOM = $(body);
// finally I have a JQuery DOM context that I can use,
// but for some reason it doesn't always seem to work quite right
How would you approach this problem? When I write this code myself and try to make it work on my own, I feel like no matter what I do, I'm doing something horribly inefficient and hacky.
Is there a way to easily return a JQuery DOM object with a GET request? or better, just return part of a document fetched with a GET request?

Just wrap it; jQuery will parse it.
$(data) // in your callback

Imagine you want to parse a <p> tag in your normal HTML web page. You probably would use something like:
var p = $('<p>');
Right? So you have to use the same approach to parse an entire HTML document and then, navigate through the DOM tree to get the specific elements you want. Therefore, you just need to say:
$.get(this.href, function(data) {
var html = $(data);
// (...) Navigating through the DOM tree
$('#slider').slideTo( HTMLportion );
});
Notice that it also works for XML documents, so if you need to download via AJAX a XML document from the server, parse the inner information and display it on the client-side, the method is exactly the same, ok?
I hope it helps you :)
P.S: Don't ever forget to put semicolons at the end of each JavaScript sentence. Probably, if you don't put them, the engine would work but it is better to be safe and write them always!