We Keep Coding

data modified after send with ajax

I am using javascript and the Flask framework
I would like to retrieve in Flask in python the bytes of one or more files that the user will have chosen in my HTML page. To do this, when the user has chosen the files and has clicked on the send button, it triggers a function that uses the FileReader API to retrieve the base64 content of the file(s) that the user has selected.
After that, I would like to send the base64 data to flask with ajax.
But here is my problem, when I get the base64 string in python and compare it with the one in javascript, I notice the number of characters is exactly the same, but some characters are different as you can see on the screenshots below
size of character strings
And when I decode the base64 string, I get different bytes :
bytes variables
This makes me think that the problem is with the use of ajax
Code Python :
#app.route('/test', methods=['POST'])
def test():
if request.method == "POST":
files = eval(request.form.get("files"))
python_data = file.get("data")
javascript_data = "" # Copy from chrome console
len_python_data = len(python_data)
len_javascript_data = len(javascript_data)
base64_bytes_javascript = base64.b64decode(javascript_data)
base64_bytes_python = base64.b64decode(python_data)
Code Javascript :
let array_files = [];
let files_input = document.getElementById("files_input");
let files = files_input.files;
let reader = new FileReader();
function readFile(index) {
if( index >= files.length ) {
let data = "files=" + JSON.stringify(array_files);
$.ajax({
url: '/test',
type: 'POST',
data: data,
success: function (msg) {
console.log(msg)
}
});
return;
}
let file = files[index];
reader.name = file.name;
reader.onload = function() {
let file_info = {};
// get file content
let bin = this.result;
console.log(bin)
let data = bin.split(";")[1].replace("base64,", "");
file_info.name = reader.name;
file_info.data = data;
array_files.push(file_info);
readFile(index + 1)
}
reader.readAsDataURL(file);
}
readFile(0);

The problem is solved
In the base64 character string, the "+" were replaced by spaces after sending it to ajax.

How to decode protobuf base64 string to js object

I'm try to decode protobuf message from server side (base64 string) to javascript object. Use for decode protobuf.js.
As doc suggest :
var builder = ProtoBuf.newBuilder({ convertFieldsToCamelCase: true });
var YourMessage = builder.build("YourMessage");
var buffer = ...; // E.g. a buffer received on a WebSocket
var myMessage = YourMessage.decode(buffer);
...
var b64str = ...; // E.g. a string fetched via AJAX
var myMessage = YourMessage.decode64(b64str);
This is how I try to do it (data - base64 string) :
var proto = dcodeIO.ProtoBuf;
var buffer = dcodeIO.ByteBuffer;
var b = buffer.wrap(data,"binary");
var builder = proto.newBuilder({ convertFieldsToCamelCase: true });
builder.define("Events");
var message = builder.build("Events");
var result = message.decode(b); //also try to decode base64 string - message.decode64(data);
I get error
decode/decode64 undefined

Missing the line to import the proto definition as below (where tests/example1.proto is your file name)
ProtoBuf.loadProtoFile("tests/example1.proto", builder);
or if loading from proto string
ProtoBuf.loadProto(...protoString..., "example1.proto");
without this the builder will say undefined because there is nothing for it to build its definition with

C# Httpwebrequest detection

What ways are there for a website to detect automated connections made through C# Httpwebrequest?
Such as c#'s default user-agent? Operating System? or what..
have this problem with any other language, just C#?
I'm being blocked from accessing a certain website using Httpwebrequest, I don't
Also it's definitely not my IP address & nor are there any faults in my code as I've tested connections to other websites which work just fine.. Also I stated above I can make connections to the website using C++, C, Vb.net, Java, Python & so on, there is also no difference in header information either.
EDIT:
// Create a request using a URL that can receive a post.
WebRequest request = WebRequest.Create ("http://services.runescape.com/m=hiscore_oldschool/overall.ws");
// Set the Method property of the request to POST.
request.Method = "POST";
// Create POST data and convert it to a byte array.
string postData = "user1=Zezima&submit=Search";
byte[] byteArray = Encoding.UTF8.GetBytes (postData);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream ();
// Write the data to the request stream.
dataStream.Write (byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close ();
// Get the response.
WebResponse response = request.GetResponse ();
// Display the status.
Console.WriteLine (((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream ();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader (dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd ();
// Display the content.
Console.WriteLine (responseFromServer);
// Clean up the streams.
reader.Close ();
dataStream.Close ();
response.Close ();

private const string Url = "http://services.runescape.com/m=hiscore_oldschool/overall.ws";
private static HttpWebRequest BuildWebRequest()
{
var request = WebRequest.Create(Url) as HttpWebRequest;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Timeout = 40000;
request.ServicePoint.Expect100Continue = true;
string body = "user1=Zezima&submit=Search";
byte[] bytes = Encoding.Default.GetBytes(body);
using (var requestStream = request.GetRequestStream())
{
requestStream.Write(bytes, 0, bytes.Length);
}
return request;
}
static void Main(string[] args)
{
try
{
HttpWebRequest request = BuildWebRequest();
var response = request.GetResponse() as HttpWebResponse;
var responseContent = new StreamReader(response.GetResponseStream()).ReadToEnd();
Console.Write("Success - " + response.StatusCode);
}
catch (Exception e)
{
Console.Write(e);
}
}
I can take the response from the website. It is not empty.

Pass PNG on form submit, Request URL Too long

So I have an interesting question. I have a form where a user draws an image on a canvas (think a signature pad). I then need to send the image to my C# Controller (I am using ASP.NET MVC 5). The code I have functions for shorter strings, but when I try to pass the PNG data, it is too long and I recieve a HTTP Error 414. The request URL is too long error. Here is my code:
Html:
<form id="mainForm" action="submitUserAnswer" method="post">
<input type="hidden" id="userOutput" name="output" value="" />
//...other form elements, signature box, etc.
</form>
Javascript:
function goToNextQuestion() {
var output = $('#signature').jSignature("getData");
$('#userOutput').val(output);
$('#mainForm').submit();
}
C#:
public ActionResult submitUserAnswer()
{
//use the userOutput for whatever
//submit string to the database, do trigger stuff, whatever
//go to next question
System.Collections.Specialized.NameValueCollection nvc = Request.Form;
string userOutput = nvc["output"];
ViewBag.Question = userOutput;
return RedirectToAction("redirectToIndex", new { input = userOutput });
}
public ActionResult redirectToIndex(String input)
{
ViewBag.Answer = input;
return View("Index");
}
My png data is very long, so the error makes sense. My question is how can I get the png data back to my controller?

Maybe you just need to increase allowed GET request URL length.
If that doesn't works I have an aspx WebForm that saves a signature, and I use a WebMethod
[ScriptMethod, WebMethod]
public static string saveSignature(string data)
{
/*..Code..*/
}
and I call it like this:
PageMethods.saveSignature(document.getElementById('canvas').toDataURL(), onSucess, onError);
also I have to increase the length of the JSON request and it works fine, with no problems with the lenght.
In MVC there isn't WebMethods, but JSON and AJAX requests do the job, just save the data in a session variable, and then use it when need it.
Hope it helps

You have error because your data is string (base64) and have max limit for send characters, better way is to create blob (png file) from base64 at client side, and send it to server. Edit. All listed code here, exists in stackoverflow posts.
function dataURItoBlob(dataURI) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
var blob = null;
// TypeError old chrome and FF
window.BlobBuilder = window.BlobBuilder ||
window.WebKitBlobBuilder ||
window.MozBlobBuilder ||
window.MSBlobBuilder;
if(window.BlobBuilder){
var bb = new BlobBuilder();
bb.append(ab);
blob = bb.getBlob(mimeString);
}else{
blob = new Blob([ab], {type : mimeString});
}
return blob;
}
function sendFileToServer(file, url, onFileSendComplete){
var formData = new FormData()
formData.append("file",file);
var xhr = new XMLHttpRequest();
xhr.open('POST', url, true);
xhr.onload = onFileSendComplete;
xhr.send(formData);
}
var base64 = $('#signature').jSignature("getData");
var blob = dataURItoBlob(base64);
var onComplete = function(){alert("file loaded to server");}
sendFileToServer(blob, "/server", onComplete)

Converting a text XHR-Response to arraybuffer

I'm making an XHR request. At the time of making the request, I don't know whether the URL will return an image or not, so I'm setting xhr.responseType="text"
If the response returns with a Content-Type of image/png [or any other image MIME type], I'm making another request with xhr.responseType="arraybuffer". I then use the arraybuffer that's returned to render the image:
var uInt8Array = new Uint8Array(arraybuffer);
var i = uInt8Array.length;
var binaryString = new Array(i);
while (i--) {
binaryString[i] = String.fromCharCode(uInt8Array[i]);
}
var data = binaryString.join('');
var base64 = window.btoa(data);
//use this base64 string to render the image
Is there any way I can avoid making the second request?
I tried doing this -
var buf = new ArrayBuffer(responseText.length);
var bufView = new Uint8Array(buf);
for (var i=0, i<responseText.length; i++) {
bufView[i] = responseText.charCodeAt(i);
}
return buf;
but the responseText isn't the same as the data in the first code sample, and the resultant ArrayBuffer doesn't render the image correctly.

I had the same problem with a PDF being corrupted, but the handler is generic and handles text, JSON and files.
The easiest way to solve this is to do it the other way round: Always make the request with:
xhr.responseType = "blob";
Then when you want the response as text, just convert the binary data to text:
xhr.response.text().then(text => {
// do something
});
Having binary as the default return type I can just convert that to text where needed.