We Keep Coding

javascript to read a text file line-by-line and display specific string

I get no credit for the code below. It was found online. It works to open a file but I'd need something to display only certain strings withing a file (i.e)
test = 2000
radio 1020
webbrowser - 1000
help needed = 2000
I'd need to modify this to display only, for example, help needed = 2000
<script>
function readBlob(opt_startByte, opt_stopByte) {
var files = document.getElementById('files').files;
if (!files.length) {
alert('Please select a file!');
return;
}
var file = files[0];
var start = parseInt(opt_startByte) || 0;
var stop = parseInt(opt_stopByte) || file.size - 1;
var reader = new FileReader();
// If we use onloadend, we need to check the readyState.
reader.onloadend = function(evt) {
if (evt.target.readyState == FileReader.DONE) { // DONE == 2
document.getElementById('byte_content').textContent = evt.target.result;
document.getElementById('byte_range').textContent =
['Read bytes: ', start + 1, ' - ', stop + 1,
' of ', file.size, ' byte file'].join('');
}
};
var blob = file.slice(start, stop + 1);
reader.readAsBinaryString(blob);
}
document.querySelector('.readBytesButtons').addEventListener('click', function(evt) {
if (evt.target.tagName.toLowerCase() == 'button') {
var startByte = evt.target.getAttribute('data-startbyte');
var endByte = evt.target.getAttribute('data-endbyte');
readBlob(startByte, endByte);
}
}, false);
</script>

First of all, web browsers does not allow you to access local files using javascript for security reasons.
So, if you really want to read the file in javascript, you should consider setting up an environment to execute javascript. You can read it here. Executing javascript without browser.

Get Byte Position during Upload Loop

I am working on a function that will write data to a remote server in chunks using a 3rd party API. Through some help on Stack Overflow I was able to accomplish this, where it is now working as expected. The problem is that I can only get a single 16kb chunk to write as I will need to advance the pos of where the next bytes are written to.
The initial write starts at 0 easily enough. Due to my unfamiliarity with this though, I am unsure if the next pos should just be 16 or what. If it helps, the API call writeFileChunk() takes 3 parameters, filepath (str), pos (int64), and data (base64 encoded string).
reader.onload = function(evt)
{
// Get SERVER_ID from URL
var server_id = getUrlParameter('id');
$("#upload_status").text('Uploading File...');
$("#upload_progress").progressbar('value', 0);
var chunkSize = 16<<10;
var buffer = evt.target.result;
var fileSize = buffer.byteLength;
var segments = Math.ceil(fileSize / chunkSize); // How many segments do we need to divide into for upload
var count = 0;
// start the file upload
(function upload()
{
var segSize = Math.min(chunkSize, fileSize - count * chunkSize);
if (segSize > 0)
{
$("#upload_progress").progressbar('value', (count / segments));
var chunk = new Uint8Array(buffer, count++ * chunkSize, segSize); // get a chunk
var chunkEncoded = btoa(String.fromCharCode.apply(null, chunk));
// Send Chunk data to server
$.ajax({
type: "POST",
url: "filemanagerHandler.php",
data: { 'action': 'writeFileChunk', 'server_id': server_id, 'filepath': filepath, 'pos': 0, 'chunk': chunkEncoded },
dataType: 'json',
success: function(data)
{
console.log(data);
setTimeout(upload, 100);
},
error: function(XMLHttpRequest, textStatus, errorThrown)
{
alert("Status: " + textStatus); alert("Error: " + errorThrown); alert("Message: " + XMLHttpRequest.responseText);
}
});
}
else
{
$("#upload_status").text('Finished!');
$("#upload_progress").progressbar('value', 100);
getDirectoryListing(curDirectory);
}
})()
};

The current position for the file on client side would be represented by this line, or more specifically the second argument at the pre-incremental step:
var chunk = new Uint8Array(buffer, count++ * chunkSize, segSize);
though, in this case it advances (count++) before you can reuse it so if you need the actual position (below as pos) you can extract it by simply rewriting the line into:
var pos = count++ * chunkSize; // here chunkSize = 16kb
var chunk = new Uint8Array(buffer, pos, segSize);
Here each position update will increment 16kb as that is the chunk-size. For progress then it is calculated pos / fileSize * 100. This of course assuming using the unencoded buffer size.
The only special case is the last chunk, but when there are no more chunks left to read the position should be equal to the file length (fileSize) so it should be pretty straight-forward.
When the ajax call return the server should have the same position unless something went wrong (connection, write access change, disk full etc.).

You can use Filereader API to read the chunks and send it to your remote server.
HTML
<input type="file" id="files" name="file" /> Read bytes:
<span class="readBytesButtons">
<button>Read entire file in chuncks</button>
</span>
Javascript
// Post data to your server.
function postChunk(obj) {
var url = "https://your.remote.server";
return new Promise(function(resolve, reject) {
var xhr = new XMLHttpRequest();
xhr.open('post', url, true);
xhr.responseType = 'json';
xhr.onload = function() {
var status = xhr.status;
if (status == 200) {
resolve(xhr.response);
} else {
reject(status);
}
};
var params = "";
// check that obj has the proper keys and create the url parameters
if (obj.hasOwnProperty(action) && obj.hasOwnProperty(server_id) && obj.hasOwnProperty(filepath) && obj.hasOwnProperty(pos) && obj.hasOwnProperty(chunk)) {
params += "action="+obj[action]+"&server_id="+obj[server_id]+"&filepath="+obj[filepath]+"&pos="+obj[pos]+"&chunk="+obj[chunk];
}
if(params.length>0) {
xhr.send(params);
} else {
alert('Error');
}
});
}
// add chunk to "obj" object and post it to server
function addChunk(reader,obj,divID) {
reader.onloadend = function(evt) {
if (evt.target.readyState == FileReader.DONE) { // DONE == 2
obj.chunk = evt.target.result;
console.log(obj);
document.getElementById(divID).textContent +=
['Sending bytes: ', obj.pos*16000, ' - ', ((obj.pos*16000)+(obj.pos+1)*obj.chunk.length),
'\n'].join('');
// post data to server
postChunk(obj).then(function(data) {
if(data!=="" && data!==null && typeof data!=="undefined") {
// chunk was sent successfully
document.getElementById(divID).textContent +=
['Sent bytes: ', obj.pos*16000, ' - ', ((obj.pos*16000)+(obj.pos+1)*obj.chunk.length),'\n'].join('');
} else {
alert('Error! Empty response');
}
}, function(status) {
alert('Resolve Error');
});
}
};
}
// read and send Chunk
function readChunk() {
var files = document.getElementById('files').files;
if (!files.length) {
alert('Please select a file!');
return;
}
var file = files[0];
var size = parseInt(file.size);
var chunkSize = 16000;
var chunks = Math.ceil(size/chunkSize);
var start,stop = 0;
var blob = [];
for(i=0;i<chunks;i++) {
start = i*chunkSize;
stop = (i+1)*chunkSize-1;
if(i==(chunks-1)) {
stop = size;
}
var reader = new FileReader();
blob = file.slice(start, stop);
reader.readAsBinaryString(blob);
var obj = {action: 'writeFileChunk', server_id: 'sid', filepath: 'path', pos: i, chunk: ""};
var div = document.createElement('div');
div.id = "bytes"+i;
document.body.appendChild(div);
addChunk(reader,obj,div.id);
}
}
// Check for the various File API support.
if (window.File && window.FileReader && window.FileList && window.Blob) {
console.log(' Great success! All the File APIs are supported.');
} else {
alert('The File APIs are not fully supported in this browser.');
}
document.querySelector('.readBytesButtons').addEventListener('click', function(evt) {
if (evt.target.tagName.toLowerCase() == 'button') {
readChunk();
}
}, false);
You can check this example in this Fiddle

Multiple XMLHttpRequest.send or eventlisteners memory leak?

I'm currently implementing an upload for files. Because I've to handle huge files here and there I've started to slice files and send them in 1mb chunks which works great as long as file are <~500MB after that it seems that memory isn't freed anyone randomly and I can't figure out what I'm missing here.
Prepare chunks
var sliceCount = 0;
var sendCount = 0;
var fileID = generateUUID();
var maxChunks = 0;
var userNotified = false;
function parseFile(file)
{
var fileSize = file.size;
var chunkSize = 1024 * 1024;//64 * 1024; // bytes
var offset = 0;
var self = this; // we need a reference to the current object
var chunkReaderBlock = null;
var numberOfChunks = fileSize / chunkSize;
maxChunks = Math.ceil(numberOfChunks);
// gets called if chunk is read into memory
var readEventHandler = function (evt)
{
if (evt.target.error == null) {
offset += evt.target.result.byteLength;
sendChunkAsBinary(evt.target.result);
}
else
{
console.log("Read error: " + evt.target.error);
return;
}
if (offset >= fileSize) {
console.log("Done reading file");
return;
}
// of to the next chunk
chunkReaderBlock(offset, chunkSize, file);
}
chunkReaderBlock = function (_offset, length, _file)
{
var r = new FileReader();
var blob = _file.slice(_offset, length + _offset);
sliceCount++;
console.log("Slicecount: " + sliceCount);
r.onload = readEventHandler;
r.readAsArrayBuffer(blob);
blob = null;
r = null;
}
// now let's start the read with the first block
chunkReaderBlock(offset, chunkSize, file);
}
Send Chunks
function sendChunkAsBinary(chunk)
{
var progressbar = $("#progressbar"), bar = progressbar.find('.uk-progress-bar');
// create XHR instance
var xhr = new XMLHttpRequest();
// send the file through POST
xhr.open("POST", 'upload.php', true);
var progressHandler = function (e)
{
// get percentage of how much of the current file has been sent
var position = e.loaded || e.position;
var total = e.total || e.totalSize;
var percentage = Math.round((sendCount / maxChunks) * 100);
// set bar width to keep track of progress
bar.css("width", percentage + "%").text(percentage + "%");
}
// let's track upload progress
var eventSource = xhr.upload || xhr;
eventSource.addEventListener("progress", progressHandler);
// state change observer - we need to know when and if the file was successfully uploaded
xhr.onreadystatechange = function ()
{
if (xhr.readyState == 4)
{
if (xhr.status == 200)
{
eventSource.removeEventListener("progress", progressHandler);
if (sendCount == maxChunks && !userNotified)
{
userNotified = true;
notifyUserSuccess("Datei hochgeladen!");
setTimeout(function ()
{
progressbar.addClass("uk-invisible");
bar.css("width", "0%").text("0%");
}, 250);
updateDocList();
}
}
else
{
notifyUser("Fehler beim hochladen der Datei!");
}
}
};
var blob;
if (typeof window.Blob == "function") {
blob = new Blob([chunk]);
} else {
var bb = new (window.MozBlobBuilder || window.WebKitBlobBuilder || window.BlobBuilder)();
bb.append(chunk);
blob = bb.getBlob();
}
sendCount++;
var formData = new FormData();
formData.append("chunkNumber", sendCount);
formData.append("maxChunks", maxChunks);
formData.append("fileID", fileID);
formData.append("chunkpart", blob);
xhr.send(formData);
progressbar.removeClass("uk-invisible");
console.log("Sendcount: " + sendCount);
}
If I attach to the debugger within Visual Studio 2015 it take a bit but soon I get an OutOfMemoryException in the send function at exactly this line: blob = new Blob([chunk]);. It's all the time the same line where the exception occures.
As soon as the Exception happens I get POST [...]/upload.php net::ERR_FILE_NOT_FOUND however I still got the chunks in my php-file.
Here's a Timeline-graph of my error
What I dont understand, I'm not able to see increasing memory inside the Task-Manager (a few mb of course but not close to 16gb ram I got).
So can anyone tell me where this leak comes from? What am I missing?

building a voice recorder using html5 and javascript

I want to build a voice recorder using HTML5 same as one found in gitHub JSSoundecorder, but what I want is for the user to be able to choose the file format before recording the voice.I can do this using ffmpeg. In other words the user must be able to select the audio format by check box (mp3,wma,pcm) and in the background code, the .wav file usually created by the program instead of displaying it, it should be converted by the format selected then displayed in the new format.this is the ffmpeg code we can use ,but I don't know how to get the .wav audio file to convert it and show it.please if someone have ideas,or if can find demos I have been looking for weeks.this is the ffmpeg code:
var fileName;
var fileBuffer;
function timeToSeconds(time) {
var parts = time.split(":");
return parseFloat(parts[0]) * 60 * 60 + parseFloat(parts[1]) * 60 + parseFloat(parts[2]) + parseFloat("0." + parts[3]);
}
// create ffmpeg worker
function getFFMPEGWorker() {
// regexps for extracting time from ffmpeg logs
var durationRegexp = /Duration: (.*?), /
var timeRegexp = /time=(.*?) /;
var duration;
var ffmpegWorker = new Worker('worker.js');
var durationLine;
ffmpegWorker.addEventListener('message', function(event) {
var message = event.data;
console.log(message.type);
if (message.type === "ready" && window.File && window.FileList && window.FileReader) {
// script loaded, hide loader
$('#loading').hide();
} else if (message.type == "stdout") {
console.log(message.data);
} else if (message.type == "stderr") {
console.log(message.data);
// try to extract duration
if (durationRegexp.exec(message.data)) {
duration = timeToSeconds(durationRegexp.exec(message.data)[1]);
}
// try to extract time
if (timeRegexp.exec(message.data)) {
var time = timeToSeconds(timeRegexp.exec(message.data)[1]);
if (duration) {
$("#progress").text("Progress: " + Math.floor(time / duration * 100) + "%");
$("#progress").show();
}
}
} else if (message.type == "done") {
var code = message.data.code;
console.log(message.data);
var outFileNames = Object.keys(message.data.outputFiles);
console.log(outFileNames);
if (code == 0 && outFileNames.length) {
var outFileName = outFileNames[0];
var outFileBuffer = message.data.outputFiles[outFileName];
var src = window.URL.createObjectURL(new Blob([outFileBuffer]));
$("#downloadLink").attr('href', src);
$("#download").show();
} else {
$("#error").show();
}
$("#converting").hide();
$("#progress").hide();
}
}, false);
return ffmpegWorker;
}
// create ffmpeg worker
var ffmpegWorker = getFFMPEGWorker();
var ffmpegRunning = false;
$('#convert').click(function() {
// terminate existing worker
if (ffmpegRunning) {
ffmpegWorker.terminate();
ffmpegWorker = getFFMPEGWorker();
}
ffmpegRunning = true;
// display converting animation
$("#converting").show();
$("#error").hide();
// hide download div
$("#download").hide();
// change download file name
var fileNameExt = fileName.substr(fileName.lastIndexOf('.') + 1);
var outFileName = fileName.substr(0, fileName.lastIndexOf('.')) + "." + getOutFormat();
$("#downloadLink").attr("download", outFileName);
$("#downloadLink").text(outFileName);
var arguments = [];
arguments.push("-i");
arguments.push(fileName);
arguments.push("-b:a");
arguments.push(getBitrate());
switch (getOutFormat()) {
case "mp3":
arguments.push("-acodec");
arguments.push("libmp3lame");
arguments.push("out.mp3");
break;
case "wma":
arguments.push("-acodec");
arguments.push("wmav1");
arguments.push("out.asf");
break;
case "pcm":
arguments.push("-f");
arguments.push("s16le");
arguments.push("-acodec");
arguments.push("pcm_s16le");
arguments.push("out.pcm");
}
ffmpegWorker.postMessage({
type: "command",
arguments: arguments,
files: [
{
"name": fileName,
"buffer": fileBuffer
}
]
});
});
function getOutFormat() {
return $('input[name=format]:checked').val();
}
function getBitrate() {
return $('input[name=bitrate]:checked').val();
}
// disable conversion at start
$('#convert').attr('disabled', 'true');
function readInputFile(file) {
// disable conversion for the time of file loading
$('#convert').attr('disabled', 'true');
// load file content
var reader = new FileReader();
reader.onload = function(e) {
$('#convert').removeAttr('disabled');
fileName = file.name;
console.log(fileName);
fileBuffer = e.target.result;
}
reader.readAsArrayBuffer(file);
}
// reset file selector at start
function resetInputFile() {
$("#inFile").wrap('<form>').closest('form').get(0).reset();
$("#inFile").unwrap();
}
resetInputFile();
function handleFileSelect(event) {
var files = event.target.files; // FileList object
console.log(files);
// files is a FileList of File objects. display first file name
file = files[0];
console.log(file);
if (file) {
$("#drop").text("Drop file here");
readInputFile(file);
}
}
// setup input file listeners
document.getElementById('inFile').addEventListener('change', handleFileSelect, false);

file upload processing file from xhr to web api

I'm using the following code to upload file to the server,
Javascript:
function FileUpload(ServerName) {
alert("Entering File Change...");
var file = this.file1[0];
alert("File Change");
var xhr = new XMLHttpRequest();
xhr.file = file; // not necessary if you create scopes like this
xhr.addEventListener('progress', function(e) {
alert("progress");
var done = e.position || e.loaded, total = e.totalSize || e.total;
console.log('xhr progress: ' + (Math.floor(done/total*1000)/10) + '%');
}, false);
if ( xhr.upload ) {
xhr.upload.onprogress = function(e) {
var done = e.position || e.loaded, total = e.totalSize || e.total;
console.log('xhr.upload progress: ' + done + ' / ' + total + ' = ' + (Math.floor(done/total*1000)/10) + '%');
};
}
xhr.onreadystatechange = function(e) {
if ( 4 == this.readyState ) {
console.log(['xhr upload complete', e]);
}
};
xhr.open('post', "http://localhost:49868/UploadFile/" + ServerName, true);
xhr.send(file);
}
Web api code:
[HttpPost]
[ActionName("UploadFile")]
public HttpResponseMessage UploadFile(string servername)
{
try
{
HttpContext.Current.Response.Headers.Add("Access-Control-Allow-Origin", "*");
var httpRequest = HttpContext.Current.Request;
HttpResponseMessage result = null;
// Check if files are available
if (httpRequest.Files.Count > 0)
{
var files = new List<string>();
// interate the files and save on the server
foreach (string file in httpRequest.Files)
{
var postedFile = httpRequest.Files[file];
//var filePath = HttpContext.Current.Server.MapPath("~/" + postedFile.FileName);
postedFile.SaveAs("C:\\Users\\Desktop\\New folder (4)\\" + postedFile.FileName);
//files.Add(filePath);
//File.Copy(filePath, RootPath + postedFile.FileName);
//File.Delete(filePath);
}
// return result
result = Request.CreateResponse(HttpStatusCode.Created, files);
}
else
{
// return BadRequest (no file(s) available)
result = Request.CreateResponse(HttpStatusCode.BadRequest);
}
return result;
}
catch (Exception ex)
{
return null;
}
}
My Question is how to handle this on the server side, i.e in the web api, how to get the file there on the server and store it in some specific location?
Now the httpRequest.Files is returned null, where to get the file and save it?