More elegant and faster way to serve data to user- Django - javascript

I have been working on one of my project and in one of its feature we retrieve the data from database and write it to a Blob instance and make it available to user to download. But as data is very very large like 15k+ pages in .JSON file with 8000000 words. the code snippet with which i am writing data to that file is given. please suggest any other elegant and faster way because this is taking too much time and also freezing the machine for a while.
function bulk_download_json(label_id) {
$.msg({
//autoUnblock : false,
clickUnblock : false,
beforeUnblock: function() {
$.ajax({
type: "GET",
url: "/initiate_download/?label=" + label_id,
success: function(data) {
var bulk_mails = data;
bulk_download_filename = label_id.concat(".json");
var bulk_json_file = JSON.stringify(bulk_mails);
var a = window.document.createElement('a');
a.href = window.URL.createObjectURL(new Blob([bulk_json_file], {
type: 'text/json'
}));
a.download = bulk_download_filename;
// Append anchor to body.
document.body.appendChild(a)
a.click();
// Remove anchor from body
document.body.removeChild(a)
remove_bulk_data(label_id);
}
});
//self.unblock();
}
});
}

You can use pagination for this where you will pass on the page_no and limit for the records being fetched
[Refer] https://docs.djangoproject.com/en/1.8/topics/pagination/

Related

Saving images uploaded using ImagesLoader

I am trying to wire up the ImagesLoader plugin, which allows uploading multiple images. It has a nice drag-n-drop UI but I just can't figure out how to get the images that were uploaded. I cannot find any documentation.
Link to the plugin page: ImagesLoader
Here is the javascript from the demo:
<script type="text/javascript">
// Ready
$(document).ready(function () {
// Create image loader plugin
var imagesloader = $('[data-type=imagesloader]').imagesloader({
minSelect: 3
,imagesToLoad: [{"Url":"./img/Nespresso001.jpg","Name":"Nespresso001"},{"Url":"./img/Nespresso002.jpg","Name":"Nespresso002"}]
});
//Form
$frm = $('#frm');
// Form submit
$frm.submit(function (e) {
var $form = $(this);
var files = imagesloader.data('format.imagesloader').AttachmentArray;
var il = imagesloader.data('format.imagesloader');
if (il.CheckValidity())
alert('Upload ' + files.length + ' files');
e.preventDefault();
e.stopPropagation();
});
});
The images are saved in the object "files". Here is a screen shot of the contents from the inspector:
I tried converting to json and posting, but that only generates an error.
$.ajax({
url: 'process-images.php',
type: 'POST',
data: JSON.stringify(files),
contentType: 'application/json; charset=utf-8',
dataType: 'json',
cache: false,
error: function() {alert("ERROR");},
success: function() {alert("OK");}
});
The rest of the code works just like the demo. Looks like everything needed for the uploaded images is stored in "files". I just need to get the data back to php and I can pull it apart from there. But right now, the original submit code just dies or my addition aborts with an error.
THANKS!
I hope it is not too late to answer.
You just need to loop through the base64 encoded image object, decoding and saving each image to the disk.
// Sample code
foreach(json_decode($request->input('files')) as $file) {
$name = $file->FileName
$imagePath = storage_path('app/public/images/');
file_put_contents($imagePath.$name, base64_decode($file->Base64));
}

Mootools/Jquery Scripting

This is my first exposure with using Mootools; I have not used Jquery much, and I am having troubles implementing a new change to the scripting.
We used a flash uploader, FancyUpload2 and I have been tasked with replacing it with a non-flash version. I am attempting to use the existing coding for the rest of the project, but I have not been able to get this to work now.
<script type="text/javascript">
$('#demo-browse').change(function(){
//on change event
formdata = new FormData();
if($(this).prop('files').length > 0)
{
file =$(this).prop('files')[0];
formdata.append("Filedata", file);
}
jQuery.ajax({
url: '../control/image_upload/gallery_script.php',
type: "POST",
data: formdata,
processData: false,
contentType: false,
success: function(response) {
var json = new Hash(JSON.decode(response, true) || {});
// alert(response);
iDependOnMyParameter(response);
}
});
function iDependOnMyParameter(param) {
// You should do your work here that depends on the result of the request!
var json = new Hash(JSON.decode(param, true) || {});
var req = new Request({
url: '../control/image_upload/gallery_script_ajax.php',
onSuccess: function(response){
var feedback = response.split('|');
var thumb = feedback[0];
var new_id = feedback[1];
var li = new Element('li', {id: 'item_'+new_id, 'class':'sort_me', 'alt':new_id});//create new list item
alert(new_id);
///// CODING WORKS UP TO HERE
sb.addItems(li);//add to sortables
var first_item = $('content_list_images').firstChild; // get current first li
$('content_list_images').insertBefore(li, first_item);// insert into list
li.set('tween',{duration: 2000});
li.tween('opacity', 0, 1);
},
onFailure: function(){
}
});
var data = 'file='+json.get('filename')+'&extension='+json.get('extension')+'&date='+json.get('date')+'&width='+json.get('width')+'&height='+json.get('height')+'&mime='+json.get('mime')+'&page_id=<?=$id?>&new=<?=$unix?>';
req.send(data);
////////////////////////////////////////////////
}
});
</script>
I have been able to hack some things together to have it work how I need, but then I get to a certain point and I am not sure how to continue. All the posts get the responses that I need, but then I am at a loss. I would greatly appreciate some assistance as to how I am able to make this work.
Everything up to alert(new_id); works.
I am fully aware that I need to review more with Jquery and Mootools, and I will do so. I am on a tight deadline and I do appreciate any and all help!
The previous tools that they used to upload files was using FancyUpload.

Read local file text (tab separated values) with d3 or ajax cause syntax error in firefox development console

The reading works.
However I got a syntax error in the firefox console (which is tiresome when I read 30 files).
The files are annotation files like (time \t value) with no headers like :
0.0 5.2
0.5 5.6
1.0 6.3
...
This is the ajax code :
function getdatafromfile(filename) {
// Read annotation file. Example : %timeinstant \t %value \n
// Return an array of string
var arraydata
$.ajax({
type: "GET",
url: filename,
dataType: "text",
async: false,
success: function(csv) {arraydata = $.csv.toArrays(csv,{separator:'\t'}); }
});
return arraydata}
And with d3:
d3.text(filename, function(text) {
var data = d3.tsv.parseRows(text).map(function(row) {
return row.map(function(value) {
return +value;
});
});
console.log(data);
});
}
It seems that I could use one of those code, but I got a syntax error in both cases (with firefox 33.1).
A file reader could work like the code below.
In the example I've added a flag to use the content of the variable instead of a file. That's just for the demo and can be removed. The same code is here as jsFiddle.
Maybe you could add some validation before or after the $.csv method. So you know that the file was a csv/tsv file.
If you need to open the file with-out user interaction, you have to look for something different because JS is not allowed to open a file with-out the user choosing the file (security concerns, see this SO question).
You could add your data to a database and read it from there. e.g. Firebase or MongoDB or use a JSON file. The code of my other answer should work for a JSON file that you host with your webpage.
var demoTxt = "0.0 5.2\
0.5 5.6\
1.0 6.3";
var flag_usedemofile = true; //if true var demoTxt will be used
function doOpen(evt) {
var files = evt.target.files,
reader = new FileReader();
reader.onload = function() {
if ( !flag_usedemofile) {
var arraydata = $.csv.toArrays(this.result,{separator:' '});
showout.value = arraydata; //this.result;
} else {
var arraydata = $.csv.toArrays(demoTxt,{separator:' '});
showout.value = arraydata;
console.log(arraydata);
}
};
reader.readAsText(files[0]);
}
var openbtn = document.getElementById("openselect"),
showout = document.getElementById("showresult");
openselect.addEventListener("change", doOpen, false);
#showresult {
width:98%;
height: 300px;
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<script src="http://cdnjs.cloudflare.com/ajax/libs/jquery-csv/0.71/jquery.csv-0.71.min.js"></script>
<input type="file" id="openselect" />
<textarea id="showresult"></textarea>
I'm not exactly sure what syntax error you are getting. But I think the error have something to do with the mime type of your json request.
I think the best way is to wrap your data in json and then use JSONP. (I have also tried to get it working with text/plain, but with-out success.)
Please check the following example for details. You can also find the same example on
jsFiddle.
(function ($) {
var url = 'http://www.mocky.io/v2/547c5e31501c337b019a63b0'; // dummy url
var jsonCallback = function (csv) {
var arraydata;
console.log(data);
$('#data').html(JSON.stringify(csv, null, 2));
arraydata = $.csv.toArrays(csv.data,{separator:'\t'});
console.log(arraydata);
};
$.ajax({
type: 'GET',
url: url,
contentType: "application/json",
dataType: 'jsonp'
}).done(jsonCallback)
.fail(function (xhr) {
alert("error" + xhr.responseText);
});
})(jQuery);
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<pre id='data'></pre>

Im not able to decode my file passed from JAVASCRIPt to PHP

I am encoding and passing a file (word document) to php.How can I read this and write to a file?
I have submit button.On submit, I am passing an ajax.Before that, Iam encoding the file with file reader.On submit button, an event 'handleFileSelect' is trtiggered.The file is read as dataurl and sent to php via ajax.
I am able to get the data as encoded.If the file is a text, i am also able to decode .But
its not able to get the contents of a word file.If I decode
How would I do this?
My code :
//File Convertion--Function to convert images to base 64 encoded format
function handleFileSelect(objEvent) {
var strFiles = objEvent.target.files; // FileList object
strInput = document.getElementById('uploaded_file');
strFile = strInput.files[0];
strFiletype=strFile.type;
strFileSize=strFile.size;alert(strFiletype);
strFiletype=strFiletype.split("/");
//Checking wheter the uploaded file is image or not
if(strFiletype[0]!='image') {
for (var i = 0, f; f = strFiles[i]; i++) {
var reader = new FileReader();
// Closure to capture the file information.
reader.onload = (function(theFile) {
return function(e) {
// Render thumbnail.
strGlobalImageData=e.target.result;
};
})(f);
reader.readAsDataURL(f);
}
} else {
alert("NOT A DOC");
}
}
//ajax call to send files to php
var app = 'contact.php';
$.ajax({
url: app,
async: false,
type:"POST",
data : "file="+strGlobalImageData,
dataType: "jsonp",
contentType: 'application/x-www-form-urlencoded',
processData:false,
jsonp: "jsoncallback",
success: function(html) {
alert("Thank you. We will be in touch with you");
},
error: function(){
alert("Thank you. We will be in touch with you");
}
});
//Php side--contact.php
<?php
$files=base64_decode($_POST['file']);
If I decode, I am getting a binary format of the word file
The issue was with character replacing.We need to replace befor decoding the data.The exact code is shown below:
In Php,
$files=trim($_POST['file']);
$strEncodedData= str_replace(' ','+',$files);
$strFilteredData = explode(',',$strEncodedData);
$strDecodedData= base64_decode($strFilteredData[1]);
$arrFiles = explode(",",$files);
file_put_contents("myfile.doc",$strDecodedData);
This content will be written into file "myfile.doc".
Thank you for all effort made by my friends

Preloading the XML via jQuery or Javascript

Hey guys, how can I preload an external XML file in Javascript/jQuery?
This is my XML loader:
jQuery.ajax({
type: "GET",
url: dictionaryList,
dataType: ($.browser.msie) ? "text/xml" : "xml",
success: function(xml) {
var xml2 = load_xml(xml);
var i=0;
$(xml2).find('wordle').each(function(){
$(xml2).find('w').each(function(){
var tmpHold = $(this).text();
if (tmpHold.substring(0, 1) == letter) {
if ($(this).attr('p') == 1) {
wordColor = 'color: #693030';
} else {
wordColor = 'color: #5a5a5a';
}
$('#wordList').append('<li class="w" style="'+wordColor+';">'+$(this).text()+'</li>');
}
});
});
}
});
one possibility, and it sounds like this is what you want, would be to send the response document, (xml) above, to a variable that could be processed on-demand at a later time based on some event.
the stored xml document, and the xml processing function, would live in an object, and the xml processing function would be called based on an event trigger rather than the ajax success event. if this doesn't make sense let me know and i can provide some sample code ...
also, i'd recommend adding an error: function to the ajax call if you don't already have one in place.
I think it's good to keep backend xml generator/retriever script in case if you want to get xml from a different domain.
jQuery.ajax({
type: "GET",
url: XML_GENERATE_BACKEND_URL, // data.xml, /generate/xml etc.
..
..
..
Sultan

Categories