We Keep Coding

Loading, paging and filtering ComboBoxField (extreact)

I want to display data in a sencha comboboxfield, however the dataset has 15,000+ records and it is not feasible to load it all at one time. What I would like to do is have the sencha extreact comboboxfield:
Load remotely (restful api)
return a subset (25 records) that can be scrolled through (infinite scrolling - get next 25 and so on).
filter data based on data typed by the user. This would be a refresh of the data not filter what has already been fetched.
I've read through the documentation and sounds like I may need to use ExtJs store and proxy but for some reason it's not getting through my thick skull :). Any examples I have found appear to have the proxy setup that requires you access the api directly. I cannot do this because I'm working within a framework where that is not possible. All calls need to go through an async function due to authentication, logging and other purposed. If anyone has some pointers or examples to share that would be greatly appreciated.

I've had to make some tweaks to our system but I have this working and thought I would share. I have to give credit to this thread for pointing me in the right direction. Forgive me in advance for formatting issues.
First off, I couldn't figure out how to get the proxy to call a function so just went with the Ext.data.proxy.Ajax approach. A bit of a work around for me but it works. Here is my store and proxy setup (this is placed into component state):
postalCodes = (props) => {
let postalCodeStore = new Ext.data.Store ({
proxy: {
type: 'ajax',
url: postalCodesApiRoutes.DROPDOWNDATA, //stores my api route information
reader: {
type: 'json',
rootProperty: 'postalCode',
totalProperty: 'totalRecordCount'
},
pageParam: 'index',
extraParams: {
pageSize: props.pageSize
},
headers: { ...authHeader(), // returns my authentication header info
<OTHER LOGGING INFO WENT HERE>
},
noCache: false
},
autoLoad: false,
pageSize: props.pageSize,
clearOnPageLoad: false
})
return postalCodeStore;
}
The following is my ComboBoxField config (note: I had to use this.handlePickerCreate.bind(this) because the sencha controls don't appear to honor the arrow function binding):
<ComboBoxField
displayField="value"
valueField="postalCodeId"
queryMode="remote"
store={this.state.postalCodes}
remoteFilter={true}
clearable
allQuery=""
triggerAction="all" // I want when the dropdown clicked on it ignores whats in the combobox text and queries the entire data set. Otherwise if a user types a value a server-side filter will be executed.
queryParam="filter"
onPickerCreate={this.handlePickerCreate.bind(this)}
/>
The following is the handlePickerCreate and onScroll functions:
handlePickerCreate = (obj, picker) => {
picker.getScrollable().on('scroll', this.onScroll);
}
onScroll = (scroller, x, y, deltaX, deltaY, opts) => {
if (y >= scroller.getMaxPosition().y) {
let sStore = this.state.areas;
if (sStore.getTotalCount() != sStore.getCount()){
sStore.nextPage({
callback: function () {
scroller.doScrollTo(0, y);
console.log('store count', sStore.getCount())
}
});
this.setState({areas: sStore})
}
}
}
Anyway, there are still some quirks that I'm trying to figure out like when a value is selected from the list then you click on the trigger/dropdown arrow the picker displays and immediately disappears and the combobox input field clears out. I see a call being made to the server and data being returned but can't figure out what is happening. Well hope this helps someone.

Handsontable is not fully rendered when using Chrome's built in search function

Weird, inconsistent problem I am running into when using Chrome's built in search function. I have some 250 lines of data to be rendered in a handsontable, more than can be displayed on your screen without scrolling or zooming out.
http://jsfiddle.net/hU6Kz/3723/
var myData = [
["", "Kia", "Nissan", "Toyota", "Honda"],
["lots of data begins here"],
];
$("#exampleGrid").handsontable({
data: myData,
startRows: 5,
startCols: 5,
minSpareCols: 1,
minSpareRows: 1,
rowHeaders: true,
colHeaders: true,
contextMenu: true
});
Observe that when you first pull up the page, you can scroll down and all the data is rendered in the handsontable.
Now hit control + f to pull up the Chrome's built in search function. Search for any character in the handsontable. Much of the data in the handsontable is no longer rendered! Occasionally the data will get rendered again if I search for something else, but it seems inconsistent and I can't find a common cause..
This does not seem to be a problem in firefox, but my company is decidedly in the Chrome camp. Help me, o wizards of the internet.

This is because Handsontable uses a wonderful technique called "Virtual Rendering" which only renders the rows you are currently looking at plus a few more. It makes it possible to display "infinitely" many rows. The problem with ctrl+f is that it searches the HTML text so you won't be able to search using this.
This is why there is a search plugin available which returns to you a list of all matching cells. From there you can do many things like on enter, scrollTo the next available matching cell (search). Another very famous application is to filter rows by recreating the table with less data (filter).

Here there is a working demo using the search feature of handsontable.
In the search input write Test and then hit Enter.
This is the method that does what I described above.
var searchField = document.getElementById('search_field');
Handsontable.Dom.addEvent(searchField, 'keyup', function (event) {
if (event.keyCode == 13) {
var queryResult = hot.search.query(this.value);
hot.selectCell(queryResult[0].row, queryResult[0].col);
}
});
http://jsfiddle.net/pdivasto/hp8ge8kk/

ExtJS Grid not refreshing after store update

I have a grid and a store. The store gets populated with some data - country - while the rest is empty. Once I receive more inputs I fill up the store with the inputs.
Assume that I have only 1 country in the store after the initial load.
Store:
feedTheGrid = new Ext.data.JsonStore({
autoDestroy : true,
autoSave : false,
autoLoad : true,
idProperty: 'country',
root: 'root',
totalProperty : 'totalcount',
fields : ['country', 'towns'],
proxy : new Ext.data.HttpProxy(new Ext.data.Connection({
url : "country/searchCountries.action",
method : 'POST',
timeout : 300000
}))
});
So the user inputs towns. I save a town in the array towns using push():
listeners: {
'change': function() {
if (feedTheGrid.getAt(0).towns === undefined) {
feedTheGrid.getAt(0).towns = [];
}
feedTheGrid.getAt(0).towns.push(this.getValue());
gridMonster.getView().refresh();
}
}
I want the grid to update every time a new town is put in.
This is part of the column model I use for gridMonster:
{ header: "Town 1", width: 150, dataIndex: 'towns',
renderer: function(val) {
if (val[0] != undefined) {
return val[0];
}
}, sortable: true},
{ header: "Town 2", width: 150, dataIndex: 'towns',
renderer: function(val) {
if (val[1] != undefined) {
return val[1];
}
}, sortable: true},
{ header: "Town 3", width: 150, dataIndex: 'towns',
renderer: function(val) {
if (val[2] != undefined) {
return val[2];
}
}, sortable: true}
The issue is that after a town is put in the grid never refreshes.
Some things I noticed so far:
The store gets updated with the new value when I check through Firebug. Also it seems the val that is given in renderer is always empty String, even after I push the new value into the array and refresh the grid view.
I am using ExtJS 3.3.1
I know its old so I am open to solutions that would work for 4.0+ and I can try to apply it to 3.3.1.
EDIT: Found the issue
Very subtle problem, spent hours trying to find what is the problem only to find out I was looking in the wrong place. The issue was in this line:
feedTheGrid.getAt(0).towns.push(this.getValue());
This line actually creates a new field towns as part of the record that you get using getAt(0). Using firebug to see if getAt(0).towns[0] is populated actually returns the value you pushed which is misleading.
ExtJS is not aware of this new field and it shouldn't be.
This line should be:
feedTheGrid.getAt(0).data.towns.push(this.getValue());
The trick is to point at the right place: .data. Javascript/ExtJS allows you to add new field without any warning because technically you are not doing anything wrong. Logically it creates a new field that does nothing.
#Lorenz Meyer I am able to display elements of the array using renderer while pointing at towns array in the column model.

There is a major misunderstanding about what a store and a grid are in ExtJs.
Both are just a representation of tabular data. A store is the representation in memory of a table, in order to get fast access to records filtering and more features. A grid is the visual representation of a table in the browser window.
Such a table could look like :
Id | Country | City
-------------------
1 | US | NYC
2 | France | Macon
3 | US | Macon
First of all on one table row, you can only have one city. In tables, you cannot have an array in a single field. If you have more than one city, this is what rows are for.
Then, your idProperty cannot be the country. If it were, this would mean that countries are unique. Obviously, they aren't. There can be more than one city in a country. (cities cannot be the idProperty neither, because there exists more than one city with the same).
Now you certainly begin to understand, that feedTheGrid.getAt(0).towns.push(this.getValue()); makes no sense whatever. The towns column cannot be an array. You insert a new row with feedTheGrid.add({country: 'US', towns: 'LA'}).
At best the towns column can be a comma separated value, in which case, you update the store with
var record = feedTheGrid.getAt(0),
currentTowns = record.get('towns');
record.set('towns', currentTowns + ', ' + value)
The method set of the record does all the magic: It passes the update event to the grid in order to rerender it. Your push in addition to be wrong will not trigger the events necessary to update the view.
About the grid: The columns of your grid should be:
{ header: "Id", width: 150, dataIndex: 'id', sortable: true},
{ header: "Country", width: 150, dataIndex: 'country', sortable: true},
{ header: "Town(s)", width: 150, dataIndex: 'towns', sortable: true}
Notes:
That listeners: looks really strange to me, but maybe it's just because the bigger picture is missing.
feedTheGrid is a strange name for a store. The store does not feed the data. It is tha data.
A renderer alway must return a value, else the cell will remain blank. Therefore your renderers cannot work.

In your renderers the val can not be an array. It's a simple variable. That's why your grid is not refreshing.

Very subtle problem, spent hours trying to find what is the problem only to find out I was looking in the wrong place. The issue was in this line:
feedTheGrid.getAt(0).towns.push(this.getValue());
This line actually creates a new field towns as part of the record that you get using getAt(0). Using firebug to see if getAt(0).towns[0] is populated actually returns the value you pushed which is misleading.
ExtJS is not aware of this new field and it shouldn't be.
This line should be:
feedTheGrid.getAt(0).data.towns.push(this.getValue());
The trick is to point at the right place: .data. Javascript/ExtJS allows you to add new field without any warning because technically you are not doing anything wrong. Logically it creates a new field that does nothing.
#Lorenz Meyer I am able to display elements of the array using renderer while pointing at towns array in the column model.

Dojo DataGrid Virtual Scrolling How-To?

I've been digging around for this one quite a bit. I'm using dojox.grid.datagrid and I have an ajax call that brings back 200-300 rows.
The grid renders and scrolls just fine in Chrome but scrolling is excruciatingly slow in IE 7 and 8. I'd like to use virtual scrolling to try and remedy the issue but can't find any sample code.
Here's what my code looks like at present.
function setupAvailableScenes(location) {
var avaiableScenesGridPane = dijit.byId("AvaiableScenesGridPane");
var availableScenesGrid = dijit.byId("AvailableScenesGrid");
if (_isFirstLoad) {
availableScenesGrid = new dojox.grid.DataGrid({
id: 'AvailableScenesGrid',
store: availableScenesStore,
query: { Id: "*" },
sortInfo: "1",
rowsPerPage: 20,
autoHeight:20,
style: "width:315px",
structure: [
{ name: "Available Scenes", field: "Name", width: "85%" },
{ name: " ",
field: "_item",
rowsPerPage: "25",
type: dojox.grid.cells._Widget,
editable: false,
width: "15%",
formatter: function (scene) {
return new dijit.form.Button(
{
label: "+",
onClick: function () {
AddSceneToSelectedScenes(scene);
}
})
}
}
]
});
avaiableScenesGridPane.set('content', availableScenesGrid);
}
var availableScenesStore = new dojo.data.ItemFileWriteStore({
url: _applicationPath + "/Location/" + location.Id + "/Scene.json",
preventUrlCache: true
});
availableScenesGrid.setStore(availableScenesStore);
}

Often one of the biggest things you can do to improve DataGrid performance is to throw away the ItemFileReadStore/WriteStore and use an optimized data store (personally I like QueryReadStore). It would mean needing a server-side servlet of some kind (PHP/JSP/etc) to handle the virtual scrolling/pagination, but I've seen major perf boosts over just using a store backed by a JSON file.
Some other things to consider, which may or may not help:
give your anonymous formatter function a name and try scrolling the table with the Chrome or Firebug profiles turned on to see if it's hogging a lot of cycles (or, like Vijay Agrawal said, you could try replacing the dijit.form.Button with a vanilla html <button> tag)
you shouldn't actually need to specify the dojox.grid.cells._Widget type for that cell; having a custom formatter returning a valid Dijit should be sufficient to make the Grid do the right thing.

Since you specified rowsPerPage=25, it is already doing virtual scrolling (it pulls the new set of rows only when user scrolls down)
Since you mention scrolling is very slow, the performance issue seems to be around rendering the new rows - you may try a couple things to improve performance:
1) remove autoHeight attribute. Instead, specify a fixed height in the style attribute
2) in the formatter function, instead of returning a dijit, try returning a simple html button/anchor styled as button
so remove the type:dojox.grid.cells._Widget attribute and in the format function return the html you want to use

How to show all rows in the jqGrid?

jqGrid exposes a property rowNum where you can set the number of rows to display for each page. How do you set the grid to just display ALL rows?
Right now I'm just setting the rowNum to something really high like <%= int.MaxValue %> but I'm wondering if there is a better way.

In the latest version of jqGrid, you can set rowNum to -1 to instruct the grid to always display all rows:
rowNum: -1
See the latest jqGrid documentation here.
Specifically:
Sets how many records we want to view in the grid. This parameter is passed to the url for use by the server routine retrieving the data. Note that if you set this parameter to 10 (i.e. retrieve 10 records) and your server return 15 then only 10 records will be loaded. Set this parameter to -1 (unlimited) to disable this checking.
Update
Unfortunately this behavior was broken in jqGrid 3.6.3. According to this post from Tony:
Yes, this is true. The reason is the new introduced scroll:1. In the future we will correct this behavior.
So the jqGrid developers are aware of this problem and apparently are planning to fix it in a future release. Unfortunately this post was from over a year ago...
At this time, all I can recommend is that you set rowNum to a very large number to simulate the behavior of -1.
You can also try whatispunk's solution below of using rowNum: ''. However, I tried this on a grid containing local data (loadonce: true). When attemping to sort the rows all of the grid's local data would disappear. So this solution does not seem to work for grids with local data, unless this defect has been fixed in a later version of jqGrid (I tested it on jqGrid 3.8.2). If you have feedback, please post a comment below!
Update - April 16, 2014
According to the jqGrid team this is now fixed:
I have added support to set different display values on pager select box including -1 for all.
I have not had a chance to test to confirm the fix, though. Presumably this change will be in the next release after jqGrid 4.6.0.

jqgrid (3.5 anyway) doesn't seem to have an elegant built in way to do this. The best I have found so far is to add something like the following to your grid options:
rowList:[10,20,30,100000000],
loadComplete: function() {
$("option[value=100000000]").text('All');
},
Where the 100000000 is some arbitrarily higher number than the maximum # of rows you will ever return, and the option[value=] line is so your user interface looks a little nicer. Jenky, but works for me.

if you dont wish to use paging at all then change you server side code to simply return all the rows. dont use the rows parameter at all.
if you want to have the rowlist but also have an option to show all then do something like this in the grid properties
jQuery("#statement_mods").jqGrid({
rowList:['ALL',30,50,100,200]
});
and then in the serverside code make sure that you ignore the rows parameter if GET['rows']='ALL'

This works:
// Step1 - defines the rows
jqGridOptions.rowList =[10, 50, 100, 500, 'All'];
...
...
// Step2 - Change the 'All' to a meaningful value
loadComplete: function (data) {
$(".ui-pg-selbox option[value='All']").val(1000);
}

setting rowNum:-1 did the trick for me

If you have set the pagination on the navbar, you can also access to the total number of rows written on the right-bottom of the grid and then append to the generated RowList option.
Do something like :
// Get the total number of rows and delete space between numbers (Split the content of the div depending of the language (for me french)
var val=jQuery("#pager_right div").text().split('sur')[jQuery("#pager_right div").text().split('sur').length-1].split(' ').join('');
// And do the appending if the option isn't already added
if(!$(".ui-pg-selbox option[value='"+val+"']").length > 0)
jQuery(".ui-pg-selbox").append($('<option></option>').val(val).html(val));

Setting rowNum: '' you get all rows.

Jqgrid.PagerSettings.PageSize = Max Row you want to display;
Jqgrid.ToolBarSettings.ToolBarPosition = ToolBarPosition.Hidden;

I've got this working:
$('#bla').jqGrid({
...
'rowNum' : 0,
'loadOnce' : true,
'loadComplete': function(data) {
$(this).jqGrid('setGridParam', 'rowNum', data.total);
},
...
});
This works with and without the loadOnce option set to true. Note that you have to set the rowNum option to 0 first, if you leave out this option it'll still default to the 20 records to show.
Also, I'm assuming you're returning the total rows from the server in the documented JSON reader format.

resolved it with simple change:
rowNum: inputDataArray.length
where inputDataArray is the array that I am providing to the Grid.

By default the JQ grid show 20 rows Max ,if you are using not using pagination:
// To over come with this problem ,you can just write the bold mark
(rowNum:10000,):
$("#MasterDataDefinationGrid").jqGrid({
url: 'FetchData.aspx/GetDataFromDB',
datatype: 'json',
mtype: 'POST',
height: 300,
autowidth: true,
serializeGridData: function (postData) {
return JSON.stringify(postData);
},
ajaxGridOptions: { contentType: "application/json" },
loadonce: true,
colNames: [Your column names],
colModel: [Your model],
formatter: 'actions',
pager: '#MasterDataDefinationPager', pgbuttons: false,pgtext:false,
multiselect: false,
ignoreCase: true,
**rowNum: 10000,**
loadtext: 'Loading ...',
gridview: true,
hidegrid: false,
jsonReader: {
page: function (obj) { return 1; },
total: function (obj) { return 1; },
records: function (obj) { return obj.d.length; },
root: function (obj) { return obj.d; },
repeatitems: false,
id: "0"
},
caption: 'Data'
});

You can also go into jquery.jqGrid.js and change "rowNum:20" to "rowNum:Some-Really-Large-Number". When you define your jqGrid, don't specify rowNum. Then return your entire dataset back to jqGrid.

Even if it still appears in the doc that you cannot set rowNum to -1 as of jqGrid 4.5.4 it works again (maybe in earlier version too).

loadComplete: function (data) {
//set our "ALL" select option to the actual number of found records
$(".ui-pg-selbox option[value='ALL']").val(data.records);
}
This changes the "ALL" option to the actual number of records in the dataset.

Setting rowNum:-1 works for me like, after that it was showing all records but it still has row number option in grid footer like this:
To remove this I just added a css option display none by getting the sector in jquery. Like this
$('#id_tableCell').css('display', 'none');
Note: This css setting should be done when the grid load is completed.