Please find below fiddle-link.
https://jsfiddle.net/zwb1hbdz/1/
[https://jsfiddle.net/zwb1hbdz/1/][1]
Range for line is time date: "08:45" to date: "08:54"
And for line 2 date: "08:44", to date: "08:53" so creates issue with second line?
how can i set dynamic time range from "08:44" to "08:54"?
You just need to sort your data in order of time. I've adjusted your parse function to match how it's stored, then used the array sort method:
var parse = d3.time.format("%H:%M").parse;
data = data.sort(function(a,b) {
return parse(a.date) - parse(b.date);
} );
See https://jsfiddle.net/zwb1hbdz/2/
Related
I have a data-frame and I want to send it in my Django template.
dummy code in views.py:
def graphs(request):
df_new = pd.read_excel("/home/cms/cms/static/Sensorik.xlsx")
times = df_new.loc[:, df_new.columns[0]].to_json(orient='records')
# columns[0] contains datetime values
data_color = df_georgian.loc[:, df_georgian.columns[2]]
color = data_color.to_json(orient='records')
context = {
'times': times,
'data_color': color,
...
}
return render(request, 'graphs/graphs.html', context)
In my template, I get these data like the following:
<script>
var times = {{ times|safe }};
console.log('index: ', typeof(index));
var color = {{ data_color|safe }};
</script>
the color variable is totally ok but the times variable when get to JSON format turns from 2018-05-29 08:09:00 format to something like it:
element: 1528108200000
I want to be able to plot the color based on times to get a line graph with plotly. i.e. I want to show times as x-ticks of my plot.
any suggeston on
1- how to send datetime dataframe to django template?
or
2- how to convert element: 1528108200000 into a datetime in js to plot it as x-ticks?
Whenever you see something that should be a date or time expressed in a large number like the 1528108200000 format, that means it is or is similar to a UNIX timestamp—the number of seconds past January 1, 1970. In this case, the length of the timestamp indicates that it's milliseconds, not seconds, so it's not exactly a UNIX timestamp but is very close.
Milliseconds since January 1, 1970 is the way that JS internally stores Date objects, which is interesting because it means some sort of conversion is probably happening on the JS side, not the python side (where you'd usually get ISO format or UNIX Timestamps).
In any case, it's pretty easy to solve on the JS side because you can simply parse the number:
dateObject = new Date(1528108200000);
If you do that parsing and pass the data to Plotly as Date objects, Plotly should be able to recognize your dates.
A simple way to do that would be:
const times = [1528108200000, 1528108200000, 1528108200000]
const parsed = times.map(time => new Date(time))
assuming your times variable is just an array of these integers.
Scenario:
I am fetching records around 50k through APIs. Each record in this has a Datetime field like this.
{
"ID": 7,
"CaseID": 7,
"Name": "ABC",
"DateTime": "2020-02-25 00:00:00.000",
},
{
"ID": 8,
"CaseID": 8,
"Name": "Skin",
"DateTime": "2020-03-12 00:00:00.000",
}
On UI, I have a dropdown that filters data based on the current quarter, last quarter, this month, last year, last month, etc and a custom range datepicker.
On the Selection of any option, I have a filter function which filters this huge record based on the start and end date of the option selected.
For example, the Current Quarter then, Start Date of the quarter and end date of Quarter.
Filter function just checks if the records date Is falling under the range of the option selected.
My Code:
const filteredData = _.filter(hugeRecords, record => {
const visualDate = moment(record.DateTime).format('YYYY-MM-DD');
return moment(visualDate).isBetween(this.periodTitleStartDate, this.periodTitleEndDate);
});
where periodTitleStartDate and periodTitleEndDate is the range of dropdown option selected in 'YYYY-MM-DD' format.
Now the problem is if I have records of around 10k then no issue. If records are around 20k, UI hangs a little bit. If records are 30k-50k, Browser UI hangs for longer.
Furthermore, I will use this filtered array to pass into highcharts to create charts in my dashboard view.
I tried many terms and other StackOverflow and Github questions/issues but not able to solve my problem.
How can I improve the performance of filtering?
Note: Passing dropdown option to rest API and fetch only filtered records is not an option for now.
Thanks in advance.
Well you could preprocess the array to have the calculated timestamp. This only has to be done once.
Then the filter will only compare numbers (timestamps) and speed up the process a lot.
Also you could then sort the array by timestamp and thus do a lot of improvements when searching between dates.
// Preprocess
const processed = Records.map( record =>{
record.timestamp = moment(record.DateTime).format('YYYY-MM-DD').unix();
return record;
});
// Filter assuming you have converted the boundary Dates to timestamps with moment and unix().
const filtered = processed.filter( record => {
return !(this.periodTitleStartDate < record.timestamp && this.periodTitleEndDate > record.timestamp
});
I have event objects in MonogDB that look like this:
{
"start": 2010-09-04T16:54:11.216Z,
"title":"Short descriptive title",
"description":"Full description",
"score":56
}
And I need to get a query across three parameters:
Time window (event start is between two dates)
Score threshold (score is > x)
Full-text search of title and description
What's the right way to approach this efficiently? I think the first two are done with an aggregation but I'm not sure how text search would factor in.
Assuming your start field is of type date (which it should be) and not a string, here are the basic components that you'd want to play with. In fact, given the ISO 8601 structure of a MongoDB date a string based comparison would work just as well.
// create your text index
db.collection.ensureIndex({
description: "text",
title: "text"
})
// optionally create an index on your other fields
db.collection.ensureIndex({
start: 1,
score: 1
})
x = 50
lowerDate = ISODate("2010-09-04T16:54:11.216Z") // or just the string part for string fields
upperDate = ISODate("2010-09-04T16:54:11.216Z")
// simple find to retrieve your result set
db.collection.find({
start: {
$gte: lowerDate, // depending on your exact scenario, you'd need to use $gt
$lte: upperDate // depending on your exact scenario, you'd need to use $lt
},
score: { $gt: x }, // depending on your exact scenario, you'd need to use $gte
$text: { // here comes the text search
$search: "descriptive"
}
})
There is an important topic with respect to performance/indexing that needs to be understood, though, which is very well documented here: Combine full text with other index
This is why I initially wrote "components of what you'd want to play with". So depending on the rest of your application you may want to create different indexes.
I would like to make a bar chart where x-axis is date e.g. 2017-08-12, and the y value is the number of data rows (i.e. records count) on the same date.
// get data into right format
var dateFormat = d3.time.format("%Y-%m-%d");
pageViews.forEach(function(d) {
d['timestamp'] = dateFormat.parse(d['timestamp'].slice(0,10));
d['timestamp'].setDate(1);
});
// Create Crossfilter instance
var cf = crossfilter(data);
var dateDim = cf.dimension(function(d) {return d['timestamp'];});
var numByDate = dateDim.group();
but if I do
console.log(numByDate.top(Infinity));
this returns 5 elements that are the first day of each month, so the group I have is in terms of Year-Month instead of Year-Month-Day, how can I resolve this?
I looked into this crossfilter.js: group data by month, but it did not work, after using .all() I still get the same thing back.
And I tried this:
var numByDate = dateDim.group(function(d) {return d['timestamp'];});
it just returns me an empty object.
So this is a result of my unfamiliarity with js and copy paste example code ...
The issue comes from misuse of the setDate method,
setDate(1)
would just set the day of the month in the data to the first day, so the group method would only have several unique dates to work with.
EDIT 4/16/2012: I solved the issue of getting the timezone abbreviated into a letter format, had to download a third party sorting method and add a few things to get the desired results. The only problem now is Daylight Savings Time handlers, but there are a bunch of subjects on that. However if anyone knows how to handle UTC Daylight Savings hanlers, please feel free to help.
Thank you everyone.
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
I've made an html table that I've binded with a javscript viewmodel using knockoutjs that pulls the info from a private server usin JSON function. I'm trying to make each column sortable (click on the column header once to get everything in descending order according to that column's info; click header again to get everything in ascending, and a third time to get everything in it's original order).
PLEASE NOTE: I have searched for my problem and have seen other solutions, but nothing so far has worked for me. I'm hoping to find a solution specific towards my code.
The Javascript ViewModel.js file is basically like this:
Event(column1, column2, ...., columnN){
var self = this;
self.column1 = column1;
self.column2 = column2;
.
.
}
//Sort column2 that has the Dates (dd (day) HHMM (hours/minutes) mmm (month) yy (year) format)
self.sortColumn = "Column2"
self.sortAscending = true;
self.SortByDates = function(){
if(self.sortColumn == "Column2")
self.sortAscending = !self.sortAscending;
else{
self.sortColumn = "Column2"
self.sortAscending = true;
}
self.rows.sort(function(a,b){
if(self.sortAscending == true)
for(self.Column2 in self.rows)
return a.Column2 > b.Column2 ? 1 : a.Column2 < b.Column2 ? -1 : 0;
else
return a.Column2 < b.Column2 ? 1 : a.Column2 > b.Column2 ? -1 : 0;
});
}
//specify location of server and info and get them
function getEvents(){
$.getJSON("http://.........",
function (data){
$.each(data.d, function(i, item){
handleEvent(item)
})
}
);
}
//pushes (AKA populates) info from server into the table
function handleEvent(item){
var newEvent = new Event(item.Column1InfoFromServer,
formatJSONDate(item.Column2DateInfoFromServer), .....)
this.Model.rows.push(newEvent);
}
//Formats the date info from server into dd (day) HHMM (hours/minutes) mmm (month) yy (year)
formatJSONDate(jsonDate){
var date = new Date(parseInt(jsonDate.substr(6)));
return date.format("dd HHMM mmm yy");
}
this.Model = new ViewModel();
this.getEvents();
ko.applyBindings(this.Model);
I'm having one hell of a hard time getting the Date in its converted form (yes it HAS to be in that form --> actually, I still need to figure out how to include the time-zone abbreviation right after the 'HHMM' part based off of UTC). So lets say I have "11 1136 Apr 12" and "22 1624 Jan 12" among other dates in the table. Right now when I try sorting the table according to the dates, they don't sort appropriately. Any help is appreciated, thank you.
EDIT: To be clear, I'm trying to display the timezones in military timezone codes (timezones 'A'-'Z'). Also, the dates being taken from the server are already in UTC.
UPDATE:
I was looking at another question, and someone created a knockout grid addon:
https://github.com/ericmbarnard/KoGrid
I bet this might help you out :-)
---OLD ANSWER for nostalgia----
There are some great helper functions in the Underscore library, one of them being sort:
http://documentcloud.github.com/underscore/#sortBy
sortBy_.sortBy(list, iterator, [context]) Returns a sorted copy of
list, ranked in ascending order by the results of running each value
through iterator. Iterator may also be the string name of the property
to sort by (eg. length).
_.sortBy([1, 2, 3, 4, 5, 6], function(num){ return Math.sin(num); });
=> [5, 4, 6, 3, 1, 2]
I'd give this a shot, along with creating a better model for your dates. It sounds like you need to store a property which is a unique point in time, along with a text value for the user.
Well, to get numeric values for your date objects, sort by pDate.valueOf(). This will give you the # of millisecond since epoch.
However, there is an issue inside of your sort function, but I'm not sure what it is supposed to do. You can't walk an object inside of a sort function and return values like that.