I am currently working on a Bokeh script that will display an earthquake catalogue on a map. So far, I have had little trouble, but I have stumbled upon a couple questions.
1) I came upon this stackoverflow Q&A: Get selected data contained within box select tool in Bokeh, which was very helpful in learning how to use CutsomJS to take the selected data and output it. I have, however encountered a bit of oddness. The script only seems to work properly for me in the Jupyter notebook. The CustomJS portion reads as follows:
source.callback = CustomJS(args=dict(p=p), code="""
var inds = cb_obj.get('selected')['1d'].indices;
var d1 = cb_obj.get('data');
console.log(d1)
var kernel = IPython.notebook.kernel;
IPython.notebook.kernel.execute("inds = " + inds);
If I try running it through and html window, the following command has no output:
subset = zip([source.data['xvals'][i] for i in inds],
[source.data['yvals'][i] for i in inds])
subset_array = np.array(subset)
outfile = 'Scripts/subset.xy'
fo = open(outfile,'w')
for i in range(len(subset_array)):
print(subset_array[i,0],subset_array[i,1],file=fo)
fo.close()
I could use some advice on how to get the selected data to output to a variable, and ultimately a text file when using html, as opposed to the Jupyter notebook.
2) I have been trying to create a legend for my plot. The earthquakes are indicated by circles (so it is a circle plot), and the circles vary in size based on their magnitude and in colour based on their depth, which I have gotten to work. What I would like to do is show circles of different sizes in the legend, each reflecting different magnitudes (Mag 1, Mag 2, Mag 3, etc.). Here's the command I use for plotting the data:
p.circle("xvals", "yvals",source=source, size="radius", fill_color=transform("depth", mapper),legend = 'Magnitude')
I've tried messing around with the legend commands, but it's all a bit difficult for me to grok, even though I feel like this should be a pretty simple thing to do.
I am utilizing the latest version of Bokeh (0.12.7) and Python 2.7.12. Thanks for any help, and I appreciate the time!
On question 1):
If you are creating your html file with Bokeh's output_file function, you cannot access any python code from it, since the resulting file is just html + javascript. The file saving functionality could be embedded in the javascript code (a bit tricky since it seems there is no solution that works in every browser). Another option would be to run a bokeh server and implement some mechanism to push the data to the client.
Below is a rough example where the file saving functionality is included in the html file. This should work on non-IE browsers.
from random import random
from bokeh.plotting import figure, output_file, show, ColumnDataSource
from bokeh.models import CustomJS, Button
from bokeh.layouts import row, column
# Create some random data and put it into a ColumnDataSource
x = [random() for x in range(100)]
y = [random() for y in range(100)]
source_data = ColumnDataSource(data=dict(x=x, y=y))
# Create a button that saves the coordinates of selected data points to a file
savebutton = Button(label="Save", button_type="success")
savebutton.callback = CustomJS(args=dict(source_data=source_data), code="""
var inds = source_data.selected['1d'].indices;
var data = source_data.data;
var out = "";
for (i = 0; i < inds.length; i++) {
out += "(" + data['x'][inds[i]] + ", " + data['y'][inds[i]] +") ";
}
var file = new Blob([out], {type: 'text/plain'});
var elem = window.document.createElement('a');
elem.href = window.URL.createObjectURL(file);
elem.download = 'selected-data.txt';
document.body.appendChild(elem);
elem.click();
document.body.removeChild(elem);
""")
# Plot the data and save the html file
p = figure(plot_width=400, plot_height=400, tools="lasso_select, reset")
p.circle(x='x', y='y', source=source_data)
plot = column(p, savebutton)
output_file("test.html")
show(plot)
Related
I am creating a bubble chart in the online Version of MS Excel with the Automate -> Code Editor Typescript language. I am trying to set the transparency for the markers in a bubble chart but I cannot figure it out and I couldn't find any info in the Office Scripts doc. In the following code I am looping through some info / colors / dataranges in the Worksheet and I am creating data series for the bubble chart dynamically.
for (let i = 0; i < categories_count; i++) {
var row = 13;
var column = 7;
var series_name = sheet.getCell(row + i, column).getValue();
var series_range_x = sheet.getCell(row + i, column + 1).getValue();
var series_range_y = sheet.getCell(row + i, column + 2).getValue();
var series_range_z = sheet.getCell(row + i, column + 3).getValue();
var series_color = sheet.getCell(row + i, column + 4).
getFormat().getFill().getColor();
var new_series = chart.addChartSeries(series_name);
new_series.setXAxisValues(sheet.getRange(series_range_x));
new_series.setValues(sheet.getRange(series_range_y));
new_series.setBubbleSizes(sheet.getRange(series_range_z));
new_series.getFormat().getFill().setSolidColor(series_color);
}
All the methods don't seem to have a transparency attribute which I could use.
It is also strange, that I cannot set the transparency manually in the online Excel version.
You could set the Bubble chart to a certain style.
function main(workbook: ExcelScript.Workbook) {
let sheet = workbook.getWorksheet("Sheet1");
let newChart = sheet.addChart(ExcelScript.ChartType.bubble, sheet.getRange("B1:D6"))
let chartStyle = newChart.getFormat().setColorScheme(25);
}
Setting the color scheme to 25 will give you MonochromaticPalette12 (see here).
This will give you a transparent-looking light blue for the Bubbles.
Excel Online (Excel on the Web) supports a subset of features currently available in Excel Desktop.
Office Script only supports APIs that can be used in Excel Online.
If you are unable to do something (like set the transparency of a chart) in the UI in Excel Online, it's unlikely this can be done though an Office Script API.
If this feature is important for your scenario please send feedback: Automate Tab -> Code Editor -> ... -> Send Feedback
For the project I'm working on I need to generate an image file from emoji (ideally Apple emoji). I thought it should be a fairly simple thing, but with each tool I use, I eventually run into a wall.
I've also considered working with an emoji set, like this one that I could query when needed. Unfortunately, the one I've linked to doesn't have Unicode 9.0 emoji such as avocado (🥑) shrimp (🦐) or harambe (🦍). Do you know of such an up-to-date set?
Code-wise, I've tried opentype.js, but it doesn't support .ttc fonts, which is the extension of the emoji font on my mac (Apple Color Emoji.ttc). I've converted the font to .ttf but that didn't work either:
var opentype = require('opentype.js');
opentype.load('./build_scripts/apple_color_emoji.ttf', function (err, font) {
if (err) {
alert('Could not load font: ' + err);
} else {
console.log("loaded the font",font);
var Canvas = require('canvas')
, Image = Canvas.Image
, canvas = new Canvas(200, 200)
, ctx = canvas.getContext('2d');
var path = font.getPath('🐐🦃', 0, 150, 72);
path.draw(ctx);
console.log('<img src="' + canvas.toDataURL() + '" />');
}
});
The result looks like this:
I've tried fontkit, which is supposed to be able to read .ttc files, but it throws an error if I try to use the Apple Color Emoji font.
var fontkit = require('fontkit');
var font = fontkit.openSync('./build_scripts/Apple Color Emoji.ttc');
// TypeError: font.layout is not a function
If I try the same with my converted .ttf file I end up with some incorrect svg:
var fontkit = require('fontkit');
var font = fontkit.openSync('./build_scripts/apple_color_emoji.ttf');
var run = font.layout('🐐🦃');
var svg = run.glyphs[0].path.toSVG();
console.log(svg);
// M-1 0ZM799 800Z
My question is, am I even approaching this the right way? Converting emoji that I already have on my machine to a .png or another format seems like something that should be fairly straightforward but I just can't get it to work.
EDIT:
I've found a list of emoji names by their hex codes in this repo (big shoutouts to rodrigopolo). Now I can simply use this:
var emoji = "😊".codePointAt(0).toString(16); //1f60a
var emojiFile = fs.readFileSync('./my-emoji-folder/' + emoji + '.png');
Still, would be great to know if somebody has a coding solution to this problem!
FURTHER EDIT:
Turns out the first solution I've found only included emoji up to Unicode 8.0, not Unicode 9.0. I've found a ruby gem gemoji that does emoji extraction. If you're not a ruby developer (I'm not), you can simply use the following commands in your shell:
git clone https://github.com/github/gemoji.git
cd gemoji
bundle install
bundle exec gemoji extract some-directory/emoji
You now have Unicode 9.0 emoji in your some-directory/emoji folder!
I was able to get this to work with fontkit by selecting a font from the font collection. I haven't found a case yet where using either of the TTFs included in the "Apple Color Emoji.ttc" gives different results.
const fs = require('fs');
const fontkit = require('fontkit');
const emoji = require('node-emoji');
const font = fontkit.openSync('./Apple Color Emoji.ttc').fonts[0];
let emo = emoji.get('100');
let run = font.layout(emo);
let glyph = run.glyphs[0].getImageForSize(128)
fs.writeFileSync('100.png', glyph.data);
I have a program where a camera is set up to constantly take pictures (about every 10 seconds or so) and the picture is sent to a folder on my server and then another program refreshes that folder constantly so that I always just have the most recent picture in that particular folder.
An HTML document exists that also constantly refreshes, and references that picture location to get and display the newest image.
What I'm trying to do is extract the EXIF data (that I've verified exists when I save the image from the active webpage and look at it's properties). I want to display the DateCreated (I believe this is DateTime) and the Latitude and Longitude (I believe is GPSLatitude and GPSLongitude).
I came across this library, exif-js, which seems like the go-to for most people trying to do this same thing in JavaScript. My code looks the same as the code at the bottom of the README.md file, except I changed out my img id="...." and variable names, (see below). It seems like it should work, but it's not producing any data. My empty span element just stays empty.
Is there an issue with the short time span that the page has before refreshing?
Thanks for any help!
Here's what my code currently looks like (just trying to get the DateTime info). I have also tried the GPSLatitude and GPSLongitude tags.
<!-- Library to extract EXIF data -->
<script src="vendors/exif-js/exif-js"></script>
<script type="text/javascript">
window.onload=getExif;
function getExif()
{
var img1 = document.getElementById("img1");
EXIF.getData(img1, function() {
var time = EXIF.getTag(this, "DateTime");
var img1Time = document.getElementById("img1Time");
img1Time.innerHTML = `${time}`;
});
var img2 = document.getElementById("img2");
EXIF.getData(img2, function() {
var allMetaData = EXIF.getALLTags(this);
var allMetaDataSpan = document.getElementById("Img2Time");
allMetaDataSpan.innerHTML = JSON.stringify(allMetaData, null, "\t");
});
}
</script>
go into ur exif.js file and then go to line 930 and then change it to
EXIF.getData = function(img, callback) {
if ((self.Image && img instanceof self.Image
|| self.HTMLImageElement && img instanceof self.HTMLImageElement)
&& !img.complete)
return false;
I know this may be already solved but I'd like to offer an alternative solution, for the people stumbling upon this question.
I'm a developer of a new library exifr you might want to try. It's maintained, actively developed library with focus on performance and works in both nodejs and browser.
async function getExif() {
let output = await exifr.parse(imgBuffer)
console.log('latitude', output.latitude) // converted by the library
console.log('longitude', output.longitude) // converted by the library
console.log('GPSLatitude', output.GPSLatitude) // raw value
console.log('GPSLongitude', output.GPSLongitude) // raw value
console.log('GPSDateStamp', output.GPSDateStamp)
console.log('DateTimeOriginal', output.DateTimeOriginal)
console.log('DateTimeDigitized', output.DateTimeDigitized)
console.log('ModifyDate', output.ModifyDate)
}
You can also try out the library's playground and experiment with images and their output, or check out the repository and docs.
I have to map a lot of different files with different structures to a db. There is a lot of different tables in those xlsx so I thought about schemeless noSQL approach, but I'm quite newbie in this field.
It should be a microservice with client interface for choosing tables/cells for parsing xlsx files. I do not have strict technology; it could be JAVA, GROOVY, Python or even a JavaScript engine.
Do you know any working solution for doing it?
Here is example xlsx (but I've got also other files, also in xls format): http://stat.gov.pl/download/gfx/portalinformacyjny/pl/defaultaktualnosci/5502/11/13/1/wyniki_finansowe_podmiotow_gospodarczych_1-6m_2015.xlsx
The work you have to do is called ETL (Extract Transform Load). You need to either find a good ETL software (here is a discussion about open source ETL) or to script your own solution in a language you are used with.
The advantage of a ready made GUI software is that you just have to drag and drop data but if you have some custom logic or semi structured data like in your xlsx example, you have limited support.
The advantage of writing your own script is you have all the freedom you need.
I have done some ETL work and I used successfully Groovy for writing my own solution with custom logic and so on, and in terms of GUI I used Altova Mapforce when I had to import some exotic file types.
If you decide to write your own solution you have to:
Convert all data to an easy to load format. In your case you have to convert each xls or xlsx tab to CSV with a naming convention.
Load your files in your chosen language for transforming
Do your logic to put data in a desirable format
Save it in a database (SQL or noSQL)
Maybe you should try Google Sheets to display excel and Google Apps Script (https://developers.google.com/apps-script/overview) to write custom add-on for parsing data to JSON.
Spreadsheet Service (https://developers.google.com/apps-script/reference/spreadsheet/) has plenty methods to access data in sheets.
Next you can send this JSON over API (https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app) or put directly into database (https://developers.google.com/apps-script/guides/jdbc).
Maybe isn't clean, but fast solution.
I had a project that done work almost the same as your problem but it seem easier as I had a fixed structure of xlsx files.
For xlsx parsing, I had experiment with Python and Openpyxl and had no struggle while working with them, they are simple, fast and easy to use.
For database, I recommend using MongoDB, you can deal with documents and collections in MongoDB just as simple as working with JSON objects or a set of JSON objects. PyMongo is the best and recommended way to work with MongoDB from Python I think.
The problem is you have different files with different structures. I cannot recommend anything deeper on this without viewing your data. But you should find the general structure of them or you have to figure out the way to classify them into common sets, each set will be parsed using appropriate algorithm.
Javascript solution, as xlsx2csv (you can make export anywhere):
var def = "1.xlsx";
if (WScript.Arguments.length>0) def = WScript.Arguments(0);
var col = [];
var objShell = new ActiveXObject( "Shell.Application" );
var fs = new ActiveXObject("Scripting.FileSystemObject");
function flush(){
WScript.Echo(col.join(';'));
}
function import_xlsx(file) {
var strZipFile = file; // '"1.xlsx" 'name of zip file
var outFolder = "."; // 'destination folder of unzipped files (must exist)
var pwd =WScript.ScriptFullName.replace( WScript.ScriptName, "");
var i,j,k;
var strXlsFile = strZipFile;
var strZipFile = strXlsFile.replace( ".xlsx",".zip").replace( ".XLSX",".zip");
fs.CopyFile (strXlsFile,strZipFile, true);
var objSource = objShell.NameSpace(pwd+strZipFile).Items();
var objTarget = objShell.NameSpace(pwd+outFolder);
for (i=0;i<objSource.Count;i++)
if (objSource.item(i).Name == "xl"){
if (fs.FolderExists("xl")) fs.DeleteFolder("xl");
objTarget.CopyHere(objSource.item(i), 256);
}
var xml = new ActiveXObject("Msxml2.DOMDocument.6.0");
xml.load("xl\\sharedStrings.xml");
var sel = xml.selectNodes("/*/*/*") ;
var vol = [];
for(i=0;i<sel.length;i++) vol.push(sel[i].text);
xml.load ("xl\\worksheets\\sheet1.xml");
ret = "";
var line = xml.selectNodes("/*/*/*");
var li, line2 = 0, line3=0, row;
for (li = 0; li< line.length; li++){
if (line[li].nodeName == "row")
for (row=0;row<line[li].childNodes.length;row++){
r = line[li].childNodes[row].selectSingleNode("#r").text;
line2 = eval(r.replace(r.substring(0,1),""));
if (line2 != line3) {
line3 = line2;
if (line3 != 0) {
//flush -------------------------- line3
flush();
for (i=0;i<col.length;i++) col[i]="";
}
}
try{
t = line[li].childNodes[row].selectSingleNode("#t").text;
//i = instr("ABCDEFGHIJKLMNOPQRSTUVWXYZ", left(r,1))
i = ("ABCDEFGHIJKLMNOPQRSTUVWXYZ").indexOf(r.charAt(0));
while (i > col.length) col.push("");
if (t == "s"){
t = eval(line[li].childNodes[row].firstChild.text)
col[i] = vol[t];
} else col[i] = line[li].childNodes[row].firstChild.text;
} catch(e) {};
}
flush();
}
if (fs.FolderExists("xl")) fs.DeleteFolder("xl");
if (fs.FileExists(strZipFile)) fs.DeleteFile(strZipFile);
}
import_xlsx(def);
I am working on a project that uses mxGraph where I am required to export a high resolution output in PDF for a service process diagram. I've tried recreating the graph using JGraphX, the Java Swing client and exporting that to a PDF, but the result is not close to what the browser displays.
There's no PDF export in JavaScript on the client, does mxGraph have any explicit support for PDF generation from JavaScript?
I'll explain the case of a client initiated request, where the diagram is displayed on the browser when the request is made. This is the standard case, mxGraph transmits an XML representation of the graph using custom graphics primitives and these are received on the server and decoded either by the Java or .NET back-ends.
The reason for the need for the graph being displayed is there are certain text measurements that are hard to recreate outside of a browser environment.
On the client side you need to create the required immediate XML using, say, the diagrameditor.html example as a guide:
var exportImage = function(editor)
{
var graph = editor.graph;
var scale = graph.view.scale;
var bounds = graph.getGraphBounds();
// New image export
var xmlDoc = mxUtils.createXmlDocument();
var root = xmlDoc.createElement('output');
xmlDoc.appendChild(root);
// Renders graph. Offset will be multiplied with state's scale when painting state.
var xmlCanvas = new mxXmlCanvas2D(root);
xmlCanvas.translate(Math.floor(1 / scale - bounds.x), Math.floor(1 / scale - bounds.y));
xmlCanvas.scale(scale);
var imgExport = new mxImageExport();
imgExport.drawState(graph.getView().getState(graph.model.root), xmlCanvas);
// Puts request data together
var w = Math.ceil(bounds.width * scale + 2);
var h = Math.ceil(bounds.height * scale + 2);
var xml = mxUtils.getXml(root);
// Requests image if request is valid
if (w > 0 && h > 0)
{
var name = 'export.png';
var format = 'png';
var bg = '&bg=#FFFFFF';
new mxXmlRequest(editor.urlImage, 'filename=' + name + '&format=' + format +
bg + '&w=' + w + '&h=' + h + '&xml=' + encodeURIComponent(xml)).
simulate(document, '_blank');
}
};
Where editor.urlImage is the URL of the image generating servlet, in the case for a Java back-end.
On the server-side, in the case of Java, look at the java/examples/com/mxgraph/examples/web/ExportServlet.java. That looks at the "format" parameter passed up, and if 'pdf', the writePdf() method is invoked.
That method creates an PdfWriter and renders the graphics primitives to a Java Swing Graphics2D using the Java favoured part of mxGraph.
This example writes the PDF result directly to the outstream of the servlet repsonse in this line:
PdfWriter writer = PdfWriter.getInstance(document, response.getOutputStream());
You could map the output to any stream.
Note that you need to setup iText to map every font you need in the PDF. This isn't always ideal for a large number of fonts. It's worth testing a few cases of the export to see if the output is good enough for your requirements. We're currently researching the use of PhantomJS for the export. If the Java export isn't good enough, please post another question regarding using PhantomJS and I'll detail the process for that.
iText is provided as an example PDF library to use, it's easier since it's under an open source library. It's possibly not the best suited library, we didn't find it easy to work with for this specific scenario. You might also want to investigate other Java PDF generation libraries.
Also note that the .NET back-end only supports raster image generation in dotnet/aspnet/Export.ashx, there's no known open source PDF library to supply as an example there.
Full vector solution:
Set mxClient.NO_FO = true;
Export SVG to PDF by svg2pdf.js
Write all DIV's text to pdf by jsPDF
Example:
let pdf = new jsPDF('p', 'pt', 'a4', false, false);
mxClient.NO_FO = true;
let graph = Draw(drawdiv, false);
let svgEl = drawdiv.children[1];
//draw svg:
svg2pdf(svgEl, pdf, {
xOffset: pdfPageDefaultOffsetX,
yOffset: pdfOffsetY,
scale: divToPdfRatio
});
//draw text:
for (let child of drawdiv.children) {
if (child.tagName === 'DIV') {
let splitText = pdf.splitTextToSize(child.innerText, Math.ceil((childSizes.width) * divToPdfRatio));
pdf.text(pdfPageDefaultOffsetX + (child.offsetLeft * divToPdfRatio), textPositionTop, splitText, {
align: child.style.textAlign,
lineHeightFactor: 1,
});
}
}
pdf.save('Test.pdf');