I am currently trying to run a python file from a deno backend using the following code...
const cmd = Deno.run({
cmd: ["python", "python.py"],
stdout: "piped",
stderr: "piped"
});
const output = await cmd.output() // "piped" must be set
const outStr = new TextDecoder().decode(output);
const error = await cmd.stderrOutput();
const errorStr = new TextDecoder().decode(error);
cmd.close();
console.log(outStr, errorStr);
const resultsAlgorithm = outStr
console.log('This is a test, python result is...',outStr)
console.log('Finished')
The code works for basic scripts like 'print("Hello")' but is unable to run imports on more complex scripts such as...
import pandas as pd # Changes call up name to pd
from yahoofinancials import YahooFinancials
from datetime import date, datetime, timedelta,time
Yahoo_Forex = pd.DataFrame()
Currency_Pair_Prices = pd.DataFrame()
print('Running')
def DataExtract(FileName, DataFrameName, DataFrameName_2, time_range, Interval, ColumnName):
print('Function started')
start = date.today() - timedelta(days=2)
end = date.today() - timedelta(days=time_range)
DataFrameName = pd.read_excel(FileName, header=None)
DataFrameName.columns = ColumnName
n = 0
for ticker in DataFrameName[ColumnName[0]]:
Currency_Pair = DataFrameName.iloc[n, 1]
Currency_Pair_Ticker = YahooFinancials(ticker)
data = Currency_Pair_Ticker.get_historical_price_data(
str(end), str(start), Interval)
Extracted_Data = pd.DataFrame(data[ticker]["prices"])
Currency_Close_Price = (Extracted_Data["close"])
DataFrameName_2[str(Currency_Pair)] = Currency_Close_Price
n = n+1 # 13
print(DataFrameName_2)
print("DataExtract Completed")
DataExtract("yahoo_Forex.xlsx", Yahoo_Forex, Currency_Pair_Prices,int(10), "daily", ["Ticker", "Pair", "Exchange"])
The python code runs successfully on it's own so must be something with deno but sure what I would need to change so any help would be appreciated!
Related
I am looking for a way to read the feather files via GoLang or Javascript, or some other languages that does not require users to do some other extra installation.
My goal is to provide a User-interface to read a feather csv file and convert it back to a human-readable csv. However I can't find much resources on how to work it out.
Currently I have a test feather file generated by below.
import pandas as pd
import datetime
import numpy as np
import pyarrow.feather as feather
# Create a dummy dataframe
todays_date = datetime.datetime.now().date()
index = pd.date_range(todays_date-datetime.timedelta(10), periods=10, freq='D')
columns = ['A','B', 'C']
df = pd.DataFrame(index=index, columns=columns)
df = df.fillna(0) # with 0s rather than NaNs
feather.write_feather(df, 'test_feather.csv')
Thanks in advance.
The Javascript package apache-arrow comes with a script that does exactly this. You can find the source for the script here: https://github.com/apache/arrow/blob/master/js/bin/arrow2csv.js
If it is not doing exactly what you want the script should serve as an example of how to use the API to read in a feather file.
Thanks for the hints from #Pace. Turns out I found that I can simply use the arrow.Table.from([arrow]) function to convert .feather file to csv.
For those people encountered same issue, you may find the code below for reference.
const apArrow = require('apache-arrow');
const fs = require('fs');
const outputDir = 'output/feather';
const writeIntoFile = (data) => {
fs.appendFileSync(`${outputDir}/test_feather.csv`, data, function (err) {
if (err) return console.log(err);
});
};
const readDataFromRow = (fields, row) => {
return fields
.map((f) => row.get(f))
.join(',');
};
const arrowReader = (filePath) => {
console.log('filePath', filePath);
const arrow = fs.readFileSync(filePath);
const table = apArrow.Table.from([arrow]);
const columns = table.schema.fields.map((f) => f.name);
let buf = columns.join(',') + '\n';
for (let i = 0; i < table.count(); i++) {
const rowData = readDataFromRow(columns, table.get(i));
buf += `${rowData}\n`;
// export to csv every 10000 rows
if (i % 10000 === 0) {
writeIntoFile(buf);
buf = '';
if (i > 0) {
break;
}
}
}
writeIntoFile(buf);
};
I have a nodeJS application and I need to run a python script in order to get a certain response. I am using python-shell in order to do that, but I am getting no response.
I have tried also using a child-process, same response.
Here I call the python script:
var ps = require('python-shell');
ps.PythonShell.run('./face_detect.py', array1, function (err, data) {
if (err) req.send(err);
req.send(data.toString())
});
This is a snippet of my python script:
import cv2
import sys
import os
import numpy as np
students = sys.argv[1]
# get the names and put them in an array ---> subjects
imagePath = "class/welcome.jpg"
cascPath = "haarcascade_frontalface_alt.xml"
faceCascade = cv2.CascadeClassifier(cascPath)
.....
for (x, y, w, h) in faces:
num = 0
crop_img = cv2.UMat(image[y-40:y+h+100,x-40:x+h+40])
cv2.imwrite("face" + str(num) + ".jpg", crop_img)
test_img = cv2.imread("face" + str(num) + ".jpg")
num = num + 1
predicted_img1 = predict(test_img)
absences["A"] = 1
for name, a in absences.items():
if a == 0:
noshow.append(name)
print(noshow)
cv2.waitKey(0)
I expect it to return an array.
Can anyone help me with this?
The correct syntax for passing argument from Nodejs python-shell to Python script is:
ps.PythonShell.run('./face_detect.py', { args: array1 }, function (err, data) { ... })
Here the value of sys.argv[1] in your Python script will not contain the Nodejs array1 value because you don't set the args property in your PythonShell options.
Note also this should probably be res.send instead of req.send, depending on your program, and I advise you to return if there is an error to prevent "headers already sent" exception.
Let's say I have a TypeScript 1.6 file containing this one single line that contains JSX (TSX):
var a = (<div></div>);
When I compile this on the command line with TypeScript 1.6 (installed via the ntypescript npm package):
ntsc --jsx react test.tsx
It produces the expected output:
more test.js
var a = (React.createElement("div", null));
However, when I try to do the same thing with the TypeScript JS API, I can't get it to work. I run node and then run these commands:
var ts = require('ntypescript');
src = 'var a = (<div></div>);'
d = []; // for diagnostics
compilerOptions = {jsx: 'react', module: ts.ModuleKind.CommonJS};
res = ts.transpile(src, compilerOptions, 'test.tsx', d);
The result I get is:
'var a = (<div></div>);\n'
Also, d is an empty array, so no diagnostic messages (i.e errors) were reported. What gives?
It turns out that I had to pass in ts.JsxEmit.React (which resolves to the number 2) instead of the string 'react' and then everything works.
So this is the working code:
var ts = require('ntypescript');
src = 'var a = (<div></div>);'
d = []; // for diagnostics
compilerOptions = {jsx: ts.JsxEmit.React, module: ts.ModuleKind.CommonJS};
res = ts.transpile(src, compilerOptions, 'test.tsx', d);
Output:
var a = (React.createElement("div", null));
Enjoy!
I currently developping a Coffeescript with nodejs that would allows me to dump all images on a Tumblr.
In a first loop it execute a request that will get a list of all the images on each page via the API.
In a second loop, it download all the images.
the problem is that, I don't know why, the first callback is never executed, the "start" counter is never incremented.
Here is the code :
xml2json = require "xml2json"
fs = require "fs"
util = require "util"
request = require "request"
tumblr_name = process.argv[2]
api_endpoint = util.format "http://%s.tumblr.com/api/read", tumblr_name
start = 0
num = 50
post_count = 50
download = (uri, filename) ->
request(uri).pipe(fs.createWriteStream(filename))
while post_count > 0
console.log util.format "post_count: %s - num: %s", post_count, num
page_uri = util.format "%s?type=photo&start=%s&num=%s", api_endpoint, start, num
do (page_uri) ->
request page_uri, (error, response, body) ->
console.log util.format "Downloading %s", page_uri
data_xml = body
data_json = JSON.parse xml2json.toJson data_xml
post_count = data_json["tumblr"]["posts"]["post"].length
for post in data_json["tumblr"]["posts"]["post"]
post_id = post["id"]
post_date = post["date-gmt"].split(" ")[0]
for photo in post["photo-url"]
if photo["max-width"] == 1280
outname = util.format "%s-%s-%s.%s", tumblr_name, post_date, post_id, photo["$t"].split(".").slice(-1)[0]
download photo["$t"], outname
start = start + num
Trying to write a file inside windows temp directory using XUL code:
function writeFile_launch_application(utility_name,utility_path)
{
var data ='tasklist /nh /fi "imagename eq '+utility_name+'" | find /i "'+utility_name+'">nul && (echo alerady running) || ("'+utility_path+'")';
//alert(data);
var file = Cc["#mozilla.org/file/directory_service;1"].
getService(Ci.nsIProperties).
get("TmpD", Ci.nsIFile);
file.append("launch_application.bat");
file.createUnique(Ci.nsIFile.NORMAL_FILE_TYPE, 0666);
// Then, we need an output stream to our output file.
var ostream = Cc["#mozilla.org/network/file-output-stream;1"].
createInstance(Ci.nsIFileOutputStream);
ostream.init(file, -1, -1, 0);
// Finally, we need an input stream to take data from.
const TEST_DATA = data;
let istream = Cc["#mozilla.org/io/string-input-stream;1"].
createInstance(Ci.nsIStringInputStream);
istream.setData(TEST_DATA, TEST_DATA.length);
NetUtil.asyncCopy(istream, ostream, function(aResult) {
if (!Components.isSuccessCode(aResult)) {
// an error occurred!
}
})
}
But getting error:
Timestamp: 11/29/2012 11:03:09 PM
Error: ReferenceError: Cc is not defined
Source File: chrome://myaddon/content/overlay.js
Line: 199
I also tried to add below lines at the top of my code but it didn't solve above error:
Components.utils.import("resource://gre/modules/NetUtil.jsm");
Components.utils.import("resource://gre/modules/FileUtils.jsm");
Cc and Ci are aliases for Components.classes and Components.interfaces respectively.
Depending on the context they might (or might not) be already defined.
In any case
const Cc = Components.classes;
const Ci = Components.interfaces;
const Cu = Components.utils;
const Cr = Components.results;
(you shouldn't tag your question as firefox-addon-sdk)
When I'm developing my addon, I always try to replace Components.utils, Components.intefaces with Cu and Ci. To do that, first line in my file is:
const { Cc, Ci, Cu } = require('chrome');
Cc is Component.classes. Than you can use now
Cu.import("resource://gre/modules/FileUtils.jsm");
Cu.import("resource://gre/modules/NetUtil.jsm");