We Keep Coding

Patch custom object with kubernetes javascrtip SDK

I'm using External Secrets to sync my secrets from azure. And now I need a programmatic way to trigger the sync. With kubectl the command is
kubectl annotate es my-es force-sync=$(date +%s) --overwrite
So, I try to use k8s js sdk to do this. I can success fully get the External Secret
await crdApi.getNamespacedCustomObject("external-secrets.io", "v1beta1", "default", "externalsecrets", "my-es")
However, when I try to update it with patchNamespacedCustomObject, it always tells me "the body of the request was in an unknown format - accepted media types include: application/json-patch+json, application/merge-patch+json, application/apply-patch+yaml"
Here's my code
const kc = new k8s.KubeConfig();
kc.loadFromString(kubeConfig);
const crdApi = kc.makeApiClient(k8s.CustomObjectsApi);
let patch = [{
"op": "replace",
"path": "/metadata/annotations",
"value": {
"force-sync": "1663315075"
}
}];
await crdApi.patchNamespacedCustomObject("external-secrets.io", "v1beta1", "default", "externalsecrets", "my-es", patch);
I am referring their patch example here

const options = {
"headers": {
"Content-type": k8s.PatchUtils.PATCH_FORMAT_JSON_PATCH
}
};
is still required.

How to upload a file into a specific location in the Google Drive using the JS API?

I'm using googleapis to upload different files to the Google Drive. The scenario is:
User provides a document and it's information through my REST API (I'm using NodeJS).
The REST API creates the directory that will contain the document, if it's not already exist.
The REST API uploads the document to that directory.
The structure of the drive is:
/root/documents/$type/$new_document
where $type is one of the user's provided fields and the $new_document is the document that was provided by the user.
The way I connect:
oauth2Client.setCredentials({ refresh_token: REFRESH_TOKEN });
drive_instance = google.drive({
version: 'v3',
auth: oauth2Client,
});
I figured how to upload the document to root folder of the Google Drive:
}
try {
const response = await drive.files.create({
requestBody: {
name: file.name,
mimeType: file.mimetype,
},
media: {
mimeType: file.mimetype,
body: file.data,
},
});
console.log(response.data);
} catch (error) {
console.log(error.message);
}
What I'm struggling is:
How to create the directory /root/documents/$type if it's not already existing?
How to upload the $new_document to /root/documents/$type?
For the second question, I know that the docs provide an option of parents[] that will contain all the folder IDs. but then, how can I get the folder ID of /root/documents/$type? Is there someway to combine the steps (like maybe mkdir -p for the directories or creating the directory will return the ID of the directory).

1. Try found folder you need via drive.files.list() method
You need set filter. Example:
add 'q': "..." in requestBody to search what you need
use "name = 'Some Folder Name'" to search by name
use "mimeType = 'application/vnd.google-apps.folder'" to search only folders
Thus, combine via and:
'q': "name = 'Some Folder Name' and mimeType = 'application/vnd.google-apps.folder'" to find all folders with name you chose
const folderName = 'Some Folder Name'
gapi.client.drive.files
.list({
'q': "name = 'Some Folder Name' and mimeType = 'application/vnd.google-apps.folder'",
'pageSize': 1000,
'fields': "files(id, name, parents)"
})
.then((response) => {
const files = response.result.files;
if (files.length > 0) {
handleSearchResult(files)
} else {
createFolder()
}
console.log(total / (1024*1024*1024))
});
About handleSearchResult() or createFolder:
It maybe more than 1 file. So you can find necessary root getting files[i].parents . That's why I added parents in 'fields': "files(id, name, parents)". https://developers.google.com/drive/api/v3/reference/files
Also you can add searching rule e.g. 'parents contain "..."''
https://developers.google.com/drive/api/v3/search-files
If search brings 0 files result so just create folder by yourself. You can create path\directory step-by step. Create first folder in drive root and remember id. After that create second folder and add in requestBody parentId that equal first folder id. And etc... Btw you can use almost the same logic to search.
2. Create folder if its necessary
Example:
// name = 'Folder Name',
// parents = ['some-parent1-id', 'some-parent2-id', ...]
function createFolder(name, parents) {
const fileMetadata = {
'name' : name,
'mimeType' : 'application/vnd.google-apps.folder',
'parents': parents
};
gapi.client.drive.files
.create({
resource: fileMetadata,
}).then((response) => {
switch(response.status){
case 200:
const file = response.result;
console.log('Created Folder Id: ' + file.id);
break;
default:
console.log('Error creating the folder, '+response);
break;
}
});
}
3. Upload file with setted parents
you should add parents = ['id-of-folder'] in requestBody
Read more in Google Drive API - Files: create
I hope it will help at least a bit:) Keep it up!

You can upload a folder inside a folder using the follow method
You should have the id of the folder you want to store the new folder in (can be extracted using nodejs api or by opening the folder and looking at the characters after last / in the url)
Use a special mimetype reserved for folders in google drive ( application/vnd.google-apps.folder )
considering your example
drive.files.create({
requestBody:{
name:"SomeFolder",
mimeType:"application/vnd.google-apps.folder"
},(error,folder)=>{
console.log(folder.data.id);
drive.files.create:({
requestBody:{
name:"SomeFolderInsideAFolder",
mimeType:"application/vnd.google-apps.folder"
},
parents:[folder.data.id]
})
})
})
You can even easily create a recursively uploading folder function by combining file upload and folder upload which can upload a whole folder

How to share a file using Web Share Target API?

How can I share a file using Web Share Target API?
I managed to make my Progressive Web App appear when I try to share a file from the gallery app on my phone. My question is how can I take the file and send it to the server or fill a HTML input with type="file" with the shared file?
This is what I have right now
In manifest.json
"share_target": {
"action": "/qtransfer-p/send.php",
"method": "POST",
"enctype": "multipart/form-data",
"params": {
"title": "name",
"text": "description",
"url": "link",
"files": [
{
"name": "fileToUpload",
"accept": [".pdf", ".png", ".jpeg", ".doc", ".docx", ".pdf", ".xls", ".xlsx", ".txt", ".mp4", ".mp3", ".wav", ".rar", ".zip"]
}
]
}
}
In sw.js (service worker)
self.addEventListener('fetch', event => {
const url = new URL(event.request.clone().url);
// If this is an incoming POST request for the
// registered "action" URL, respond to it.
if (event.request.method === 'POST' &&
url.pathname === '/qtransfer-p/send.php') {
event.respondWith(Response.redirect('/qtransfer-p/send.php'));
event.waitUntil((async () => {
console.log("1");
const data = await event.request.clone().formData();
console.log("2");
console.log(data);
const client = await self.clients.get(event.resultingClientId || event.clientId);
console.log("3");
const file = data.get("fileToUpload");
console.log("file", file);
client.postMessage({ file, action: 'load-file' });
})());
console.log("ok");
return 0;
}
});
Right now, when I share a photo, I get in console:
1
ok
Uncaught (in promise) TypeError: Failed to fetch
So I think the problem is on the "const data = await event.request.clone().formData();".
Someone please help! I am trying to solve this for 2 weeks!

If you'd like to send a file directly to your web server, you can skip the service worker involvement.
Your current manifest.json setup will result in a POST request being sent to the /qtransfer-p/send.php at your origin, encoded as multipart/form-data, whenever someone shares a supported file to your installed PWA.
As long as you don't intercept that request in a service worker, I believe it will just be sent directly like any other HTTP POST, and could be processed server-side.

Error serving HTML files from an Azure function

I am trying to open, read and return an HTML files using Azure functions. I am developing locally and the logs says that the function executed successfully however on the browser I am getting 500 internal server error. Am I doing something wrong in here?
const fs = require('fs');
const path = require('path');
const mime = require('../node_modules/mime-types');
module.exports = function (context, req) {
const staticFilesFolder = 'www/build/';
const defaultPage = 'index.html';
getFile(context, req.query.file);
function getFile(context, file) {
const homeLocation = process.env["HOME"];
if(!file || file == null || file === undefined){
context.done(null,{status:200,body:"<h1>Define a file</h1>",headers:{
"Content-Type":" text/html; charset=utf-8"
}});
}
fs.readFile(path.resolve(path.join(homeLocation, staticFilesFolder, file)),
(err, htmlContent) => {
if (err) {
getFile(context, "404.html");
}
else {
const res = {
status: 200,
body: htmlContent,
headers:{
"Content-Type": mime.lookup(path.join(homeLocation, staticFilesFolder, file))
}
}
context.done(null,res);
}
})
}
};
Note
I am sure that 404.html exists and index.html exists. When I log the contents of htmlContent it is giving the correct output.
functions.json
{
"disabled": false,
"bindings": [
{
"authLevel": "anonymous",
"type": "httpTrigger",
"direction": "in",
"methods":["get"],
"route":"home",
"name": "req"
},
{
"type": "http",
"direction": "out",
"name": "res"
}
]
}
Response on Chrome
If I removed "Content-Length" header the status code changes to 406.
Update 1 The code seems to be running normally on Azure Portal but it is not working when running it locally.

It looks like you are combining two methods of returning data from an http triggered function(context.res and context.done()): https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-node#accessing-the-request-and-response
Since you are using context.res, try removing context.done();

You are making an incorrect use of context.res, you shouldn't be overwriting it but instead leveraging the methods provided by the Response class provided in the Azure NodeJS worker. If you are using using VSCode you'll get intellisense for these methods. Otherwise see: https://github.com/Azure/azure-functions-nodejs-worker/blob/dev/src/http/Response.ts
Your code should look something like this instead.
context.res.setHeader('content-type', 'text/html; charset=utf-8')
context.res.raw(htmlContent)
Using context.res.raw or context.res.send will already perform the context.done call for you.
Make sure you use content-type=text/html; charset-utf8 instead of content-type=text/html or you'll trigger an issue with the returned content-type. Instead of returning content-type=text/html you end up getting content-type=text/plain which will fail to render your html.
Addressed on: https://github.com/Azure/azure-webjobs-sdk-script/issues/2053

Using logstash and elasticseach

I'm actually using node-bunyan to manage log information through elasticsearch and logstash and I m facing a problem.
In fact, my log file has some informations, and fills great when I need it.
The problem is that elastic search doesn't find anything on
http://localhost:9200/logstash-*/
I have an empty object and so, I cant deliver my log to kibana.
Here's my logstash conf file :
input {
file {
type => "nextgen-app"
path => [ "F:\NextGen-dev\RestApi\app\logs\*.log" ]
codec => "json"
}
}
output {
elasticsearch {
host => "localhost"
protocol => "http"
}
}
And my js code :
log = bunyan.createLogger({
name: 'myapp',
streams: [
{
level: 'info',
path: './app/logs/nextgen-info-log.log'
},
{
level: 'error',
path: './app/logs/nextgen-error-log.log'
}
]
})
router.all('*', (req, res, next)=>
log.info(req.url)
log.info(req.method)
next()
)
NB : the logs are well written in the log files. The problem is between logstash and elasticsearch :-/
EDIT : querying http://localhost:9200/logstash-*/ gives me "{}" an empty JSON object
Thanks for advance

Here is how we managed to fix this and other problems with Logstash not processing files correctly on Windows:
Install the ruby-filewatch patch as explained here:
logstash + elasticsearch : reloads the same data
Properly configure the Logstash input plugin:
input {
file {
path => ["C:/Path/To/Logs/Directory/*.log"]
codec => json { }
sincedb_path => ["C:/Path/To/Config/Dir/sincedb"]
start_position => "beginning"
}
}
...
"sincedb" keeps track of your log files length, so it should have one line per log file; if not, then there's something else wrong.
Hope this helps.

Your output scope looks not complete. Here's the list of the output parameters http://logstash.net/docs/1.4.2/outputs/elasticsearch
Please, try:
input {
file {
type => "nextgen-app"
path => [ "F:\NextGen-dev\RestApi\app\logs\*.log" ]
codec => "json"
}
}
output {
elasticsearch {
host => "localhost"
port => 9200
protocol => "http"
index => "logstash-%{+YYYY.MM.dd}"
}
}
Alternatively, you can try the transport protocol:
output {
elasticsearch {
host => "localhost"
port => 9300
protocol => "transport"
index => "logstash-%{+YYYY.MM.dd}"
}
}
I also recommend using Kibana as a data viewer. You can download it at https://www.elastic.co/downloads/kibana