How to paginate XML responses using digital ocean spaces api? - javascript

I am having trouble making sense of the documentation for the DO Spaces API in my fetch request to obtain all records which exceed the max-keys value of 1000 in the response. https://docs.digitalocean.com/reference/api/spaces-api/
I'm hoping to fetch the content without having to use s3 for this task.
const spacesURL = 'https://xxx.sfo3.digitaloceanspaces.com'
const xml = await fetch(
spacesURL
).then(res => res.text())
const parser = new XMLParser()
const paths = parser
.parse(xml)
['ListBucketResult']['Contents'].filter(({ Key }: { Key: string }) =>
Boolean(Key)
)
.map(({ Key }: { Key: string }) => Key.trim())

Related

Fetch api in javascripts

So i am creating a javascript searchbar that uses a fetch command to fetch data from a server. This server is not my own and i just used it as a template. This script works fine.But what i want to do is replace the fetch api with my own
Demonstation of code
const userCardTemplate = document.querySelector("[data-user-template]")
const categoriesSearch = document.querySelector("[data-user-cards-container]")
const searchInput = document.querySelector("[data-search]")
let users = []
searchInput.addEventListener("input", e => {
const value = e.target.value.toLowerCase()
users.forEach(user => {
const isVisible =
user.name.toLowerCase().includes(value) ||
user.email.toLowerCase().includes(value)
user.element.classList.toggle("hide", !isVisible)
})
})
fetch("https://jsonplaceholder.typicode.com/users")
.then(res => res.json())
.then(data => {
users = data.map(user => {
const card = userCardTemplate.content.cloneNode(true).children[0]
const header = card.querySelector("[data-header]")
const body = card.querySelector("[data-body]")
header.textContent = user.name
body.textContent = user.email
categoriesSearch.append(card)
return { name: user.name, email: user.email, element: card}
})
})
I think ive made my own fetch api via github which follows.
https://gist.github.com/UllestReal/09ca3d968dda94535e8fc25b998a6ce5#file-gistfile1-txt
But when i replace the first template api with my own, it wont fetch the code. Can someone tell me what im doing wrong?
I just wrote everything in one go it was alot easier. Just look above for my entire problem
What you have there is just a json file. But whats missing is a server that would serve the file upon a request.
Go to the link you pasted
Click Raw button
Copy the url
const url = 'https://gist.githubusercontent.com/UllestReal/09ca3d968dda94535e8fc25b998a6ce5/raw/705e7b335cd24e382c15e851ea8888fbdc9cdae4/gistfile1.txt'
fetch(url).then(res => res.json()).then(console.log)

NextJS (v13) How do I dynamic request data inside of a server component

Been experimenting with some server components and nextjs (v13) and have encountered something that i'm not sure how to do. Let's say in theory I wanted to have something with this functionality (with the requests running serverside)
const ClientComponent = () => {
// this value cannot be hardcoded into the server component, nor can we change
// this value at any point in time during the clients components life (just using const for example)
// in reality it'd be set in some other format
const id = "some-id"
return <ServerComponent id={somethingId} />
}
const fetchData = async (id: string) => {
// writing this off top of head, not sure if correct syntax but some basic post request using the id
const res = await fetch("someUrl", { data: id });
const data = await res.json();
return { data };
}
const ServerComponent = async ({ id }: { id: string }) => {
if (!id) return null;
const { data } = await fetchData(id);
return (
<div>
{data.someValue}
</div>
);
}
How would I go about doing something of this nature? is this considered "improper" / "bad practice"? if so what would be a better way to go about doing this? Keep in mind that ClientComponent could be three-four nodes down (with each node being a client component)
Thanks :)

How to create internationalized blog with Strapi using Next.js getStaticPaths & getStaticProps

I have a problem figuring out how to localize my content with Next.js and Strapi.
my current file structure looks like so:
pages/
blog/
- [post].tsx
- portfolio.tsx
Inside portfolio I am fetching data for all posts created on Strapi, depending on which locale is currently set, like so:
export const getStaticProps = async ({ locale }) => {
const res = await fetch(
`https://strapi.com/api/articles?locale=${locale}&populate=*`
);
const data = await res.json();
return {
props: {
articles: data.data,
},
};
};
In [post] it looks like that:
export const getStaticPaths = async ({ locales, locale }) => {
const res = await fetch(
`https://strapi.com/api/articles?locale=${locale}&populate=*`
);
const data = await res.json()
const ids = data.data.map((post: any) => post.id);
const paths = ids
.map((id: any) =>
locales.map((locale: any) => ({
params: { post: id.toString() },
locale,
}))
)
.flat();
return {
paths,
fallback: false,
};
};
export const getStaticProps = async (context: any) => {
const id = context.params.post;
const res = await fetch(
`https://strapi.tincors.com/api/articles/${id}?populate=*`
);
const res_blocks = await fetch(
`https://strapi.tincors.com/api/articles/${id}?populate[blocks][populate]=*`
);
const data = await res.json();
const blocks_data = await res_blocks.json();
const block_data_slider = blocks_data.data.attributes.blocks[0].files.data;
return {
props: { article: data.data, slider: block_data_slider },
};
};
Note that each article on Strapi has different id for it's localized version - so as an example:
article_1_pl has id of 10
but it's english variant has id of 12.
fun fact - on portfolio.tsx data is fetched successfully, and post miniature cards are properly displaying based on the current domain (on dev I am using two different hosts for i18n - localhost:3000 for PL & example.com:3000 for EN).
However once I try to redirect myself to the full article, by clicking on the post miniature card, I get the 404 page not found error in the browser, with these errors poping each second in the console logs Error.
It doesn't matter which local host I am currently at.
cards are wrapped in a <Link href="/blog/${id}" locale={route.locale} />
However the moment I change my code in [post].tsx by removing the "locale" from the endpoint:
const res = await fetch(
`https://strapi.com/api/articles?locale=${locale}&populate=*`
);
to:
const res = await fetch(
`https://strapi.com/api/articles?populate=*`
);
suddenly each of my PL articles on localhost:3000 are displayed properly, and only the english variants aren't working (404 page not found).
I assume that it is due to the fact, that by removing the "locale" from the API endpoint it only fetches the polish articles, not the english ones, but it baffles me why it's not working at all, when I use the localized endpoint source.
How I want my app to work:
I want article description (which is generated as a dynamic route) to display in active language, by fetching localized data by the post id.
My question is:
How do I fetch the proper localized data from Strapi in [post].tsx
when using getStaticPaths & getStaticProps. What is wrong with above code and how do I fix it?
I apologize if the above description is too chaotic to understand - feel free to ask for more explanations if necessary :)

How to fetch all the documents with unique id from firestore database using React?

[Firestore SS][1]
[1]: https://i.stack.imgur.com/EI1Dm.png
I want to fetch each document as displayed in SS it's stored as Pets + unique_userId.
I am unable to fetch all data together. Just able to fetch one data of a particular user using the code below.
const [info,setInfo]=useState([]);
useEffect(() => {
db.collection("pets ESYXOPqlJpZ48np8LfNivnh9pvc2").onSnapshot((snapshot) =>
setInfo(snapshot.docs.map((doc) => doc.data()))
);
},[]);
Here ESYXOPqlJpZ48np8LfNivnh9pvc2 this is the userID of each unique user
Please help me out to fetch all the Pets data instead of hardcoding and fetching one particular data.
Try the following code,
const [docs, setDocs] = useState([]);
useEffect(() => {
const querySnapshot = await getDocs(collection(db,"pets ESYXOPqlJpZ48np8LfNivnh9pvc2"));
const document =[];
querySnapshot.forEach((doc) => {
document.push({
...doc.data(),
id: doc.id
});
});
setdocs(document);
}, []);
I'm guessing the appended id is a reference to the owner's id? In this case, would it be an option to fetch the owner list and use everyone's id to build a list of collection ids and then get all of their data?
If not, I only see to options:
Rethink your database structure - maybe use a unified pets collection and have a reference with/to that id in the pet documents.
Create a cloud function in which use #google-cloud/firestore to get the list of collections. There are tons of resources out there to help you get started with firebase cloud functions. Their documentation is pretty good also, and probably the most up-to-date
const functions = require('firebase-functions')
const { Firestore } = require('#google-cloud/firestore');
module.exports = functions
.region('europe-west3') // use the region you want here
.https.onRequest(async (request, response) => {
try {
const firestore = new Firestore();
const collections = (await firestore.listCollections()).map(collection => collection.id)
response.json({ data: collections })
} catch (error) {
response.status(500).send(error.message)
}
})
You'll get and endpoint which you can use to fetch the collection ids (e.g.: https://your-project-name.cloudfunctions.net/collections)
const [pets, setPets] = useState([]);
const [collectionIds, setCollectionIds] = useState([])
useEffect(() => {
fetch('https://your-project-name.cloudfunctions.net/collections')
.then(response => response.json())
.then(({ data }) => setCollectionIds(data))
}, [])
useEffect(() => {
collectionIds.forEach((collectionId) => {
// There are better ways to do this,
// I'm just using your approach so you can focus on the rest of the code
db.collection(collectionId).onSnapshot((snapshot) => {
setPets((currentPets) => [...currentPets, ...snapshot.docs.map((doc) => doc.data())])
})
})
}, [collectionIds])
Please note that these are very high-level implementations, there's no error handling, no teardowns or anything, so keep that in mind. Hope it helps, good luck!

Is it possible to read/decode .avro uInt8Array Container File with Javascript in Browser?

I'm trying to decode an .avro file loaded from a web server.
Since the string version of the uInt8Array starts with
"buffer from S3 Objavro.schema�{"type":"record","name":"Destination",..."
I assume it's avro Container File
I found 'avro.js' and 'avsc' as tools for working with the .avro format and javascript but reading the documentation it sound's like the decoding of a Container File is only possible in Node.js, not in the browser.
(The FileDecoder/Encoder methods are taking a path to a file as string, not an uInt8Array)
Do I get this wrong or is there an alternative way to decode an .avro Container File in the browser with javascript?
Luckily I found a way using avsc with broserify
avro.createBlobDecoder(blob, [{options}])
[avro.js - before browserifying]
var avro = require('avsc');
const AVRO = {
decodeBlob(blob) {
let schema, columnTitles, columnTypes, records = []
return new Promise((resolve) => {
avro.createBlobDecoder(blob, {
// noDecode: true
})
.on('metadata', (s) => {
schema = s
columnTitles = schema.fields.map(f => f.name)
columnTypes = schema.fields.map(f => f.type)
})
.on('data', (data) => {
records.push(data)
})
.on('finish', () => {
resolve(
{
columnTitles: columnTitles,
columnTypes: columnTypes,
records: records
}
)
})
})
}
}
module.exports = AVRO
[package.json]
"scripts": {
"avro": "browserify public/avro.js --s AVRO > public/build/avro.js"
}
[someOtherFile.js]
//s3.getObject => uInt8array
const blob = new Blob([uInt8array]) //arr in brackets !important
const avroDataObj = await AVRO.decodeBlob(blob)
Thanks for posting!
Below is my integration of avro/binary with axios, in case it helps anyone else trying to implement browser side decoding:
[before browserifying]
const axios = require('axios')
const avro = require('avsc')
const config = {
responseType: 'blob'
};
const url = 'https://some-url.com'
axios.get(url, config)
.then(res => {
avro.createBlobDecoder(res.data)
.on('metadata', (type) => console.log(type))
.on('data', (record) => console.log(record))
})
.catch(e => console.error(e))

Categories