We Keep Coding

REST API Best practice ? return Empty object vs no object

let's say i have an API that returns users wallet balance and List of wallet transactions which is going to expiry
response = {
user_balance: 20
expiring_credits : [
{object 1 }
{object 2}
]
}
in case if user dont have any expiring transactions we can format respose in 2 ways
option 1
response = {
user_balance: 20
}
option 2
response = {
user_balance: 20
expiring_credits : []
}
which is the ideal option or best practices? and why? looking for some expert insights. many thanks.

Your returned data should reflect the shape of the data being asked for. If they're asking for a user's information, and that includes a user balance as one-to-one and expiring credits as a one-to-many, you should include those relationships. If there is nothing to be included, specifically call it out with a null if it's a one to one to let the dev know "there is a lack of data here" and if it is a one to many, return an empty array, as there were no dependant records but there was a master record.
An example endpoint:
/user/[id]/credits
GET
id: the user's id
{
user_balance: null | number, // o:o
expiring_credits: credits[] // o:m
}
This way the data shape is always the same for the consuming developer and they would not have to worry about top-level keys no existing on the returned object. It'll always be there, and it will always be consistent with the type returned.
If it exists, it will be this type. If it would be an array, it will always be an array. This lets people code to the data shape, not to the possibility of the data shape.

which is the ideal option or best practices?
REST doesn't care.
What you have here is a question about schema design, and specifically whether or not your expiring credits field should be optional or mandatory.
For instance, OpenApi uses optional parameters by default; your specification must explicitly "opt-in" to using required parameters (the "required" field is optional). This pattern holds for objects in your schema, just as it does for parameters in your URI (the "required" field is optional).
The choice between optional vs mandatory can impact you later if you discover that you need to modify the schema, and want to do so in a way that doesn't break existing clients. The XML community explored this question back in the day, so you will want to look into their conclusions (and in particular policies like must-ignore and must-forward).

I can't think of any good reason to omit a field just because it's empty. You might be able to save a few bytes in the response, but that is a very weak argument.
On the other hand: omission is not the same as absence. API's can return partial responses:
Partial response allows you to give application developers just the information they need.
source: Web API Design: The Missing Link
With partial responses:
api.example.com/user/1234 returns all fields by default:
response = {
user_balance: 20,
expiring_credits : [
{object 1 },
{object 2}
]
}
api.example.com/user/1234?fields=user_balance only sends user_balance, omitting expiring credits even if expiring credits exist:
response = {
user_balance: 20
}

It would be always a good practice to make json response structure intact so that client does not need to understand whether is the attribute is empty as attribute itself missed our due to some authorization or can it is because of no data is present
If attribute is present and not data is present array it will be give more clarity to calling user that removing attribute itself.

The better practice is to send an empty array. The reason is when someone calls your API and expect expiring_credits field to be present in the response, if you did not send it because it is empty, they can assume that they sent a bad request because the empty array is a valid value.

Note: Anything that I'm saying here is based on my own experience on web application and api development.
In my experience, always sending a static structure to the front-end or every other apis or web applications or wherever is better than not sending them at all.
I mean if you are working on a project and you have to send your data to the next implemented department, you have to have a standard for your responses and that means that you promise if for example you send some data with 200 response code from /me/ url, you surely have to promise that you always send ["username", "email"] fields. (even when they are null or empty strings)
This makes the other department(that can be anything) to always trust the responses from your api.
response = {
user_balance: 20
expiring_credits : []
}
So this is better.

response = {
user_balance: 20
expiring_credits : []
}
Option 2 is the best practice because of the following reasons
You don't have to write extra code to handle the undefined situation. This will simply render an empty grid.
Less prone to bugs.
Now the consumer is confirmed that ok the API is providing the correct response. It's just that there are no expiring_credits as of now.
While using libraries like Typescript, you don't have to mention an optional field.

How to get table data as rows and columns from wikipedia api?

When I tried to get table data as json, I could find distinguishable children in json output of the following query:
https://en.wikipedia.org/w/api.php?action=parse&page=List_of_football_clubs_in_India&prop=wikitext&section=3&format=json
I want to get the rows and columns of this table (the text) :-
https://en.wikipedia.org/wiki/List_of_football_clubs_in_India#Assam
The JSON output seems complicated and I don't find a good way to extract text from it.
(I am doing this in Javascript (Node.js)
Please help..

I'm not sure, what you expect. Your API request to the page is actually returning the wikitext encapsulated into a JSON structure. However, the wikitext (where the table is part of) is not JSON, so you can not really interpret it as such.
I'm also not quite sure, what information you want to have. If you want to have the football clubs in the table, then your only bet is to parse the wikitext (you can also return the actual parsed HTML from the API to make it "easier") and go through the data yourself. However, this is probably an error prone and not fun task.
So, if you want to get all football clubs of india in a structured data format, I would probably better try Wikidata for that. It allows you to crunch structured data for the information you need (and also get you the links to Wikipedia articles, if the objects has a link to a Wikipedia page). In your use case, it's probably a good idea to try out the Wikidata Query service.
There you could issue a query like:
SELECT ?itemLabel ?sitelink WHERE {
?item wdt:P31 wd:Q476028;
wdt:P17 wd:Q668.
?sitelink schema:isPartOf <https://en.wikipedia.org/>;
schema:about ?item.
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
which queries a list of all football clubs in India and returns you a list with the item label as well as the link to the english Wikipedia article:
https://query.wikidata.org/#SELECT%20%3FitemLabel%20%3Fsitelink%20WHERE%20%7B%0A%20%20%3Fitem%20wdt%3AP31%20wd%3AQ476028%3B%0A%20%20%20%20%20%20%20%20wdt%3AP17%20wd%3AQ668.%0A%20%20%3Fsitelink%20schema%3AisPartOf%20%3Chttps%3A%2F%2Fen.wikipedia.org%2F%3E%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20schema%3Aabout%20%3Fitem.%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%22%5BAUTO_LANGUAGE%5D%2Cen%22.%20%7D%0A%7D

Assume that this is the res is the data that you get from the wiki
//This will get you the innermost part of the object which is the text you want
let wikiText = res.parse.wikitext['*'];
//This will strip out all the numbers and non-alphabet charater.
let pureText=wikitext.replace(/[^a-zA-Z\s]+/g, ' ');
The above code can give you clean access to the text; however, how you are going to separate the column and row is up to you.

This will slow down performance a bit (It seems, but I'm not sure if any other faster way exists).
This can be done by setting prop=text and then parsing the obtained HTML using JSDOM (comes with/for Node.js)

I know this question is old but there is an API for this. You can supply a page title and it will return the tables of your choice in JSON.

Why and when do we need to flatten JSON objects?

I am surprised that no one on StackOverflow asked this question before.
Looking through the JSON object documentation and a quick google search did not yield satisfactory results.
What's the advantage of it? How does it work?
Edit: To make it clear, take a look at this flatten/un-flatten example.
Fastest way to flatten / un-flatten nested JSON objects
Thank you.

There are many situations where you get JSON text that was automatically built by some library. Throughout the programming languages, there are many libraries that build JSON text (one example is here).
Whenever libraries add some additional object or array wrappings, you might want to get rid of them maybe because you send the JSON to the server and your code there crashes because it expects a primitive value instead of an object (or an array). Or, if your JSON is a server response, you don't want the resulting Javascript code having to differ between object/array or not object/array. In all these cases, flattening is helpful as it will save you time. You will have to implement lesser if/elses, and you can reliably expect your data structure to be as flat as possible.
The other approach to improve code for the scenario mentioned is to write the code in a maximal robust way so there is no way for it to crash by superfluous wrappings ever. So always expect some wrappers and get it's contents. Then, flattening is not needed.
You see, it depends on what is building the JSON and what is parsing it. The building may be out of your scope.
This leads also to data model questions. I've worked with XML code that needed to be parsed quiet a different way if there where 0 entries of some XY, or if there were >0 entries of some XY. Having a wrapper that is allowed to have 0 or more entries of some XY will make live easier. These are data model desicions.
In all cases where the JSON represents an object structure that I've combined manually, I expect it not to change. So flattening something I've designed in detail would be disturbing. Standard operations as far I've seen them do not need flattening (e.g. JSON.stringify(), json_encode() etc.)

Here's a simple scenario: In a web app you have an HTTP POST that is updating a complex relational object.
POST
update=1
&user.id=12345
&user.email=testmail#domain.tld
&user.profile.name=Mr. Test
&user.profile.age=42
&user.profile.friend.0.email=tom#domain.tld
&user.profile.friend.1.email=sally#domain.tld
&user.profile.friend.2.email=bob#domain.tld
&user.profile.skill.0.id=100
&user.profile.skill.0.name=javascript
&user.profile.skill.1.id=200
&user.profile.skill.1.name=piano
Everything is already in a flat structure, so why not have a simple one-to-one binding? If you had a list of constraints or security requirements that you needed to enforce you could validate them by searching directly on the sorted key list.
Flat structures are easier for people to understand and work with there's even some cross-over with database de-normalisation. It also allows for context specific security and constraints to be implemented in a readable, but more verbose way.
When showing a user's view in full you may want to hide the display of the primary key ID for the user's list of skills.
"user.profile.skill.#.id": { hidden: true, readonly: true }
But when looking directly at a skill (to possibly edit it as an administrator) you may want to see the ID.
"skill.id": { readonly: true }
If you were writing a user-centric/self-service type CMS application you'd get more users on board and able to contribute using a straightforward flat model (flat abstraction of the underlying nested relational model) than you would with just the nested model.
TLDR: Flat is easier to read than nested. While programmers can handle nested schemas, recursive parsing and processing; end-users and admins usually prefer that part abstracted away.

I realize this is a 5 year old question at this point, but I figured, I'd add my thoughts to it, in case someone runs into a similar use case and finds this useful.
One of the use cases why you would want to flatten a JSON object, is for dynamic template binding via Regular Expression (RegEx) string interpolation. Well wasn't that a mouthful 👀😅? It simply translates to "template filling a string without hardcoding".
Ok Imagine a scenario, you have a template string like this for an email:
Hello {{firstName}},
It is amazing you chose to join our site. We are happy to have you on board.
To get started, we would really love it if you can confirm your email address
by clicking on the link: {{confirm_url}}.
Welcome aboard
The Team!
Given the following JSON object in memory:
{
"user" : {
"prefix" : "Dr.",
"firstName" : "Awah",
"lastName" : "Teh",
"email" : "awah#superduperubercoolsite.com",
"address" : {
"street": "100 Main St",
"city" : "PleasantVille",
"state" : "NY",
"phone" : "+1-212-555-1212"
}
},
"meta" : {
"confirm_url" : "http://superduperubercoolsite.com/confirm/ABC123"
}
}
it seems super simple to do a Regular Expression replace like so (assuming our email template string was stored in a variable named template and the json object was stored in a variable called templateData:
template = template.replace(new RegExp('{{firstName}}', 'g'), templateData.user.firstName);
template = template.replace(new RegExp('{{confirm_url}}', 'g'), templateData.meta.confirm_url);
Easy right? --> Actually yes! How about this email had 10 templated fields, or you wanted to decouple the template from the code, by storing it in a separate system like SendGrid, where your cool head of marketing can access the template and make changes to the copy-language, without having to call someone from engineering to make changes to the code, test the code and redeploy to production (what a hassle).
This is exactly where flattening of the JSON comes save the day!
Now there are many ways to flatten JSON, I have attached a link to a codepen I wrote that has logic to flatten JSON (actually, I demonstrate two similar but different approaches in the methods flattenJSONIntoKVP and flattenJSONIntoRAW check 'em out!).
That said, there are other implementations out there, and it is worth remembering that the focus on this post is to discuss the WHY JSON flattening could be useful, not the HOW.
Moving on! Assume you flattened the JSON from above (using my implementation that results in key value pairs) to something like this:
[
{ "key": "user.prefix", "value": "Dr."},
{ "key": "user.firstName", "value": "Awah"},
{ "key": "user.lastName", "value": "Teh"},
{ "key": "user.email", "value": "awah#superduperubercoolsite.com"},
{ "key": "user.address.street", "value": "100 Main St"},
{ "key": "user.address.city", "value": "{PleasantVille"},
{ "key": "user.address.state", "value": "NY"},
{ "key": "user.address.phone", "value": "+1-212-555-1212"},
{ "key": "meta.confirm_url", "value": "http://superduperubercoolsite.com/confirm/ABC123"},
]
Now, my friend, you are cooking with GAS!
Why, cause now you can dynamically interpolate the template string with values from the JSON object without giving too much worry to the structure of the JSON (if it changes due to the application evolving, you don't have to also remember to come down here and change this interpolation code -- you simply just have to update the email template itself, which mind you, is on SendGrid [per this example]).
So how to do it you say?: Simple, iteratively. Let's assume that flattened from above was stored in a variable called flatJSON:
///Notice how I use Javascripts native string interpolation to create my RegExp
///Also note that I am replacing the dot (.) in my flattened JSON variable names with a double underscore (__), I only do this because my intended target is SendGrid, and I don't believe it likes dots in its template placeholders.
flatJSON.forEach(kvp=>template = template.replace(new RegExp(`{{${kvp.key.replace(/\./g, '__'}}}`, 'g'), kvp.value));
That's it, one line of code to replace possibly 10 or even hundreds or even thousands (ok.. maybe not thousands, but you get the point).
Ohh! almost forgot, we need to update our template string.
Notice how now, in our new templated string we can use a somewhat FQDN style variable to map back to our original JSON (Ideally if SendGrid supported dots in their template placeholders, this would look super sweet but alas, can't always win everything!😭.
Hello {{user__firstName}},
It is amazing you chose to join our site. We are happy to have you on board.
To get started, we would really love it if you can confirm your email address
by clicking on the link: {{meta__confirm_url}}.
Welcome aboard {{user__prefix}} {{user__lastName}}!
The Team!
Et Voila!
Just like that, we have accomplished some good here today; we have:
Answered the WHY of flattening JSON objects
We dibble-dabbled into the how, with the codepen example
And we even overviewed a use case where taking advantage of JSON flattening can help you write durable dynamic code, that evolves as your underlying object structures change -- and that doesn't require you to leverage the big bad ugly eval method (we can talk about big bad ugly eval on another post).

Better design for data stored using HTML5 localStorage

I have a scenario on my web application and I would like suggestions on how I could better design it.
I have to steps on my application: Collection and Analysis.
When there is a collection happening, the user needs to keep informed that this collection is going on, and the same with the analysis. The system also shows the 10 last collection and analysis performed by the user.
When the user is interacting with the system, the collections and analysis in progress (and, therefore, the last collections/analysis) keep changing very frequently. So, after considering different ways of storing these informations in order to display them properly, as they are so dynamic, I chose to use HTML5's localStorage, and I am doing everything with JavaScript.
Here is how they are stored:
Collection in Progress: (set by a function called addItem that receives ITEMNAME)
Key: c_ITEMNAME_Storage
Value: c_ITEMNAME
Collection Finished or Error: (set by a function called editItem that also receives ITEMNAME and changes the value of the corresponding key)
Key: c_ITEMNAME_Storage
Value: c_Finished_ITEMNAME or c_Error_ITEMNAME
Collection in the 10 last Collections (set by a function called addItemLastCollections that receives ITEMNAME and prepares the key with the current date and time)
Key: ORDERNUMBER_c_ITEMNAME_DATE_TIME
Value: c_ITEMNAME
Note: The order number is from 0 to 9, and when each collection finishes, it receives the number 0. At the same time, the number 9 is deleted when the addItemLastCollections function is called.
For the analysis is pretty much the same, the only thing that changes is that the "c" becomes an "a".
Anyway, I guess you understood the idea, but if anything is unclear, let me know.
What I want is opinions and suggestions of other approaches, as I am considering this inefficient and impractical, even though it is working fine. I want something easily maintained. I think that sticking with localStorage is probably the best, but not this way. I am not very familiar with the use of Design Patterns in JavaScript, although I use some of them very frequently in Java. If anyone can give me a hand with that, it would be good.
EDIT:
It is a bit hard even for me to explain exactly why I feel it is inefficient. I guess the main reason is because for each case (Progress, Finished, Error, Last Collections) I have to call a method and modify the String (adding underline and more information), and for me to access any data (let's say, the name or the date) of each one of them I need to test to see which case is it and then keep using split( _ ). I know this is not very straightforward but I guess that this whole approach could be better designed. As I am working alone on this part of the software, I don't have anyone that I can discuss things with, so I thought here would be a good place to exchange ideas :)
Thanks in advance!

Not exactly sure what you are looking for. Generally I use localStorage just to store stringified versions of objects that fit my application. Rather than setting up all sorts of different keys for each variable within localStore, I just dump stringified versions of my object into one key in localStorage. That way the data is the same structure whether it comes from server as JSON or I pull it from local.
You can quickly save or retrieve deeply nested objects/arrays using JSON.stringify( object) and JSON.parse( 'string from store');
Example:
My App Object as sent from server as JSON( I realize this isn't proper quoted JSON)
var data={ foo: {bar:[1,2,3], baz:[4,5,6,7]},
foo2: {bar:[1,2,3], baz:[4,5,6,7]}
}
saveObjLocal( 'app_analysis', data);
function saveObjLocal( key, obj){
localStorage.set( key, JSON.stringify(obj)
}
function getlocalObj( key){
return JSON.parse( localStorage.get(key) );
}
var analysisObj= =getlocalObj('app_analysis');
alert( analysisObj.foo.bar[2])

javascript: array of object for simple localization

I need to implement a simple way to handle localization about weekdays' names, and I came up with the following structure:
var weekdaysLegend=new Array(
{'it-it':'Lunedì', 'en-us':'Monday'},
{'it-it':'Martedì', 'en-us':'Tuesday'},
{'it-it':'Mercoledì', 'en-us':'Wednesday'},
{'it-it':'Giovedì', 'en-us':'Thursday'},
{'it-it':'Venerdì', 'en-us':'Friday'},
{'it-it':'Sabato', 'en-us':'Saturday'},
{'it-it':'Domenica', 'en-us':'Sunday'}
);
I know I could implement something like an associative array (given the fact that I know that javascript does not provide associative arrays but objects with similar structure), but i need to iterate through the array using numeric indexes instead of labels.
So, I would like to handle this in a for cycle with particular values (like j-1 or indexes like that).
Is my structure correct? Provided a variable "lang" as one of the value between "it-it" or "en-us", I tried to print weekdaysLegend[j-1][lang] (or weekdaysLegend[j-1].lang, I think I tried everything!) but the results is [object Object]. Obviously I'm missing something..
Any idea?

The structure looks fine. You should be able to access values by:
weekdaysLegend[0]["en-us"]; // returns Monday
Of course this will also work for values in variables such as:
weekdaysLegend[i][lang];
for (var i = 0; i < weekdaysLegend.length; i++) {
alert(weekdaysLegend[i]["en-us"]);
}
This will alert the days of the week.
Sounds like you're doing everything correctly and the structure works for me as well.

Just a small note (I see the answer is already marked) as I am currently designing on a large application where I want to put locals into a javascript array.
Assumption: 1000 words x4 languages generates 'xx-xx' + the word itself...
Thats 1000 rows pr. language + the same 7 chars used for language alone = wasted bandwitdh...
the client/browser will have to PARSE THEM ALL before it can do any lookup in the arrays at all.
here is my approach:
Why not generate the javascript for one language at a time, if the user selects another language, just respond(send) the right javascript to the browser to include?
Either store a separate javascript with large array for each language OR use the language as parametre to the server-side script aka:
If the language file changes a lot or you need to minimize it per user/module, then its quite archivable with this approach as you can just add an extra parametre for "which part/module" to generate or a timestamp so the cache of the javascript file will work until changes occures.
if the dynamic approach is too performance heavy for the webserver, then publish/generate the files everytime there is a change/added a new locale - all you'll need is the "language linker" check in the top of the page, to check which language file to server the browser.
Conclusion
This approach will remove the overhead of a LOT of repeating "language" ID's if the locales list grows large.

You have to access an index from the array, and then a value by specifying a key from the object.
This works just fine for me: http://jsfiddle.net/98Sda/.
var day = 2;
var lang = 'en-us';
var weekdaysLegend = [
{'it-it':'Lunedì', 'en-us':'Monday'},
{'it-it':'Martedì', 'en-us':'Tuesday'},
{'it-it':'Mercoledì', 'en-us':'Wednesday'},
{'it-it':'Giovedì', 'en-us':'Thursday'},
{'it-it':'Venerdì', 'en-us':'Friday'},
{'it-it':'Sabato', 'en-us':'Saturday'},
{'it-it':'Domenica', 'en-us':'Sunday'}
];
alert(weekdaysLegend[day][lang]);