We Keep Coding

Best way to structure fetching data that has duplicates without spamming an API?

This is a methodological question about the best way to structure fetching and displaying a list of data with duplicates.
On a browse/display page - it calls an API which returns 25 items at a time - the item structure looks like this:
{
title: "some text", //can be duplicates
id: always_unique_value, //never duplicates
subtitle: "some text" //can be duplicates
}
The issue is that there can be thousands of duplicate titles. Something like:
{
id: 1,
title: "the same"
}
{
id: 2,
title: "the same"
}
{
id: 3,
title: "the same"
}
Since we can only fetch 25 items at a time - I want to group duplicates together when displaying them, but herein lies the problem.
When we group 10 items that are duplicates, the page will only have 15 unique items being displayed. So we need to fetch more items from the API. But if there are 100 duplicates next in the API, we would have to repeatedly call the API 5 times, to get the remaining 10 unique items.
Meaning you can run into a situation where to display 25 items, you might be calling the API 100s of times over and over, to fetch a unique set of 25 items.
So my question is - without changing the API database, how can we design this system so that its displaying 25 items on the page, without having to call this API potentially hundreds of times to get those unique items?
The API can be called with the following params:
from - where to start, so 0 starts at the beginning, and 25 would be page 2.
search values - can search for values but this won't help in displaying items
size - can request 100, 2000 items from the API - but we only need to display 25 per page.
I don't want to repeatedly call the API as it could be stressed and potentially enter a warning/failed state when multiple people are querying it 100 times every second

comparing two tables with mysql

I tried to search for an answer to this but I'm not sure if there are any answers to this question because I'm not quite sure how to word it correctly... Here I have two tables
Recipe ingredient table
Recipe_id|ingredient_id
ifqvv |1
ifqvv |2
User ingredient table
User_id|ingredient_id
1 |1
1 |2
2 |1
3 |3
I need to compare these table to where if both recipe and user ingredients_id are a complete match it will return a 1, or if the ingredients are greater than 0 it'll return a 2, and 3 for no matches. For Example, a query for User 1 will return a 1, User 2 a 2, and User 3 a 3. I'm not sure if this is something I'll have to code but I was told by someone that this is possible with little information, which led me here

Assuming you also have a user table, you could achieve this by cross-joining the user with the recipe_ingredients table and then left-joining the user_ingredients table, e.g.:
SELECT u.user_id, ri.recipe_id,
COUNT(ui.ingredient_id) AS available_ingredients, -- Number of ingredients the user has that are required to cook this recipe
COUNT(ri.ingredient_id) AS required_ingredients, -- Number of ingredients that are required to cook this recipe
CASE
WHEN COUNT(ui.ingredient_id) = COUNT(ri.ingredient_id) THEN 'can_cook'
WHEN COUNT(ui.ingredient_id) > 0 THEN 'has_some_ingredients'
ELSE 'has_no_ingredients'
END AS state
FROM users u
CROSS JOIN recipe_ingredients ri
LEFT JOIN user_ingredients ui ON(ri.ingredient_id = ui.ingredient_id AND u.user_id = ui.user_id)
GROUP BY u.user_id, ri.recipe_id
ORDER BY u.user_id, ri.recipe_id
If you want to limit it to a certain user / recipe, just use a where clause:
WHERE u.user_id = 1 AND ri.recipe_id = 'ifqvv'
You can try it live here: DB Fiddle
If you don't have an users table then you can replace
FROM users u with FROM (SELECT DISTINCT user_id FROM user_ingredients) u
DB Fiddle

Firebase data: denormilization and duplication

I have a hard time getting my data right for Firebase and need some guidance. I'm confident around relational database and find that I'm just replicating that way of thinking to firebase.
I'm creating a fitness app. The structure I have is as follows: Exercises, Workouts and Programs.
Exercises is quite simple, it's a name, description and a photo of a simple movement, like squats or benchpressing.
exercise
--e1
--name:"Squats"
--description: "..."
--e2
--name:"Benchpress"
--description: "..."
Then I have a list of workouts. A workout is a set of exercises performed in a special order with specific reps and sets. One exercise could be used in many workouts with different parameters (reps, sets, rest)
workouts
--w1
--name:"Easy workout"
--description: "Design for beginners"
--exercises:
--we1:
--exercise: e1
--reps: 12
--rest: 60
--order: 1
--we2
--exercise: e2
--reps:6
--rest: 30
--order: 2
--w2
--name: "Hard exercise"
...
So here I have w1, a workout that just uses exercise 1: First you perform 12 reps of exercise 1, then rest for 60 seconds, then 6 reps of exercise 2 and rest for 30 seconds.
Finally I have a program/plan. A program consist of multiple workouts that you can follow. For example your program could consist of 3 workouts that you should on monday, wednesday and friday.
programs
--p1
--name: "The Stack Overflow Fitness Plan"
--description: "I am a fancy text"
--workouts:
--w1: true
--w2: true
So as stated above I'm just putting up relational data up in firebase. Which isn't the correct way. I just have trouble how I should maange this data in flat way. Are duplicates for example ok?
At the moment I have a really expensive read(not best practice) since I query the program location, map it, query the workouts, map it,query the exercises. It becomes quite cumbersome fast.
Any thoughs and help is greatly appreciated.

You could take a look into the normalizr library. It was originally designed to help in working with rest API and redux : the same problem of duplication in a non relationnal environnement occurs.
Basically what I would do is have each different kind of entity stored as a Map in firebase.
const tree = {
exercises: { 1: { name: ... }, ... },
workouts: { 1: ... },
programs: { 1: ... },
}
Then you can have each of your components make subscriptions to their corresponding entity. With promises and/or streams it's pretty straightfoward.
Each of your relationships are simply an id in an Array and you can use the normalizr library to normalize / denormalize when you want to read / write in the db.
Note that with this pattern you can easily use the firebase update api to update several entities at once using a query such as :
const updateQuery = {
"workouts/id": ...,
"exercises/id": ...,
"programs/id": ...,
}

AngularJS - Sorting ng-repeat on string with numbers in them

I have a list of objects in a table-view i would like to sort properly.
Amongst other things, my objects contain a name-field. This name field can also contain numbers, so for example:
Chairman
Seat 1
Seat 2
Seat 3
Seat 11
Seat 12
Seat 23
Secretary
This is however sorted like this:
Chairman
Seat 1
Seat 11
Seat 12
Seat 2
Seat 23
Seat 3
Secretary
This doesn't seem like a natural way of sorting my list when sorting by name.
Now i'm using the ng-repeat like this:
seat in seats | orderBy:orderField:orderReverse track by seat.id
Where orderfield is some variable that is set when clicking the table header and orderReverse is too for reversing.
I've tried making a custom filter to makes sure it behaves properly but i failed. It seems that Javascript just won't order it normally unless i break up the string. But i'd rather not because the data is often updated by polling. Another way would be to force leading zero's but since users are entering this stuff manually i'm not sure if i should.
Also, its not only names with numbers in them, so i can't just completely cut them off
So, any suggestions on how to make this list show normally?
Edit: cleared up some info about the problem.

You can use a custom sort function with orderBy (it can take custom sorting function too)
define a sort function in your controller:
$scope.sorter = function (a){
return parseInt(a.replace( /^\D+/g, '')); // gets number from a string
}
and then in your template
seat in seats | orderBy:sorter track by seat.id
Edit as per the modified problem statement:
Do manual sorting in controller instead of with ng-repeart using naturalSort
$scope.seats = [{"id": "Seat 12"},{"id": "Seat 3"},{"id": "Seat 1"},{"id": "Seat 2"},{"id": "Secretary"}];
$scope.seats.sort(function(a,b) {
return naturalSort(a.id, b.id);
})
or check this angular module http://blog.overzealous.com/post/55829457993/natural-sorting-within-angular-js and the fiddle demonstrating it - http://jsfiddle.net/wE7H2/3/

Create table with nested rows for each parent row

Table 1
orderid customerName totalCost
----------------------------------
1 Jonh £200.00
2 Ringo £50
Table 2
orderlineid orderid productName productPrice Quantity
-------------------------------------------------------
1 1 Product1 £150 1
2 1 Product2 £50 1
3 2 Product3 £50 1
Table 3
orderid customerName totalCost
---------------------------------------
1 John £200
---------------------------------------
+ 1 1 Product1 £150 1
+ 2 1 Product2 £50 1
---------------------------------------
2 Ringo £50
---------------------------------------
+ 3 2 Product3 £50
Is it possible (given tables 1 and 2) to create an HTML table similar to table 3? Where for each order underneath there are the orders corresponding order lines information
Thanks

It's not possible to make a single table with real* nested relationships in SQL. Each SQL table is nothing more than a big two dimensional grid, no exceptions.
The correct way to store this type of relationship is in multiple tables, as you already have done.
You haven't said why you are trying to do this, but it sounds like it is a requirement for display rather than for storage. If so, the answer is to have whatever is producing the display query multiple tables as appropriate to get the information you need. This might be a single SQL query with a join, or it might be multiple queries. That depends on having more information about what you are doing.
*Sometimes nested relationships can be modeled within the rigid structure of a single table, but that isn't appropriate in your case.