Use Javascript to Scrape table by selecting some other elements and clicking on button - javascript

I'm working with financial data and i want to scrape data from this site web using javascript and add the sript.js to my index.html file (https://www.sikafinance.com/marches/historiques?s=BRVMAG).
I want to scrape the data of a table from above site that takes four arguments
•Ticker who is BRVMAG in the minimal example
•dlPeriod
•datefrom
•dateto
And finally click on the button
btcChange ="OK".
After trying the code below with R I get the table but how can a achieve the same script with fetch?
I would like to be able to get the other tables when I change the start and end date using now javascript in Visual Studio Code.
Since yersterday i'm looking for a reponse searching every here but now result.
Does anyone have any idea how to recover the whole table and return table in of html?
The bottom images show what I noticed when i inspected their site.
I think either the whole table is available and it does a filter for the dates(gap between dates must not exceed 3 months)
library(httr)
library(rvest)
first_date<-as.Date("2022-02-01")
end_date <- as.Date("2022-03-29")
query_params <- list(dlPeriod = "Journalière",
datefrom = first_date,
dateto = end_date,
btnChange = "OK")
parameter_response <- GET("https://www.sikafinance.com/marches/historiques?s=BRVMAG", query_params)
parameter_response1<- httr::content(parameter_response, as = "text", encoding = "UTF-8")
parameter_response2 <- read_html(parameter_response1)%>%
html_node('#tblhistos')%>%
html_table()
parameter_response2
# Date Clôture `Plus bas` `Plus haut` Ouverture `Volume Titres` `Volume FCFA` `Variation %`
# <chr> <chr> <chr> <chr> <chr> <chr> <int> <chr>
# 1 29/04/2022 312,09 312,09 312,09 312,09 - 0 2,53%
# 2 28/04/2022 304,38 304,38 304,38 304,38 - 0 0,00%
# 3 27/04/2022 304,38 304,38 304,38 304,38 - 0 2,69%
# 4 26/04/2022 296,42 296,42 296,42 296,42 - 0 0,81%
# 5 25/04/2022 294,05 294,05 294,05 294,05 - 0 1,34%
# 6 22/04/2022 290,17 290,17 290,17 290,17 - 0 0,36%

Related

In plotly, is it possible to have an action where a user clicks and sees a different time range in the chart?

I'm creating a plotly python chart for a web app. Right now I am using the 'rangeselector' option to display different views to go back 7 days, 14 days, etc. However - what I actually want is a way to show the data for "this week", "last week", "the week before last week", etc. For the range selector, it always seems to start backward from today. Is there a way to do this in plotly? If not, is there a different way to go about this (i.e., a different charting library, some javascript)?
I'm new to web development, so thank you for the help.
rangeselector=dict(
buttons=list([
dict(count=7,
label="7d",
step="day",
stepmode="backward"),
dict(count=14,
label="14d",
step="day",
stepmode="backward"),
dict(count=1,
label="1m",
step="month",
stepmode="backward"),
dict(count=6,
label="6m",
step="month",
stepmode="backward"),
dict(count=1,
label="YTD",
step="year",
stepmode="todate"),
dict(step="all")
])
),
One workaround would be to add buttons that modify the xaxis range, which will also shift the rangeslider accordingly. We can do this by iterating backwards in 7 day increments from the end date to the start date with as many buttons as you like.
In the below example, we create buttons labeled ["All", "This Week", "Last Week", "The Week before Last Week"], and use some sample timeseries data. We need an "All" button to go back to the starting state of the figure.
import plotly.graph_objects as go
import pandas as pd
# Load data
df = pd.read_csv(
"https://raw.githubusercontent.com/plotly/datasets/master/finance-charts-apple.csv")
df.columns = [col.replace("AAPL.", "") for col in df.columns]
df['Date'] = pd.to_datetime(df['Date'])
df['Timedelta'] = (df['Date'].max() - df['Date']).dt.days
# Create figure
fig = go.Figure()
fig.add_trace(
go.Scatter(x=list(df.Date), y=list(df.High)))
# Set title
fig.update_layout(
title_text="Time series with range slider and selectors"
)
## ensure that the button date range includes the last data point
buttons = [
dict(
label="All",
method="relayout",
args=[
{
"xaxis.range": [df['Date'][0], df['Date'].iat[-1]]
}
],
)
]
button_labels = ["This Week", "Last Week", "The Week before Last Week"]
recent_date = df['Date'].iat[-1]
for i, label in enumerate(button_labels):
button_end_date = recent_date
button_start_date = recent_date - pd.Timedelta("7D")
if button_start_date > df['Date'][0]:
buttons += [
dict(
label=label,
method="relayout",
args=[
{
"xaxis.range": [button_start_date, button_end_date]
}
],
)
]
recent_date = recent_date - pd.Timedelta("7D")
# Add range slider
fig.update_layout(
xaxis=dict(
rangeselector=dict(
),
rangeslider=dict(
visible=True
),
type="date"
),
updatemenus=[
dict(
type="buttons",
direction="right",
x=0.7,
y=1.2,
showactive=True,
buttons=buttons
)
]
)
fig.show()
If you are willing to consider a plotly-dash based solution, you can create two buttons that modify the date range of the x-axis every time they are clicked. For example:
import plotly.graph_objects as go
from dash import Dash, html, Input, Output, dcc, ctx
import pandas as pd
app = Dash(__name__)
# Load data
df = pd.read_csv(
"https://raw.githubusercontent.com/plotly/datasets/master/finance-charts-apple.csv")
df.columns = [col.replace("AAPL.", "") for col in df.columns]
df['Date'] = pd.to_datetime(df['Date'])
df['Timedelta'] = (df['Date'].max() - df['Date']).dt.days
# Create figure
fig = go.Figure()
fig.add_trace(
go.Scatter(x=list(df.Date), y=list(df.High)))
date_range = [df['Date'].min(), df['Date'].max()]
# Set title and range slider
fig.update_layout(
title_text="Time series with range slider and selectors",
xaxis=dict(
rangeselector=dict(
),
rangeslider=dict(
visible=True
),
type="date"
)
)
app.layout = html.Div([
html.Button('Last Week', id='last-week', n_clicks=0),
html.Button('Next Week', id='next-week', n_clicks=0),
dcc.Graph(figure=fig, id="time-series-range-slider")
])
#app.callback(
Output('time-series-range-slider', 'figure'),
Input('last-week', 'n_clicks'),
Input('next-week', 'n_clicks'),
prevent_initial_call=True
)
def update_figure(last_week, next_week, date_range=date_range):
print(f"current date range: {date_range}")
if "last-week" == ctx.triggered_id:
date_range[0] -= pd.Timedelta("7D")
date_range[1] -= pd.Timedelta("7D")
if "next-week" == ctx.triggered_id:
date_range[0] += pd.Timedelta("7D")
date_range[1] += pd.Timedelta("7D")
print(f"new date range: {date_range}")
fig.update_layout(
xaxis_range=date_range
)
return fig
if __name__ == '__main__':
app.run_server(debug=True)

Select the opposite of the union data

I had product, vendor, vendor's available day and booking tables. Currently, my search function only can search through vendor available day, but not search through booking table. I used left joins between vendor and vendor available day and compare with the selected day, time and hour. The search result works. How do I search through booking table?
Product Table
id | vendor_id | title | description
Vendor Table
id | name
Vendor Available Table
id | vendor_id | d_day | d_start_hour | d_start_minute | d_end_hour | d_end_minute
Booking Table
id | vendor_id | d_date | d_day | d_start_hour | _start_minute
Below is how I select product based on the selected day, start_hour and start_minute:
SELECT * FROM product
LEFT JOIN ( SELECT * FROM vendor GROUP BY id ) vendor
ON vendor.id = product.vendor_id
LEFT JOIN ( SELECT * FROM vendor_available GROUP BY id) vendor_available
ON vendor_available.vendor_id = product.vendor_id
WHERE ( d_day = '4'
AND (CONCAT(d_start_hour, '.' , d_start_minute) + 0.0) <= '12.15'
AND (CONCAT(d_end_hour, '.', d_end_minute) + 0.0) >= '12.15')
4 is the selected day and 12.15 is the selected hour and selected minute on the client side.
When the user selects 12.15 which is 12.15pm, the query will check which product is available between the vendor available time.
How do I use opposite union with booking table, so that the product which already booked doesn't display to users?
When using LEFT JOIN tbl, it is normal to add this to the WHERE clause (not the ON clause)
AND tbl.id IS NULL
This says that you want to return results only when the is no row in tbl.

calculation to show image on website every 3rd week

I have a grails project where need to show on call team icon on home page. there are three images for three team. teams rotate every 3 week so for example team 1 logo will show first (1)and last week(4) of month and so one..
I have tried putting in db table for all 52 weeks and team no and putting it in session-
code in service-
sql.query("SELECT * FROM tbl_ImageForTeamUser where weekno=?",[wkno])
{ rs ->
while (rs.next()) {
def userinf = new UserInfo()
userinf.weekno= rs.getInt("weekno")
userinf.teamname= rs.getString("teamname")
userinf.teamno= rs.getInt("teamno")
items.add(userinf)
}
}
Saved in session-
session.weekno= items[0].teamno()
UI looks like-
<tr>
<g:if test="${session.weekno == "1" }">
<td class="team1" align="right" nowrap></br></br></td>
</g:if>
<g:if test="${session.weekno == "2"}">
<td class="team2" align="right" nowrap></br></br></td>
</g:if>
<g:if test="${session.weekno == "3" }">
<td class="team3" align="right" nowrap></br></br></td>
</g:if>
but somehow this is not looking a good solution. please suggest a better ,coding suggestion .dont want to use DB and handle it on UI side.
The cal.get(Calendar.WEEK_OF_MONTH) will give you 1 2 3 4 5 or the the week of the month it is.
You can use this to print the correct order without having to store it in a database table. because you only have 3 unique ordering of the pictures if I understand your question correctly.
Calendar cal = Calendar.getInstance()
def weekOfMonth = cal.get(Calendar.WEEK_OF_MONTH)
def teamImageCountShown = weekOfMonth % 3
if(teamImageCountShown==1){
//Display 1 2 3
}
if(teamImageCountShown==2){
//Display 2 3 1
}
if(teamImageCountShown==0){
//Display 3 1 2
}

Using Javascript to configure a googleVis event listener in Shiny

Basically, I have a gvisCalendar Chart from the googleVis package in a Shiny app, and I want to display a dataTable underneath the chart that corresponds to a selected box.
I can add an event listener by setting the gvis.listener.jscode argument to a variable that holds a string of javascript code. For example, using this code, I can pull up the wikipedia page for a selected calendar date:
output$dates_plot <- renderGvis({
gvisCalendar(calendar.ddply,
options = list(
colorAxis = "{
minValue: 0,
colors: ['E9967A', 'A52A2A']
}",
gvis.listener.jscode = jscode2 )
)
})
jscode2<- "window.open('http://en.wikipedia.org/wiki/'
+ data.getValue(chart.getSelection()[0].row,0)); "
Using this code, I ran my program, selected the "June 16, 2015" box, and a new tab came up on my browser for this website: https://en.wikipedia.org/wiki/Tue_Jun_16_2015_00:00:00_GMT-0400_(EDT)
I don't actually want to do anything with wikipedia, I was just using that as an example.
All I want to do is save the date of the selected calendar box as an R object so that I can then display a data table of data that corresponds to that date.
I have almost no experience with javascript.
Thank you!
You can use Shiny.onInputChange to send data back to the server.
Here is an example:
library(shiny)
library(googleVis)
server <- function(input, output) {
output$dates_plot <- renderGvis({
gvisCalendar(Cairo,
options = list(
colorAxis = "{
minValue: 0,
colors: ['E9967A', 'A52A2A']
}",
gvis.listener.jscode = "
var selected_date = data.getValue(chart.getSelection()[0].row,0);
var parsed_date = selected_date.getFullYear()+'-'+(selected_date.getMonth()+1)+'-'+selected_date.getDate();
Shiny.onInputChange('selected_date',parsed_date)")
)
})
output$date <- renderText({
input$selected_date
})
}
ui <- shinyUI(fluidPage(
htmlOutput("dates_plot"),
textOutput("date")
))
shinyApp(ui = ui, server = server)
In this example I parsed the date to YYYY/M/D, if you want to keep the javascript long date format you can also return selected_date.toString() instead of parsed_date.

transferring value from a google map to a rchart in shiny application

I have a shiny app in which I plot a map with plotGoogleMaps and a rChart, with values related
to some markers in the map, rendered via rPlot.
The app user can click on a marker in the map to show a tooltip.
I'd like that when he clicks on the marker, the pertinent value in the chart would be highlighted.
Anybody knows how to perform this task?
Thank you.
#jdharrison
The R-code can be like this:
suppressPackageStartupMessages(library(googleVis))
## Hurricane Andrew (1992) storm track with Google Maps
AndrewMap <- gvisMap(Andrew, "LatLong", "Tip", options = list(showTip = TRUE,
showLine = TRUE, enableScrollWheel = TRUE, mapType = "hybrid", useMapTypeControl = TRUE))
print(AndrewMap, "chart")
The example was taken from here. You can download the package here. In this example, library googleVis is used, but I think that the answer can be similar.
Shiny example
server.R
# Author: Denis Petrov
# Date: 2014-07-04
# Example is based on http://rpubs.com/gallery/googleVis
library (shiny)
suppressPackageStartupMessages (library (googleVis))
# Define server logic required to summarize and view the selected dataset
shinyServer ( function (input, output, session)
{
GetDataAndrew <- function ()
{
# Hurricane Andrew (1992) storm track with Google Maps
AndrewMap <- gvisMap (Andrew, "LatLong", "Tip",
options = list(showTip = TRUE,
showLine = TRUE,
enableScrollWheel = TRUE,
mapType = "hybrid",
useMapTypeControl = TRUE))
return (AndrewMap)
}
output$viewchart <- renderGvis({
GetDataAndrew ()
})
output$info <- renderPrint ({
cat ('Hurricane\n')
cat ('Pressure=937\n')
cat ('Speed=120')
})
}
)
ui.R
# Author: Denis Petrov
# Date: 2014-07-04
# Define UI for dataset viewer application
shinyUI(pageWithSidebar(
# Application title
headerPanel('Hurricane Andrew (1992)'),
# Sidebar with controls to provide a caption, select a dataset, and
# specify the number of observations to view. Note that changes made
# to the caption in the textInput control are updated in the output
# area immediately as you type
sidebarPanel(
p ('Hurricane Andrew (1992) storm track with Google Maps'),
p ('I would like the following output be changed based on
selected pin.'),
verbatimTextOutput ('info')
),
# Show the caption, a summary of the dataset and an HTML table with
# the requested number of observations
mainPanel(
htmlOutput ('viewchart')
)
))

Categories