We Keep Coding

How can a web page find what value to POST?

I'm currently making an application for a client to automatically fill some web forms on the website he uses to store his item pricing. The website doesn't have a documented public API, and there doesn't seem to be a way to add bulk pricing on the website itself. In order to accomplish this, I'm making a simple python application that reads his data, then sends POSTs to the website.
Their website is giving me a hard time, however, because it's sending payloads containing dozens of fields, while the form used to enter the pricing information only has 4 input fields. On top of that, their website uses angularjs to generate most of the web page, so I can't just find the <form>[...]</form> block and look at what's being sent, because that's not what they use.
Here is what the payload json looks like:
{
"entities":
[
{
"Price_Line_ID":"{}",
"Price_List_ID":"{}",
"Item_ID":"{}",
"Uofm_ID":"{}",
"Amount":"{}",
"Dtstamp":"{}",
"Tenant_ID":"{}",
"Created_On":"null",
"Created_By":"null",
"Changed_On":"null",
"Changed_By":"null",
"Seq":"0",
"Begin_Qty":"0",
"End_Qty":"0",
"Customer_ID":"null",
"Tax_Before_Discount":"false",
"Discount_Target":"All",
"Max_Discount_Amount":"null",
"Min_Discount_Amount":"null",
"Customer_Name":"null",
"Uofm":"null",
"Item_Number":"null",
"Uofm_Schedule_ID":"null",
"Uofm_Schedule":"null",
"Inactive":"false",
"entityAspect":
{
"entityTypeName":"PriceLine:#SalesPad.Spo.Api.Model",
"defaultResourceName":"PriceLines",
"entityState":"Added",
"originalValuesMap":
{},
"autoGeneratedKey":
{
"propertyName":"Price_Line_ID",
"autoGeneratedKeyType":"Identity"
}
}
}
],
"saveOptions":
{}
}
The 7 values at the top (with values of "{}") are found when I do a GET or POST action on the website's other pages. I've managed to find where all the values originate from, except the "Price_Line_ID" one, because it appears to change from page to page (and it changes after a price is added).
I know a web page can get the data needed for a POST event either in its own html (when using tags like <form>), and it can get them from other GET and POST events. Is there any other way for a web page to determine a value that will be sent in a POST event?
I'm not very familiar with angularjs, although from what I understand it only creates a bunch of Javascript for the page. Does it offer different ways of determining what values are sent in a POST or GET event?
Edit: I've already tracked all responses from GET and POST events from logging-in to adding a price, the Price_Line_ID field changes from page to page, and adding the price appears to use an ID different to the one received in the GET event. I just want to know the different ways that a web page (specifically one using angularjs might use to determine the value of the data sent in POST events.

Javascript control page and user view page

Currently I'm working on a project where a user enters a lot of data constantly for a hour long window. I'm looking to have one user control all the data via some control panel and then have a link they can distribute to other users that will allow them to view that data without the ability to edit it.
Right now I'm doing some extremely weird methods. I have an XHR request on the control page that fires whenever a field is finished being edited. From there the data is sent to a php file that converts the data into a simple text file. Then the distributed link file will load that file one time and translate it into the necessary format.
Some potential problems I've run into are it seems odd that I'm sending starting as javascript data then going to a php file then to a text file then translating the data all the way back into javascript data again. Another problem I've come into is I'm not sure of a way to force users to reload the page when a field is edited in the control panel after the user has opened the view page.
Have I totally gone overboard here? What are some better concepts I could employ to accomplish this task?

If i understand what you want to do this is how i will do this:
First the data entry
if you have lot of fields you better use a form wizard, i don't have a particular one in mind right now but there is lot of them just search jQuery Form wizard
Here is an example:
http://i.stack.imgur.com/Luk2b.jpg
The concept of the form wizard is to guide user via multiple page and also validate the data. And click save when and the end.
Then save date in database.
Display content
All you need to do is to create a global separate page to display your content.
Let see something like: http://yourserver.com/view/{id}
where id is the identifier of the particular row in your database.

i'm not sure if i totally understand what u about to do. i'm trying to make your work description shorter here:
want to build a website that one person can edit a single page's content in 1 hour, and others can view the content change in that 1 hour.
if this is what u want to build, here's the module:
teacher: the one who can edit the page
student: the one who can only view the page
server: information center
teacher client edits page -> teacher client sends update data to server -> server saves data -> server sends update notice to student client -> student client receives update notice -> student fetches update data from server
to make this module work well, i suggest try socket instead of http reqeust, just like online games or IMs do.
well, try socket.io

Passing a variable from one page to the next page

I am developing a Question & Answer website where a user is presented five puzzles on a page and the score is calculated using JavaScript as he attempts the questions on the page. The score is saved in a Javascript variable score. I have a paging system like this:
Now when the user clicks on 3 I want to send the variable score to the next page, where I have require scoreUpdateInDatabase.php on each such page such that the score of previous page is made permanent to the databse and the following PHP script presents him the next 5 questions.
How can I pass that score variable in secure way? I can't use GET because the user will modify it. I am satisfied with POST if this can be used, even though POST data can be modified but I just want minimal security.
P.S. Please do not suggest making AJAX call where in one side I will send score and while returning carries next 5 questions. I don't want to use AJAX to refresh content because it is not SEO friendly.

The simplest solution would be cookie based. Writing the value to a session cookie and the reading it.
You could use jquery cookie. It also gives you the option to require https if desired.

Save it in a session. POST would work equally well in this particular case but my preference would be storing it in the session.

The only secure way to do this is to pass the actual answers to the server using a POST or AJAX, do the calculation of the score also on server side and keep it in a SESSION variable.
More information on sessions in PHP

Try looking into Jquery - You should be able to return the value to the server scripting language (as you listed PHP as a tag, I assume you're using PHP). By using Jquery, you can get the javascript variable to the form BEFORE submitting the form to the next page.
Assuming you have used PHP to generate the form to submit initially rather than create the form in javascript. I would use Jquery - to get this file ( http://jquery.com/ ) and to include("jquery.js"); etc... in your PHP script for it to be used.
I would then convert the javascript variable(s) to a php variable and assign this to a hidden field in the form to be submitted to the next page using a $_POST[] variable.

However It will not be SEO friendly (POST and SESSION is not SEO friendly, but you should use them, continue reading)
We are talking of a game. No-one want that the Search engine index the last page of a game... because everyone can search on google (for example) for the last page of your game without playing.
You have to use ajax or post, but don't let google index every page of your game. It's nonsense.
Only the first page of your game should be indexed.

Dynamic response with tracking pixels?

I am testing out some tracking pixel functionality in an ASP.Net 4 MVC architecture.
This article gives a nice way of setting a tracking pixel (image) that you can use to read a visitor's environment parameters and do some logging on the server side before completing the response.
What I would like to do is inject some Javascript, based on the account ID that the pixel came from. In the example above, the ID would be set by setting some query string parameters.
By the looks of that code, it can only be used to log data, as the response type is of type image.
Is it possible to accomplish this using the method shown above? If not, can I get some recommendations/sources on how to accomplish this using Javascript and tying this back into my .Net architecture where based on some logic, I can add some additional Javascript to the response?
If I have no other choice to go the JS route, I'm guessing it would be something along the lines of the Google Analytics tracking script that includes some parameters sent back through JS.
Thanks.

If the client is requesting an image and expecting an image, then that is what you need to return. Look at this type of HTML that would generate an image request:
<img src="test.jpg">
Clearing the client is expecting image bits to come back and anything besides that is going to mess up the display of that image.
If you want to put server-supplied javascript into the page, then simply have the client request some javascript like this:
<script src="test.js"></script>
Your server can then do it's logging upon that request and return whatever javascript it wants to from that request. If you want to return different javascript for every request, then you will need to defeat caching in the browser (there are a number of was to do that) so that the javascript is always requested from the server.
In general, I'm guessing that you don't need to return different javascript for every request. But rather, you can put a common block of javascript in the client page and that javascript can examine the environment and branch based upon what it finds. That's how Google Analytics works. One common piece of javascript is served to the client, that code examines the environment and then makes an ajax request with different parameters set that causes the right information to be recorded on the server.

How far can I go with JavaScript?

I need to do as much as possible on the client side. In more details, I would like to use JavaScript to code an interface (which displays information to the user and which accepts and processes response from the user). I would like to use the web serve just to take a date file from there and then to send a modified data file back. In this respect I would like to know if the following is possible in JavaScript:
Can JavaScript read content of a external web page? In other words, on my local machine I run JavaScript which reads content of a given web page.
Can JavaScript process values filled in a HTML form? In other words, I use HTML and JavaScript to generate an HTML form. User is supposed to fill in the form and press a "Submit" button. Then data should be sent to the original HTML file (not to a web server). Then this data should be processed by JavaScript.
In the very end JavaScript will generate a local data-file and I want to send this file to a PHP web server. Can I do it with JavaScript?
Can I initiate an execution of a local program from JavaScript. To be more specific, the local program is written in Python.
I will appreciate any comments and answers.

It could technically, but can't in reality due to the same origin policy. This applies to both reading and writing external content. The best you can do is load an iframe with a different domain's page in it - but you can't access it programmatically. You can work around this in IE, see Andy E's answer.
Yes for the first part, mmmm not really for the second part - you can submit a form to a HTML page and read GET arguments using Javascript, but it's very limited (recommended maximum size of data around 1024 bytes). You should probably have all the intelligence on one page.
You can generate a file locally for the user to download using Downloadify. Generating a file and uploading it to a server won't be possible without user interaction. Generating data and sending it to a server as POST data should be possible, though.
This is very, very difficult. Due to security restrictions, in most browsers, it's mostly not possible without installing an extension or similar. Your best bet might be Internet Explorer's proprietary scripting languages (WScript, VBScript) in conjuction with the "security zones" model but I doubt whether the execution of local files is possible even there nowadays.

Using Internet Explorer with a local file, you can do some of what you're trying to do:
It's true that pages are limited by the same origin policy (see Pekka's link). But this can be worked around in IE using the WinHttpRequest COM interface.
As Pekka mentioned, the best you can manage is GET requests (using window.location.search). POST request variables are completely unobtainable.
You can use the COM interface for FileSystemObject to read & write local text files.
You can use the WScript.Shell interface's Exec method to execute a local program.
So just about everything you asked is attainable, if you're willing to use Internet Explorer. The COM interfaces will require explicit permission to run (a la the yellow alert bar that appears). You could also look at creating a Windows Desktop Gadget (Vista or Win 7) or a HTML Application (HTA) to achieve your goal.
Failing all that, turn your computer into a real server using XAMPP and write your pages in PHP.

see i got what you want to do
best things is do following
choose a javascript library (eg:jquery,dojo,yui etc), i use jquery.this will decrease some of your load
inspite of saving forms data in in a local file, store them in local variables process them and send them to server (for further processing like adding/updating database etc) using XMLHttp request, and when webservice returns data process that data and update dom.
i am showing you a sample
--this is dom
Name:<input type='text' id='name' />
<a href='javascript:void(0)' onClick='submit()'>Submit Form</a>
<br>
<div id='target'></div>
--this is js
function submit()
{
var _name=$('#name').val();// collect text box's data
//now validate it or do any thing you want
callWebservice(_name,_suc,_err);
//above call service fn has to be created by you where you send this data
//this function automatically do xmlHttprequest etc for you
//you have to create it ur self
}
//call this fn when data is sucessfully returned from server
function _suc(data)
{
//webservice has returned data sucessefully
//data= data from server, may be in this case= "Hello user Name"; (name = filled in input box);
//update this data in target div(manipulate dom with new data);
$('#target').html(data);
}
function _err()
{
//call this fn when error occurs on server
}
// in reality most of the work is done using json. i have shown u the basic idea of how to use js to manipulate dom and call servcies and do rest things. this way we avoid page-reloads and new data is visible to viewer

I would answer saying there's a lot you can do, but then in the comment to the OP, you say "I would like to program a group game."
And so, my answer becomes only do on the client side what you are able and willing to double check on the server side. Never Trust the Client!
And I do not want to do my job twice.
If you are going to do things on the client side, you will have to do it twice, or else be subject to rampant cheating.

We had the same question when we started our project.In the end we moved everything we could on the JS side. Here's our stack:
The backend receives and send JSON data exclusively.We use Erlang, but Python would be the same. It handles the authentication/security and the storage.
The frontend, is in HTML+CSS for visual elements and JS for the logic.A JS template engine converts the JSON into HTML. We've built PURE, but there are plenty of others available. MVC can be an overkill on the browser side, but IMO using a template engine is the least separation you can do.
The response time is amazing. Once the page and the JS/CSS are loaded(fresh or from the cache), only the data cross the network for each request.