Results are POST now

[ShootingKing-AM]

This should not longer work since Gitam changed its form submitting to POST from GET, that too over HTTPS.

...
url = "http://doeresults.gitam.edu/onlineresults"
url += "/pages/NewReportviewer1.aspx?"
url += "&sem={sem}&reg={id}".format(sem=str(sem), id=str(id))
...

[Ethcelon]

Yeah! I'm aware that the system has changed. The future plan is to use a headless browser to do the scraping

[Ethcelon]

@ShootingKing-AM ^

[ShootingKing-AM]

@Ethcelon Headless browser is too much, cURL should do the trick. (node-curl or PycURL)

[Ethcelon]

Yes, I'm aware curl works. curl is amazing after you get a session token. I was wondering how to get a session token in the first place. I don't know if uploading my cookie is the right option.

[Ethcelon]

woot! Made some progress. On a post with text as "Select", we get an sql error as a reply. However, the server also sends us a nice little session ID, that can be used to make further requests ;) 👍 to us 👎 to gitam!

[ShootingKing-AM]

Hmm, Actually, the correct way is, after you POST sem and regid to /onlineresults/pages/Newgrdcrdinput1.aspx, you will get a Session ID in response, like,

HTTP/1.1 302 Found
Age: 0
Cache-Control: private
Connection: keep-alive
Content-Length: 165
Content-Type: text/html; charset=utf-8
Date: Thu, 15 Oct 2015 15:31:59 GMT
Location: /onlineresults/pages/NewReportviewer1.aspx
Server: ATS/5.3.0
Set-Cookie: ASP.NET_SessionId=wau0y0qfg5notrmkt41lpc45; path=/; HttpOnly
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET

You can use that for further communication. Let me upload my version of ResultsScanner :D

[ShootingKing-AM]

@Ethcelon, Completed removing crappy debugging lines and added my ResultsScanner. Its in PHP but cURL syntax remains same in any language :D