Month: April 2013

urlopen error

dondonliuLeave a comment

Trying to download following url:

http://www.simplyhired.com/a/job-details/view/jobkey-15a23ff5ab31691f84e867e52367c392cd34adcc/cjp-52/hits-329401

<urlopen error [Errno 1] _ssl.c:504: error:140773E8:SSL routines:SSL23_GET_SERVER_HELLO:reason(1000)>
name error

Code I used:

an=0
while an&lt;len(agents):
agent=agents[an]
headers = { 'User-Agent' : agent }
req = urllib2.Request(url, headers=headers)
finalurl=url
try:
  fp = urllib2.urlopen(req,timeout=20)
  finalurl = fp.geturl()
  try:
   fp._sock.recv = None
  except Exception,ex: # in case it's not applicable, ignore this.
    print ex
except HTTPError, e:
  print e.getcode()
  an+=1
except URLError, e:
  print e
  an+=1
finally:
try:
  fp.close()
except NameError:
  print 'name error'

return finalurl