urllib requets selenium的应用场景
cookie识别用户身份和记录用户状态 driver.get_cookies() 获得cookie信息 add_cookie(cookie_dict) 向cookie添加会话信息 delete_cookie(name) 删除特定(部分)的cookie delete_all_cookies() 删除所有cookie ie 和chrome 需下载对应浏览器驱动,firefox不需要。 Phantomjs设置代理IP: from selenium import webdriver from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
dcap = dict(DesiredCapabilities.PHANTOMJS) dcap = { "phantomjs.page.settings.userAgent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 " \ "(KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36", "phantomjs.page.settings.loadImages": False, "phantomjs.page.settings.resourceTimeout": 5000 } proxy = [ '--proxy=120.27.142.209:82', '--proxy-type=http', '--ignore-ssl-errors=true', '--ssl-protocol=tlsv1' ] driver = webdriver.PhantomJS(desired_capabilities=dcap,service_args=proxy)
firefox设置IP代理 socks类型的代理:
profile = webdriver.FirefoxProfile() profile.set_preference('network.proxy.type', 1) #默认值0,就是直接连接;1就是手工配置代理。
profile.set_preference('network.proxy.socks', ip) profile.set_preference('network.proxy.socks_port', port) profile.set_preference('network.proxy.ssl',ip) profile.set_preference('network.proxy.ssl_port', port) profile.update_preferences() browser = webdriver.Firefox(profile) browser.get("http://www.ip138.com") 如果是http类型的代理,则将proxy.socks改为proxy.http