- Python Web Scraping Cookbook
- Michael Heydt
- 122字
- 2021-06-30 18:43:57
There's more...
Requests and urllib3 are very similar in terms of capabilities. it is generally recommended to use Requests when it comes to making HTTP requests. The following code example illustrates a few advanced features:
import requests
# builds on top of urllib3's connection pooling
# session reuses the same TCP connection if
# requests are made to the same host
# see https://en.wikipedia.org/wiki/HTTP_persistent_connection for details
session = requests.Session()
# You may pass in custom cookie
r = session.get('http://httpbin.org/get', cookies={'my-cookie': 'browser'})
print(r.text)
# '{"cookies": {"my-cookie": "test cookie"}}'
# Streaming is another nifty feature
# From http://docs.python-requests.org/en/master/user/advanced/#streaming-requests
# copyright belongs to reques.org
r = requests.get('http://httpbin.org/stream/20', stream=True)
for line in r.iter_lines():
# filter out keep-alive new lines
if line:
decoded_line = line.decode('utf-8')
print(json.loads(decoded_line))
推薦閱讀
- CorelDRAW X6 中文版圖形設計實戰從入門到精通
- Building Django 2.0 Web Applications
- 重新定義Spring Cloud實戰
- Oracle SOA Suite 11g Performance Tuning Cookbook
- 計算機網絡與數據通信
- Go Web Scraping Quick Start Guide
- 網絡安全應急響應技術實戰
- 語音信號處理及Blackfin DSP實現
- 5G非正交多址接入技術:理論、算法與實現
- Laravel Application Development Cookbook
- Intelligent Mobile Projects with TensorFlow
- 現場綜合化網絡運營與維護:運營商數字化轉型技術與實踐
- 新媒體交互藝術
- 從實踐中學習Kali Linux網絡掃描
- Learning IoT with Particle Photon and Electron