requests库

模拟请求获取网页源代码

如下

import  requestsr = requests.get('https://spa1.scrape.center/page/1')
print(r.text)

 源码如下

 GET请求

示例网站为 https://httpbin.org

import requests
r = requests.get('https://httpbin.org/')
print(r.text)

所显示的结果如下

{"args": {}, "headers": {"Accept": "*/*","Accept-Encoding": "gzip, deflate","Host": "httpbin.org","User-Agent": "python-requests/2.32.3","X-Amzn-Trace-Id": "Root=1-681dec13-1f8d5f796c04c6d73f09a6b3"},"origin": "183.250.93.64","url": "https://httpbin.org/get"
}

如果想添加两个参数，其中name是germey，age是25

URL可以写成  https://httpbin.org/get？/name=germey&age=25

import requests
date = {'name': 'germey','age': 25
}
r = requests.get('https://httpbin.org/get', params=date)
print(r.text)

结果如下

{"args": {"age": "25","name": "germey"},"headers": {"Accept": "*/*","Accept-Encoding": "gzip, deflate","Host": "httpbin.org","User-Agent": "python-requests/2.32.3","X-Amzn-Trace-Id": "Root=1-681ded56-2771b27b0f18cd9b71461ee7"},"origin": "183.250.93.64","url": "https://httpbin.org/get?name=germey&age=25"
}

如果想要直接解析返回结果得到一个JSON格式的数据的话，可以直接调用json方法

import requestsr = requests.get('https://httpbin.org/get')
print(type(r.text))
print(r.json())
print(type(r.json()))

结果如下

<class 'str'>
{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.32.3', 'X-Amzn-Trace-Id': 'Root=1-681dee2d-03cd762c0490679f0ef17d91'}, 'origin': '183.250.93.64', 'url': 'https://httpbin.org/get'}
<class 'dict'>

或取网页

import requests
import rer = requests.get('https://ssr1.scrape.center/')
pattern = re.compile('<h2.*?>(.*?)</h2>', re.S)
titles = re.findall(pattern, r.text)
print(titles)