requests库

模拟请求获取网页源代码

如下

import  requests

r = requests.get('https://spa1.scrape.center/page/1')
print(r.text)

 源码如下

 GET请求

示例网站为 https://httpbin.org

import requests
r = requests.get('https://httpbin.org/')
print(r.text)

所显示的结果如下

{
  "args": {}, 
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.32.3",
    "X-Amzn-Trace-Id": "Root=1-681dec13-1f8d5f796c04c6d73f09a6b3"
  },
  "origin": "183.250.93.64",
  "url": "https://httpbin.org/get"
}

如果想添加两个参数，其中name是germey，age是25

URL可以写成  https://httpbin.org/get？/name=germey&age=25

import requests
date = {
    'name': 'germey',
    'age': 25
}
r = requests.get('https://httpbin.org/get', params=date)
print(r.text)

结果如下

{
  "args": {
    "age": "25",
    "name": "germey"
  },
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.32.3",
    "X-Amzn-Trace-Id": "Root=1-681ded56-2771b27b0f18cd9b71461ee7"
  },
  "origin": "183.250.93.64",
  "url": "https://httpbin.org/get?name=germey&age=25"
}

如果想要直接解析返回结果得到一个JSON格式的数据的话，可以直接调用json方法

import requests

r = requests.get('https://httpbin.org/get')
print(type(r.text))
print(r.json())
print(type(r.json()))

结果如下

<class 'str'>
{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.32.3', 'X-Amzn-Trace-Id': 'Root=1-681dee2d-03cd762c0490679f0ef17d91'}, 'origin': '183.250.93.64', 'url': 'https://httpbin.org/get'}
<class 'dict'>

或取网页

import requests
import re

r = requests.get('https://ssr1.scrape.center/')
pattern = re.compile('<h2.*?>(.*?)</h2>', re.S)
titles = re.findall(pattern, r.text)
print(titles)