Golang Colly下载小红书详情页面图片小爬虫

发布于:2024-05-05 ⋅ 阅读:(35) ⋅ 点赞:(0)

语言:Golang

库:Iris/Colly

目前网上相关内容比较少,查了很久才找到解决方法。

期间曾尝试使用 selenium 解决,但需要搭建 selenium server,就没有再折腾了。

直到找到:https://juejin.cn/post/7230780828795584567

虽然不是Colly实现的,但起码提供了解决思路~~

抱着试一试的心态,没想到真成功了。

代码输出日志:

Received Url: https://www.xiaohongshu.com/explore/66069af9000000001a017909
text/html; charset=utf-8
0 http://sns-webpic-qc.xhscdn.com/202404281403/770fd190d3a75c303f0a90b8617e4e00/1040g008310u4ij1c6k4g5orbdnunrdaapen0kv0!nd_dft_wlteh_webp_3
It is image 20240428140308_1.jpg
Saved file: images\20240428140308_1.jpg
Completed 200
1 http://sns-webpic-qc.xhscdn.com/202404281403/e6f11d10d7c186b8ca96351375bad89c/1040g008310u4ij1c6k3g5orbdnunrdaavkr2dr0!nd_dft_wlteh_webp_3
It is image 20240428140308_2.jpg
Saved file: images\20240428140308_2.jpg
Completed 200
2 http://sns-webpic-qc.xhscdn.com/202404281403/4c15493ed3f2c378233828b8967e1667/1040g008310u4ij1c6k305orbdnunrdaaa6hn8s0!nd_dft_wlteh_webp_3
It is image 20240428140308_3.jpg
Saved file: images\20240428140308_3.jpg
Completed 200
3 http://sns-webpic-qc.xhscdn.com/202404281403/945ecc47c07553017edf8d76b458eda1/1040g008310u4ij1c6k205orbdnunrdaaka5kfe0!nd_dft_wlteh_webp_3
It is image 20240428140308_4.jpg
Saved file: images\20240428140308_4.jpg
Completed 200
4 http://sns-webp