There is a problem that we can't get the playback information continuously when using bilibili api to obtain the playback information.

api: http://api.bilibili.com/x/web.
there are already 70w aid, in the library every morning to get video playback updates by aid
, and then there is a sudden problem in the early hours of this morning. Every time we get 200,300 pieces of data, there will be an error, and then all the data will go wrong. For example:
error number: 403
user IP: 112.28.149.110
Front end server: cn-jswx-cmcc-w-02
processing server: shylf-ops-slb-09
request address: http://api.bilibili.com/x/web.

come back in 2 minutes, the error will be "solved", and after getting another 200-300, the problem will occur again.

is this a new anti-crawler mechanism today?
the error above is not an error from python, but an error from the browser page. That is, a set of json data should have appeared, but now it is "very sorry, the page you are looking for is missing"

does this have anything to do with my ip
is there a solution

Mar.02,2021

this is a very common anti-crawling strategy. If the same IP is accessed too frequently in a certain period of time, you can limit how long the request lasts
if you do not require high timeliness of the data, you can set the request interval, such as 0.5 second, 1 second
. If the requirement for timeliness is high, you can use proxy ip

.
Menu