I wrote a distributed crawler to keep him fetching data from the queue.

is how does while True keep fetching data from the queue all the time?

because other people don"t know when the data will be put into the queue, I have been fetching data from the queue

Jan.14,2022

you have said to use while True,. I don't know if your specific problem means the following

.
while True:

    -sharp 
    {{getData}}
    
    time.sleep(1) -sharp 1

I can't believe you go to V station to ask this question.
don't you know how to use scrapy-redis? Stuffing it directly into the redis will naturally deal with it

.
Menu