Message queuing application scenario, right?

now there are scheduled tasks in the project to regularly scan qualified records in the database, perhaps thousands at a time, and then loop these thousands of data into rabbitmq. When consumption is again consumed, the source third-party API will be called, and finally the call log will be recorded in the log table. Excuse me, is it necessary to remove the message queue and directly call the third-party interface, and then the database records logs, which has a great impact on the database? It should be a connection for the database, right?

May.14,2022

it's not necessary if you don't worry about dealing with halfway through the collapse. If it is required that once the first round is started, it must be all processed before the next round can be started, then it is still necessary to use mq, which can simplify many operations


whether it is necessary, you can consider: 1. Why do you want to add it? What is the cost? two If not, what's the problem? Is there any other way to solve it?

on the face of it, you are worried about the impact on the database if you do not use it, which basically has no effect. Because you are traversing serial calls to third parties. It's the same whether you use the queue or not.

if you are worried about data loss, you can also write thousands of selected items to text, then traverse the call, record the result, and fail to retry (to take this example, just to show that your goal may have multiple solutions, just choose what you think is right, for example, when you build a message queue, you should also consider persistence and usability, etc., the solution is complex, if simple can be done, try to use simple)

Menu