Kafka and mysql operations, how to ensure the sequence

problem description

In the

A project, there is a method that first saves the data in mysql, and then sends the id of the data returned by the insert operation of mysql as the message body through kafka. (this method has been @ Transactional, so inserting data and sending messages should)

After receiving the message, the

B project queries the MySQL for the details of the message and performs subsequent actions.

when deployed online, mysql is the remote database server, while the kafka,A project and Project B are deployed on the same server. One problem is that project B has received the message sent by An and started to query the database according to the id in the message, but at this time, the insert operation of the database has not been completed, resulting in empty query results.

in this case, how to control the database insert operation and kafka sending messages to ensure that when B receives the message, the database insert operation has been completed, and when the kafka message fails, the correct fallback operation can be performed.

at present, my approach is to manually control, remove the @ Transactional annotation, and manually delete the data added in the previous database insertion operation when there is an exception in message delivery.

is there a more elegant solution?

Jun.14,2022

first analyze why the order problem occurs:
plus @ Transactional transactions that belong to the database. The pseudo code is as follows:

 

@ TransactionalEventListener

or take the way of local event table, write the message to the local event table, and the polling thread handles the delivery problem

Menu