. The system architecture diagram when using Rabbitmq
is binding the switch and queue through the routing key to achieve message sending and receiving.
. The basic concept of Rabbitmq
Rabbitmq is an open source implementation of the AMQP protocol, so its internal is actually the basic concept in AMQP, as shown in the figure below:
1, Message
The message is not known, it consists of message head and message body. The message is opaque, and the message head consists of a series of optional attributes. These attributes include routing-key (routing key), priority Message may require persistence storage) and so on.
2. Publisher
Message producers are also a client application that posted messages to the exchange, which is a program that deliver messages.
3. Exchange
The switch is used to receive messages sent by the producer and give these messages routing to the queue in the server. Message switch, what rules it specifies the message, which queue is route to.
4. Routing key
Routing keywords, Exchange submits message according to this keyword.
5. Binding (binding)
is used for the association between the message queue and the switch. A binding is a routing rule based on the routing key to connect the switch and the message queue, so you can understand the switch as a routing table composed of binding.
The function is to bind exchange and queue in accordance with the routing rules.
The binding is actually related to Exchange and Queue, or say this: Queue is interested in Exchange's content. Exchange wants its message deliver to Queue.
6. Queue (message queue)
The carrier of the message will be cast to one or more queues, waiting for consumers Walk. It is the container of the message and the end of the message.
7,
network connection, such as a TCP connection.
8. Channel (channel, channel)
Message channels, every connection of the client can be established.
The independent two -way data circulation in multi -road reuse connection. The channel is a virtual connection built in the real TCP connection. The AMQP command is sent out through the channel. Whether it is published, subscribing queues or receiving messages, these actions are completed through channels. Because the establishment and destruction of TCP is very expensive for the operating system, the concept of the introduction of channels is introduced to achieve the purpose of reusing a TCP connection.
9. Consumer
Message consumers, indicating a client application that obtained a message from the message queue, is the program that accepts the message.
10. Virtual host
virtual host, indicating a batch of switches, message queues and related objects. There can be multiple VHOST in a broker for the separation of permissions for different users. Virtual hosting is an independent server domain that shares the same identity authentication and encrypted environment. Each VHOST is essentially a MINI version of the RabbitMQ server with its own queue, switch, binding and permissions mechanism.
Vhost is the basis of the AMQP concept. It must be specified during connection. The default vhost of Rabbitmq is /.
11. Broker
means the message queue server entity. It provides a transmission service. Its role is to maintain a route from producers to consumers to ensure that data can be transmitted in a specified manner.
three. Message in AMQP
The producer published the message to Exchange. The message finally reached the queue and was accepted by consumers. Send to that queue.如下图所示：rn rn 四、Exchange类型rn rn Exchange分发消息时根据类型的不同分发策略有区别，目前共有四种类型：direct、fanout、topic、 Headers. Headers match the header of the AMQP message instead of the routing key. In addition, the headers switch and the Direct switch are completely the same, but the performance is much different. At present, it is hardly used. Look at the three types of Direct, Fanout, Topic.
1. Direct type
The route key Routing Key in messaging If it is consistent with the binding key in binding, the switch sends the message to the corresponding queue. The routing key is fully matched with the queue name. If a queue is bound to the switch requires the routing key to be "DOG", only the news that Routing Key is marked as "DOG" will not be reposted "Dog.Puppy" and so on. It is a fully match and single communication mode.
Drique Exchange's routing algorithm is very simple: the complete matching of the pass can be used to illustrate:
EXCHANGE and two queues are binded together. Q1 is orange. Q2's Binding Key is Black and Green.
Is when the procer publish key is Orange, Exchange will put it on Q1. If it is Black or Green, it will reach Q2, and the remaining Message will be discarded.
2. FANOUT type
Message to each of the FANOUT type switch will be divided into all binding queues. The FANOUT switch does not handle the routing key, but simply bind the queue to the switch, and each message sent to the switch is forwarded to all queues bound to the switch. Similar to Ziwang Broadcast, the host in each net has received a copy of the copy. The FANOUT type forwarding message is the fastest. As shown in the figure below:
3. Topic type
topic switches to match the route key attributes of allocating messages through mode matching The queue needs to be bound to a pattern. It is divided into words in string of routing key and binding keys, which is separated by points between these words. It also recognizes two compatriots:#and*,#to match 0 or multiple words,*can only match one word.
The restrictions on the ROUTING_KEY of Message and cannot be arbitrarily. The format is a character table with ".". For example: "stock.usd.nyse", "nyse.vmw", "quick.orange.rapbit". You can let go of any of the key in routing_key, of course, the longest cannot exceed 255 bytes. For routing_key, there are two special characters#and*,#match 0 or multiple words,*can only match one word. As shown in the figure below:
Perr need to set routing_key when sending messages, routing_key contains three words and two points numbers. ), The third is species.
The were we created two binding: Q1's binding key is ".range."; Q2 was ". Rabbit" and "lazy.#": q1 interested in all Orange colors. Animals; Q2 Interested in all Rabbits and all lazy.
For example: Rounting_key is "quick.orange.rabbit" will be sent to Q1 and Q2. ROUNTING_KEY is "lazy.orange.rabbit.hujj.ddd" will be delivered to Q2,#matches 0 or multiple words.
five ... Channel
, and channels are the most basic objects in the API provided by Rabbitmq.
1,
is the socket connection of Rabbitmq, which encapsulates the related part of the related part of the socket protocol.
2.
is a manufacturing plant.
3. Channel
Channel is the most important interface to deal with Rabbitmq. Define Exchange, bind Queue and Exchange, publish messages, etc.
. Sixth, task distribution mechanism
1. Round-Robin cyclic distribution
Rabbbbitmq's distribution mechanism is very suitable for expansion, and it is specifically complicated by concurrent concurrency The program is designed. If LOAD is now aggravated, you only need to create more consumer for task processing.
2. MESSAGE message confirmation
In order to ensure that the data is not lost, the RabbitMQ support message confirmation mechanism is to ensure that the data can be properly processed instead of being received by Consumer. This needs to send a confirmation ACK after processing the data.
It sends ACK after processing the data, that is, telling RabbitMQ data that has been received and processing is completed. Rabbitmq can remove the message from the queue. If the consumer exits but does not send ACK, then RabbitMQ will send this message to the next consumer, so that the data will not be lost under the condition of the Consumer abnormal exit.
Rabbitmq is not used in timeout mechanism. It only confirms that the message is not processed correctly through the connection interrupt of the consumer. Other consumers, that is, Rabbitmq gives Consumer enough time to do data processing. If you forget the ACK, when the consume exits, the mesage will be re -distributed, resulting in more and more information in the queue, and then Rabbitmq will take up more and more memory.
3. MESSAGE Rability Message persistence
If we want to lose the message even if the RabbitMQ service restarts, we can set the Queue and Message as a The persistence (rable), so that in most cases, our rabbitmq messages will not be lost. However, it still cannot solve the occurrence of a small probability loss event (for example, the RabbitMQ server has received the news of the producer, but the RabbitMQ server is off powerless before it can be durable). If you also want to manage this small probability event, you need to use transactions. To persist the queue, you need to specify rable = true when the statement is declared; here should note that the name of the queue must not exist in the broker, otherwise it cannot change any attribute of this queue. The queue and switch have a specified logo RABLE when it was created. The only meaning of RABLE is to let the queue and exchange opportunities with this logo be re -established after restarting.
In the durability of the message includes 3 parts
(1) Exchange durable, specify rable => true
channel. (, ",", "," Direct ", rable: true,: false, arguments: null); // State the message queue, and the lasting
(2) Queue is persistent, specify rable => true
channel. (Queuename, rable: true, exclusive: false,: false, arguments: null); // State the message queue, and the lasting
(3) The message is durable and specifies the delvity_mode => 2 (1 is non -persistence) during delivery.
. If exchange and queue are durable, the binding between them It is also persistent; if there is a durability between Exchange and Queue and a non -persistence, binding is not allowed.
Note: Once the queue and switch are created, the logo cannot be modified. For example, a non-rable queue was created, and then it wanted to change it to rable. The only way was to delete this queue and re-create it.
4. Fair Dispath fair distribution
You may also notice that the distribution mechanism is not so elegant. In the default state, Rabbitmq distributes the n MESSAGE to the N Consumerr Essence N is after the remaining, and it has the unacked message regardless of whether the consciouser is, but it is distributed according to this default mechanism. Then if there is a consumer working heavier, then some consumer is basically fine to do, but some consumer has no chance of rest. So what does Rabbit deal with? Basic.qos method Set the prefetch_count = 1, as if
channel.basic_qos (preFETCH_COUNT = 1)
A message, in other words, it will not distribute the new Message to it before receiving the Consumer. But this method may lead to Queue full. Of course, in this case you may need to add more consumer or create more to refine your design.
5. It is distributed to multiple Consumer
direct Exchange: Direct matching, sending and receiving messages through Exchange name .
FANOUT Exchange: Broadcasting subscribe to all consumers, but only consumers bind the queue to the router to receive the message and ignore ROUTING Key.
Topic exchange: The theme matching subscription. The theme here refers to the use of generals, such as:*or#. The router can only receive the message when the matching rules are specified.
heads exce: message head subscription, the message head defines one or more key values pairs before the message is released, and then consumers receive the message. At the same time
x-mactch = all or x_match = Any). Only when the request header matches the message header, can you receive the message and ignore it.
The default Exchange: If you use an empty string to declare an exchange, then the system will use "amq.direct". When we create a queue, there will be a default one that is bound to this default Exchange with the same name as the new Queue. As follows:
channel. ("", "Taskqueue", bytes);
because the default Exchange was selected in the first parameter, and the queue we declared was called Taskqueue, so the default, it wants to create a new one called Taskqueue and bind it on the default Exchange, which leads to we can write Taskqueue in the second parameter, so that it will find the defined queue of the same name and give the message with the message put in.
If two receiving programs use the same Queue and the same to bind Direct Exchange, the distribution behavior is a load balancing, that is, the first is program 1 collection By the time, the program 2 was received, and so on.
If there are two receiving programs using their own queue, but using the same one to bind Direct Exchange, the distribution behavior is copied, that is, each program will receive a copy of the message. The behavior is equivalent to the EXCHANGE of the FANOUT type.
M multiple queue binding the same key is also possible. For the examples in the figure below, Q1 and Q2 are bound to Black. For Routing Key The rest of the MESSAGE will be discarded.
7. The remote process of RPC
MQ itself is based on asynchronous message processing. All producers (P) in the previous example will send the message to Rabbitmq. Knowing that consumers (C) handle success or failure (even whether there are consumers to handle the news). However, in the actual application scenario, we are likely to need some synchronous processing, and we need to wait for the server to process my message to complete my message and then proceed the next step. This is equivalent to RPC (Remote Procererr, remote process call). RPC is also supported in Rabbitmq.
The mechanism of implementing the RPC in RABBITMQ is shown in the figure below:
When the client sends a request (message), the attributes of the message (, define 14 species in the AMQP protocol, defined 14 species in the AMQP protocol These attributes will be sent with the message) Set two value replyto (a Queue name, which is used to tell the server processing to send the message to this queue) and (the identification number of this request, server processing, server processing After completion, you need to return this attribute, and the client will understand which request is successfully executed or executed according to this ID).
The server side receives the message and process it. After the server processs the message, a response message will be generated to the Queue specified by the replyto. At the same time, it will bring the attribute. After receiving the response message from the server, which request was executed based on the attributes, and then subsequent business processing was performed according to the execution results.
Rabbitmq is a distributed system
. The system architecture diagram when using Rabbitmq
is binding the switch and queue through the routing key to achieve message sending and receiving.
. The basic concept of Rabbitmq
Rabbitmq is an open source implementation of the AMQP protocol, so its internal is actually the basic concept in AMQP, as shown in the figure below:
1, Message
The message is not known, it consists of message head and message body. The message is opaque, and the message head consists of a series of optional attributes. These attributes include routing-key (routing key), priority Message may require persistence storage) and so on.
2. Publisher
Message producers are also a client application that posted messages to the exchange, which is a program that deliver messages.
3. Exchange
The switch is used to receive messages sent by the producer and give these messages routing to the queue in the server. Message switch, what rules it specifies the message, which queue is route to.
4. Routing key
Routing keywords, Exchange submits message according to this keyword.
5. Binding (binding)
is used for the association between the message queue and the switch. A binding is a routing rule based on the routing key to connect the switch and the message queue, so you can understand the switch as a routing table composed of binding.
The function is to bind exchange and queue in accordance with the routing rules.
The binding is actually related to Exchange and Queue, or say this: Queue is interested in Exchange's content. Exchange wants its message deliver to Queue.
6. Queue (message queue)
The carrier of the message will be cast to one or more queues, waiting for consumers Walk. It is the container of the message and the end of the message.
7,
network connection, such as a TCP connection.
8. Channel (channel, channel)
Message channels, every connection of the client can be established.
The independent two -way data circulation in multi -road reuse connection. The channel is a virtual connection built in the real TCP connection. The AMQP command is sent out through the channel. Whether it is published, subscribing queues or receiving messages, these actions are completed through channels. Because the establishment and destruction of TCP is very expensive for the operating system, the concept of the introduction of channels is introduced to achieve the purpose of reusing a TCP connection.
9. Consumer
Message consumers, indicating a client application that obtained a message from the message queue, is the program that accepts the message.
10. Virtual host
virtual host, indicating a batch of switches, message queues and related objects. There can be multiple VHOST in a broker for the separation of permissions for different users. Virtual hosting is an independent server domain that shares the same identity authentication and encrypted environment. Each VHOST is essentially a MINI version of the RabbitMQ server with its own queue, switch, binding and permissions mechanism.
Vhost is the basis of the AMQP concept. It must be specified during connection. The default vhost of Rabbitmq is /.
11. Broker
means the message queue server entity. It provides a transmission service. Its role is to maintain a route from producers to consumers to ensure that data can be transmitted in a specified manner.
three. Message in AMQP
The producer published the message to Exchange. The message finally reached the queue and was accepted by consumers. Send to that queue.如下图所示：rn rn 四、Exchange类型rn rn Exchange分发消息时根据类型的不同分发策略有区别，目前共有四种类型：direct、fanout、topic、 Headers. Headers match the header of the AMQP message instead of the routing key. In addition, the headers switch and the Direct switch are completely the same, but the performance is much different. At present, it is hardly used. Look at the three types of Direct, Fanout, Topic.
1. Direct type
The route key Routing Key in messaging If it is consistent with the binding key in binding, the switch sends the message to the corresponding queue. The routing key is fully matched with the queue name. If a queue is bound to the switch requires the routing key to be "DOG", only the news that Routing Key is marked as "DOG" will not be reposted "Dog.Puppy" and so on. It is a fully match and single communication mode.
Drique Exchange's routing algorithm is very simple: the complete matching of the pass can be used to illustrate:
EXCHANGE and two queues are binded together. Q1 is orange. Q2's Binding Key is Black and Green.
Is when the procer publish key is Orange, Exchange will put it on Q1. If it is Black or Green, it will reach Q2, and the remaining Message will be discarded.
2. FANOUT type
Message to each of the FANOUT type switch will be divided into all binding queues. The FANOUT switch does not handle the routing key, but simply bind the queue to the switch, and each message sent to the switch is forwarded to all queues bound to the switch. Similar to Ziwang Broadcast, the host in each net has received a copy of the copy. The FANOUT type forwarding message is the fastest. As shown in the figure below:
3. Topic type
topic switches to match the route key attributes of allocating messages through mode matching The queue needs to be bound to a pattern. It is divided into words in string of routing key and binding keys, which is separated by points between these words. It also recognizes two compatriots:#and*,#to match 0 or multiple words,*can only match one word.
The restrictions on the ROUTING_KEY of Message and cannot be arbitrarily. The format is a character table with ".". For example: "stock.usd.nyse", "nyse.vmw", "quick.orange.rapbit". You can let go of any of the key in routing_key, of course, the longest cannot exceed 255 bytes. For routing_key, there are two special characters#and*,#match 0 or multiple words,*can only match one word. As shown in the figure below:
Perr need to set routing_key when sending messages, routing_key contains three words and two points numbers. ), The third is species.
The were we created two binding: Q1's binding key is ".range."; Q2 was ". Rabbit" and "lazy.#": q1 interested in all Orange colors. Animals; Q2 Interested in all Rabbits and all lazy.
For example: Rounting_key is "quick.orange.rabbit" will be sent to Q1 and Q2. ROUNTING_KEY is "lazy.orange.rabbit.hujj.ddd" will be delivered to Q2,#matches 0 or multiple words.
five ... Channel
, and channels are the most basic objects in the API provided by Rabbitmq.
1,
is the socket connection of Rabbitmq, which encapsulates the related part of the related part of the socket protocol.
2.
is a manufacturing plant.
3. Channel
Channel is the most important interface to deal with Rabbitmq. Define Exchange, bind Queue and Exchange, publish messages, etc.
. Sixth, task distribution mechanism
1. Round-Robin cyclic distribution
Rabbbbitmq's distribution mechanism is very suitable for expansion, and it is specifically complicated by concurrent concurrency The program is designed. If LOAD is now aggravated, you only need to create more consumer for task processing.
2. MESSAGE message confirmation
In order to ensure that the data is not lost, the RabbitMQ support message confirmation mechanism is to ensure that the data can be properly processed instead of being received by Consumer. This needs to send a confirmation ACK after processing the data.
It sends ACK after processing the data, that is, telling RabbitMQ data that has been received and processing is completed. Rabbitmq can remove the message from the queue. If the consumer exits but does not send ACK, then RabbitMQ will send this message to the next consumer, so that the data will not be lost under the condition of the Consumer abnormal exit.
Rabbitmq is not used in timeout mechanism. It only confirms that the message is not processed correctly through the connection interrupt of the consumer. Other consumers, that is, Rabbitmq gives Consumer enough time to do data processing. If you forget the ACK, when the consume exits, the mesage will be re -distributed, resulting in more and more information in the queue, and then Rabbitmq will take up more and more memory.
3. MESSAGE Rability Message persistence
If we want to lose the message even if the RabbitMQ service restarts, we can set the Queue and Message as a The persistence (rable), so that in most cases, our rabbitmq messages will not be lost. However, it still cannot solve the occurrence of a small probability loss event (for example, the RabbitMQ server has received the news of the producer, but the RabbitMQ server is off powerless before it can be durable). If you also want to manage this small probability event, you need to use transactions. To persist the queue, you need to specify rable = true when the statement is declared; here should note that the name of the queue must not exist in the broker, otherwise it cannot change any attribute of this queue. The queue and switch have a specified logo RABLE when it was created. The only meaning of RABLE is to let the queue and exchange opportunities with this logo be re -established after restarting.
In the durability of the message includes 3 parts
(1) Exchange durable, specify rable => true
channel. (, ",", "," Direct ", rable: true,: false, arguments: null); // State the message queue, and the lasting
(2) Queue is persistent, specify rable => true
channel. (Queuename, rable: true, exclusive: false,: false, arguments: null); // State the message queue, and the lasting
(3) The message is durable and specifies the delvity_mode => 2 (1 is non -persistence) during delivery.
chanel. ("", queuename, ._text_plain, msg.getbytes ());
. If exchange and queue are durable, the binding between them It is also persistent; if there is a durability between Exchange and Queue and a non -persistence, binding is not allowed.
Note: Once the queue and switch are created, the logo cannot be modified. For example, a non-rable queue was created, and then it wanted to change it to rable. The only way was to delete this queue and re-create it.
4. Fair Dispath fair distribution
You may also notice that the distribution mechanism is not so elegant. In the default state, Rabbitmq distributes the n MESSAGE to the N Consumerr Essence N is after the remaining, and it has the unacked message regardless of whether the consciouser is, but it is distributed according to this default mechanism. Then if there is a consumer working heavier, then some consumer is basically fine to do, but some consumer has no chance of rest. So what does Rabbit deal with? Basic.qos method Set the prefetch_count = 1, as if
channel.basic_qos (preFETCH_COUNT = 1)
A message, in other words, it will not distribute the new Message to it before receiving the Consumer. But this method may lead to Queue full. Of course, in this case you may need to add more consumer or create more to refine your design.
5. It is distributed to multiple Consumer
direct Exchange: Direct matching, sending and receiving messages through Exchange name .
FANOUT Exchange: Broadcasting subscribe to all consumers, but only consumers bind the queue to the router to receive the message and ignore ROUTING Key.
Topic exchange: The theme matching subscription. The theme here refers to the use of generals, such as:*or#. The router can only receive the message when the matching rules are specified.
heads exce: message head subscription, the message head defines one or more key values pairs before the message is released, and then consumers receive the message. At the same time
x-mactch = all or x_match = Any). Only when the request header matches the message header, can you receive the message and ignore it.
The default Exchange: If you use an empty string to declare an exchange, then the system will use "amq.direct". When we create a queue, there will be a default one that is bound to this default Exchange with the same name as the new Queue. As follows:
channel. ("", "Taskqueue", bytes);
because the default Exchange was selected in the first parameter, and the queue we declared was called Taskqueue, so the default, it wants to create a new one called Taskqueue and bind it on the default Exchange, which leads to we can write Taskqueue in the second parameter, so that it will find the defined queue of the same name and give the message with the message put in.
If two receiving programs use the same Queue and the same to bind Direct Exchange, the distribution behavior is a load balancing, that is, the first is program 1 collection By the time, the program 2 was received, and so on.
If there are two receiving programs using their own queue, but using the same one to bind Direct Exchange, the distribution behavior is copied, that is, each program will receive a copy of the message. The behavior is equivalent to the EXCHANGE of the FANOUT type.
M multiple queue binding the same key is also possible. For the examples in the figure below, Q1 and Q2 are bound to Black. For Routing Key The rest of the MESSAGE will be discarded.
7. The remote process of RPC
MQ itself is based on asynchronous message processing. All producers (P) in the previous example will send the message to Rabbitmq. Knowing that consumers (C) handle success or failure (even whether there are consumers to handle the news). However, in the actual application scenario, we are likely to need some synchronous processing, and we need to wait for the server to process my message to complete my message and then proceed the next step. This is equivalent to RPC (Remote Procererr, remote process call). RPC is also supported in Rabbitmq.
The mechanism of implementing the RPC in RABBITMQ is shown in the figure below:
When the client sends a request (message), the attributes of the message (, define 14 species in the AMQP protocol, defined 14 species in the AMQP protocol These attributes will be sent with the message) Set two value replyto (a Queue name, which is used to tell the server processing to send the message to this queue) and (the identification number of this request, server processing, server processing After completion, you need to return this attribute, and the client will understand which request is successfully executed or executed according to this ID).
The server side receives the message and process it. After the server processs the message, a response message will be generated to the Queue specified by the replyto. At the same time, it will bring the attribute. After receiving the response message from the server, which request was executed based on the attributes, and then subsequent business processing was performed according to the execution results.
forward: // P/13280745