ZFConf 2010: Using Message Queues in Day-to-Day Projects (Zend_Queue)

Editor's Notes

• #3: http://en.wikipedia.org/wiki/FIFO http://en.wikipedia.org/wiki/Message_queue Message queues provide an  asynchronous   communications protocol , meaning that the sender and receiver of the message do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them. These message queueing systems typically provide enhanced  resilience  functionality to ensure that messages do not get "lost" in the event of a system failure. Очередь в программировании используется, как и в реальной жизни, когда нужно совершить какие-то действия в порядке их поступления, выполнив их последовательно. Примером может служить организация событий в Windows. Когда пользователь оказывает какое-то действие на приложение, то в приложении не вызывается соответствующая процедура (ведь в этот момент приложение может совершать другие действия), а ему присылается сообщение, содержащее информацию о совершенном действии, это сообщение ставится в очередь, и только когда будут обработаны сообщения, пришедшие ранее, приложение выполнит необходимое действие. Many of the more widely-known  communications protocols  in use operate  synchronously . The  HTTP  protocol – used in the  World Wide Web  and in  web services  – offers an obvious example. In many situations this makes perfect sense; for example, a user sends a request for a web page and then waits for a reply. However, other scenarios exist in which such behaviour is not appropriate. For example, an application may need to notify another that an event has occurred, but does not need to wait for a response. Another example occurs in  publish/subscribe  systems, where an application "publishes" information for any number of clients to read. In both these examples it would not make sense for the sender of the information to have to wait if, for example, one of the recipients had crashed. Alternatively, an interactive application may need to respond to certain parts of a request immediately (such as telling a customer that a sales request has been accepted, and handling the promise to draw on inventory), but may queue other parts (such as completing calculation of billing, forwarding data to the central accounting system, and calling on all sorts of other services) to be done some time later. In all these sorts of situations, having a subsystem which does asynchronous message-queuing (or alternatively, a broadcast messaging system) can help improve the behaviour of the overall system.
• #4: http://docs.amazonwebservices.com/AWSSimpleQueueService/2008-01-01/SQSDeveloperGuide/index.html?AboutVT.html When a consuming component in your system receives and processes a message from the queue, the message remains in the queue. Why doesn't SQS automatically delete it? Because your system is distributed, there's no guarantee that the component will actually receive the message (it's possible the connection could break or the component could fail before receiving the message). Therefore, SQS does not delete the message, and instead, your consuming component must delete the message from the queue after receiving and processing it. Immediately after the component receives the message, the message is still in the queue. However, you don't want other components in the system receiving and processing the message again. Therefore, SQS blocks them with a  visibility timeout , which is a period of time during which SQS prevents other consuming components from receiving and processing that message. The following figure and discussion illustrate the concept. The visibility timeout clock starts ticking once SQS returns the message. During that time, the component processes and deletes the message. But what happens if the component fails before deleting the message? If your system doesn't call  DeleteMessage  for that message before the visibility timeout expires, the message again becomes visible to the  ReceiveMessage calls placed by the components in your system and it will be received again. If a message should only be received once, your system should delete it within the duration of the visibility timeout. Each queue starts with a default setting of 30 seconds for the visibility timeout. You can change that setting for the entire queue. Typically, you'll set the visibility timeout to the average time it takes to process and delete a message from the queue. When receiving messages, you can also set a special visibility timeout for the returned messages without changing the overall queue timeout. We recommend that if you have a system that produces messages that require varying amounts of time to process and delete, you create multiple queues, each with a different visibility timeout setting. Your system can then send all messages to a single queue that forwards each message to another queue with the appropriate visibility timeout based on the expected processing and deletion time for that message. The following table lists the API actions to use to manipulate the visibility timeout.
• #5: http://en.wikipedia.org/wiki/Publish/subscribe Publish/subscribe  (or pub/sub) is a  messaging   paradigm  where senders (publishers) of messages are not programmed to send their messages to specific receivers (subscribers). Rather, published messages are characterized into classes, without knowledge of what (if any) subscribers there may be. Subscribers express interest in one or more classes, and only receive messages that are of interest, without knowledge of what (if any) publishers there are. This  decoupling  of publishers and subscribers can allow for greater  scalability  and a more dynamic  network topology . Pub/sub is a sibling of the  message queue  paradigm, and is typically one part of a larger  message-oriented middleware  system. Most messaging systems support both the pub/sub and message queue models in their  application programming interface  (API) (e.g.  Java Message Service  (JMS)). In the  point-to-point or queuing model , a  sender  posts messages to a particular queue and a  receiver  reads messages from the queue. Here, the sender knows the destination of the message and posts the message directly to the receiver's queue. It is characterized by the following: Only one consumer gets the message The producer does not have to be running at the time the consumer consumes the message, nor does the consumer need to be running at the time the message is sent Every message successfully processed is acknowledged by the consumer The  publish/subscribe model  supports publishing messages to a particular message topic.  Subscribers  may register interest in receiving messages on a particular message topic. In this model, neither the publisher  nor the subscriber know about each other. A good analogy for this is an anonymous bulletin board. The following are characteristics of this model: Multiple consumers can get the message There is a timing dependency between publishers and subscribers. The publisher has to create a subscription in order for clients to be able to subscribe. The subscriber has to remain continuously active to receive messages, unless it has established a durable subscription. In that case, messages published while the subscriber is not connected will be redistributed whenever it reconnects. Using Java, JMS provides a way of separating the application from the transport layer of providing data. The same Java  classes  can be used to communicate with different JMS providers by using the  JNDI  information for the desired provider. The classes first use a  connection factory  to connect to the queue or topic, and then use populate and send or publish the messages. On the receiving side, the clients then receive or subscribe to the messages.
• #6: Тут красиво будет разместить картинки + фичи каждого адаптера Также рассказать про фичи каждого и когда какой удобнее использовать MemcacheQ - So we have a limit on the message body size with a max of a bit less than  64K . Redis – 110K SETs/second, 81K GETs/second in an entry level Linux box
• #7: http://framework.zend.com/wiki/display/ZFPROP/Zend_Queue+-+Justin+Plock
• #8: Стратегия ,  Strategy  — поведенческий  шаблон проектирования , предназначенный для определения семейства  алгоритмов ,  инкапсуляции  каждого из них и обеспечения их взаимозаменяемости. Это позволяет выбирать алгоритм путем определения соответствующего класса. Шаблон Strategy позволяет менять выбранный алгоритм независимо от  объектов -клиентов, которые его используют. Мотивы Программа должна обеспечивать различные варианты алгоритма или поведения Нужно изменять поведение каждого экземпляра класса Необходимо изменять поведение объектов на стадии выполнения Введение интерфейса позволяет классам-клиентам ничего не знать о классах, реализующих этот интерфейс и инкапсулирующих в себе конкретные алгоритмы
• #9: http://framework.zend.com/manual/en/zend.queue.adapters.html http://rediska.geometria-lab.net/ Любые адаптеры на все случаи жизни  Можно легко переключаться между ними, изменяем только конфиг Apache ActiveMQ – быстрый, расширяемый, удобный, много API (Ajax API, REST) База данных – с использованием Zend_Db – простой вариант с неограниченным местом MemcacheQ - Очередь задач Zend Platform Массив – удобен для тестирования Redis – говорят что очень быстрый
• #10: Примеры – Прозрачная архивация данных Конвертация JSON данных Снятие огранчения не отправку только текстовых сообщений (возможность отправлять объекты)
• #11: http://framework.zend.com/manual/en/zend.queue.custom.html
• #16: Описание системы мониторинга пользователя на сайте И системы подсчёта кликов
• #17: Напримере турстанка – кеширование блоков Как говорил Илья Lifetime – неделя, но актуальный lifetime – 1day Никаких блокировок, обновление кэша из внешних программ