Force subscription & surprises with duplicate messages in Azure Service Bus

We've been using Azure Service Bus since 2 years and still discovering new surprises (aka designed not how we would have expected to).

Back in the past, we faced the situation of being able to replay dead messages from a subscription within Service Bus Explorer. No standard way was available and we decided to implement it through SQL Filtering. An SQL Filter action is defined with a flag forcedestination with the actual subscription name (no limitation here, only consistency).
When a message is sent to a topic, message get an extra customProperty called forcedestination which can be filtered if message being replayed.

  • filter : NOT EXISTS([forcedestination]) OR [forcedestination] = 'subscription1'
  • action : SET [forcedestination] = 'subscription1'

A message send without forcedestination property will be propagated to both subscription1 and subscription2.

below an active message from subscription1 with expected forcedestination property. By design, RuleName get the rule name which matched.

Any subscription can handle several rules, recently some subscription rule started to be complex to read and we decided to move forcedestination to its own rule for better clarity. We were confident of knowing impacts and limited our test to minimum and then we blow up our development environment due to a wrong comprehension of how were linked rules altogether :)

For one year, we thought filtering was actually done by concatenating rules with a OR operator: rule1 OR rule2 OR rule3 however, that was wrong.
Each message is run against all rules, one by one and if filtering is valid, message is forwarded. Sounds like OR operator isn't it ? pretty much, except that each valid rule will generate one message.
If 3 rules match, then active queue will get 3 copies of the same message. We were only able to reproduce this behavior by replaying a message from deadletter (not active) for unknown reason.

Beware of this limitation, your colleagues might be surprised getting duplicate when replying messages.

Fabien Camous

Read more posts by this author.