Monthly Archives: June 2019

PrincipalObjectAccess and double mailboxes

PrincipalObjectAccess table – The stuff that nightmares are made of πŸ™‚

Ok So first of all if you don’t know what PrincipalObjectAccess is (POA from now on), Go ahead and write it down in your favorite search engine, do some reading in the endless blog posts and articles available out there – and then get back here, as I’m going to assume that you already know what it is and how it works.

Think of the following scenario – You have 10 users in your Organization but want to use only 1 mailbox, and you want all the users to use the same email address and see the same emails in the system. Yes you should definitely use a Queue for this, but maybe you don’t feel like using a queue or just don’t know exactly how it works and how to set it up – and hey, the system does not prevent you from setting the same email address for multiple users, right…?

So can you do it this way? Probably yes
Will it actually work? Likely it will
Is it a good idea? Nope

Here’s what will happen
** Obviously this is for demonstration only DO NOT DO THIS IN YOUR ENVIRONMENT **

Step 1: Assign 10 users the same email address. (I used a yahoo.com email for testing – feel free to spam my email address :))

Step 2: Create your server-side-sync profile and assign to the users, then activate, approve, test & enable all the mailboxes – Basically do all the steps you need to do for a mailbox to start working.

Step 3: Change all the user’s settings to automatically track all emails in the mailbox (the scenario would also work for emails in reply to CRM emails).

Step 4: Send an email to the newly crated mailbox and wait a few minutes.

Step 5: Check your POA table by running a query on your DB:

SELECT TOP 100 * FROM PrincipalObjectAccess
ORDER BY ChangedOn DESC

Results:
The email that entered the system receives a POA share record for every user that owns the email address. Not so great!

This is a small example of what will happen. I found it at a customer with more then 400 users that were assigned the same mailbox and something in between a few hundreds to thousand email threads – every day!
Needless to say that in this scenario their POA table grew at a rate of about 500,000 records per day.

This could definitely be causing additional side effects but I didn’t bother to check any further πŸ™‚

The fact that the system allows you to do things in a certain way does not always mean it’s a good practice, and if there is a mechanism built in the system to address a specific scenario – you should probably use it as there is a reason behind it.

If this post prevents from even one person setting up a system in this way – I’ve done my job πŸ™‚

Happy POA’ing

Michael

Improving Server Side Sync performance

The following blog was created after helping a customer of mine to drastically improve their server-side-sync performance by modifying the polling intervals of the mailboxes. let’s go πŸ™‚

Understanding the mechanism

Server side sync polls mailboxes for Emails and ACT’s (Accounts, Contacts, Tasks) in sync cycles. In each sync cycle a mailbox will be inspected for new items and according to your settings for that specific mailbox it will sync these items to Dynamics.

This mechanism has an internal prioritization logic that increases and decreases the time between each polling for each mailbox according to activity that is observed on that specific mailbox. This behavior is described briefly in the SSS whitepapers and I will expand on this in this post.

A busy mailbox that has constant activity on it should be polled for items (emails) approx. every 5 minutes. But when there is no activity on the mailbox for several consecutive cycles – The mechanism will kick in and start increasing the time between each poll. At this point the mailbox enters a state of an IdleMailbox – and for these mailboxes type the sync cycle can increase to up to 6 Hours. That’s right, 6 hours. This same behavior is also relevant for ACT’s and has a separate setting with separate intervals.

This means that you can end up in a scenario in which a mailbox becomes Idle at 7:00 AM because there were no emails flowing in, and from that point the mailbox will be polled again only at 13:00 (1:00 PM) 6 hours later in the worst case scenario.

This mechanism is in place for a reason – to decrease the utilization on the email integration servers and unnecessary calls to EWS. Without it a customer that has for example 5,000 configured mailboxes but only few of them actually active – would end up with massive utilization of the servers and huge amount of calls to EWS.Β ((5000 x 12 email polls per hour) + (5000 x 5 ACT polls per hour)) = 85,000 polls per hour.

Luckily – we can control these settings, and it helped me solve an issue for a customer that actually needed to poll ~1000 mailboxes at a very high and consistent rate, without any delays or surprises. As explained above changing the setting caused the Async servers to soar in terms of resource consumption, so this is something you need to take into account and make sure your infrastructure can handle the change.

Explaining the parameters

The actual polling settings are stored in the DeploymentProperties table in the MSCRM_CONFIG database and are represented in seconds.

Although the Minimum values for Emails and ACT’s are 1 minute & 5 minutes – In reality Iv’e always seen that the MaximumBackoff values for polls are being used for the Active mailboxes.
Default Values

Changing the setting

You can use PowerShell on your Dynamics servers to adjust the settings. In this example we will change the IdleMailboxMaximumBackoff time from 21600 (6 Hours) seconds to 1800 seconds (30 minutes)

Add-PSSnapin Microsoft.Crm.PowerShell
Get-CrmSetting -SettingType ServerSideSyncEmailSettings
$set = Get-CrmSetting -SettingType ServerSideSyncEmailSettings
$set.IdleMailboxMaximumBackoff = “1800”
Set-CrmSetting -Setting $set
Get-CrmSetting -SettingType ServerSideSyncEmailSettings

Result after change
* not that the column name is ECidlemailboxMaximumBackoff

You could also change those settings on the DB but for safety and supportability reasons it would be a better to do it VIA PowerShell.

MailboxstatisticsBase

The MailboxStatisticsBase table is an excellent source of insights regarding the internal works of the polling mechanism. Download and run This query to see all the polls that were done on all the mailboxes and how many items were processed in each poll.  You can also filter it by a time interval to show you all the times in which a single sync cycle on any mailbox took more then X minutes – This is very useful when you need to troubleshoot sync issues. Just read the comments in the SQL query.

Additional Notes

If your MailboxStatisticsBase table is empty and not populating then it’s probably disabled for data collection – You can enable it with the OrgDBSettings tool by setting the MailboxStatisticsPersistenceTimeInDays to the number of days you want to save data for (Lot’s of data!) 0 means no data is collected.

Needless to mention – Those changes are only applicable for Dynamics on-premises deployments. And as mentioned above – If you make changes be sure you are ready for the extra resource consumption on the servers.

It’s been a long post! hope you find this useful πŸ™‚

Michael

Back to business…

It’s been a while,

2.5 years to be precise since I wrote my last post here, and quite a few things have changed. First of all I started to work for Microsoft – a big accomplishment and a personal goal that I was very happy and proud to achieve. I’m a Dynamics 365 Premier field engineer and am enjoying my work very much – So an update to my “About” section is also coming. And Of course many changes to the product, the movement to Dynamics 365 online with it’s Azure & Office 365 ecosystem, New versions and features for on-premise versions, The relatively new V9 and Unified interface, App for Outlook for both online and on-premises deployments and much much more – In other words, lot’s of stuff to write about πŸ™‚

So as the headline above implies, I’m going to start writing again and already have some great ideas in mind that will hopefully help you out with your Dynamics 365 ventures, whether it’s online or on-premise

Watch this space & see you soon!

Michael