Tag Archives: dynamics

PrincipalObjectAccess and double mailboxes

Leave a reply

PrincipalObjectAccess table – The stuff that nightmares are made of 🙂

Ok So first of all if you don’t know what PrincipalObjectAccess is (POA from now on), Go ahead and write it down in your favorite search engine, do some reading in the endless blog posts and articles available out there – and then get back here, as I’m going to assume that you already know what it is and how it works.

Think of the following scenario – You have 10 users in your Organization but want to use only 1 mailbox, and you want all the users to use the same email address and see the same emails in the system. Yes you should definitely use a Queue for this, but maybe you don’t feel like using a queue or just don’t know exactly how it works and how to set it up – and hey, the system does not prevent you from setting the same email address for multiple users, right…?

So can you do it this way? Probably yes
Will it actually work? Likely it will
Is it a good idea? Nope

Here’s what will happen
** Obviously this is for demonstration only – DO NOT DO THIS IN YOUR ENVIRONMENT **

Step 1: Assign 10 users the same email address. (I used a yahoo.com email for testing – feel free to spam my email address :))

Step 2: Create your server-side-sync profile and assign to the users, then activate, approve, test & enable all the mailboxes – Basically do all the steps you need to do for a mailbox to start working.

Step 3: Change all the user’s settings to automatically track all emails in the mailbox (the scenario would also work for emails in reply to CRM emails).

Step 4: Send an email to the newly crated mailbox and wait a few minutes.

Step 5: Check your POA table by running a query on your DB:

SELECT TOP 100 * FROM PrincipalObjectAccess
ORDER BY ChangedOn DESC

Results:
The email that entered the system receives a POA share record for every user that owns the email address. Not so great!

This is a small example of what will happen. I found it at a customer with more then 400 users that were assigned the same mailbox and something in between a few hundreds to thousand email threads – every day!
Needless to say that in this scenario their POA table grew at a rate of about 500,000 records per day.

This could definitely be causing additional side effects but I didn’t bother to check any further 🙂

The fact that the system allows you to do things in a certain way does not always mean it’s a good practice, and if there is a mechanism built in the system to address a specific scenario – you should probably use it as there is a reason behind it.

If this post prevents from even one person setting up a system in this way – I’ve done my job 🙂

Happy POA’ing

Michael

Improving Server Side Sync performance

Leave a reply

The following blog was created after helping a customer of mine to drastically improve their server-side-sync performance by modifying the polling intervals of the mailboxes. let’s go 🙂

Understanding the mechanism

Server side sync polls mailboxes for Emails and ACT’s (Accounts, Contacts, Tasks) in sync cycles. In each sync cycle a mailbox will be inspected for new items and according to your settings for that specific mailbox it will sync these items to Dynamics.

This mechanism has an internal prioritization logic that increases and decreases the time between each polling for each mailbox according to activity that is observed on that specific mailbox. This behavior is described briefly in the SSS whitepapers and I will expand on this in this post.

A busy mailbox that has constant activity on it should be polled for items (emails) approx. every 5 minutes. But when there is no activity on the mailbox for several consecutive cycles – The mechanism will kick in and start increasing the time between each poll. At this point the mailbox enters a state of an IdleMailbox – and for these mailboxes type the sync cycle can increase to up to 6 Hours. That’s right, 6 hours. This same behavior is also relevant for ACT’s and has a separate setting with separate intervals.

This means that you can end up in a scenario in which a mailbox becomes Idle at 7:00 AM because there were no emails flowing in, and from that point the mailbox will be polled again only at 13:00 (1:00 PM) 6 hours later in the worst case scenario.

This mechanism is in place for a reason – to decrease the utilization on the email integration servers and unnecessary calls to EWS. Without it a customer that has for example 5,000 configured mailboxes but only few of them actually active – would end up with massive utilization of the servers and huge amount of calls to EWS. ((5000 x 12 email polls per hour) + (5000 x 5 ACT polls per hour)) = 85,000 polls per hour.

Luckily – we can control these settings, and it helped me solve an issue for a customer that actually needed to poll ~1000 mailboxes at a very high and consistent rate, without any delays or surprises. As explained above changing the setting caused the Async servers to soar in terms of resource consumption, so this is something you need to take into account and make sure your infrastructure can handle the change.

Explaining the parameters

The actual polling settings are stored in the DeploymentProperties table in the MSCRM_CONFIG database and are represented in seconds.

Although the Minimum values for Emails and ACT’s are 1 minute & 5 minutes – In reality Iv’e always seen that the MaximumBackoff values for polls are being used for the Active mailboxes.
Default Values

Changing the setting

You can use PowerShell on your Dynamics servers to adjust the settings. In this example we will change the IdleMailboxMaximumBackoff time from 21600 (6 Hours) seconds to 1800 seconds (30 minutes)

Add-PSSnapin Microsoft.Crm.PowerShell
Get-CrmSetting -SettingType ServerSideSyncEmailSettings
$set = Get-CrmSetting -SettingType ServerSideSyncEmailSettings
$set.IdleMailboxMaximumBackoff = “1800”
Set-CrmSetting -Setting $set
Get-CrmSetting -SettingType ServerSideSyncEmailSettings

Result after change
* not that the column name is ECidlemailboxMaximumBackoff

You could also change those settings on the DB but for safety and supportability reasons it would be a better to do it VIA PowerShell.

MailboxstatisticsBase

The MailboxStatisticsBase table is an excellent source of insights regarding the internal works of the polling mechanism. Download and run This query to see all the polls that were done on all the mailboxes and how many items were processed in each poll.  You can also filter it by a time interval to show you all the times in which a single sync cycle on any mailbox took more then X minutes – This is very useful when you need to troubleshoot sync issues. Just read the comments in the SQL query.

Additional Notes

If your MailboxStatisticsBase table is empty and not populating then it’s probably disabled for data collection – You can enable it with the OrgDBSettings tool by setting the MailboxStatisticsPersistenceTimeInDays to the number of days you want to save data for (Lot’s of data!) 0 means no data is collected.

Needless to mention – Those changes are only applicable for Dynamics on-premises deployments. And as mentioned above – If you make changes be sure you are ready for the extra resource consumption on the servers.

It’s been a long post! hope you find this useful 🙂

Michael

Back to business…

Leave a reply

It’s been a while,

2.5 years to be precise since I wrote my last post here, and quite a few things have changed. First of all I started to work for Microsoft – a big accomplishment and a personal goal that I was very happy and proud to achieve. I’m a Dynamics 365 Premier field engineer and am enjoying my work very much – So an update to my “About” section is also coming. And Of course many changes to the product, the movement to Dynamics 365 online with it’s Azure & Office 365 ecosystem, New versions and features for on-premise versions, The relatively new V9 and Unified interface, App for Outlook for both online and on-premises deployments and much much more – In other words, lot’s of stuff to write about 🙂

So as the headline above implies, I’m going to start writing again and already have some great ideas in mind that will hopefully help you out with your Dynamics 365 ventures, whether it’s online or on-premise

Watch this space & see you soon!

Michael

Dynamics CRM 2016 Step-By-Step installation guide

3 Replies

Hi everyone!

To be honest – The installation process & requirements of Dynamics CRM 2016 is practically identical to 2015 – But most people that are new to Dynamics don’t know that.

So here is a new video tutorial for Dynamics CRM 2016 installation

Move AX 2012 to a different Server / Domain

10 Replies

As you probably know Microsoft Dynamics AX 2012 R2/R3 does not have a redeployment utility (like the Dynamics CRM deployment manager) meaning that if you want to move an AX Deployment from one domain to another – you’ll have to get your hands a little dirty.

Some important notes before we start :

  • You can use these steps also in AX 2012 – Just ignore the Modelstore database.
  • You can also use these steps to move a deployment in the same domain – ignore step 4.
  • You need to make sure that the AX installation version in the new server matches the installation
    on the previous server.
  • I’m assuming that you have the basic required knowledge in SQL Server, your Windows Serverenvironments and of course some AX 2012 architecture understanding to do these steps 🙂

Let’s go:

1. Open the SQL management studio and Backup your AX Databases [BusinessData Database + Modelstore Database] for example: AXProd + AXProd_model
2. Stop the AX Service on the new server and restore the databases in the new SQL server – You can restore them with different names just remember that they have to match – for example: AXProdNEW
AXProdNEW_model.
3. In the SQL management studio run the following queries on the BusinessData Database (AXProdNEW):

DELETE FROM SYSSERVERCONFIG
DELETE FROM SYSSERVERCONFIG

— This will delete the AOSID & SERVERID of the previous installation and will
— repopulate the tables with the new server name when the AX service on the new server is started.

Also consider running:

DELETE FROM SRSSERVERS

— This will clear the Reporting Services properties form – you will need to reconfigure this
— later if you want to redeploy the reports.

DELETE FROM BATCHSERVERGROUP
DELETE FROM BATCHSERVERCONFIG

— This will delete the batch server properties of the previous server – You should do this if
— the redeployment is in the same domain because the old server might be reachable by the new one and may use it as a batch server.

DELETE FROM SYSEMAILPARAMETERS

— This will delete the SMTP server parameters for sending emails

4. Now you need to configure your user in the AX Database in the new domain (You can skip this step if you re-deployed AX in the same domain).
Open CMD with “Run as admin” and type:

wmic useraccount where (name='your_windows_username' and domain='your_domain') get name,sid

Example: wmic useraccount where (name='administrator' and domain='contoso') get name,sid

Copy the SID to notepad.

In the SQL management studio run the following query on the BusinessData Database (AXProdNEW):

UPDATE USERINFO
SET SID='your_sid', NETWORKALIAS='your_windows_username', NETWORKDOMAIN='your_domain', NAME='your_full_name'
WHERE ID = 'Admin'

Example:

UPDATE USERINFO
SET SID='S-1-5-21-916693446-2822637979-541647007-1001', NETWORKALIAS='administrator', NETWORKDOMAIN='contoso', NAME='John TheAdmin'
WHERE ID = 'Admin'

— This will replace the AX Admin user to your user – you can modify this query to replace other
— users as well, or you can later add new users from the AX GUI.

5. Grant the AX AOS account permissions over the databases as follows:
AXProdNEW:
db_datareader
db_datawriter
db_ddladmin
public

axpermissions1

AXProdNEW_model:
public

axpermissions2

You can identify the AOS account by going to the services on the application server (start->
services.msc) and check the Log On tab:

axservices1
If you are using NETWORK SERVICES as the service account you will need to grant permissions to the machine$ account – Use the following query to do it:

CREATE LOGIN [Domain\axservername$] FROM WINDOWS

6. All done! Go to services on the application server and start the Microsoft Dynamics AX Service –
it might take a few minutes for it to start. Once it’s running – you can start the application.
If you have any issues starting the service check the server’s event log for clues, and if you have
any questions – feel free to post them here and i will do my best to help you out.

Further optional steps:

1. You should synchronize your Database to make sure everything is working properly.
2. If you want to deploy your reports – you need to fill the Report Servers form and then run the
following command from the Dynamics AX 2012 management Shell:

Publish-AXReport –ReportName *

I hope you find this post helpful! Good luck, Michael.

CRM async service high CPU usage

Leave a reply

There are many scenarios that can cause a high CPU utilization by the CRM asynchronous service, here are some
of the most common ones i encountered:

1. Check if the AsyncOperationBase table doesn’t have too many records in it. Once this table has millions of records – things may start working very slow. Use the following article to solve this issue and to create a proper maintenance plan to prevent this from happening:

http://support.microsoft.com/kb/968520

2. Use the CRM Advanced Find to search for locked system jobs – this is a very common issue that can cause the
async service to go havoc and utilize 100% of the CPU on the server ALL THE TIME – even when there is
absolutely zero activity in the system.

If you find any locked system jobs with the advanced find – delete them and restart the async service.

Image

3. Check the email router and the emails in the system:

Use the Advanced find to see if there are any emails stuck in sending status, if so – try deleting them and investigate why they got stuck in the first place. You should also try stopping the email router service to eliminate it as the cause and check the event log to see if there are any “suspicious” emails in the queue. Sometimes emails with special characters or some sort of corruption can cause issues with the async service (although they are not supposed to, i did encounter it).

4. RESTART! try restarting the server/asynchronous service. Sometimes restarting the system is the simplest
solution, so before you start investigating the problem and wasting hours on hours of your time – make sure you do system reboot first.

5. If none of the above helps you should try investigating further and farther.

  • Check your SQL server logs, performance & event log for any suspicious entries.
  •  Try tracing the CRM and analyze the trace file for any errors or exceptions. Use this article to enable tracing for CRM: http://support.microsoft.com/kb/907490
  • Try to see if maybe a 3rd party entity is causing the problem: Antivirus, Firewall etc.

Good Luck 🙂

Michael.