Beware of Dynamics 365 Web API: When attribute and entity have same names

With increasing maturity of the OData Web API exposed by Dynamics 365 CE/CRM its adoption does also steadily increase. From my perspective especially from JavaScript client codes. While the Web API doubtlessly has its sweetspots, it also comes with a lot of pitfalls and limitations (yet) we must get comfortable with and become aware of.

One of these pitfalls which I want to recall today, was already covered back in 2017 by Tip #970: When attribute and entity collide and goes like this:

While (technically) the CRM platform allows an attribute to have the same name as the entity it is defined on, the OData Web API is not capable of handling this and simple throws an “Could not find a property” HTTP 400 error.

Keep in mind when designing the entity model: Avoid naming collisions by never giving an attribute the same schema name as its parent entity.

Test and mock (custom) organization messages with FakeXrmEasy in Dynamics 365 CRM

FakeXrmEasy supports many of the common CRM standard organization messages out of the box. With “supports” I mean that it not only exposes simple mock/fake/stub/whatever facilities for these messages — like f.e. intercepting executions thereof –, but rathermore FakeXrmEasy provides runnable implementations of them which behave much like the “real” CRM message pendant. Using these, we can test workflows, plugins and general client code which internally use the CRM organization service for executing organization messages or reacting to them without having to write any message processing logic by ourselves. As said … FakeXrmEasy supports many messages out of the box, but by far not all and in more or less similar behaviour.

FakeXrmEasy supports common messages like f.e. Create, Retrieve, RetrieveMultiple, Update and Delete and therefore provides kind of a “virtual” mini CRM system including things like an organization message processing pipeline which behaves similar to “real” CRM and wherein all organization messages are executed. Of course my comparison is very simplistic, but I hope you got the idea. 😉 

How to test and mock organization messages

However, there are also several messages which are not implemented (yet) by FakeXrmEasy and thus have to be mocked/stubbed/faked by our test code, depending on our testing scenario. FakeXrmEasy uses an extensible strategy pattern for registering message handlers and when determining which handler implementation to use for a given organization message.

Mocking any of the missing CRM standard messages or any of our custom message (like calling an Action) can be done in two different ways:

  1. Returning some predefined response data 
  2. Implement a full fledged request request handler 

To be more specific, using the following methods provided by class XrmFakedContext:

  • AddExecutionMock<T>(ServiceRequestExecution mock
    • Purpose: Register delegate function of type ServiceRequestExecution (which takes OrganizationRequest object as input and returns OrganizationResponse object as output) which returns some (predefined) response data for every message of type <T> whereas T is OrganizationRequest or a derivate of it. 
  • AddFakeMessageExecutor<T>(IFakeMessageExecutor executor)
    • Note: Generic method variant does also exist, where message name is passed as additional paramter to method → AddGenericFakeMessageExecutor(string message, IFakeMessageExecutor executor) 
    • Purpose: Register full fledged message executor implementation for messages of type <T> whereas T is OrganizationRequest or a derivate of it. This interface plays a central role in FakeXrmEasy since all standard organization messages which are supported out of the box, are implement as classes which implement IFakeMessageExecutor. These are organized using the chain of responsibility pattern, where a message in the message processing pipeline is handed over to every registered message executor instance until one is found that `CanExecute` it.

How to implement a custom message handler

Given we have a CRM process of type Action which is called “xy_GetEnergyProductDetailsBySapRateCode”. Now how should we for example test a workflow which internally calls this action or any custom CRM client code which calles this action? But now step by step. First we create two specialized messages classes which makes it easier for us to execute the message using organizationService.Execute(…):

Well, let’s assume we just want to return some predefined response inside our test code. We can do this like:



Summary

FakeXrmEasy is a very helpful and extensible testing solution for Dynamics CRM plugins, workflows and general SDK client code. It not only is around for a long time and has matured noticeably over time, it is also backed by a comprehensive test suite and does support CRM 2011 until latest version 9.x including all their specifics. While the official documentation may shed some light onto general questions for new users, they will leave the reader with more questions than he came. But that’s okay, because its open source and freely available on GitHub, so everybody can should dive into the internal FakeXrmEasy test suites and analyze how it’s implemented. It’s very insightful. When this is not enough or you want to get up and running asap, the awesome Jordi Montana offers training courses and also several premium support plans –> https://dynamicsvalue.com/

When any of the XRM/CRM standard organization messages is not already included out of the box, we can easily add support for them by ourselves by registering a message executor instance of a simple execution mock delegate.

Note: when we attempt to execute an orgnization message which is not supported (read: no message executer is registered for) FakeXrmEasy throws a very charming not implemented exception.

CRM Online does arbitrarily reduce batch request size / How to intelligently adjust batch size of ExecuteMultipleRequest when request limit is hit

As most of you know, yesterday the Microsoft Azure platform and several of its services/resource types like VSTS and Dynamics 365 were affected by outages and connectivity issues.

I have been affected by this since I was performing regression and penetation tests on a Azure-hosted integration system to a Dynamics 365 system which had its go-live last week. The aforementioned outages manifested as miscellanous network connectivity issues/timeouts and several CRM organization services not responding.

However, one effect caught my attention:

In our custom developed CRM/ERP integration system we make heavy use of ExecuteMultipleRequests and thus, of course, know all restrictions, particularities and limitations inside out. Especially regarding the maximum batch size (ImportSetting.BatchSize), I thought to be aware of. To be more precise, CRM Online by default has a fixed limit of 1000 Organization Requests that may be — simply said — “bundled” into a single ExecuteMultipleRequest and then are executed together in a single physical request to the CRM organization.

Yesterday however, I noticed that many ExecuteMultipleRequests suddenly began to raise service faults saying that the maximum batch size is exceeded which turned on the alert lamps in my mind. Since I conducted the tests and the used fixtures/data by myself and thus know them, and the fact that we used a maximum of 999 requests per ExecuteMultipleRequest, I could surely exclude the reason for these faults to be on our side.

This leads me to the conclusion, the Microsoft could automatically decrease the maximum batch size limit in Dynamics 365 Online organizations in situations where they need to reduce pressure/resource consumption in their cloud landscape. I did not find similar reports on the interweb yet, but will keep this in my backhead for clarification at given time.

Did you encounter a similar behaviour?

Solution

As a solution I wrote a simple proof-of-concept for a mechanism that “intelligently” lowers the number of organization requests put into a single ExecuteMultipleRequest when it attempts a service fault related to MatchBatchSize transgressions. Funny detail: It needs no privileged user account for retrieving a deployment setting but instead uses the “MaxBatchSize” value contained in the service fault detail data object.

Howto: Attribute fields in detail window of workflow action “Update Record” not editable

Let’s end the week with a tricky issue found even in latest Dynamics 365 / CRM.

Once upon a time, there was a system workflow action called “Update Record”. When users added it to their workflow and opened the detail window for inserting value placeholders, certain fields were mysteriously greyed out so users could not edit them. Coincidentially, the fields being disabled, were exactly those, that have been configured as “read-only” on the entities main form. So obviously CRM is re-using the entities main form definition for building up the detail window of said standard workflow action.

Side note: CRM does this not only for the said “Update Record” workflow action, but also on several more places too, like e.g. bulkoperation distribution window and the outlook client dialogs.

Problem:

Fields marked as “read-only” on entities main form are not editable in detail window of standard workflow action “Update Record”.

Solution:

  1. Open developer tools of your browser (f.e. press F12 in Chrome)
  2. Pick the DOM element representing the attribute input field you want to edit
  3. Delete HTML element attribute “disabled=true”
  4. Do whatever you want to do
Whilst this is no “real” solution to a “real” problem, it’s rather more a clever workaround for editing the detail window form. Whether this is a bug or just a (too?) consequent behaviour of the read-only option of a form control might be debatable. 

(Undocumented) Maintenance job types of CRM Async Processing Services

In a customer’s CRM2016 On-Premise cluster deployment the CRM Asynchronous Processing Service gets stuck in a more or less periodic interval. Sounds like kind of leakage somewhere … So I started with problem analysis by crunching Windows Server event logs and CRM systemjob logs into co-related timeseries data, visualizing them and then considering typical suspects like auditing CRM maintenance job configurations, custom-developed CRM processes and general MSSQL database state. While doing so I noticed several log entries for a CRM maintenance task with type code “61“. The fact that this task type is not documented in any of the official sources nor Google did yield some search results made me curious.

But first let’s quickly recap the list of officially documented maintenance task types:

1 System Event
2 Bulk Email
3 Import File Parse
4 Transform Parse Data
5 Import
6 Activity Propagation
7 Duplicate Detection Rule Publish
8 Bulk Duplicate Detection
9 SQM Data Collection
10 Workflow
11 Quick Campaign
12 Matchcode Update
13 Bulk Delete
14 Deletion Service
15 Index Management
16 Collect Organization Statistics
17 Import Subprocess
18 Calculate Organization Storage Size
19 Collect Organization Database Statistics
20 Collection Organization Size Statistics
21 Database Tuning
22 Calculate Organization Maximum Storage Size
23 Bulk Delete Subprocess
24 Update Statistic Intervals
25 Organization Full Text Catalog Index
26 Database log backup
27 Update Contract States
28 DBCC SHRINKDATABASE maintenance job
29 DBCC SHRINKFILE maintenance job
30 Reindex all indices maintenance job
31 Storage Limit Notification
32 Cleanup inactive workflow assemblies
35 Recurring Series Expansion
38 Import Sample Data
40 Goal Roll Up
41 Audit Partition Creation
42 Check For Language Pack Updates
43 Provision Language Pack
44 Update Organization Database
45 Update Solution
46 Regenerate Entity Row Count Snapshot Data
47 Regenerate Read Share Snapshot Data
50 Outgoing Activity
51 Incoming Email Processing
52 Mailbox Test Access
53 Encryption Health Check
54 Execute Async Request
49 Post to Yammer
56 Update Entitlement States
57 Calculate Rollup Field
58 Mass Calculate Rollup Field
59 Import Translation
62 Convert Date And Time Behavior
63 EntityKey Index Creation
65 Update Knowledge Article States
68 Resource Booking Sync

Source: MSDN asyncoperation entity type

After doing some quick deep dive into miscellaneous assembly sources I unveiled the description of CRM maintenance task type 61:

61 Check FullText Column Status