Importing Data into the MSCRM Application - Part 2

There are several data exchanges for bringing data into MS CRM periodically or just having it access external data storage. Based on your business needs the best approach for data migration should be determined:

1.    Native MSCRM Imports & Data Migration Manager can be used when the data load is less and business logic is minimal. This can be used for one time migration and for periodic end user imports.

2.    Custom application tools can be used when the data load is moderate and to accommodate the business logic through extension.  This will be used when the number of entity objects is less in number for first time and ongoing migrations. These tools can be automatically started via a job scheduler.

3.    SSIS can be used when the data load is huge and to accommodate the business logic through extension. This will be used when the number of entity objects is more in number and for during the first time and ongoing migrations. The SSIS jobs will be created and scheduled to run for the day to day migration. 

InfoStrat, a Microsoft Federal Partner that does a lot of IT consultation to U.S. government has published a white paper on Data Migration & Systems Integration for MS CRM last month. You can access it here. It evaluates some widely used tools for specific customer considerations of data migration and systems integrations: the Biz Talk Server, Scribe, Simego Data Synchronization Studio, among others.

You can utilize a service best suited for your needs especially if you need to consolidate data from multiple sources, cleanse it, transform it and load it into the MSCRM. The most common data migration functionalities desired by most customers but not limited to, are:

• Cleanse the data to match the data with CRM entity objects
• Creation of data maps for entity objects
• Data mapping of source objects with MSCRM entity objects.
•  Duplication rule for entity objects during data import.
• Implementation of functional business rules during the imports.
• Microsoft supported way of associating the relationship between entity objects during the migration.
• Automated jobs for day to day integration.
•  Auto association of CRM owners with the respective CRM entity records during the imports.

You will be completing the task of importing sample data into the test environments, designed per you customer’s data requirements during this 'Design' phase.

Update:

This sample data import also assumes the import of business units, users, teams to the MS CRM application. Besides, Email router or Outlook configurations will make sense only when you have some sample data in your application though we saw them in the 'Analysis' Phase to review all the installation related configurations together.
My friend Andy Schultz explains the CRM Security Model thoroughly and how you can stretch its capabilities to work for your business situation, in his blog here. Kudos to his storytelling writing style!:)

Importing Data into the MSCRM Application - Part 1

Importing data marks the beginning of the next phase, 'Design' in the MSCRM Implementations:

Data Import Wizard:

Import using the MSCRM 2011 Web application tool ‘Import Data Wizard’ is the standard import utility tool on native CRM. You use this tool to import data records from one or more comma-separated values (.csv), XML Spreadsheet 2003 (.xml), or text files. Zip files can also be imported but all files have an 8MB size limit for import.

Here is a blog link with screenshots that explains CRM 2011 Data Import Wizard for the following types of Import data:

·                Import data for any entity from a file

·                Import data and create new entity and new fields,

·                Import attachments with notes and do special imports.

To implement data import, you typically do the following:

·         Create a comma-separated values (CSV), XML Spreadsheet 2003 (XMLSS), or text source file.

·         Create a data map or use an existing data map.

·         Associate an import file with a data map.

·         Upload the content from a source file to the associated import file.

·         Parse the import file.

·         Transform the parsed data.

·         Upload the transformed data into the target Microsoft Dynamics CRM server.

You can import data from one source file or several source files. A source file can contain data for one entity type or multiple entity types.

Parsing, transforming, and uploading of data is done by the asynchronous jobs that run in the background.

  

Additional Capabilities:

MS CRM Web services provide additional capabilities that are not available in the Import Data Wizard:

· Create data maps that include complex transformation mapping, such as concatenation, split, and replace.

· Define custom transformation mapping.

· View source data that is stored inside the temporary parse tables.

· Access error logs to build custom error reporting tools with improved error logging views.

· Run data import by using command line scripts.

· Add LookupMap XML tags in the data map to indicate that the data lookup will be initiated and performed on a source file that is used in the import.

· Add custom OwnerMetadata XML tags in the data map to match the user records in the source file with the records of the user (system user) in Microsoft Dynamics CRM.

· Use optional validation checks. Validation is not optional in the Import Data Wizard.

ISV Solutions:

To meet custom code development, Data Import utilities may also be custom developed and utlilized by ISV's.  A spreadsheet of CRM 2011 entities for Data Import can be found in the Sure Step templates and documents, for detailed Data Planning.

Then there are third-party solutions for many data migration needs that simply cannot be covered by the native CRM wizard.

Below are a few examples and you can evaluate which one best fits your needs.

· Scribe Insight

· InaPlex Inaport

· CRM Extensions Import Manager

· Starfish ETL

· eOne SmartConnect

· QuickBix Integration Suite

Blogger Joel Lindstorm  lists some lessons learned migrating data to MS Dynamics CRM 2011 in his blog link. and here's another link

Outlook and Email Router Configurations

Outlook CRM Installation and Configuration in MS CRM 2011

Installation files can be found in this location, here.

In addition, latest rollups will need to be applied after installation to ensure all hotfixes and performance enhancements are in place.

For an on-premise installation there are two steps to complete the Outlook for CRM besides setting it up on the Outlook itself:

1. Setup to install the files on the computer.

2. Next, you run the Microsoft Dynamics CRM Configuration Wizard to configure the application and complete the installation.

Installing MS Dynamics CRM for outlook using command prompt is explained here in this msdn library location that details silent commands & scripts to help roll it out to all users.

Outlook Client Configured =dword:00000001 gets created/set for users whose outlook is configured under [HKEY_CURRENT_USER\Software\Microsoft\MSCRMClient] and not for those who are not configured, and this information can be utilized for configuring outlook using Microsoft System Center Configuration Manager or other similar products that enable administrators to effectively control deployment and security.  

The next question to ask is if your customer will need their CRM contacts to not synchronize with their outlook contacts, because by default CRM synchronizes My Contacts to Outlook. Now users can create additional filters to expand the range of contacts that they synchronize, and it is possible to do this for all users. The quickest way to accomplish this is by creating a System Filter. However, if the answer does not care that their CRM contacts synchronize with their Outlook then no further changes are necessary after initial setup.

There is ready-to-use toolkit that is a collection of custom workflow activities to help manage views, filters, filter templates and local data groups in CRM 2011. The toolset contains the following custom workflow activities and should be imported as a solution within MS CRM application:

·  DeleteSystemView: Delete a System View

·  DeploySystemView: Deploy a System View

·  DeployUserView: Deploy a Personal View

·  InstantiateFilter: Instantiate Outlook and Offline filters for a user

·  ResetUserFilters: Reset all Outlook filters for a particular user (online filters, offline filters or both)

·  RetrieveView: Extract information about a System or Personal View

·  SetFilterTemplateDefault: Set/Reset the IsDefault attribute for a Filter Template

Additional information about each tool can be found in this blog (xrm2011.wordpress.com) as well as in the Documentation pages.

You can follow the deployment instructions in his blog for planning deployment for all users.

Email Router Installation and Configuration in MS CRM 2011

Please find Microsoft documents for this configuration located here and that includes troubleshooting tips.

These documents list steps to configure MS CRM 2011 or MS CRM 4.0 e-mail in different deployment scenarios. The scenarios included in these documents focus on the following:

• The Microsoft Dynamics CRM E-mail Router

• The forward mailbox

• The Microsoft Dynamics CRM Outlook client.

You can install the E-mail Router and Rule Deployment Wizard on any computer that is running one of the following operating systems, and that has network access to both Microsoft Dynamics CRM and the e-mail server:

· Windows 7 32-bit and 64-bit editions

· Windows Server 2008 (x64 versions) or Windows Server 2008 R2

Screenshots of the router configuration can be found in many blogs. Here's a link on how to configure the MS Dynamics CRM 2011 E-mail Router and here's a link on how to configure users to use the e-mail router and the various tracking options. Screenshots on installation including configuration within CRM user settings and email router can be found here.

Training Sandbox Environment Installation Pre-Requisites & more

MS Dynamics CRM Installations for creating Training Sandbox Environments is the next task once architecture assessments and basic performance benchmarkings are completed.

Microsoft offers a thorough list of materials for installation along with lots of preinstall and worksheet documents, within this download location: http://www.microsoft.com/en-us/download/details.aspx?id=3621

The document CRM2011IG_Installing within the MS downloads includes troubleshooting section for known issues during upgrade and installation.
It also explains installation of MS Dynamics CRM Server 2011 on multiple computers to balance the processing load across several servers to increase performance and availability. This can be a desirable feature if the customer source you are working with will experience very high transaction volumes everyday.

Microsoft Sure Step also offers templates for Installation Pre-Requisites checklist/templates that can be used as your guide to prepare for the MS CRM component installations:

The following blog contains step by step instructions with screenshots for installation of the various components, click here for details.

The following Pre-Requisite chart applies for MS CRM 4.0 but a similar quick chart of reference can be prepared & utilized, keeping in mind that current versions will be more recent versions of all the components utilized for the installation.

Installation Pre-Requisites

Item	Details
Hardware and Operating Systems and Service packs	As per Environment Specification.
Internet Access	All Servers should have Internet access to allow the installer to apply additional pre-requisites: ·         SQL Server 2005 Reporting Services Report Viewer control ·         Microsoft SQL Server Native Client ·         Microsoft Application Error Reporting tool ·         Microsoft Visual C++ Runtime Library ·         MSXML 6 This is also required for product registration.
Firewall Configuration	No firewalls should be present between the CRM Application and SQL Servers.  If a firewall exists then it is recommended that this initially allows open access from each CRM Server to the SQL Server for the installation process and then is locked down once installation is complete.  Client access to the CRM Servers is required on Ports 80/443.
Windows Updates	Windows Updates should have been applied in accordance with [Customer Name] standards.
CRM Servers – Application Server Role (IIS and ASP.NET)	The Application Server Role (including IIS and ASP.NET) should be enabled on all CRM Servers in all environments.  It is assumed that the default web site on Port 80/443 will be used for the CRM implementation.  Any specific requirements different to this should be provided prior to the installation.
CRM Servers - ASP.Net 3.0 redistributable	ASP .Net redistributable
CRM Servers – Indexing Service	The Indexing Service should be set to automatic start and should be running on all CRM Servers.
CRM Servers – SMTP Service	The SMTP Service should be set to automatic start and should be running on all CRM Servers.
SQL Server 2005 Enterprise x64 SP2	SQL Server 2005 Enterprise Edition x64 SP2 should be installed and configured with Reporting Services (Default Configuration applied), Integration Services and Analysis Services on all environments.
SQL Servers – SQL Agent	The SQL Agent Service should be running in each environment
SQL Servers – Full text Indexing	The SQL Server Full Text Indexing Service should be running in each environment
SQL Server - Domain	The SQL Server must be in the same domain as the CRM Servers for the environment.
SQL Server Service Account	The service account that SQL Server uses to log on to the network must be either a local system/Network Service account or a domain user account. Installation of Microsoft Dynamics CRM Server will fail if the SQL Server service account is the local administrator.
SQL Server Network connectivity from CRM Server	Microsoft Dynamics CRM Server Setup requires at least the Named Pipes network library to authenticate using SQL Server. By default, both Named Pipes and TCP/IP network libraries are enabled when you install SQL Server 2005.
Domain Membership	All Components of each environment (i.e. SQL Server and CRM Servers should be in the same domain).  Details to be provided before arrival onsite.
Organization Unit configured in AD for each environment	Each environment should have an Organizational Unit created to store Security Groups related to that environment.  A naming standard of the following is recommended: \MSCRM\MSCRMv4\<EnvironmentName>.  i.e. ·         MSCRM\MSCRMv4\Integration ·         MSCRM\MSCRMv4\UAT ·         MSCRM\MSCRMv4\Production Any constraints around the creation of a new OU or the capability of the application to modify security groups should be provided before arrival onsite.
CRM Service Accounts	A domain CRM Service Account should be created that has local administrator rights to the CRM Servers and SQL Server and full rights to the Organization Unit and all Child objects (for the related environment). Ie. ·         Svc_CRM_Integration ·         Svc_CRM_UAT ·         Svc_CRM_Production Details to be provided before arrival onsite.
Installation Accounts	An Installation Account should be provided that have local administration rights on the CRM Servers, the SQL Server and full access to the related OU and all child objects (for each environment).  The installation account must also have read access in the AD to navigate to the provided OU for each environment.
CRM Installation Media	CRM should be installed from the official media, to be provided by [Customer Name]as this should include the required License key and local copies of some of the pre-requisites.
License keys	Confirm that the installation set includes a license key for each environment.  If you purchased Microsoft Dynamics CRM through a Microsoft Volume Licensing program, the license key is provided in the license.txt file, which is included in the Microsoft Dynamics CRM installation files.
Client Machines	Sample Client machines (As per standard Build) on the correct domains should be available for each environment.  These machines must have network connectivity to each environment (i..e Port 80/443 to CRM Servers and Load Balanced VIPs).  The URL for each environment should be added to the trusted sites and should be enabled for Integration Authentication in this zone. Access to change IE setting may be required to troubleshoot any unexpected issues.
Test Accounts	A selection of at least 5 user test accounts should be created in each test domain to allow access to the CRM implementations in the local domain. Details of these test accounts should be provided to Microsoft to allow them to map these to specific roles and test scenarios.