Unfortunately, and as far as I can see, there is no straight forward way to do bulk records deletion on Dynamics CRM 4.0 using the out of the box functionality and interface of Dynamics CRM 4.0.

To bulk delete records in Dynamics CRM 4.0, you have the following main options:

• Get a third party tool or CRM add-on to bulk delete records. This option is a straight forward one but you might have to pay for purchasing or using the tool. It may also have security issues. I would not recommend it to my clients as most probably the tool is created by a small company or an individual which I don’t know. Hence, it will be rather difficult to put this tool on a live Production environment or client server. Let alone adding it to CRM Online or to a CRM hosted solution by a partner.
• Use CRM SDK to write a .NET application (or a .NET console application) that will run and delete all records for a specified entity or entities. This is a more robust way of doing it, but it may take longer time and is probably not suitable for people who do not come from .NET development background.
• Use Scribe Insight. This is what this post is about really.. Using Scribe Insight to bulk Delete Dynamics CRM records.

Please Note: This is a work around. It is not supported by Scribe and the advice in this post is provided as is with no warranty. I have tried it and it works perfectly but can not guarantee it will have the same acceptable results in any other environment.

Here is what you need to do:

1. Create a new Scribe workbench DTS (or Job). Point to your usual source file (even a sample one) and point to CRM: either IFD Forms for hosted CRM or direct connection.
2. Configure the targe: Create one delete step on the target.
3. Make sure that the option to “Allow multiple record matches on updates/deletes” is ticked under the All steps tab.
4. Under Step control tab, leave failure to go to next row but change all the success records (Success (0), Success (1) and Success (>1) ) to End Job. Select success radio button at the bottom and write a message to your log such as: “All records Deleted”.
5. No Data links are important as you are only deleting.
6. On the Lookup link, just make the lookup condition impossible. Such as: where Account Name = 123456789 or whatever.
7. Run the DTS.

The Job will read the first source line. Will then try to find this record at the target (remember it is update/delete). Since we have setup the lookup link to look for something “impossible to find”, the result of the update will be Success (0).

Once this happens, Scribe will go and delete all records for your chosen entity (or CRM table). This will be a complete bulk delete of all CRM records using Scribe.

Remember, it’s a work around… that works.

Again, I have looked online and could barely see any information or help in this regard (probably my mistake as I didn’t search properly!). I found out that the key for moving Scribe project files including source across to a new location or a new server, the key is always the QETXT.ini file. This file is vital for pointing the DTS to which source file it should be looking at. QETXT.ini files have all the field names, so mapping S1 to the name that you have chosen for the first source field. It also has the source file name, the ODBC name and the table name. From there you can do almost everything.

When you move the files across, you will obviously need to re-point to the new source location, but by editing QETXT.ini, you will be able to put back all the source (S1 to field name) mappings, point to the new source file name, ODBC name and everything else.

This has proved very efficient and have worked now with my whole deployment.

One more important piece of advice, always try to get a dedicated folder for every source file. So if you have more than one DTS jobs, make sure that the source for each DTS job is in a separate dedicated folder. This will ensure you have separate QETXT.ini file for each one of them, hence, you can easily update the information inside it. It will still work with one large QETXT.ini file but it’s always better to separate the sources and their associated QETXT.ini files. You can always manually split this file into source specific files and put each source in a separate folder later on (which is what I have done after ”inventing” this best practice of having separate source folders!).

Have you ever looked in Scribe Insight for a way to rename a source file before processing it. Scribe console creates collaborations where integration processes can be configured so that they wait for a file to be added to a specific location and then run a specified job. Once the file is added the job will run and process the file. Also, Scribe DTS jobs can only be setup to process a source file that has its name always fixed and unchanged. So a DTS can be setup to process a source file named: customersdata.txt. It will never run if another source file is added to the location the Scribe console is looking at. In this case, if you get a source file with the date (and time) stamp in its name will need to rename it so the the DTS can detect it and run. So if the source file comes with time and date stamp that varies every day (for example: customers_1453_21092009.txt), you will need to rename “customers_1453_21092009.txt” to “customersdata.txt” only to get the DTS to work.

After some research, I found out that you can only do this using pre and post processing commands step of the process. Every integration process has step 2 in it called pre and post processing commands. In this step, you can specify a pre-processing and post-processing commands or scripts. This feature lets you specify a pre and post processing file that can do this renaming for you. Accepted pre and post files are: *.vbs, *.js, *.vbe, *.bat, *.cmd, *.exe, *.com

You will then create a pre-processing script that finds all files that start with “customers*” in our example and rename it to customersdata.txt which is the source file name the job is expecting. Post processing can be to rename the file to something else so that you keep a record of processed files.

In step 3 of the Integration process, Scribe also gives you two options (in the form two check boxes) that I find very useful. You can either select to delete the source file after processing or you can select to rename it. So the source file will be processed and renamed to something like customersdata.L1.txt. Unfortunately Scribe doesn’t give you the choice to choose the new name of the file. Hence, you will need to write a post processing script again for renaming it afterwards if you are after a specific file name.

I’m not sure if there is any other way of renaming source files (also called event files by Scribe) before processing them or specifying a new name for them after processing.

The source in this case is SAP. The target is Microsoft Dynamics CRM. I am using Microsoft Dynamics CRM Scribe Adaptor.

I’m not an expert in Scribe (as yet) and I wanted to do a simple step. I wanted to get the GUID of a record in Microsoft CRM entity using a lookup field value.

The whole Scribe step is just an update to the Account form. I have the Contact name and I need to get the Contact GUID to input this GUID value (pointer to the Contact) into the Account form primary contact field. This is an N:1 relationship between Account and Contact.

So if the primary Contact name of an Account record is John Smith, the question was: how can I get the GUID of the John Smith contact to update the Account record.

Thanks to my colleague John Ball, the answer is to use the simple DBLookup function (formula).

To do that: Select the target Scribe CRM adaptor field. In my case this is custom_account_primarycontactid. Select it and then click formula. In the formula window look for the DBLooukup function. You can use the DBLookupCached function for an enhanced performance if your values are not changing before or after in any of the steps.

The formula should be something like that:

DBLOOKUPCached(S2 , “T”, “contactentity”, “contactname”, “custom_account_primarycontactid” )

That should be it. It’s quite simple.

The quick answer is yes. SCRIBE is a good application that can be used to provide an integration between Microsoft Dynamics CRM and SAP.

If you want the report I created studying the strength of SCRIBE and the possibility of using it to do such integration, please request it via a comment on this page and I will email you the document. I will also email you a technical specification document that SCRIBE has sent me which details such integration. This technical document is not publicly available on their website as far as I know but you can always request it directly from them.

My technical report/document also has some examples of case studies in which Microsoft CRM system has been integrated successfullywith SAP. It also lists some white papers and technical documents/documentation that has covered the subject in general and the specific integration between SAP and CRM using SCRIBE.

The document does not include any reference to the client, the exact project specification or any information that could be confidential. It’s just simple facts and findings on SCRIBE and the possibility of using it for SAP and CRM integration.

*****  Updated 18/02/2010:

The document is now available on Scribe Insight blog as a guest blog post: http://blog.scribesoft.com/2010/02/guest-post-crmsap-integration-using-scribe.html  .. 

I can still send you the document if you want, just request it via a comment below please.

****** Mohamed Ibrahim Mostafa

I thought about a list of general questions that most (if not all) consultants working on any integration solutions will need to have complete answers for before starting the design phase, let alone the development phase of the project.

In my opinion, the list of questions are as follows (not in real order – just a braindump!):

• How many environments do you have? Development, Test and Live? (recommended) or is the project is still in development so you can use live environment for development? Where will be the test environment later on?
• Is this a direct integration or in-direct integration? Is this an instant, event driven integration or a periodic scheduled integration between two systems? Are their queues for data to be migrated?
• What backup and restore operations can you do? The ability to Backup and restore data is vital.
• What integration application or tool are you going to use or is available? SCRIBE, SQL Server Integration Services, Web Services (Microsoft .NET Web Services), console applications, plugins? What SDK will you need? CRM SDK and CRM API for example?
• The Environment structure: How many physical servers? Where are these servers located? Where is the integration tool or application installed?
• How and When can I get access to the environment? Access to all servers is required including access to all databases and to all applications. For example: Access to Microsoft Dynamics CRM application (via webclient) is essential to confirm that data imported to a CRM has been migrated successfully.
• What type and format of extracts and data imports? CSV (Comma separated Values), XML, i-Doc, sql flat files, batch files, etc…
• Where will the extracts be imported? directly using the tool or via an FTP server? Is an SFTP server required?
• Are their duplicates? If so where, and what classifies a duplicate?
• Are there data entry standards for each application in the overall integrated system?
• Are there fields that are required in each system part of this integration?
• Are there fields that aren’t used?
• Are there any fields with null values?
• What relationship does the data has? are there fields which are dependant on others?
• What are the primary and forign keys of all tables in each system that will be part of the integrated system? Any field that does not allow null, business required (and preferably business recommended) must have a data upon migration (Defauls can be used then).
• Overall high level mapping between the different systems.
• What is the value, length, and format of fields/columns in the source system? What is the corresponding value, length and format in the target system?
• Are there any Pick Lists? A cross reference is required to map source and target values.
• What Data validation is required and is acceptable by the client and the project stakeholders?
• Differencing: What are the business rules for differencing? What data does not need to be updated and when? what data is needed to be updated based on the business requirements?
• using Default values for all required fields and columns in the target system to avoid causing any errors.

This is the list I have thought of so far. I will keep on updating this list as and when I think of something important that needs to be considered.

Let me know if you have any comments or feedback on these questions and tell me whether or not are these questions helpful.

Thanks for reading.