Technical Paper
Background
A large generic pharmaceutical manufacturer needed to migrate a subset of documents from their legacy GxPharma solution to a FirstDoc FDQM 5.0 system. Sitrof is a vendor neutral integrator, always looking for the best approach for individual customers. We also believe in building on past successes.
After much research and consideration with a previous customer (See previous case study), we decided to build a custom migration utility rather than use an off the shelf product like Buldoser or FME’s migration-center. That approach proved to be successful and we had envisioned using this custom utility for future migrations.
This particular client preferred to use CSC Configuration Management Utility (CMU) to perform the migration. They had confidence in CMU and had performed a similar migration several years ago. They were looking to adopt a consistent migration approach and toolkit to perform the FDQM migration along with any other migrations needed in the future. While Sitrof had experience using the CMU to move FirstDoc configurations from environment to environment, we had never used the tool to perform a migration to or from a repository outside of FirstDoc.
The FirstDoc solution was already in production, presenting an interesting set of circumstances. Other complicating factors were the GxPharma system was running Documentum version 4.2.8 while the FDQM system was running Documentum version 6.0 SP1 as illustrated below.
The Sitrof Approach
We had hoped to use CMU 5.0 to perform the entire migration. However, CMU 5.0, which uses DFC 6.0 SP1, cannot connect to a 4.2.8 repository. We worked around this issue by using CMU 4.2 to perform the export while using CMU 5.0 to perform the import and any post import processing steps.
A typical E-T-L (extract, transport, load) model for this type of migration would be to export the content, apply metadata transformation rules and then import into the FirstDoc repository. With past migrations, we exported the metadata into database tables which provides an efficient and convenient way to update metadata values based on migration rules (e.g. existing documents with r_object_type of sop will have a document_type value of Standard Operating Procedure).
CMU does not provide a way to write metadata to a database but instead, writes it to an XML file. Fortunately, rather than having to modify values within the XML export file, the CMU provides a means to transform values during the export itself so that the metadata in the resultant XML already has the transformed value. Using the ‘transform’ attribute in the CMU ObjectExporter allows you to call your own JavaScript code where more advanced transformation rules can be applied. Many of our transformation rules were handled in JavaScript code where either the rule was hard coded or read in from spreadsheets that contained the mapping rules.
Once the export was complete, the exported metadata resided in XML and the content resided on a file system. The import process was very straightforward; we just used the CMU ObjectImporter to import the exported XML files. The ObjectImporter provided a handy “save_object_id_map” feature that produced a CSV file that created a map of the old r_object_id to the new r_object_id. This map can be useful if you wish to import audit trail values but need to update the old object Id to the new object Id.
Once the documents were imported, there were several post import steps that had to be performed. These tasks included creating FirstDoc relationships between documents (e.g. an SOP related to a Form), calling the FirstDoc methods AutoNumber (to assign a new document number), SetACL (to apply the appropriate document security based on the document’s type and subtype) and SetLinks (to link the document to the correct folder location). The CMU TaskManager was used to call a series of JavaScript programs to perform these tasks.
Observations
We were pleasantly surprised by the number of CMU options available to facilitate a migration. For this particular migration, 40,000 documents (all versions) and tens of thousands of audit trail records were migrated. While the performance was adequate for a migration of this size, the CMU would not be our tool of choice for a larger scale migration. Also, the CMU would not be a viable option if there was a need to migrate from a non-Documentum based source system. Nonetheless, since most FirstDoc customers already own the CMU it should be considered as an option for your next migration.
