Of late, I have needed to test against Microsoft’s Active Directory - so I setup a virtual machine hosting Windows 2003 Server and configured as a domain controller. The domain controller is connected to small number of machines (also VMs) on a private network - pretty much what you would expect for a tightly controlled test setup.

One surprise in the experience was the difficulty in getting the domain controller (and DHCP setup) properly working. Admittedly, setting up a domain controller is a somewhat unusual task. As a rare task perhaps not so important to Microsoft as a process that needs the user experience optimized. Still, given I have long been well-acquainted with how this all works at a network level, I ended up wasting a surprising amount of time trying to find the right knobs to turn. (Much searching with Google to get a key Microsoft product to work … the irony.)

Oddly, all the nice GUI interfaces to administering a domain server actually make the process more obscure than the equivalent activity on Unix (where I am accustomed to configuration kept in text files). I do now better understand my past conversations with network administrators who did not know how to do (what I considered) simple tasks with Microsoft’s domain controller … and why they often got it wrong.

Since my aim was testing, I wrote programs to populate Active Directory with large/variable numbers of users and groups, and to cleanup after. Though you could make direct Active Directory or LDAP calls, I find Microsoft’s ldifde utility is quite useful for this purpose. Note that Active Directory uses tombstones to support distributed replication (which is good), and that scripted changes will leave behind many tombstones - which may perturb your testing. You can limit the scope of the impact by changing the tombstone lifetime - the period of time tombstones are kept before cleanup - from the default down to 2(?) days. (Finding and changing that single setting ate up the large part of day.)

(Had a flashback to the old original text-based Adventure game. “You are in a maze of twisty passages, all alike.”)

Another approach is to take a snapshot of the VM when Active Directory is in a “clean” state, and restore the snapshot later. In principle this could be simpler/faster than the scripted restore (using delete operations fed via ldifde). Found a quite thorough article on virtualizing Active Directory, and ran across the USN ROLLBACK problem.

To aid in distributed replication, Active Directory keeps a count of local changes (the USN number) to track replication of remote changes. All quite usual and normal for this problem domain. Given that Active Directory has been around for a decade, you would expect the obvious use-cases to be covered, but not quite….

Turns out if you distribute changes among more than one Active Directory controller, you have a problem in the not-unusual case of taking a backup (or a snapshot) and restoring it later. Thus the many items returned by the above web search, including developers at Microsoft.

Um … what?!? Seems to me this is a pretty common case. That Microsoft would have got this wrong in the first version back in 2000 is only slightly disappointing. That this is still a problem a decade later is very surprising indeed.

Given that I may for future test purposes need to add a second domain controller VM, this looks to be another problem to avoid.