11 weeks, 1000 computers and Windows 7

April 11th, 2013 at 7:51 AM  No Comments


I moved back to Colorado and took over two call center sites in September. Just as a little background I work for a outsourcing company that manages call centers for other companies (amongst many other things).  It provides some IT challenges since we use the customers software on our machines alongside our internal software to run the business.

I received notice a few months after I took over these sites that our client wanted all of our desktop machines upgraded from Windows XP to Windows 7 by “April 15th” (we will get to the quotation marks later).  Half way kidding I told my boss we would get it done in 60 days.  I thought the 90 days that we were given was more than enough time even running as lean as we do, with two desktop technicians supporting a site of over 1000 desktops.  I had built XP images and rolled it out to 800 machines in less than 60 days.  90 days for Windows 7 should be a piece of cake.  I was in for a big surprise.

First 30 Days

The first 30 days consisted of upgrading the 3 domain controllers from Windows Server 2003 R2, to Windows Server 2008 R2.  We also built a new WDS server since our old WDS server was also still running Windows Server 2003 R2 and built the base image and did internal testing.  We built new group policies and WMI filters to make sure these new policies only affected the new machines.  Everything was smooth sailing.  I sent my boss a status email outlining our plan of completion by “April 15th”

Week 5 and 6

Over the next week we rolled it out to about 5% of the first two business unit groups (about 30 desks) and made a few small tweaks to the GPO’s and the image.  The next week we rolled it out to the remaining machines in those two groups, and like that we were sitting at over 60% of our machines completed.  Proud of our status and since we met the goal I set for the first two business groups I sent an update to my boss reassuring him we were on track for the “April 15th” deadline.

Week 7 and 8

After completing the first two business groups, we had to slightly modify the group policies for the next two business groups.  Luckily we didn’t have to modify the image at all so it went pretty quickly.  We took the two weeks we had slated for these last two groups (around 100 machines) and made sure it was done right.  We also pushed it to another 140 training machines.  Again I sent my boss an email with our status and reassuring him of the “April 15th” deadline.  This time I got an email back that we needed to get it done by April 7th.  When I asked my boss why it changed he told me the date had always been April 7th, and he wondered why I kept forecasting completion for the 15th.  I am not sure where I got the 15th from or why he didn’t mention it before now but hey, 1 week won’t make THAT much different, right?

The final 3 weeks

The final 3 weeks we had the least amount of machines to deploy but we had to modify the base image with additional software and when we rolled it out we also had to use Symantec PGP WDE (I hate this software) to fully encrypt the station.  This adds about 3.5 hours to each machine setup time.  We got the image modified in week 9, rolled it out to a test group half way through week 10 and then finished rolling it out to the support team, supervisors and managers in week 11 and finished with 12 hours to spare on April 7th.  Everything was good (so we thought).

The Aftermath

It has only been a few days since we completed.  After the roll out we received some strange reports from everything from corrupt OST files, corrupt Office installs to programs stating they were not valid Windows applications.  None of which that were reported in our extremely short 5 day “soak period”.  It appears the problems were caused by how we encrypted the stations and PGP wasn’t happy with it.  To get up and running in time we had to log in as an administrator, install PGP, start the encryption process and use Windows Fast User Switching to Switch User and allow the users to immediately start using the machine.  We are still troubleshooting and testing but we believe that was the cause.


There are a few things I learned throughout this process.  It was a good experience and I know next time I will do a few things different.

  1. When you are faced with a roll out of this size, take your firstestimate of time and double it just to be safe.
  2. Make sure you always let the image “soak” for a minimum of a week (two would be better) with a test group before you start the roll out.
  3. During the soak period make sure you restart the soak timer anytime you make a change.
  4. Think through the roll out and make sure you setup the machine to soak exactly how you will deploy it to the floor.  There should be no variation.
  5. Make sure you get your boss to respond to any deployment plans that have to do with time frames.  Silence is not golden.

Why lazy sysadmins and IE 6 make the net unsafe

January 16th, 2010 at 11:14 AM  3 Comments

The number of businesses still using Internet Explorer 6 is painful to see. Coupled with the fact that all of them are on Windows XP or Windows 2000, it turns from pain into terror, especially when it comes to security.

For a lot of system administrators, the reasons to stay outweigh the reasons to upgrade. Websites that break, plugins that won’t load, old software that isn’t updated anymore. Trust me, I’ve been there. However, a lot of it boils down to lazy and poor practices of system administration.

Yes, you’re lazy and you’re bad at your job. Internet Explorer 6 was released in 2001. Yes, 2001, most of us don’t even drive cars that old, let alone unleash people on the “information superhighway” with a browser that old. It was designed at a time when security was not the issue it is today. It was designed to work on operating systems like Windows 98 and Windows ME. Would you let people use Windows ME on your network? No! So why are you letting them use a browser that was built for it?! (more…)

Microsoft highlights MED-V features for Windows 7

January 7th, 2010 at 1:33 PM  No Comments

Microsoft Enterprise Desktop Virtualization (MED-V), is a component of the Microsoft Desktop Optimization Pack (MDOP) for Software Assurance customers. It allows administrators to provide a virtualized desktop image to users and manage them from a central console. The upcoming Service Pack 1 for MED-V will expand support for Windows 7 (both 32 and 64-bit) as a host platform. Considering most large companies held off on upgrading to Windows Vista and opted to wait for Windows 7, this technology will help boost the migration since they can do so and run older programs that may no longer be supported or have not been certified for Windows 7.

In V1 SP1, MED-V continues to employ Virtual PC 2007 as the virtualization engine but unlike the consumer “Windows XP Mode” it does not require hardware-assisted virtualization like Intel VT or an AMD-V to be present in the processor. This allows even those with lower end or older processors to take advantage of enterprise desktop virtualization.

Microsoft has put together a screencast demonstrating running a MED-V workspace using a V1 SP1 client in the Windows 7 environment.

Get Microsoft Silverlight

Natural User Interface & Microsoft Research

January 7th, 2010 at 1:11 PM  1 Comment

The QWERTY keyboard and the mouse have been the primary methods of interaction with computers for a long time. But there is a group at Microsoft seeking to make that a thing of the past. Enter the team at Microsoft Research.

Larry Larsen over at Microsoft’s Channel 9 has a great interview with Bill Buxton, one of the Principle Researchers at MSR and the author of Sketching User Interfaces. It talks about their work with what Microsoft has dubbed “Natural User Interface” and how the multi-touch technologies in products like the Apple iPhone and Windows 7 will eventually become a regular part of computing, as well as new technologies like those in the Xbox 360 Project Natal.

Get Microsoft Silverlight

Microsoft CES keynote fails to excite

January 7th, 2010 at 8:28 AM  No Comments

If you couldn’t get a chance to watch the Microsoft CES pre-show keynote last night, you didn’t miss much. If you were actually at the event, I feel sorry for you, it must have been hard to stay awake.

After starting late due to power issues (which fried one of the Microsoft demo units on stage) the keynote got off to a rather boring start with Steve Ballmer, Microsoft CEO, giving various statistics about how well recently released products like Windows 7 and Bing are doing. For the first half hour, the audio stream for the webcast was so bad, it kept cutting out and then required constant volume adjustment. Note to Microsoft, hire a decent sound engineer next time.

If you’d like to watch the keynote for yourself, you can see the saved version on the Microsoft website.

It was all pretty much downhill from there. The much discussed “Courier” tablet that many in the tech press was excited they would announce never came, and there were no details about Windows Mobile 7… at all. Only “we’ll have more about mobile at Mobile World Congress.” So overall, the keynote failed to deliver much of anything that we didn’t know or have not seen already. But, here is a breakdown of what was covered, after the break.


Snow Leopard lacks security features present in Windows Vista/7

September 17th, 2009 at 10:23 PM  2 Comments

Random_Access_MemoryNoted Apple security analyst Charlie Miller, author of The Mac Hackers Handbook and two-time winner of the Pwn2Own hacking contest has said, in an interview with TechWorld, that the latest version of Apple OS X (10.6 AKA Snow Leopard) lacks full and proper implementation of memory address space layout randomization (ASLR). ALSR is a technology, present in Windows Vista and Windows 7, that randomly assigns data to memory to make it difficult for attackers to determine the address of critical operating system functions being stored in memory, and therefore making it harder for them to create exploits.

“It’s the exact same ASLR as in Leopard, which means it’s not very good,” Miller said, “Apple didn’t change anything. I don’t understand why they didn’t. But Apple missed an opportunity with Snow Leopard.”

When OS X 10.5 (Leopard) was released, Miller and others were critical of Apple not fully implementing ASLR. While there is ASLR present in both Leopard and Snow Leopard, they fail to the heap, the stack and the dynamic linker, the parts of the operating system that are most open to attack. Linux also has what many consider a weak implementation of ASLR since kernel version 2.6.12, although some distributions include better ASLR then the stock kernel based on third party code.

Miller did say that there are elements of Snow Leopard that show Apple did do some things to improve security, most notably the inclusion of data execution prevention or DEP, which utilizes both processor-hardware and software based security programming to help prevent buffer overflow attacks by blocking code from running in memory spaces that’s supposed to contain only data.

However, Apple may be late to the game with implementation of DEP, as it has been present in Windows operating systems since Windows XP Service Pack 2, with further refinements made in Windows Vista and Windows 7.

By incorporating both technologies, Miller says it becomes extremely difficult to craft memory attack exploits. “If you don’t have either, or just one of the two [ASLR or DEP], you can still exploit bugs, but with both, it’s much, much harder. Snow Leopard’s more secure than Leopard, but it’s not as secure as Vista or Windows 7.”

Attend a Microsoft launch event and receive a free copy of Windows 7

August 22nd, 2009 at 6:54 PM  No Comments

Microsoft is running a launch campaign for Windows 7, Server 2008 R2 and Exchange 2010. The campaign is called “The New Efficiency” and is focused toward IT Professionals and Developers. There are three different tracks that you can take at this event, Windows 7, Server 2008 R2 and Exchange 2010.

There are 25 events across the country so I am sure there is one near you. Sorry international folks, Microsoft will not be holding similar events abroad.

For more information or to register for an event, visit the Microsoft event home page.