Comments (0)

By Scott, April 9, 2010 8:35 am

Recently I was involved in a upgrade of SSL at the network level. This whole project has been in the works for several weeks, testing the software, documenting the changes, going before the change board to explain what this will entail. This was and is a huge hairy deal.But everyone and everything was in place to implement the change. My portion of the upgrade was minor since all of our SSL functions are done by the networking equipment in front of our service. The majority of all the changes needed at the application level were done in about 20 minutes.

The problems began when we had to test against some of the browsers that are used by organizations that support the users. Most of the software that myself and my coworkers use are modern browsers and operation systems, Firefox 3.x, IE 7.x-8.x. But as always there are those outliers. During our testing we encountered a major support organization that was still using IE 6. This seems very strabge since the company that I work for is very strick about keeping software current, I seem to be upgrading my tomcat about every 4-6 months. So it is strange that a large support organization was and still is using old outdated software.

Well needless to say we had to call off the upgrade of SSL because of that. This whole episode wasted 4 hours of mantenance time all because of IE 6. Why in an organization the size of this do we continue to used such crusty old moldy exploitable software? The world may never know.

Comments (0)

By Scott, August 12, 2009 2:04 pm

Completing documentation is never high on the priority list of any system admin. This is usually the last thing that is done after the product is in place. Sometimes it is done weeks or even months after the install is finished. It is the vital part of the project that the program manager never puts into the Gantt charts. If you are like most admins I know, the only time you document something is because you are tired of explaining the process to someone else or are forced to by your manager.

In either case because even though it is not part of the project does not mean you can ignore it. On my current project we are expected to document, but as usual we are given no time to accomplish that goal. Here are some ways to block that time and get your documentation done.

Add the time you will need to do documentation into the schedule

This sounds easy but in reality it is the hardest thing to do. From working in the computer field for 20+ years, I can tell you that telling your customer that you will need X hours in addition to the time to install to do documentation, will be a hard sell. Remember your customer could not only be the client, but could be your manager, or your PM. The key to this, is to sell them on the benefit that documenting this process will bring to them. This could be many things, faster installs in the future, better problem resolution, not being reliant on a single person (you) for problem resolutions, lower cost of training. The reason will have to be tailored to the audience, but to get time to do the documentation you will need to show some benefit for what in the past they were getting for free.

Add the hours to your estimate of the time spent on the project.

This may sound a lot like the previous point, well it is. The main difference is in how it is presented to the client. There are going to be clients that want a breakdown of every minute you are working on a project, and there are some clients that just want to bottom line, How long will it take?. In that instance you add your documentation time into your estimate for completion. If you know it is going to take 8 hours to complete the project, and you guess it will take 2 hours to document, then 10 hours will complete the project.

Is there a documentation Genie?

Um.. no. After you get buy in from everyone, then you just have to slog through the process of documentation. One of the best things you can do is to take lots of notes during the install process, NEVER rely on your memory as a note taking apparatus. You will forget things, usually minor things, but they will make your documentation incomplete. Set aside some time that you will not be disturbed. Writing documentation means you are reliving your install process and you will need a quiet time to process your thoughts and get them in the right order.

Writing documentation is never fun, but keeping a well documented process will help in the long run.

Comments (0)

By Scott, July 28, 2009 5:19 pm

I have a couple of status meetings that occur at the that same time every week. I also have monthly status reports that need to be written at the end of the month. I am expected to update a daily “shift log” even though I do not work a shift. Even with all this reporting, all the meetings that I am required to attend, it seems nothing gets communicated.

At one of these weekly meetings, someone from one of our datacenters was describing a networking project, a project that seemed to have been in the works for sometime. This project was going to effect my servers and I was told that I have 2 days to prep for that change. I was also told that it would not effect our servers, but I knew better.

The time came for the change and like I expected it crashed our server. When I recovered the boxes, they could no longer communicate over multicast. That change that would not effect us, required us to reengineer a solution on our end.

Next time guys, give us more than a 2 day heads up and lets get some communication to make sure your changes will not screw us over.

Comments (0)

By Scott, July 14, 2009 9:05 pm

Today was one of those days where things came from several directions. One of those things that carried over from yesterday was having our XMPP messaging server launch a chat window from a link on a page. I have been searching for some solutions, and none seem to be elegant. Most are like this from Ralph Meijer, it is a solution but requires a modification on each browser,  not practical at our current client.

So, for now, it seems our only solution is to work with the web team to launch a flash client rather than a hard client. I will keep searching for a more elegant solution, so check back often.

Comments (0)

By Scott, July 13, 2009 11:14 am

I had to do a simple update to a xmpp messaging server this past week to fix a problem with IE hanging on a close of a ajax window. The problem can be described here, here and here.  Basically when we open our flash IM window and close that window within the same browser session, a new launch of the IM window would not occur.

It was suppose to be a quick update, tweak one parameter to lower the BOSH timeout and restart the tomcat server. It should have taken 10 minutes, maybe 20 with testing, to complete this update. I left the office two and a half hours after I started the update. Seems the networking guys were doing some firewall work that “should not have any effect” on my work, but rather than pointing the VIPs to one data center at time, well they pointed the external DNS name to both data centers.

So I wasted about 2 hours chasing ghosts. Sometimes work is just too much work. Oh by the way.. It looks like the update did not work