Mike’s World

Today got off to a great start. When I got home from speed dating last night I was going to set my alarm clock, because I had to be at work at 8:00am for a server upgrade (more on that later). Well, I forgot to set my alarm clock, and instead of getting to work at 8:00am, I woke up at about 8:00am. The missing scotch in my house just might explain the extra 2 hours of sleep.

We are working on upgrading one of our environments from AIX 5.1 to AIX 5.2. We did an initial upgrade of the development server in the environment, the application tested out fine, and all was good. Because we have a limited window for the production server, we decided to restore the server from a mksysb backup and do the upgrade again to get some better timings and get some documentation done up.

Well, after sleeping in I send Mandy a quick AIM message to find out the mksysb restore was a complete failure, and that was just the start of this little adventure.  With AIX, you can boot off of CD, Network (NIM), local drives or a bootable mksysb tape. With remote support in mind, we had the idea to attempt to boot off of network and restore our backup off of tape. We booted the server off of the network, and in the server install options, the restore from tape option was missing. We selected what would be the tape option, and it worked. The server started to read the tape and restore the operating system. It turns out after about 15%, the restore failed. We attempted again, and it failed again. Looking at the system, it looks like when AIX netboots, it loads the boot environment in to a ram disk, and during the restore this ram disk filled up. We then gave up on network booting, and booted off of CD to do the restore. This time it worked.  When I left work, the AIX upgrade was going right along, and we should be all set for our production upgrade on Sunday.

This mornings weigh in, 239.4. Yay, I broke 240!

Today has been kind of a trying day for me. I have seen co-workers come and go, it is a part of the job. In todays day and age, you do not stay at a job for 20 years and wait for a nice pension to come up, people are lucky to stay at the same place for 5 years, especially in the IT industry. I have gotten to know the people I have worked with quite well, and have developed some good friendships. A good friend of mine has been looking at a opening that crossed their path, and it looks like they are going to move on to that position. In the end it is a good opportunity. I have lost touch with several people over the years due to job changes, and this is a case where I do not want this to happen this time, as I truly enjoy being around this friend and their family.

So, enough about that, and on to the second half of my day :-)

This evening I gave speed dating another shot. The last time I did this a month or so ago, and did not have much luck. The last time around, the age ranges where 20-30 year old, and this time it was 25-35. I think with a more mature crowd, I might click better with people, and I think I did. Of the 9 people that attended, I think I would not mind seeing 4 of them again, which is better than last time. In fact, I had a great conversation with 2 or 3 folks. A bonus was that the place the event was at had 18 year old scotch :-)
When I got home, I chatted with my friend for a bit, and watched a good movie called Saint Ralph. I decided I could use a nice film this evening as I finish off a bottle of wine I had in the fridge.

With my latest effort to start loosing weight, I figure I will start updating my blog entires with my weight loss progress. Todays count was 240.0lbs, which is a loss of 13 lbs since I have started working at it a month or so ago. I would love to get below 200 again.

Today was kind of a reality check day for me. Today the day started off all fine and good, until I went to read my email.  A friend of mine got laid off from his job today :-(

He has been at his current job for about 3 years, and was the kind of person to stick up for the people around him. I am sure he will be missed by the people he worked with, but it gives him a chance to move to something he might enjoy a bit more.

Today at work was not all that exciting. I worked on writing a script to help detect a Veritas storage foundation bug we ran in to. The bug is that a veritas config file gets corrupt, and the veritas disk ID does not match the OS disk ID.  It is a very annoying bug.

After work I went to my personal trainer, and I had a very good session. I was able to do 10 push-ups, and I can almost do sit-ups all the way without assistance. This may not sound like much, but for me it is a big step. When I lost weight I lost quite a bit of muscle, and I have been able to recover from that. Now that I am working on loosing weight again, it is very helpful not loosing the muscle mass again.

For the evening, I went to my friends house for moral support. We ended up watching two classic movies as we drank whatever was in the house. I was smart and picked up two bottles of coke zero on my way over, so i was set for the evening, as I could use a bit of help relaxing as well :-).

The movie selections where better off dead, and canadian bacon.

Today the plan was to take a half day at work, get up north, and get out of the cities before traffic got to bad. The plan for the most part went as I wanted.

I got a bit of work done this morning, and headed out at about 12:30pm, just in time to get to my 1:00pm appointment with my trainer for a session before heading out for the long weekend. It was nice getting one more workout session before heading out. Got home, all set to get out, then I ended up chatting with a friend for a half our or so while I packed, then once I got all packed up, I chatted with my neighbor for about 20 minutes before getting my father and getting out of town.

We got out early enough to avoid most of the traffic, the only traffic we ran in to was just on the north side of the cities, which is much better then past years. The only exciting thing about the drive up north was that the wind turbine in elk river, mn was just booking. It was going faster than I have ever seen it go before. We got up to the cabin at about 7:30pm or so, and was able to chat with family for a bit before bed.

For some reason, today felt like a real good day.

I sleped in a bit this morning, so I got started off very well rested. At work I got a somewhat complex project completed today, a customer request to consolidate several systems, and I put together a solution that will do quite a bit more than they asked for, which was nice to get taken care of. It was nice getting that finished up before the long weekend. Today was a day that was somewhat relaxed, but still productive.

Today the weather was just perfect. We went out for a walk for lunch, and you could not have asked for a better afternoon for a walk. Perfectly clear Sky’s, nice breeze, not humid at all. After work, I went with a friend to pick up her Son from daycare, which is a walk we have not done in what feels like forever. I can not believe how big her son gets every time I see him, every time he has more personality than before. It is kind of cool seeing someone develop from a infant, to a baby, to now a toddler, just how much a human can adapt and learn is just simply amazing. After we got back with her son, we hung out with her neighbors for a bit, and again a very nice family.

After hanging out with my friend from work, it was off for our normal Thursday night with friends. Tonight was quite a bit of fun. We grilled, hung out and chatted on the patio until about 9:00 or so, which was real nice. We then went inside and watched some tv, and then a movie, serenity.

Icing on the cake, my sister called me this evening, which was real nice, as I had not talked to her in like 2 weeks it seems. I enjoy days like today when I get to spend a good chunk of the day with good friends, it is nice to have the opportunity present its self every so often, and I had forgotten how much I missed it. With being so busy, and vacations, it seems like I had not had the opportunity in quite some time.

This month has been a very interesting one at work. Late last month our customer canceled a large I.T project due to cost over-runs and changing business needs. This project had been very trying to design around both on a technical architecture point of view, and on a infrastructure side of things. The true scope of the project was not known at initial implementation and every time we would start to get a handle on the size of the project, it would grow. We had constant challenges working with this moving target. I think we did a good job at being flexible to meet the needs of the project, in the end due to reasons outside of us, the project is not off of our plate. It is unfortunate, as if they could have made it work, I think in the end the customer could have seen some good cost savings by moving ahead, if they could have kept the initial scope under control. In the middle of the project they acquired another company, and I think rolling that company in to the project might have contributed to some of the scope creep.

As with any large company, any time you announce you are taking a large write-off, the investors want cost control right out of the box. This has caused a complete re-focus on what I am working on. This month has been all about finding ways to save money for our customer everything from large consolidation plans that could save millions to rethinking on how we handle disks which save hundreds of dollars. It sounds like everything is on the plate. We will see in the end what works, and what does not. It is always difficult balancing the supportability of an environment with trying to reduce costs in the environment. I always tend to lean towards higher availability, and therefore more costs, and it is hard for me to change that mindset.

Every year for work, we perform a disaster recovery validation, which allows us to validate our processes and procedures and verify we are able to bring up the critical computer resources in a disaster environment, should our main data center become unavailable. This test is done in a remote physicality in Philadelphia, so it gives us a chance to get out and travel as well.

In past years, I have been the main driver for the test for our team, so previous years this was a very stressful time for me. This year I decided to take a back seat, and let others in the team have more of an involvement in the process. I found that for both prep time and during the exercise, this made things much less stressful for me. In past years, I was very stressed for weeks before the exercise, and until a week after the exercise ended, then I would wind down. This year because I handed off many more of the components to the test, I was more of a participant than anything, and this made things much easier on me, I found I was much more relaxed this year versus previous years.

This year we got out on Tuesday June 20th, with the test starting Wednesday June 21st and running through Sunday June 25th. One thing I hate is that every year, the test seems to fall on the same weekend as the ham radio field day, which is something else I would like to participate in. Tuesday we had dinner at The Capital Grille, which was a very nice. I was very impressed with the scotch selection they had.

Wednesday morning we had to switch hotels. We had been scheduled to stay in the Residence Inn Philadelphia Center City but due to rooms being full, we spent the first night in the Sheraton Suites Philadelphia Airport. After the hotel room switch, it was off to Sungard to start the test. First for us is to get the environment handed over from the sungard tech’s, and get the backup and recovery environment put together. After that, we then do our server recoveries off of tape and disk, and then the end users do testing. All very boring stuff. This year I did not take over the environment build like I have done past years, I more went with the flow of things.

I worked the day shift for the most part, and was able to find time to go out and walk every day, which was very nice. I have found getting out and unwinding helps me quite a bit. I saw quite a bit of downtown Philly, just walking around, both during the daytime and at night. I also ate halfway decently, which makes these recovery tests bearable. In past years, food has been an issue (getting lunch on time, etc) but this year that was not a problem.

In the end, we had some issues which pushed us to the end of the window for our recovery, but that happens every year, so I am use to it at this point. It is a matter of timing things so I got time to spend time relaxing a bit between shifts I had to work, because I knew I would not have a chance to have much time to unwind at the end of the test. The week went by very quickly, and I was exhausted after things had wound up. I was asleep in my bed within a hour of getting back home from Philadelphia.

Today was shutdown and travel day. Got the systems cleaned up and shut down, then went ahead and waited for the plane ride. Overall it was a nice and smooth plane ride.

The test this year went amazingly well, I am very happy with the results, and happy its going to be another year until I need to see Philly again.

It was also very nice getting back home and relaxing a bit.

Today was a bit more relaxed, we are down to the last application. We have had all of our apps restored and tested, and worked on one final app that is going to be tested tonight.

Today we made it out to the liberty bell, which was very nice. We got to see the bell, plus all the old house and senate buildings. It was a very nice walk, but a bit hot.

Later in the evening, we finished up testing for our last app, and it was off to the bar (lets just say we had to get a night of drinking done in about a hour).

Well, I got back after a bit of sleep, and the outstanding tape drive issue was still outstanding. Spent some time getting that all fixed up, then both backup and recovery environments where all set to rock and roll. Unfortunately the delay did cause timelines to get pushed back a bit, but nothing we can not make up.

By the end of the day, most of our apps where recovered, and some of the configuration had started. Another late night.