So the next step is getting the TV server running again. I created a 2008R2 Standard VM for this and installed Mezzmo.
The trick is getting the 1TB or so of video, music photos and software installers that were previously shared on the network by the previous server onto the new TV server box. I tried moving the HDD physically over to the ESX box but for whatever reason you can't add a raw disk to an ESX vm the way you can in workstation. Sure you can add a SAN as a raw disk but not a NTFS disk which I found odd.
Ok... Have to do it the hard way. I created a big temporary disk for the TV Server on the main HDD of the new server, cranked the old server up again and began copying the files across the network. About 6 hours later this process was complete.
Then I put the HDD into the new server, added it as a datastore (which formats it for VMFS and wipes anything that was on it - god I hope I got everything off!) then I added it as a disk to the TV Server. I then began copying the files BACK from the temporary space I created onto the HDD from the old server. In a few hours it might be complete... sigh...
Mean while I was looking the the new VT-D magic. Now the Asrock pro3 says it supports it and in the BIOS there is a setting for it but the BIOS says it isn't supported by the processor. I found this odd so I did some more digging.
For some bizarre reason the i7-3770K model (which is a tiny bit faster and unlocked) DOES NOT support vt-d but the cheaper 3770 (without the K) DOES. I am talking to the store now to see if I can swap it. I don't like my chances. Probably not the end of the world but it would be nice in terms of IO speed (both disk and LAN).
Tom
Thursday, 28 June 2012
Wednesday, 27 June 2012
Server Rebuild
So last night I began the process of moving my VMs (test domain controller, two Oracle DBs) onto the new server i7-3770 ESXi server from the old HP xw8200 machine.
VMWare vCentre Converter does the job - simply a matter of pointing it at the VM file and the ESXi server and letting it go. The only problem was the first VM (the main DB) took a little over 8 hours to complete!
I use the domain controller as an SSH hub as well as for testing so I thought I'd copy the VM before I began. This also took 2 hours! It was only 15GBs or so. Hmm there is something going on here. A disk speed test shows the old server drive bandwidth at a little less than 4MB/s - that's pitiful!
So I kill some apps, I pull the server apart and go looking for loose SATA connectors etc but no luck.
I noticed that in the device manager there are the two ultra ATA storage controllers and two primary/secondary channels. The Primary/Secondary channel devices, in their Advanced Settings tab said the transfer mode selected was DMA if available but the current transfer mode was PIO. Hmm that's not right.
I dig a bit further (which involved downloading and re-installing device drivers from HP, lots of rebooting and poking things). Eventually I find in the BIOS that the configuration of the drives is PIO. I found an article saying that after updating from 2.02 - 2.04 these settings can be messed up and the performance suffers. It recommended using the option to reset the BIOS to defaults so I did this.
Resetting the BIOS to defaults didn't seem to help much and it did enable the damned SCSI adapter (which I don't use and don't have a driver installed for). I shutdown and dive back into the BIOS configuration. I changed both the default settings and the settings for each drive so it read 16 blocks at a time and it uses Max UDMA. I reboot and this works! Disk speed back up to 20MB/s. Copied 40GB of data from an IDE disk to a SATA disk in about 40 minutes or so. I kicked off the next conversion and this is running much more quickly!
I also started work on the UPS configuration. Turns out Cyberpower have a 'business' edition of their software suite that works on linux and ESXi. It comes as an 'Agent', 'Client' and 'Centre', From what I can tell you need an Agent to talk to the UPS (as mine is USB. Some have network cards) and you install a client on each VM (or computer) that you want to shutdown when the power goes.
To get ESXi itself to shutdown (and to get it to talk to the USB connection) you have to install a (free) VMWare Management Assistant (vMA) image. It is just Linux with extra tools for managing the ESX server.
The next trick was getting the installer in there. You have to copy it into a folder somewhere on the data store and then use vifs command inside the vMA to get it into the vMA VM. Then you can install it.
I gave the vMA a fixed IP address and then when i installed the client I just point them at the IP of the vMA and then can get information on the state of the UPS.
Ok the rebuild is getting closer to completion!
VMWare vCentre Converter does the job - simply a matter of pointing it at the VM file and the ESXi server and letting it go. The only problem was the first VM (the main DB) took a little over 8 hours to complete!
I use the domain controller as an SSH hub as well as for testing so I thought I'd copy the VM before I began. This also took 2 hours! It was only 15GBs or so. Hmm there is something going on here. A disk speed test shows the old server drive bandwidth at a little less than 4MB/s - that's pitiful!
So I kill some apps, I pull the server apart and go looking for loose SATA connectors etc but no luck.
I noticed that in the device manager there are the two ultra ATA storage controllers and two primary/secondary channels. The Primary/Secondary channel devices, in their Advanced Settings tab said the transfer mode selected was DMA if available but the current transfer mode was PIO. Hmm that's not right.
I dig a bit further (which involved downloading and re-installing device drivers from HP, lots of rebooting and poking things). Eventually I find in the BIOS that the configuration of the drives is PIO. I found an article saying that after updating from 2.02 - 2.04 these settings can be messed up and the performance suffers. It recommended using the option to reset the BIOS to defaults so I did this.
Resetting the BIOS to defaults didn't seem to help much and it did enable the damned SCSI adapter (which I don't use and don't have a driver installed for). I shutdown and dive back into the BIOS configuration. I changed both the default settings and the settings for each drive so it read 16 blocks at a time and it uses Max UDMA. I reboot and this works! Disk speed back up to 20MB/s. Copied 40GB of data from an IDE disk to a SATA disk in about 40 minutes or so. I kicked off the next conversion and this is running much more quickly!
I also started work on the UPS configuration. Turns out Cyberpower have a 'business' edition of their software suite that works on linux and ESXi. It comes as an 'Agent', 'Client' and 'Centre', From what I can tell you need an Agent to talk to the UPS (as mine is USB. Some have network cards) and you install a client on each VM (or computer) that you want to shutdown when the power goes.
To get ESXi itself to shutdown (and to get it to talk to the USB connection) you have to install a (free) VMWare Management Assistant (vMA) image. It is just Linux with extra tools for managing the ESX server.
The next trick was getting the installer in there. You have to copy it into a folder somewhere on the data store and then use vifs command inside the vMA to get it into the vMA VM. Then you can install it.
I gave the vMA a fixed IP address and then when i installed the client I just point them at the IP of the vMA and then can get information on the state of the UPS.
Ok the rebuild is getting closer to completion!
Tuesday, 26 June 2012
Server Rebuild
So all the hardware for my new server arrived yesterday.
The first hurdle was that the power connectors on my old Antec True power supply are different. The PSU has a 20 pin main power connector (instead of 24) and 4 pin 12v CPU connector (instead of 8). I did some searching through the manual and it indicated that if I install the connector at at the bottom and leave some holes open it will work.
I was curious what risk I might be facing doing this so I went googling. Turns out the additional pins are there just to provide higher current carrying capacity. See here http://www.playtool.com/pages/psuconnectors/connectors.html. Now this guy's estimates on the max power for each connector are far outside what I will be running. As I am using the built-in graphics hardware the machine will be using much less power than the potential maximum. So I decided to give it a go and upgrade if the wires/connectors got hot.
I pulled all the old Pentium 4 parts out of the case and began sorting out the wires. The case has a window and a cold-cathode light plus an illuminated window fan which are nice but generate a load of wires. I began tidying these up with cable ties to get them out of the way. I can't do all of them until I get all the drives in there and this won't happen until after I retire the old server.
The first hurdle was that the power connectors on my old Antec True power supply are different. The PSU has a 20 pin main power connector (instead of 24) and 4 pin 12v CPU connector (instead of 8). I did some searching through the manual and it indicated that if I install the connector at at the bottom and leave some holes open it will work.
I was curious what risk I might be facing doing this so I went googling. Turns out the additional pins are there just to provide higher current carrying capacity. See here http://www.playtool.com/pages/psuconnectors/connectors.html. Now this guy's estimates on the max power for each connector are far outside what I will be running. As I am using the built-in graphics hardware the machine will be using much less power than the potential maximum. So I decided to give it a go and upgrade if the wires/connectors got hot.
 |
| Robin Helping |
The next challenge the CD drive. The old PC had an IDE drive which is fine for the few times I would use it but the new motherboard has no IDE connector! The solution was unfortunately to rip my desktop open and steal the DVD drive out of that just long enough to install the OS.
I left all the drive wires loose and only put in the new 2TB drive that will be the system drive for the new server. I removed the old floppy drive at the same time and I found covers (from the old motherboard box) for the hole where the floppy drive went. While at it I found spare covers for the expansion board spots where the old video and wireless card went (don't need these any more).
 |
| Cleaned out |
Robin helped me plug all the case connectors on (HDD light, power switch power LED, reset switch etc). It was getting on for bed time for Robin so I sent him on his way and continued with the installation of the fron USB connectors. No firewire on the new motherboard so I just left that one loose. I connected the USB to the USB 2.0 connector (not 3.0) as I'm not sure of compatibility - will check that later.
Next step is boot! I connected the server to a screen, pulled the wireless USB keyboard/mouse thing from my desktop and plugged it into the new server, plugged it in, turned on the power switch at the back and hit the power switch on the front panel. I put in the ESXi CD and it booted!
ESX goes through its thing, asks me to sign in blood and then tells me there are no drives it can install on. It basically doesn't seem to recognize the SATA drive. I reboot, go into BIOS and sure enough the BIOS sees it. I do some googling (from my work laptop as my desktop is in pieces) and find an article saying that ESXi doesn't support SATA on the Z77 chipset and you have to use a USB drive! But I checked this - other people said it was compatible!
After a while it occurred to me that the ESXi CD is one I downloaded ages ago to install on the old HP box. I put the DVD back into my desktopm boot it and go to the VMWare site and there is an ESXi 5.0 Update 1. I download and burn that, move the DVD drive back to the new server and try to boot again. Phew it now detects the SATA drive and we are in buisness.
About 5 minutes later ESX is installed. Step 1 complete.
Monday, 25 June 2012
Wired
Just finished reading Wired by Richard E Douglas on Kindle (on my phone).
Lately I've been working my way through the cheap Kindle titles and have had some luck. For example the Star Force series by B.V Larsonm while not fantastically written, is pretty entertaining. In other cases I've not been as lucky and I feel this is one of them.
On a side note - has anyone else noticed how the $0.99 books all became $2.99 books? And how the new releases for $9.00 all moved to $13 and higher? Did you notice this increase in price was timed to be just after the demise of the major paper book retailers? (Especially in Australia).
Anyway back to the book (some spoilers) - the basic premise is that a slightly damaged ex-special forces dude gets brought back out of retirement to hunt down a scientist that has gone rogue and is threatening mass destruction. He discovers she has actually developed technology to massively enhance her intellect for short periods (among other things). She has been setup and they (the special forces dude, the scientist a hacker and another army guy) all go on a meandering journey to find who set them up and right the wrongs.
The pace of the book is excellent and certainly at the start some of the situations setup by the scientist to avoid capture were very clever. The action scenes are tense and occasionally the characters are in some pretty tight situations. As the book progresses it relies more and more on the brain enhancement tech to get the characters out of trouble which quickly gets tiresome.
I have a bunch of problems with this book. The main one I think is that for whatever reason I found it hard to accept the basic premise of brain enhancement implemented *genetically* via a retrograde virus. The capabilities they had just seemed implausible and at the end of the day no matter how big or fast your brain is you can't know what you can't know. Sure the capability gives them perfect and total recall but occasionally I feel the result doesn't follow.
Now let me preface this with the fact that whilst I was brought up a catholic that my views on religion are at best quirky and probably undeveloped. The book however argues that the more intelligent people are the more anti-social they become. Super-enhanced people begin to see the people around them like normal people would apes. Furthermore it argues that if you are super-enhanced you see the fallacy of after-life and therefore tend to be less altruistic and more selfish. Then having absolute power (cough) and a sense of limited time compels you to act more selfishly as there are no risks associated with this behaviour.
First of all there is just so much research out there on co-operation and how it increases the overall effectiveness of a group that I would have thought a hyper-intelligent being would see this. Secondly if the author argued religion was invented so people could cope with their own mortality or loss of loved ones or if the author said religion was invented to explain the reason of your existence I could bite but really, there are enough examples of humans acting selflessly independently of religion. Even arguing religion was constructed to enhance altruism is a stretch if you consider early religions.
Overall I was pretty disappointed and won't be going back for the sequel.
Lately I've been working my way through the cheap Kindle titles and have had some luck. For example the Star Force series by B.V Larsonm while not fantastically written, is pretty entertaining. In other cases I've not been as lucky and I feel this is one of them.
On a side note - has anyone else noticed how the $0.99 books all became $2.99 books? And how the new releases for $9.00 all moved to $13 and higher? Did you notice this increase in price was timed to be just after the demise of the major paper book retailers? (Especially in Australia).
Anyway back to the book (some spoilers) - the basic premise is that a slightly damaged ex-special forces dude gets brought back out of retirement to hunt down a scientist that has gone rogue and is threatening mass destruction. He discovers she has actually developed technology to massively enhance her intellect for short periods (among other things). She has been setup and they (the special forces dude, the scientist a hacker and another army guy) all go on a meandering journey to find who set them up and right the wrongs.
The pace of the book is excellent and certainly at the start some of the situations setup by the scientist to avoid capture were very clever. The action scenes are tense and occasionally the characters are in some pretty tight situations. As the book progresses it relies more and more on the brain enhancement tech to get the characters out of trouble which quickly gets tiresome.
I have a bunch of problems with this book. The main one I think is that for whatever reason I found it hard to accept the basic premise of brain enhancement implemented *genetically* via a retrograde virus. The capabilities they had just seemed implausible and at the end of the day no matter how big or fast your brain is you can't know what you can't know. Sure the capability gives them perfect and total recall but occasionally I feel the result doesn't follow.
Now let me preface this with the fact that whilst I was brought up a catholic that my views on religion are at best quirky and probably undeveloped. The book however argues that the more intelligent people are the more anti-social they become. Super-enhanced people begin to see the people around them like normal people would apes. Furthermore it argues that if you are super-enhanced you see the fallacy of after-life and therefore tend to be less altruistic and more selfish. Then having absolute power (cough) and a sense of limited time compels you to act more selfishly as there are no risks associated with this behaviour.
First of all there is just so much research out there on co-operation and how it increases the overall effectiveness of a group that I would have thought a hyper-intelligent being would see this. Secondly if the author argued religion was invented so people could cope with their own mortality or loss of loved ones or if the author said religion was invented to explain the reason of your existence I could bite but really, there are enough examples of humans acting selflessly independently of religion. Even arguing religion was constructed to enhance altruism is a stretch if you consider early religions.
Overall I was pretty disappointed and won't be going back for the sequel.
Sunday, 24 June 2012
Server rebuild
I have an old HP xw8200 machine that I use as a server at home. It runs the two Oracle DB VMs I use for work and our cache of TV we can watch via Mezzmo (a DNLA server).
Unfortunately the stupid thing is starting to get flaky. I bought it before ESX was free and so I ran XP on it with VMWare Server (which was free) to run the VMs. This worked Ok as some software I wanted to run on it was only available in XP (the TV card software) but it still ran some servers to support my work needs.
It really needs more RAM and a while back I bought 8GBs to upgrade it and planned to rebuild it using ESX. Well unfortunately ESX won't run on it! While the machine is 64 bit it is only sort-of 64 bit and doesn't support the VT-X instruction. I tried 2008R2 and even 64 bit XP but nothing worked. Even worse was that when I put 8GB of RAM in the machine XP thinks there is just 2GBs and consequently all the VMs run quite badly.
So what to do? Well while the processors in the machine are not quite what I need, the hardware is very robust. My office is not air-conditioned and can get pretty hot in summer (40C on a bad day). The HP survived all that.
My plan was to get another 'tower' server. First of all I was alarmed to discover *none* of the HP tower servers are listed as compatible with ESXi. Also the cost is nearly double a desktop equivalent. For example an i3 based server like a Proliant ML110 G7 is around $AUD900-$1000 with a pitiful amount of RAM. A basic Xeon one is more like $AUD1500 and over $2000 with a decent amount of RAM.
BTW I found this site enormously useful when working out the relative performance of different processors as well as comparing the performance of server/desktop line processors.
http://www.cpubenchmark.net/high_end_cpus.html
I looked at Dell servers as they do appear to be ESX compatible but again a E3-1220 based unit (which is roughly equivalent to a i3) with 16GB of RAM is around $AUD1400.
Ok do I really want an off the shelf server? I have an old Antec case around the house that has a dead P4 motherboard etc in it. What about if I build a server inside that box? Hmm.
I could buy a server Mobo, Xeon chip ECC RAM right and build one. I looked at the ASUS P8B WS motherboard http://www.asus.com/Motherboards/Intel_Socket_1155/P8B_WS/ and looked at a few Xeons (like a E3-1280) and while this would work and give me a server with reasonable specs the cost ends up being well over $AUD1200.
I was talking to a friend about this and he asked if I really need a server machine? Ok so non ECC RAM may not be as reliable when running for hours but my desktop runs for many hours a day and I shutdown the server over night anyway. Maybe a custom built desktop would do the job.
So then I dive into the world of desktop chipsets, Ivy bridge vs Sandy Bridge and so on. Turns out that as Ivy Bridge integrate GPU functionality into the chip, most Mobo include graphics output which is nice for a server as I don't have to waste money/power on a Gfx card.
Also I can get a pretty fast processor (i7-3770) for the cost of the Xeon and a Z77 motherboard plus fast ram for 3/4 the cost of the server stuff.
Then I do more hunting... Apparently many Z77 motherboards don't work with ESXi. Sigh... The problem appears to be the LAN card. The ones with a Realtek based LAN card apparently work but with others either you need to hack the install or they simply don't work at all.
Thankfully I found this spreadsheet (https://docs.google.com/spreadsheet/ccc?key=0AjymuQhfM0vYdHZtNThGSllMeU1SMU9ldVltUmp4NWc#gid=0) which lays this out. In the last three builds or so I have tended to go with ASUS motherboards but it turns out the ASRock unit is the best one as it has a Realtek card. The ASRock one is also quite a cheap unit at a little over $110 too.
By this point my head was positively spinning with compatibility matrices, motherboard chipsets, processor chip lineups etc.
I don't understand why it has to be so hard? Anyway I've ordered the desktop stuff based on an ASRock z77 motherboard and the intel processor. I bought some fast G.Skill RAM as when I did this in the past I found the performance boost worthwhile.
Should be here in the next couple of days and I'll post more then.
Unfortunately the stupid thing is starting to get flaky. I bought it before ESX was free and so I ran XP on it with VMWare Server (which was free) to run the VMs. This worked Ok as some software I wanted to run on it was only available in XP (the TV card software) but it still ran some servers to support my work needs.
It really needs more RAM and a while back I bought 8GBs to upgrade it and planned to rebuild it using ESX. Well unfortunately ESX won't run on it! While the machine is 64 bit it is only sort-of 64 bit and doesn't support the VT-X instruction. I tried 2008R2 and even 64 bit XP but nothing worked. Even worse was that when I put 8GB of RAM in the machine XP thinks there is just 2GBs and consequently all the VMs run quite badly.
So what to do? Well while the processors in the machine are not quite what I need, the hardware is very robust. My office is not air-conditioned and can get pretty hot in summer (40C on a bad day). The HP survived all that.
My plan was to get another 'tower' server. First of all I was alarmed to discover *none* of the HP tower servers are listed as compatible with ESXi. Also the cost is nearly double a desktop equivalent. For example an i3 based server like a Proliant ML110 G7 is around $AUD900-$1000 with a pitiful amount of RAM. A basic Xeon one is more like $AUD1500 and over $2000 with a decent amount of RAM.
BTW I found this site enormously useful when working out the relative performance of different processors as well as comparing the performance of server/desktop line processors.
http://www.cpubenchmark.net/high_end_cpus.html
I looked at Dell servers as they do appear to be ESX compatible but again a E3-1220 based unit (which is roughly equivalent to a i3) with 16GB of RAM is around $AUD1400.
Ok do I really want an off the shelf server? I have an old Antec case around the house that has a dead P4 motherboard etc in it. What about if I build a server inside that box? Hmm.
I could buy a server Mobo, Xeon chip ECC RAM right and build one. I looked at the ASUS P8B WS motherboard http://www.asus.com/Motherboards/Intel_Socket_1155/P8B_WS/ and looked at a few Xeons (like a E3-1280) and while this would work and give me a server with reasonable specs the cost ends up being well over $AUD1200.
I was talking to a friend about this and he asked if I really need a server machine? Ok so non ECC RAM may not be as reliable when running for hours but my desktop runs for many hours a day and I shutdown the server over night anyway. Maybe a custom built desktop would do the job.
So then I dive into the world of desktop chipsets, Ivy bridge vs Sandy Bridge and so on. Turns out that as Ivy Bridge integrate GPU functionality into the chip, most Mobo include graphics output which is nice for a server as I don't have to waste money/power on a Gfx card.
Also I can get a pretty fast processor (i7-3770) for the cost of the Xeon and a Z77 motherboard plus fast ram for 3/4 the cost of the server stuff.
Then I do more hunting... Apparently many Z77 motherboards don't work with ESXi. Sigh... The problem appears to be the LAN card. The ones with a Realtek based LAN card apparently work but with others either you need to hack the install or they simply don't work at all.
Thankfully I found this spreadsheet (https://docs.google.com/spreadsheet/ccc?key=0AjymuQhfM0vYdHZtNThGSllMeU1SMU9ldVltUmp4NWc#gid=0) which lays this out. In the last three builds or so I have tended to go with ASUS motherboards but it turns out the ASRock unit is the best one as it has a Realtek card. The ASRock one is also quite a cheap unit at a little over $110 too.
By this point my head was positively spinning with compatibility matrices, motherboard chipsets, processor chip lineups etc.
I don't understand why it has to be so hard? Anyway I've ordered the desktop stuff based on an ASRock z77 motherboard and the intel processor. I bought some fast G.Skill RAM as when I did this in the past I found the performance boost worthwhile.
Should be here in the next couple of days and I'll post more then.
Work at home Zombie?
Welcome to my inaugural post!
For those who are curious - Some years ago I took an overseas posting with an up-and-coming startup. The company had interesting products but a typical dot-bomb business plan and so its market cap graph looked like the cross section of the Matterhorn except it very much made it back down to sea-level on the after-side.
Consequently when I returned from my secondment there was actually no company to employ me as it had been sold off/chewed up/spat out. Instead they kept me on but I worked from home (10 hours out of sync with the office!)
When I left to return home from my overseas posting I found it a bit like leaving the company - they had a lunch and wished me well etc. I wasn't quite leaving though as I was still in contact via conference calls and email. I found it very much like the employment version of being un-dead (un-fired).
And hence the WAHZ label was born.
Anyway I am a software developed specializing in PKI software, security, C++ and general Windows development. I like Java, have written way more perl than I'll admit and I like designing stuff before I cobble it together. I have threes children that very much fill my schedule outside of work and a busy wife that teaches primary schoolers.
My plan is to use this blog to publish tid-bits of software techniques or design information I come up with. Also I'll publish information about my few home projects (both technical and construction) and stuff the kids are up to and what I'm reading.
Hope you enjoy it. I'll try not to get bored with posting to it too soon.
For those who are curious - Some years ago I took an overseas posting with an up-and-coming startup. The company had interesting products but a typical dot-bomb business plan and so its market cap graph looked like the cross section of the Matterhorn except it very much made it back down to sea-level on the after-side.
Consequently when I returned from my secondment there was actually no company to employ me as it had been sold off/chewed up/spat out. Instead they kept me on but I worked from home (10 hours out of sync with the office!)
When I left to return home from my overseas posting I found it a bit like leaving the company - they had a lunch and wished me well etc. I wasn't quite leaving though as I was still in contact via conference calls and email. I found it very much like the employment version of being un-dead (un-fired).
And hence the WAHZ label was born.
Anyway I am a software developed specializing in PKI software, security, C++ and general Windows development. I like Java, have written way more perl than I'll admit and I like designing stuff before I cobble it together. I have threes children that very much fill my schedule outside of work and a busy wife that teaches primary schoolers.
My plan is to use this blog to publish tid-bits of software techniques or design information I come up with. Also I'll publish information about my few home projects (both technical and construction) and stuff the kids are up to and what I'm reading.
Hope you enjoy it. I'll try not to get bored with posting to it too soon.
Subscribe to:
Posts (Atom)