I’m pretty much out of ports on my TV. I’ve got a Motorola set-top box/DVR from ComcastXfinity (a crappy early model with hardly any disk space), a PlayStation 3, an Apple TV, and a Roku 2XS. I need the ATV for AirPlay from my iPad or MacBook Air, plus YouTube; I need the Roku because I watch a lot of Amazon streaming video. I can get Netflix and Hulu on any of the devices. For recent pay-per-view movies, I prefer Amazon to the alternatives (Comcast or Apple), simply because of price and variety. I’d love to dump one of these devices, which would have to be the Roku, but that would mean using the PS3 for Amazon Video, and the PS3 UI is utter crap.
So I’m stuck with this device setup. Unfortunately the Roku 2XS has been a disaster. I bought it 18 months ago to replace an original Roku which had fried, and it’s always been glitchy. It becomes catatonic about once a week, requiring a power cycle to fix it. But this evening the power cycle failed to produce the normal startup screen of bouncing purple letters. I unplugged and replugged it a couple of times, then tried a factory reset by sticking a paperclip in the “reset” hole for the officially-recommended 15 seconds (and then some). Nothing.
In frustration I grabbed my MBA and started to browse reviews on Amazon.com, looking for alternatives to the Roku. About 15 minutes later, the Roku 2XS suddenly came to life, and displayed a white (not purple) logo. I re-paired the remote, configured the Roku and all of my services (really strong passwords are great until you have to enter them repeatedly using an on-screen keyboard!), and I was back in business.
That weird 15 minute delay suggests to me that the Roku 2XS has some kind of hardware problem, probably heat-sensitive. Since it only has a 90-day warranty (what kind of nonsense is that?), I’m going to have to replace it. I wish I could find another device which would do the job, but I guess I’ll be going for a Roku 3. Hopefully we’ll be getting an official YouTube app for it soon.
Thought for the day
The tl;dr version: Arguably all interesting advances in computer science and software engineering occur when a resource that was previously scarce or expensive becomes cheap and plentiful.
The longer version:
This particular thought was provoked by a series of exchanges on blogs and in Twitter yesterday. It started with a piece at Information Week in which Joe Emison bemoaned the fact that Netflix was holding back progress in cloud computing. The Clouderati jumped all over this, and Adrian put together a detailed response which he also posted to his blog. By the time I got around to responding, IW had closed comments on the original piece, and so I followed up on Adrian’s blog.
Joe’s criticism was based on two points:
Netflix’s cloud architecture[…] is fundamentally (a) so intertwined with AWS as to be essentially inseparable, and (b) significantly behind the best *general* open options for configuration management and orchestration.
Point (a) is pretty silly: Netflix is a business, not a charity. Of course they’re going to work with the best of breed. But it was Joe’s second point that really bugged me. I responded (and here’s where the “Thought for the day” comes in):
Amazon and Netflix are dramatically ahead of the curve, not behind it. The configuration management pattern you seem to prefer – just-in-time customization using Chef or Puppet – was pretty old school when Sun acquired CenterRun and built out N1 and Grid Engine. It’s incredibly inefficient compared with early-bound EBS-backed AMIs.
Arguably all interesting advances in computer science and software engineering occur when a resource that was previously scarce or expensive becomes cheap and plentiful. We’ve seen it with graphical user interfaces, interpreted languages, distributed storage, and SOA. Traditional late-bound configuration management treats machine images and VM instances as expensive; AWS and Netflix invite you to imagine the possibilities if they’re effectively free. Welcome to the real Cloud 2.0…
In a subsequent Twitter exchange, I said:
@adrianco We used to talk about “specific excess MIPS” driving change. Now it’s “specific excess VMs”
… to which Adrian replied:
@geoffarnold with SSD excess IOPS can be used in interesting ways
A new home (for the box)
I’m composing this while surrounded by Ethernet cables, sitting on the (raised) floor of the Layer 42 colo in Mountain View. This is the new home of the box (grommit) that hosts email, blogs, and various stuff for me, friends and family. Steve Lau and I (but mostly Steve) are working to sort out the kinks that are introduced by changing the IP addresses of the various zones running in this OpenSolaris server. As always, DNS propagation means that cause and effect are temporally vague, but eventual consistency is being achieved.
Cleaning up after a hack
I spent several hours today disinfecting my other website (Speaking of Clouds, also reachable as GeoffArnoldConsulting.com) after a WordPress hack attack. As is often the case, I was saved by the incompetence of the hackers, who had modified my .htaccess files in such a way that it created an infinite redirection loop. (Hint: you’re not going to get far if your URL begins with “htttp:”.) This loop meant that the site became inaccessible, which was immediately noticed by Montastic, the service I use to monitor all of my sites. (Highly recommended.)
Unlike this blog, Speaking of Clouds is hosted at DreamHost. This is not particularly significant: DreamHost has always provided excellent service, and their customer service guys were immediately responsive when I contacted them. However I’m running on a multiuser system, rather than in my own virtual machine or zone, which meant that certain diagnostic and troubleshooting tools weren’t available. I couldn’t restart the Apache process, or compare logs across multiple websites.
The eventual cleanup was relatively straightforward. Ssh in to the host. Take a recursive listing of the entire filespace, so that I could tell what was changed when. Back up everything. Examine logs. Clean up all of the .htaccess files. Change the keys. Log in to the dashboard. Reinstall WordPress 3.4.1. Identify all of the bogus PHP and HTML files (made easier by the atrocious spelling and grammar of the hackers). Change all the passwords. Reinstall all the plugins and themes. Delete (rather than disabling) everything I’m not actually using. And then back everything up. And all the while, I had three terminal windows tailing the relevant log files.
I must say that I would rather been slogging through the mud at Silverstone, though….
UPDATE July 12, 2012:
This story continues to develop. Yesterday I received an email from a Russian Lithuanian company (evuln.com), advising me that my site appeared to be hacked, and providing a little bit of more-or-less accurate advice on cleaning it up. The email concluded:
If you are not able to fix this “redirect” problem on your own then we will be glad to help you for a reasonable price.
Oddly, the description that they gave of how I was hacked was slightly inaccurate, and so I ssh’d back into speakingofclouds.com to check. Sure enough, it had been hacked again. I cleaned up as before; this time I touched every file in my WordPress subtree, so that any changes would be immediately apparent.
This morning, I logged back in, and found that my .htaccess files had been changed again. This time I was able to match the modification time to the exact HTTP log entries, and this is what I saw:
94.23.116.27 - - [12/Jul/2012:05:45:44 -0700] "POST /wp-content/uploads/.cache_000.php HTTP/1.1" 200 365 "-" "Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20100101 Firefox/8.0"
So somehow an executable PHP file had been hidden away in my uploads directory, and was being used to inject stuff into my WordPress configuration. I quarantined the file, then looked around to see if this was a known exploit. I only came across one blog reference, here.
It seems like one really obvious security fix for PHP would be to prevent it from executing hidden files. A quick check suggests that this hasn’t been implemented, though.
UPDATE December 11, 2012:
I’ve received several emails from eVuln.com complaining about this blog piece:
I am concerned about your blog post. It influence our online reputation. I’m sorry about our letter, we just wanted to inform you about security issues in your website.
Since I did no more than state the facts, accurately, I’m not sure what they’re complaining about. In the unlikely event that anyone actually reads this piece and cares about what I wrote, I encourage you to visit the eVuln Labs website and draw your own conclusions.
More anon.
Live, learn, fix, repeat – my thoughts on the recent AWS outage.
On Monday, I had a conversation with Derrick Harriss of GigaOM, which he published here. I’ve followed up with a piece on my tech blog, adding a few thoughts and ripping the naivety of certain analysts…
Just published: "My Short Century" by Lorna Arnold
After years of work by many people, we’ve finally published my mother’s memoirs. “My Short Century” by Lorna Arnold is now available from Lulu. A Kindle version is on the way, and both of them should show up on Amazon quite soon.
Quick tip: if your Mac disk goes south…
Yesterday I had an unrecoverable file system error on my MacBook Air running 10.8 Preview 2. Disk Utility instructed me to back up what files I could and then reformat. Since I had an up-to-date Time Machine backup, I wasn’t worried. I reformatted, and reinstalled the OS. However, I discovered that the OS installation had left me with a copy of 10.7. Would I be able to successfully restore all of my 10.8 files onto 10.7? Probably not. (Many of the settings have changed a lot.) So I decided to complete the installation first, upgrade to 10.8, and then recover my files.
Here’s the tip – something I forgot to do which caused me to waste time. While I was reinstalling 10.7, I created my normal “Geoff Arnold” user account. That was silly, because eventually I wanted to restore that account (apps, settings, files) from Time Machine. I should have set up a disposable account called something like “Super User”, performed the upgrade to 10.8 as this user, and then restored “Geoff Arnold” from Time Machine. As it was, I had to juggle accounts before running Migration Assistant: create “Super User”, log out, log in as “Super User”, delete the “Geoff Arnold” account, etc.
I actually ran into one more problem: trying to restore across the LAN didn’t work, because Migration Assistant hung while looking for computers. So I copied the backup sparsebundle to a USB HD, and restored from that. From checking the Apple Support discussions, it appears that using Migration Assistant with Time Machine is (still) mostly broken.
Back into the walled garden, with enthusiasm (and an emptier wallet)
Yesterday I finally had enough. I headed over to the local AT&T store, indulged myself in a mild rant about the POS (Samsung Infuse) that they’d sold me last summer, and then paid through the nose to upgrade it early to an iPhone 4S. Since I don’t intend to replace this one any time soon, I went for the top-of-the-line: a 64GB white one. Did I really need that? Well, when I replaced my iPad with an iPad 2, I opted for a 32GB rather than 64GB, and I’ve been running into space constraints ever since. So 64GB seemed safer.
Anyone want to buy a Samsung Infuse 4G in good condition, complete with desktop cradle? You’ll need to root and flash it to make it usable, of course….
New year, new direction
I’ve just left Yahoo, mostly because it became clear that I wouldn’t be able to do what I was originally hired to do. Frustrating, but never mind. So now I’m checking out the alternatives (of which there are quite a few), and in the meantime I’ve joined US Venture Partners as an entrepreneur-in-residence.
Time to root – or dump – my AT&T Samsung Infuse 4G phone
Well, 2011 has given way to the New Year, and AT&T have failed to fulfill their promise to upgrade the Android software on all of the 4G phones which they sold in 2011. Back in the summer I embarked on an experiment to see what life outside Apple’s walled garden would be like. The results are in: it sucks. Battery life is awful, system freezes are common (often with the phone feeling dangerously hot), and app management is broken (somehow I have acquired two copies of several apps). I could go on, but why bother?
The main takeaway from this is that Samsung and AT&T (and probably other carriers and manufacturers) haven’t understood that Apple changed the rules with the iPhone, by bringing the PC (and Mac) upgrade model to mobile communications. Backward compatibility is mandatory. Software and hardware upgrades are decoupled. Bugs are fixed. OS and app features are delivered regularly. I’m sure Google hoped that the Android ecosystem would follow this path, but if so they’ve completely failed to convince their partners.
So what to do next? Yes, of course I can root the device, find and install a ROM image of unknown provenance, etc. But I resent the need to do this*, and I’m distinctly uncomfortable doing so on a device which is used for corporate communications. I could dump the Infuse and buy an iPhone 4S, but after only 6 months on the contract it’s a relatively expensive proposition. And the final insult is that most of the tools for hacking Android phones seem to be Windows based, and I don’t have any Windows machines lying around.
File under #FAIL.
—
* And that’s assuming that I don’t inadvertently brick the device. For those who haven’t explored this stuff, here’s the simple version of the instructions for a popular ROM:
- Ensure you have both root and CWM. See the reference post if you do not have both of these.
- Copy ROM .ZIP to SD card
- Shut phone off. Hold Vol Up + Vol Down and Power on device
- Wipe Data and Cache (Wiping data will remove your installed applications and settings. You have been warned!)
- Flash CM7 zip
- Reboot. You will get stuck at Samsung screen. This is normal.
- Pull battery, and reboot into recovery (Hold: VOL+ VOL- Power)
- You should now be in ORANGE -OR- BLUE CWM
- Go to “mounts and storage”
- Select format /system
- Reflash CM7 zip
- Don’t forget Google Apps as well. You can get the gapps easily using Rom Manger -> Download ROM -> Scroll down to Google Apps). Google Apps download link is also at the bottom of this post
- Reboot into CM7 goodness, made possible by LinuxBozo