tag:blogger.com,1999:blog-73841423225538508562024-03-26T00:28:05.144-07:00Greg's Tech MumblingsGreg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-7384142322553850856.post-43002090361220869582021-02-14T21:21:00.002-08:002022-10-07T08:48:25.120-07:00Ceiling Projection of Blood Glucose Readings<p> <strong class="markup--strong markup--h4-strong">a.k.a. MQTT-RF Bridge for Ceiling-Projecting Alarm Clock Temperature</strong></p><p class="graf graf--p" name="5715">By Greg J. Badros <<a class="markup--anchor markup--p-anchor" data-href="mailto:badros@gmail.com" href="mailto:badros@gmail.com" target="_blank">badros@gmail.com</a>><br />First draft 2020–03–29; last updated 2020–04–08; moved to blogger 2021-02-14</p><p class="graf graf--p" name="5715">Update: If you or someone you love has diabetes, checkout out <a href="https://gluroo.com"><span style="color: #6aa84f;">Gluroo</span></a>! It's an <a href="https://gluroo.com"><span style="color: #6aa84f;">iOS and Android mobile app that makes managing diabetes as easy as messaging</span></a>.</p><h3 class="graf graf--h3" name="6c27">Motivation</h3><p class="graf graf--p" name="7aa9">In February 2020, my son was diagnosed with Type 1 Diabetes. Within 4 days we had him wearing a Dexcom G6 Continuous Glucose Monitor (CGM). My wife and I each follow his CGM readings 24/7, but at night time, our primary user interface to information is a ceiling projection alarm clock. I decided to find a way to project our son’s real-time blood glucose readings unobtrusively on the ceiling of our bedroom.</p><h3 class="graf graf--h3" name="82c0">First approach — Build a simple ceiling projector</h3><p class="graf graf--p" name="5b57">At first I tried to find a ceiling projection multi-segment LCD component of some kind. Though I was hopeful, I soon became discouraged and was unable to find something easy to use in a project. I found various tutorials about making a simple projector, but the part list and the required experimentation around brightness, etc., and having to package it up to be something I’d be willing to put on the nightstand made for another challenge and would add to the cost and complexity.</p><h3 class="graf graf--h3" name="818f">Second approach — Use a Ceiling-Projecting Alarm Clock</h3><p class="graf graf--p" name="ea50">After becoming more pessimistic on the first idea, I had an insight that I could re-use an existing Ceiling-Projecting Alarm Clock and just connect in to the display or replace the display with my logic. Then I realized it was even easier than that: many ceiling-projecting alarm clocks have the ability to display a temperature from a remote module that is meant to sense the outdoor temperature. If I could reverse-engineer that outdoor-thermometer + temperature transmitter, I could transmit whatever number I was as the temperature and the ceiling projecting alarm clock would show that number of interest on the ceiling. So “all” I had to do was find a ceiling-projecting alarm clock that had three digits of temperature (e.g., 24.3 degC), reverse-engineer the outdoor thermometer module and protocol, and create a new transmitter to send a different “temperature” to the alarm clock. Here’s the end result:</p><figure class="graf graf--figure" name="468f"><img class="graf-image" data-height="3024" data-image-id="1*uEgHCefukSA67oTHJsKqWw.jpeg" data-width="4032" height="300" src="https://cdn-images-1.medium.com/max/1600/1*uEgHCefukSA67oTHJsKqWw.jpeg" width="400" /><figcaption class="imageCaption">The “Outside” temperature of 11.3 really means BGL of 113 coming from NightScout</figcaption></figure><h3 class="graf graf--h3" name="7c62">Selecting the Alarm Clock</h3><p class="graf graf--p" name="c5c9">I already owned and like a circa 2005 Oregon Scientific projection clock (Model BAR338PA), but the outdoor temperature sensor/transmitter had died long ago. I searched for a replacement with no luck but found that any of models THC138, THR138, RT918 were supposedly compatible with our projection clock. I’d still love to reverse engineer those transmitters so it’d work with that clock, but for the immediate term, I needed another projection clock.</p><p class="graf graf--p" name="3a73">After a quick look through options on Amazon, I decided to try the <a class="markup--anchor markup--p-anchor" data-href="https://smile.amazon.com/gp/product/B07CJ2ZNLK" href="https://smile.amazon.com/gp/product/B07CJ2ZNLK" rel="noopener" target="_blank">Smartro Time & Temperature Projection Alarm Clock</a> — it’s pretty basic, but has a small footprint, displays temperatures with 3 digits in equal size (on the projected view) and has a separate transmitter. I bought one to try.</p><h3 class="graf graf--h3" name="adc0">Reverse Engineering the RF Protocol</h3><p class="graf graf--p" name="0e4d">I’d previously identified <a class="markup--anchor markup--p-anchor" data-href="https://github.com/jopohl/urh" href="https://github.com/jopohl/urh" rel="noopener" target="_blank">URH (Universal Radio Hacker)</a> as a software tool to help in reverse engineering, and from its device compatibility list, chose the tiny and inexpensive <a class="markup--anchor markup--p-anchor" data-href="https://smile.amazon.com/gp/product/B01B4L48QU" href="https://smile.amazon.com/gp/product/B01B4L48QU" rel="noopener" target="_blank">NooElec NESDR Nano2+</a> software-defined-radio (SDR) receiver to listen in to the RF transmissions.</p><p class="graf graf--p" name="c5c1">I made a couple of assumptions before learning how to use URH:</p><ol class="postList"><li class="graf graf--li" name="e4fd">The projection clock devices are made in China, so I suspected that the temperature would be transmitted in Celsius/Centigrade (not Fahrenheit).</li><li class="graf graf--li" name="1cee">These devices are inexpensive so I suspected that they’d probably send the temperatures as an integer with the decimal point shifted and not bother with floating point representations. E.g., 23.9 degC will be sent as the integer 239.</li><li class="graf graf--li" name="3d43">From the user manual and FCC registration of the device, it was using the common 433.92MHz frequency for radio communications.</li></ol><p class="graf graf--p" name="7b3e">So I fired up URH with the SDR in the USB port and started the spectrum analyzer looking at (and listening to using CubicSDR which the Nooelec device suggested — it has a good spectrum analyzer that lets you play through your speakers the “heard” RF signals) around 433.9MHz. Sure enough, I saw the RF antenna receive a blurt of data every 55 seconds or so at 433.92MHz. Just after each transmission, I saw the clock update the displayed outdoor temperature. The transmissions stopped when I removed the batteries to the transmitter, so I was confident I was seeing the right signal, and there were three bursts immediately after inserting the batteries, providing further confirmation.</p><p class="graf graf--p" name="d0a5">The raw RF signal looked like this in URH when the temperature was 25.8 degC:</p><figure class="graf graf--figure" name="691e"><img class="graf-image" data-height="364" data-image-id="0*72QBbFDtMi5roYQ7" data-width="1600" height="91" src="https://cdn-images-1.medium.com/max/1600/0*72QBbFDtMi5roYQ7" width="400" /></figure><p class="graf graf--p" name="0ab7">You can clearly see a modulated sine carrier wave, and it’s also clear that it’s amplitude (not frequency) modulation — technically this is called OOK (On-Off Keying) — so I applied URH’s ASK demodulation and got a waveform showing a sequence of bits corresponding to the RF chatter. I used URH’s “autodetect parameters” to help find the red/green 0/1 split point, and the time between peaks of 500 microseconds:</p><figure class="graf graf--figure" name="7649"><img class="graf-image" data-height="364" data-image-id="0*JwBEpSdL0ZQAE4b4" data-width="1600" height="91" src="https://cdn-images-1.medium.com/max/1600/0*JwBEpSdL0ZQAE4b4" width="400" /></figure><p class="graf graf--p" name="3a16">From this URH gives me the following bits:</p><p class="graf graf--p" name="4057">10010010010010010000100100001000010010010010010010010000100100100100100100100001001000010000100001000010000100001001000010000100001000010000101000000001001001001001001000010010000100001001001001001001001000010010010010010010…</p><p class="graf graf--p" name="b4e5">It’s immediately obvious that there are lots more 0s in this representation than 1s, thus suggesting that these aren’t the logically-important bits in the transmission and that there’s some other decoding to be done. At this point I recorded another handful of different transmissions at different temperatures. In particular, I put the transmitter in the freezer for 15 minutes to get a much different temperature reading. Here are the bits sent for 2.4 degC:</p><p class="graf graf--p" name="630f">1001001001001001000010010000100001001001001001001001001001001001000010000100100100100001000010000100001001001001000010000100100100001010000000010010010010010010000100100001000010010010010010010010010010010010000100001001001…</p><p class="graf graf--p" name="986b">And then I converted these two temperatures to binary using the assumptions I made upfront:</p><p class="graf graf--p" name="9a33">25.8 degC = 258 = 0b100000010</p><p class="graf graf--p" name="033d">2.4 degC = 24 = 0b11000</p><p class="graf graf--p" name="0946">I did those conversions to know what the binary sequence was that I was looking for in each of the two demodulated sequences of bits (presumably after another decoding step). (I had 7–8 more transmission waveforms at other temperatures but I really focused on just those two to keep the scale manageable.)</p><p class="graf graf--p" name="d5d4">My first idea for further decoding of these raw transmitted bits was based on grouping them into triplets of 3 bits. I did a histogram after splitting these up into 3 bits and got:</p><p class="graf graf--p" name="a5ee"><strong class="markup--strong markup--p-strong">Count: Pattern</strong><br />152: 010<br />148: 001<br />147: 100<br />126: 000<br />4: 101</p><p class="graf graf--p" name="5b16">On the one hand, this was encouraging to see a relatively even split among 010, 001, and 100, with 000 (quiet on the RF) being an outlier, the 4 occurrences of 101 were weird and given the proximity of the transmitter to the snooping antenna, I discounted the possibility they were errors or noise. I also wasn’t quite clear how to think about this since one has to expect the RF communication would be designed to be robust against chopping off the beginning or end of the communication and thinking about the encoding in terms of % 3 = 0 offsets from some “start” position seemed fragile. But that’s where I quit for the day.</p><p class="graf graf--p" name="547d">The next day I decided to go back to the demodulated wave form and look at it some more. That’s when I had my key insight: the peaks were just separating two differently-lengthed periods of quiet: either 1000 microseconds (1ms or 2 bits) or 2000 microseconds (2ms or 4 bits). Maybe, I thought, interpreting the shorter quiet periods as “0” and the longer ones as “1” would result in a higher-level representation of the intended transmission. I.e., in the bit sequence, “00” between two “1”s is a “0” and “0000” between two “1”s is a “1”. I wrote a quick perl script to do that conversion and got the below for 25.8 degC:</p><p class="graf graf--p" name="c84c">RAW:<br />1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<em class="markup--em markup--p-em">0000</em>1<strong class="markup--strong markup--p-strong">00</strong>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<em class="markup--em markup--p-em">0000</em>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<strong class="markup--strong markup--p-strong">00</strong>1<em class="markup--em markup--p-em">0000</em>1<strong class="markup--strong markup--p-strong">00</strong>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<strong class="markup--strong markup--p-strong">00</strong>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>1<em class="markup--em markup--p-em">0000</em>101000000001</p><p class="graf graf--p" name="fbb5">(bold marks “00”s which are interpreted as one “0”; italics “0000”s are interpreted as one “1”)</p><p class="graf graf--p" name="16c8">00000101 <strong class="markup--strong markup--p-strong">1000000100 </strong>00001011111101111110..</p><p class="graf graf--p" name="eb2f">And I bolded the binary for 258 (remember 25.8 degC) in the above decoding — it showed up! I repeated on the other readings and each time I found the temperature’s binary representation inside the decoded RF transmission. I did some further experimentation to confirm that negative temperatures are, as you’d guess, encoded with 2s complement. And the signal repeats itself about 10 times per transmission, each time separated by a 6 “0” quiet period (3ms).</p><p class="graf graf--p" name="f877">Across multiple different temperature readings, the prefix to the temperature bits was consistent, but the suffix varied. The same temperature seemed to give the same suffix, but I could not decipher how to compute the suffix from the temperature, so I decided to ignore that puzzle for a bit and see if it mattered.</p><p class="graf graf--p" name="d64e">Next up, I needed to see if I could use my understanding of the protocol to synthesize an RF transmission that would be interpreted by the clock as coming from the temperature transmitter.</p><h3 class="graf graf--h3" name="f73c">Transmitting Reverse-Engineered Protocol</h3><p class="graf graf--p" name="fb0e">Next up I needed a transmitter for testing the sending of these signals. My friend recommended a <a class="markup--anchor markup--p-anchor" data-href="https://www.greatscottgadgets.com/yardstickone/" href="https://www.greatscottgadgets.com/yardstickone/" rel="noopener" target="_blank">YardStickOne</a> which probably would’ve sufficed both for receiving and transmitting, but I hedge a bit and only bought a receiver at first since I wasn’t sure that I’d be able to decode the protocol and figured I might just proceed to the next step (of using a small IC transmitter and microcontroller) and skip this step of prototyping the transmission of the signal from my laptop entirely. In parallel, I ordered some <a class="markup--anchor markup--p-anchor" data-href="https://smile.amazon.com/gp/product/B019SX6Y22" href="https://smile.amazon.com/gp/product/B019SX6Y22" rel="noopener" target="_blank">Arduino-compatible 433 MHz transmitters and receivers</a> and as it turns out, those arrived first, so I skipped the step of using the YardStickOne to test transmission entirely, and proceeded to just code up the sketch for the Particle Photon.</p><h3 class="graf graf--h3" name="6cf5">Designing the BGL Interface to the Transmitter</h3><p class="graf graf--p" name="ef26">I wanted to RF transmit the BGL from a Dexcom G6, and there’s plenty of open-source code out there (e.g., <a class="markup--anchor markup--p-anchor" data-href="http://www.nightscout.info/" href="http://www.nightscout.info/" rel="noopener" target="_blank">NightScout</a>) for being a Dexcom Follower to get the value from the CGM through the Internet. So I wrote a tiny script that fetches the BGL level and publishes it as a message on an MQTT bus that I already have running for my smart home. I wrapped that up as a docker container so it stays running, and that simplified my microcontroller device to only have to RF transmit MQTT messages that it receives (rather than having to do HTTPS queries — an HTTP query on Particle Photon is pretty trivial but the SSL handshake adds significant complexity to an otherwise simple sketch).</p><p class="graf graf--p" name="16ec">Thus, the design for the microcontroller-based component is simple: it listens to a pre-specified MQTT channel for a message that is simply and integer and it broadcasts that integer via the connected RF radio.</p><h3 class="graf graf--h3" name="057b">Building the MQTT-RF Bridge Device</h3><p class="graf graf--p" name="bc0e">My go-to microcontroller of choice is the <a class="markup--anchor markup--p-anchor" data-href="https://store.particle.io/?utm_term=&utm_campaign=Dynamic+Search+Ads+-+S8&utm_source=adwords&utm_medium=ppc&hsa_grp=67675299738&hsa_acc=5060272854&hsa_tgt=dsa-835611334005&hsa_kw=&hsa_src=g&hsa_mt=b&hsa_cam=1697512981&hsa_ver=3&hsa_ad=329924078356&hsa_net=adwords&gclid=Cj0KCQjwsYb0BRCOARIsAHbLPhFPJZnBnvQmV3sy09pyCVGR5HIGtmkexR_NzboEh8GiFfrH4RYoJgoaAuQWEALw_wcB" href="https://store.particle.io/?utm_term=&utm_campaign=Dynamic+Search+Ads+-+S8&utm_source=adwords&utm_medium=ppc&hsa_grp=67675299738&hsa_acc=5060272854&hsa_tgt=dsa-835611334005&hsa_kw=&hsa_src=g&hsa_mt=b&hsa_cam=1697512981&hsa_ver=3&hsa_ad=329924078356&hsa_net=adwords&gclid=Cj0KCQjwsYb0BRCOARIsAHbLPhFPJZnBnvQmV3sy09pyCVGR5HIGtmkexR_NzboEh8GiFfrH4RYoJgoaAuQWEALw_wcB" rel="noopener" target="_blank">Particle Photon</a> and this project was no different. I used just four parts. Here’s the BOM (Bill of Materials):</p><ul class="postList"><li class="graf graf--li" name="e189">Particle photon microcontroller</li><li class="graf graf--li" name="2f16"><a class="markup--anchor markup--li-anchor" data-href="https://smile.amazon.com/gp/product/B019SX6Y22/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1" href="https://smile.amazon.com/gp/product/B019SX6Y22/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1" rel="noopener" target="_blank">Aukru 3X 433MHz RF Wireless Transmitter and Receiver Module Kit</a></li><li class="graf graf--li" name="33af">micro breadboard</li><li class="graf graf--li" name="73f0">plastic case</li><li class="graf graf--li" name="a97f">USB power supply and cable</li></ul><p class="graf graf--p" name="0c59">The transmitter is low power enough it can be powered by the photon itself, the connections are simple:</p><figure class="graf graf--figure" name="0eae"><img class="graf-image" data-height="1200" data-image-id="0*sNlLD4VwYZ7UWWZx" data-is-featured="true" data-width="1600" height="300" src="https://cdn-images-1.medium.com/max/1600/0*sNlLD4VwYZ7UWWZx" width="400" /></figure><p class="graf graf--p" name="4eda">The sketch will (TODO) soon be available on github. Originally, I planned to use the RadioHead library, but it wasn’t immediately trivial to make that work with the Particle Photo microcontroller, so I decided to try just driving the RF transmitter directly by driving the D7 pin, and that worked (in the sense that I could see the transmissions on my NooElec RF snooper).</p><p class="graf graf--p" name="b085">At this point, I was debugging by comparing the demodulated waveforms of the same temperature as transmitted by the stock outside thermometer transmitter vs. my homebrewed device. There were a couple of small tweaks to the analog signal — e.g., between the repetitions of the temperature with its padding, the signal is actually an exception to the short/long pause for 0/1 rule: it is low for 500 microseconds, then high for 500, then low again for 4ms. I still wasn’t sure if this would work with the clock, but making the waveforms look the same when snooped seemed likely to drive the receiver in the projection clock.</p><p class="graf graf--p" name="d20a">Finally, I tested with the clock and had some trouble getting consistent results. I was always able to get the transmitted reading to show up when I put the clock in “synchronize” mode (by holding the “+” button on the clock for 3 seconds), but it would stop working some time later. I spent a bunch more time believing that maybe those extra bits were checksums of some sort (but they weren’t an easy binary function and I doubted that they bothered with a lookup table to compute the checksum) or a sequence number (but they didn’t appear to be monotonically increasing.</p><p class="graf graf--p" name="67c3">After some further experimentation, it turns out that the projection clock is actually only listening for the radio transmission exactly every 57 seconds or so! With that realization, I updated my MQTT publishing script to publish exactly every 57 seconds. Although that worked better, the timing wasn’t reliable and really it’s a property of the the connection between the RF transmitter and the projection clock, so it’s a better design to have the RF transmitter Particle photon sketch be responsible for managing the 57 second timing itself. I reworked the sketch to remember the MQTT messages but instead of doing the RF transmission in the callback for a message, just do the RF transmission of the last-received message exactly every 57 seconds. That works reliably… done!</p><h3 class="graf graf--h3" name="fdef">Improvements</h3><p class="graf graf--p" name="ecab">It would sure be nice to know what those extra bits in the RF transmission message are for, but ultimately they don’t seem to make a difference. I also wish the projection clock only projected the outdoor temperature instead of alternating between the Inside temperature and the outside temperature. I may choose to repeat the process with a different clock radio after researching the options even more carefully.</p><p class="graf graf--p" name="44cd">The fact that the indoor and outdoor temperatures alternate is even worse when projecting in celsius, which is how I started using the clock: 19.7 degC is a reasonable temperature for indoors, and 197 is also a reasonable (though high) blood glucose level. Thus, I switched the code to convert the incoming MQTT message BGL to a centigrade temperature before sending the number to the clock so that it when it converts it to Fahrenheit it shows correctly. Unfortunately, this is a slightly lossy conversion: e.g., if the BGL is 128, we can tell the clock it’s -10.7 degC (12.7 degF for 127) or -10.6 degC (12.9 degF or 129) but we can’t actually get the clock to show 128. This isn’t that big of a deal in practice since a single digit difference in BGL isn’t important, but it’s a little annoying.</p><h3 class="graf graf--h3" name="6ff6">Source Code</h3><p class="graf graf--p" name="74a8">See <a class="markup--anchor markup--p-anchor" data-href="https://github.com/gjbadros/particle-photon-mqtt-rf-bridge.git" href="https://github.com/gjbadros/particle-photon-mqtt-rf-bridge.git" rel="nofollow noopener noopener" target="_blank">https://github.com/gjbadros/particle-photon-mqtt-rf-bridge.git</a></p>Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com0tag:blogger.com,1999:blog-7384142322553850856.post-34799696257698369972016-07-23T20:47:00.001-07:002016-07-23T20:47:19.274-07:00Multi-Featured iPad Wall Mount for Home Automation<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<b style="font-family: arial; font-size: 14.6667px; line-height: 1.38; white-space: pre-wrap;">Disclaimer: I make no promises whatsoever that this does anything useful and anything you do with these instructions are your own decision. I take no responsibility for any injuries (including death!), property damage, wasted money, wasted time or any other negative consequences from your reading or acting on this -- use at your own risk!</b></h2>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Overview</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I describe a multi-featured iPad Wall-Mount designed for use as a home automation user interface. The mount provides power and wired networking to the tablet, digital audio extraction from the tablet, and four customizable buttons. It also provides an auto-on feature to smartly and automatically turn on the tablet’s display when a user approaches its location and turns it off after they have left. The mount can be built for about $200 worth of parts and requires light general electronics skill to assemble. I first built this in January of 2016 and have improved it through a half dozen iterations since then.</span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Problem Statement</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Contemporary tablet wall mounts solve very few of the problems facing home automation user interfaces. Among those problems are:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<ul style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">P1: Tablets rely on WiFi which is inferior to a hard-wired connection for a permanently wall-mounted tablet.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">P2: Tablets often are used to control a separate music player instead of directly providing their digital audio output to an amplifier for use directly as a music source.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">P3: Tablets generally have no physical customizable buttons</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">P4: Tablets manage power by turning off their display when not in use, focusing on direct user interaction with the screen to define use and do not automatically turn on when a user is about to engage with the tablet.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: disc; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">P5: Tablets require power to remain charged and functional.</span></div>
</li>
</ul>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Generally, most permanent tablet wall mounts only address the last of these problems, yet all are important. My solution addresses all five of these problems using a custom 3D-printed frame, various off-the-shelf components that get hidden in the wall behind the tablet, and light general electronics skill to assemble.</span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Design and Key Insights</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Solution #1: Support wired ethernet</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Wireless networking is great for a tablet on the move, but when it’s stationary, the combination of lower reliability, wasted wireless bandwidth, and unnecessary RF interference make using a wireless connection unsatisfactory.</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">More recent versions of iOS support wired Ethernet networking. You simply must use USB OTG (on-the-go) so that the iPad can work as a USB host (rather than client). The Apple Lightning-to-USB Camera adapter is an OTG adapter for the iPad that duplicates the Lightning input (to provide power to the client device and the iPad) and provides a USB jack for devices. The Apple USB Network Adapter plugs into that USB jack and receives an RJ45 plug connected to a network switch, and the iPad will use that Ethernet port for connectivity.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Solution #2: Extract Digital Audio</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Oftentimes a home-automation tablet is used to control a separate music player such as a Sonos or Squeezebox (via iPeng or similar). Generally, the best player interfaces are those that are native to the software outputting music directly on the phone or tablet, so it’s a shame to not be able to harness that output audio in its highest-quality digital form.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Recent versions of iOS also support external audio devices. That same OTG USB port can be used to an audio adapter that has a variety of features. I like the Muse USB DAC because it has an Coax and Optical Digital S/PDIF output and RCA analog outputs. It’s then easy to use a pair of wires from a CAT6 cable to carry the electrical (not the optical) S/PDIF output back to a receiver to drive speakers or distribute the audio signal.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Solution #3: Add customizable Buttons</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">For issuing a quick command at a wall switch, a physical button that can be customized to do anything is hard to beat.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Arbitrary physical buttons are a favorite IoT project and are straightforward to set up once you decide how you want them to signal the action when they’re pressed. For this mount, I simply provide a layout for four momentary switches, and wire them either a) to digital inputs of a micro-controller and program the microcontroller to make configurable HTTP requests when each button is pressed; or b) wire them directly to relays to provide dry contact inputs to a separate controller that then accepts the contact closure as a trigger for an arbitrary action in its universe.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Solution #4: Turn the Display On When a User Approaches It</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">If an iPad screen is dark on a conventional wall mount and a user wishes to issue a command to the currently-running app, they first must wake the tablet up by touching the screen and perform the relevant action in that app. If the currently-running app is, for example, virtual switches for the current room’s lights or an audio play where the user might have a simple desire such as pausing or restarting music, the single extra click to wake up the tablet more than doubles the interaction time. This is unacceptable in a high-use environment, as is leaving the display on all the time (distracting plus energy inefficient).</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The foundations of my approach to this problem are: 1) An infrared distance sensor with a range of 20-150cm that reports the distance as an analog voltage; and 2) a pair of magnets -- one of which is physically moved by a servo -- that emulate an iPad cover and tickle the iPad’s magnetic sensors to turn the display on and off just like the manual covers do. The distance sensor is connected to an analog input of a Particle photon microcontroller, and the microcontroller is coded to watch for objects under a certain distance away. When something is close, it assumes a user has approached the tablet and triggers the servo to move one of the permanent magnets away from the home key on the iPad thus deactivating the second magnetic sensor and turns the iPad on (when both magnetic sensors on the iPad sense magnets -- like they do when the iPad’s cover is on) the iPad display goes off.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(There are several alternative possibilities to this solution that I explain in </span><a href="https://docs.google.com/document/d/17xs9KgPdsvAxj3zAh0s2_F-9qPI4xYlC9dvVi71CONQ/edit#bookmark=id.tv8y1eszgz5r" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">alternative designs</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">, later.)</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Solution #5: Provide Power to the Tablet</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">This is straightforward and I simply use a passive POE (power-over-ethernet) solution to power the powered USB hub and then plug the lightning connector that comes with the iPad into the hub and into the tablet or Lightning-to-Camera USB Adapter used for solving Problems #1 and #2. There’s some modest trickery in ensuring that the Apple device will take the charge, but when you’re just trying to trickle-charge the iPad on the wall, many solutions suffice.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">All of these solutions must, of course, be embedded in an attractive and functional mount that actually holds the iPad and the components necessary to interact with the iPad (e.g., cable management, mounting the distance sensor, servo, magnets, and buttons, etc.). I designed this part in Sketchup and 3D-printed it using Shapeways’ service. My mount focuses on the bottom-left corner of the mount so that it fits inside of a 18cm x 10cm x 2cm bounding box which accelerates the speed of delivery from Shapeways and reduces its cost during prototyping. </span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Parts List (BOM) for the Wall Mount</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">My Custom 3D Printable iPad Mini Mount Frame</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: ~$50 to print in strong and flexible plastic from Shapeways</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="http://www.bhphotovideo.com/bnh/controller/home?O=&sku=1241761&gclid=CjwKEAjwp-S6BRDj4Z7z2IWUhG8SJAAbqbF3HxsIyT1pmvs7nhyApAYvsu-6AoV4AlZ7qtAnyjPPxhoCovnw_wcB&is=REG&ap=y&m=Y&c3api=1876%2C92051677562%2C&Q=&A=details" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Apple Lightning to USB Camera Adapter</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $39</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/Apple-USB-Ethernet-Adapter-MC704LL/dp/B00W7W9FK0/ref=sr_1_1?ie=UTF8&qid=1465534888&sr=8-1&keywords=apple+usb+network" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Apple USB Network Adapter</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $28</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/gp/product/B018HD4RP0/ref=oh_aui_search_detailpage?ie=UTF8&psc=1" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Lightning Extension Cable</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $20</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/USB-DAC-PCM2704-Optical-Converter/dp/B0093KZTEA/ref=sr_1_1?ie=UTF8&qid=1465534798&sr=8-1&keywords=digital+audio+usb+muse" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Muse USB DAC</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $24</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/gp/product/B005P2BY5I/ref=oh_aui_search_detailpage?ie=UTF8&psc=1" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Plugable 4-port Powered USB Hub</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $17</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://store.particle.io/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Particle Photon Microcontroller</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $19</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://store.particle.io/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Particle Relay Shield for Photon Microcontroller</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">(optional if you need contact-closures for the buttons - $30)</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://www.adafruit.com/products/65" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Tiny Breadboard</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $4</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://www.adafruit.com/products/1031" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Sharp IR Distance Sensor (20-150cm)</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $16</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/TowerPro-SG90-Mini-Servo-Accessories/dp/B001CFUBN8?ie=UTF8&sa-no-redirect=1" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">SG90 Servo</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $4</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/gp/product/B00B9GFJAU/ref=oh_aui_search_detailpage?ie=UTF8&psc=1" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">RJ45 Screw Terminal Breakout Board</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> x2 </span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $12 each = $24</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/gp/product/B000RB7E5Q/ref=oh_aui_search_detailpage?ie=UTF8&psc=1" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Circular Magnets with center Hole</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $8 for 25 (so <$1 for the two you need)</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/gp/product/B008DGA9UY/ref=oh_aui_search_detailpage?ie=UTF8&psc=1" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Momentary Tactile Switches</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $6 for 20 (so $1.20 for the four you need)</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<a href="https://smile.amazon.com/Generic-Premium-External-Supply-1000mA/dp/B005ODHJFM/ref=sr_1_3?s=electronics&ie=UTF8&qid=1465536228&sr=1-3&keywords=5v+power+supply" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">5V Power Supply</span></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Price: $5</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Thin paperclip, glue, 22 gauge solid patch wire, RCA audio cable</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Sketchup Model</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Here is </span><a href="https://drive.google.com/file/d/0B_LdZpU007UsN1FuR1ZOOUxyZ1U/view?usp=sharing" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">the model as an STL file</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Front View:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="509" src="https://lh5.googleusercontent.com/6ceYfV63RkR_4wMEjWihIGsohKK5iuYhHKdmhPVM_h_ByE1tiAtp5A-aG9aHZZ2mbogzYPmcira8XfwbPsS2A8S_sP2zPoVgaGeZ8XlQ5IKsA1zXUTadOXBsaZC8mK6pTEdNlF5l" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Back View:</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="587" src="https://lh6.googleusercontent.com/YRt8shYTKUh-g6MJFZZnFunV_NiOs_2qMD4GObsljFuqW18dAz-MysonPSFVdKcOnOrXZxtNvRXVnK8fTUwJ8FHU2Udvyfr3m5QJG3aCsiDYnELWY9nscRdY7Q2sXZpS4YeeNFZ3" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The Circuit</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Using a Particle Photon microcontroller, the circuit is straightforward. We wire D1, D2, D3, and D4 to the momentary contacts (i.e., the pushbuttons). We wire the IR distance sensor to analog input A0. Finally, we wire the pulse output for the servo motor from analog output A4, one of the pins that supports PWM output and the servo API. See the Breadboard and circuit views that I drew using </span><a href="http://fritzing.org/home/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Fritzing</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">. </span></div>
<h3 dir="ltr" style="line-height: 1.38; margin-bottom: 4pt; margin-top: 16pt;">
<span style="background-color: transparent; color: #434343; font-family: "arial"; font-size: 18.666666666666664px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Breadboard View </span><span style="background-color: transparent; color: #434343; font-family: "arial"; font-size: 18.666666666666664px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="265" src="https://lh4.googleusercontent.com/0O5g8sVPVEks-QQBv4dV-dTWbyTdwHX4smpzar9ekIe6uUfFN5gVd5afuqu2kqkEWAzEsRWx5Y_75GAQIw25DFxBVPU4-E1bTl08yw98jf16BJZfekfj_ADrrDeSIVGz2CNx6K03" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></h3>
<h3 dir="ltr" style="line-height: 1.38; margin-bottom: 4pt; margin-top: 16pt;">
<span style="background-color: transparent; color: #434343; font-family: "arial"; font-size: 18.666666666666664px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Schematic View</span></h3>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="478" src="https://lh6.googleusercontent.com/ugyhFqLDwRCk8kW4ibLMoi5E7z5ZBp4fSJmhY3abmUfJJez5H6gHA9StDMbpBYK9e9ZeUY8oZyiBsRndteCsYKCRkbt8HYb4QlwZH2MdN3cNaIGP6oDndUokaritP7WGCTTYWlxA" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="560" /></span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Assembly Instructions</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">After printing out the 3D model, assembly is a simple matter and takes about 30-40 minutes (probably allocated 2-3x for this the first time you try):</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<ol style="margin-bottom: 0pt; margin-top: 0pt;">
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Attach the servo bar to the top of the servo, bend a small paperclip into a right angle and attach it to the servo (see the finished prototype photos).</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Push the four pushbuttons through the opening in the mount and glue the bottoms of the buttons to mount.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Solder the buttons to the power and D1 through D4 (note that the prototype photos only show one button soldered -- I didn’t bother connecting all 4 when I took the photo).</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Assemble a mini breadboard with the circuit including the Particle Photon microcontroller and connect the IR sensor and the servo wires to the breadboard.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Connect the lightning port of the iPad to a digital camera adapter (use a lightning extension cable if you prefer). Plug another lightning cable into the digital camera adapter to power the adapter (and charge the iPad Mini), plug the other end into the 4-port powered hub.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Plug the digital audio USB adapter into the hub, connect the S/PDIF output of the adapter to the appropriate wires on the RJ45 screw connector.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Plug the Apple USB ethernet adapter into the hub, and plug a networking ethernet cable into the RJ45 jack on the USB ethernet adapter (the other end goes in your networking router/switch).</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Connect the +5V and GND power wires from the RJ45 screw connector to the power connectors on the powered USB hub and also to the + and - on the breadboard.</span></div>
</li>
<li dir="ltr" style="background-color: transparent; color: black; font-family: Arial; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; list-style-type: decimal; text-decoration: none; vertical-align: baseline;"><div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Glue the one fixed magnet into the upper/higher magnet location in the mount, put the second, moving, magnet into the lower magnet location where it can slide around and put the other end of the paperclip through the hole from the back of the mount.</span></div>
</li>
</ol>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Early Prototype Photos</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="588" src="https://lh4.googleusercontent.com/SDrP-B30blNxqbBx8DndPEERcYtJUiprIk_EwLfDDc-A4szraodAu_xNAYnwpDiBTSX_YXqdg8tDp5v5PRV4QR3L0MYSwMH9znTvfHGt08JYm9gK1jdeawxyvkra6DjYc_y79ujX" style="border: none; transform: rotate(1.57rad);" width="331" /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="351" src="https://lh5.googleusercontent.com/sdHodLbyCTnEgwMgO7W414FtnaeEeWMMd0E_vFmmzEfngm_jbFYiVcYSeEPQNzdVJN2yHfo-pkIOiMqXnCXTqG4nsLl3Y6K3JAmlhL26tF_Gg7S8aNxNjGCZyR48CzUGZ3q8DXdH" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="351" src="https://lh6.googleusercontent.com/DBDTvoDcHBAoGdsC9RkV-vxc3eA2avx2jkLHmgHl--K9jvE0C2J5K-lmDRYFi5SVSJhxNV30asWYk2cmoN6_uG5iGCVucGqaP0vEZGbnX4BfCagbA46LLKPPPgDP38yyDI8eNL8f" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Finished Prototype Photos and Video</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="416" src="https://lh3.googleusercontent.com/iY_IXhuz4p9lhN4doM1tMDXjlEhdFWHbdCCOGKmVscYwKf04awiy8rvDKtTfjAKQ0QhjpKX2jH8e9PL9biCo_uNTlWIR0E2svA5kylRepTGV7m3n-12ppLfcSgpxh869VVG9ubXf" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="416" src="https://lh4.googleusercontent.com/L-X_rAfJMstWn8yoADQB9cbCBkWMDwaj9wwz4HSZh_aE_6jLtixukG8146MfMmkKtyfcHkrsBaVGXwh2AjGiDEapo5tJGP3IIxvEwrTPQlg_kWC7DhxgmwKj-sR8HRa6ITbeJCxu" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><img height="416" src="https://lh3.googleusercontent.com/Pwjc4zYvJq6QoIYBusqU_HFfALclWId0rHkTz6nJsbYY-uES15WiWxpsVsGev5TIgPn8Pvw0L5esUwHqm_1vYAiE9MTHicRwIcpvLnM8XzJNdBL3MKNuMJTLnY1H4FW1cevCV4by" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="624" /></span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Alternative Designs</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">In this section I explain a couple alternative approaches that I considered and either discarded or might explore further.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Turning on the iPad screen</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">My first design used an electromagnet that either is activated (to turn the display off) or deactivated (to turn the display on) instead of a permanent magnet moved by a servo. The flaw with that design is that the solenoid required about 2 watts of power from behind the tablet to trigger its magnetic sensor. Drawing 2 watts in order to keep a display turned off seems unacceptable, hence the newer design using two permanent magnets and a servo.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Another alternative design worth exploring further is simply interrupting the charging current to the iPad and then reinstating it. iPads appear to turn on their display when plugged in, so this is also an easy solution to turning the screen on. However, the screen won’t turn off until it times out per your display settings. This approach may work well with a short timeout for turning the display off.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 700; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Sensing a User in front of the screen</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I didn’t explore the alternatives for this much in designing this iPad mount because I had some experience with different approaches from an earlier project where I designed and built a directional motion sensor for counting the number of people in a room with a single doorway for entry and exit. </span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><br class="kix-line-break" /></span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The sensor choices worthy of contemplating are: 1) Passive infrared (PIR); 2) Ultrasonic; and 3) IR Laser. </span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The disadvantage of PIR is that it’s hard to tune distance and far away larger heat footprints look identical to smaller up-close heat masses. PIR is great for its low power and completely passive nature. Note that for Android tablets, it’s possible to configure a motion detection application that uses the (visual spectrum) front-facing camera and then uses </span><a href="http://tasker.dinglisch.net/" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Tasker</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> to turn on the display. Apple’s more limited power-user API precludes this approach. Using simple motion detection on the camera is flawed in similar ways to PIR, but a more heavyweight facial-recognizer would likely reduce false positives (at the expense of computational and energy expense.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Ultrasonic has a number of great properties including wider visibility and accurate distance measurement, but depending on the frequency, the sound may bother pets and more importantly, multiple uncoordinated ultrasonic sensors within earshot results in bad readings. See the </span><a href="http://www.alice.virginia.edu/~whitehouse/research/buildingEnergy/hnat12doorjamb.pdf" style="text-decoration: none;"><span style="background-color: transparent; color: #1155cc; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap;">Ultrasonic Doorjamb</span></a><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> for more of a discussion of this issue (which I only found after doing my doorjamb sensor that I ultimately switched to using IR laser). </span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Those shortcomings ultimately led me to use an IR laser sensor. One downside that is rumored that I’ve not encountered is the worry that it can confuse IR receivers on electronics and interfere with IR cameras.</span></div>
<h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 21.333333333333332px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Next Steps</span></h2>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I’d love to see a consumer electronics company pick up on some of these ideas and make a powerful and flexible wall mount using these ideas. I’m happy to be contacted by folks interested in productionizing this.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I’m still slightly partial to tablet-stands for tablets and may turn this more into a tablet-stand to place in a wired display shelf rather than mounted on the wall. </span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 14.666666666666666px; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">If the push buttons on the mount are not of interest or do not need to be labelled, the mount could shrink horizontally by mounting the distance sensor vertically and using a right-angle Lightning connector. In particular, that would be easier if making a full-height frame which would allow more vertically room for the sensor. (Thanks to Eric Badros for that observation.)</span></div>
<br />Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com1tag:blogger.com,1999:blog-7384142322553850856.post-49785866960482712822016-01-07T12:06:00.000-08:002016-01-07T12:16:19.924-08:00Meet Alexa, the Amazon Echo - New Features for Home AutomationIn early January 2015, I received the <a href="http://smile.amazon.com/gp/product/B00X4WHP5E/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B00X4WHP5E&linkCode=as2&tag=preparedmind-20&linkId=FQJWIHXRBPU6ZZGN&sa-no-redirect=1">Amazon Echo</a>. I'm always curious about new consumer electronics devices, and this looked to be something pretty different. It purported to be a music player (competing with Sonos and many others, it seemed) but also announced voice recognition, natural language understanding, and interactive question answering capabilities that put it in a different category.<br />
<br />
When the device did arrive, I was pleasantly surprised. In particular, it's a good quality single-channel bookshelf speaker: it can pair with a bluetooth device and act as a speaker for your phone or iPad as well as play music from your personal music collection or Amazon Prime Music or several other online music catalogs. But more importantly, it has a wicked-good array of microphones that pick up your voice commands after saying the hotword "Alexa" or "Amazon". I'd been trying lots of microphone solutions to integrate voice into my home automation system, and back in 2010 gave up on open-air speech solutions, relenting after configuring <a href="http://badros.blogspot.com/2010/05/skype-as-whole-house-wireless.html">Skype as a whole house microphone</a> which requires speaking into a phone or iPod Touch. The Echo microphones coupled with their voice recognition engine works beautifully from across the room even with some significant background noise. Off the shelf, you can ask it about the weather, sports scores, to play music or a specific song, and lots more. Everything you say shows up in a companion app on your phone that also lets you interact with some of the features via a classical small-screen user-interface.<br />
<br />
Jump ahead a couple months and Amazon released the <a href="https://developer.amazon.com/appsandservices/solutions/devices/echo">Echo SDK</a> which made it possible to do integrations as extensions to the grammar Amazon provided. Even the earliest versions of the SDK were sufficiently solid that I was able to code an integration into <a href="http://www.homeseer.com/">Homeseer</a>/<a href="http://rover-for-homeseer.blogspot.com/">Rover</a> in a short afternoon, so now we can say, e.g., "Alexa, ask house to turn fireplace lights on" and exactly that happens. It understands various devices, events, and scenes given the set of sample utterances I auto-generate from metadata of my home control software. Sidenote: I use <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> as the execution platform for the code -- I love not needing an always-running server to handle this computationally-simple and infrequently executed actions.<br />
<br />
<h3>
But lots is missing...</h3>
<div>
The <a href="http://www.amazon.com/gp/product/B00X4WHP5E/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B00X4WHP5E&linkCode=as2&tag=preparedmind-20&linkId=FQJWIHXRBPU6ZZGN" rel="nofollow">Amazon Echo</a><img alt="" border="0" src="http://ir-na.amazon-adsystem.com/e/ir?t=preparedmind-20&l=as2&o=1&a=B00X4WHP5E" height="1" style="border: none !important; margin: 0px !important;" width="1" /> is already a compelling gadget, and because it's off to a great start, I realized that it could be an important component in the whole-house automation system I'm working on for a new home we're building now. Here's my list of improvements to the hardware and software for the Echo that will make the Echo's successor be able to serve among the underpinnings of any smart home.</div>
<div>
<br /></div>
<h3>
Whole House Audio Support</h3>
<div>
The Echo could take a play from the playbook of Sonos and enable two Echos to work as a stereo pair, or perhaps pair with a speaker-only companion that can be the second speaker in a pair. For lots of folks, though, what would be better is simply enabling digital-audio output from the single Echo itself. The on-board mono speaker is great for talking back to the user, but for any at-length music listening experience, you really want to use an amp or receiver with your preferred speakers. Ideally the <b>digital-audio would be a coax out (not optical) since the coax digital plays nicer with inexpensive baluns</b> (gadgets on both ends that let you use the copper on a CAT5/6 cable to transmit signals other than ethernet) for a centrally-wired whole house audio system. The Echo should still turned down the music when it hears the hotword since that's an essential feature to being able to control the music after its started.<br />
<br />
In addition to wired digital audio-out, I'd love to see the Echo pair with other bluetooth devices as a music player (as opposed to as a speaker). I.e., to support the music being played by the Echo to be transmitted via Bluetooth (ideally with the aptX low-latency codec) to a bluetooth receiver connected to your preferred speaker system.<br />
<br />
An alternative to having Echo drive the music itself is to integrate Echo as a controller for SqueezeBox or Sonos music systems. Those systems are already in place in many houses driving speakers as desired, but don't have good voice integration. Asking "Alexa, play One by U2" (a tough sentence to parse for sure) should queue up that song on the SqueezePlayer serving the same room as the Alexa.<br />
<br /></div>
<div>
<h3>
Echo App Improvements</h3>
</div>
<div>
The companion Echo app on the phone/tablet also needs to be improved to be competitive with the other music apps out there. In particular, it needs to 1) start up in less than 1.5 seconds -- right now on a LG G4, I wait 10 seconds or more to do anything with the app; 2) integrate with Android Wear for pausing, volume changing, and confirmation of what you just said coupled to Undo functionality; 3) support multiple Echos including switching which one you're controlling. <br />
<br />
I'd also like to see the Echo app have a display mode where it's reporting about what's happening on the echo in that room. This mode would be useful for mounted tablets near each Echo for when voice control isn't sufficient or you just want to know what song is playing, pause it, change the volume, or whatever.<br />
<br />
Even better is for the Echo to be able to pair to a tablet as its display partner to support multi-modal interfaces rather than just having the tablet (or phone) reporting its status. For example, the voice command "Alexa, buy tickets to The Force Awakens" is best served by a continued interaction on a screen rather than reading off a list of possible venues, show times, and viewing options. If a screen isn't available, the more tedious voice interface could continue, but by using a touch screen in collaboration with voice commands, the interaction becomes much more natural allowing a click or a response like "The first one looks good" to finalize the purchase.</div>
<h3>
<br /></h3>
<h3>
Multiple Echo Support</h3>
<div>
One can easily imagine having an Echo in each room of the house, enabling music to follow you around the house (based on Bluetooth beacons or another signal identifying your location). An important part of this is enabling micro-location awareness of each of the Echos so that they know how they relate to other devices you wish to control. For example, in my bedroom saying "Ask house to turn the lights on" should have a different affect than in the living room: unqualified device descriptions need to use the context of the room to disambiguate. (And fully-qualified device names should work anywhere: "Ask house to turn the master bedroom ceiling lights off" should work from anywhere in the house.)<br />
<br />
<h3>
Person Identification</h3>
Related to multiple Echos and having per-room context is having per-person context. Minimally, certain functionality should be able to be limited to certain speakers (perhaps with an override code word when the voice is a close match but not close enough?) For example, "Alexa, disarm the security system" should do just that but only if a recognized voice issues the command (and perhaps only if a camera near the entrance also confirms facial identification of a household member).<br />
<br />
<h3>
Voice Notifications</h3>
Another useful feature is enabling push voice notifications to an Echo. If my garage door is open for more than 15 minutes, I have tablets in my house that are running my automation control software report "Garage door is open!" Ideally such notifications could be pushed to any/all of the Echos in a house, and have each local Echo support do-not-disturb functionality to block or delay those notifications from a specific room.<br />
<br />
<h3>
More Hotwords</h3>
Having just two hotwords is somewhat limiting (especially since my daughter's name is Alexis -- we can't use Alexa as the hotword, and the word Amazon comes up in our daily conversation too often). It would also be great if some hotwords could tie directly into a skill so instead of saying "Alexa, ask house to turn lights off" one could say "House, turn lights off" where "House" replaces "Alexa, ask house". Probably having two or three such skill-tied hotwords would make a significant practical difference for high-use skills. Note also that this approach sidesteps the more complex goal of extending the core grammar with developer-defined utterances -- that seems challenging to do in a scalable way across multiple third-party providers of skills.<br />
<br />
<h3>
Better Control of other Devices</h3>
<div>
There are already a handful of integrations of the Echo with other device ecosystems (e.g., <a href="https://support.smartthings.com/hc/en-us/articles/205275404-Amazon-Echo">with SmartThings</a>, <a href="https://ifttt.com/amazon_alexa">with IFTTT</a>), but I'd love to see the Echo just have better support natively for various IP-controllable devices like TiVos, HDTVs, etc. <a href="https://www.simplecontrol.com/">SimpleControl</a> (formerly Roomie Remote) does a fantastic job with a massive library of controllable devices (including support for IR blasters for older components), and Echo should be able to control the same class of gadgets that don't require new antennas, protocols, or additional hardware support. If they really want to play in the smart home hub space, adding ZWave (my favorite right now), Zigbee, Weave, Bluetooth mesh, etc., support would be pretty useful.</div>
<div>
<br /></div>
<h3>
Conclusion</h3>
<div>
I'm hopeful that the next generation Echo will have some set of these features, and certainly others from the clearly insightful forward-thinking team that came up with v1. I can't wait!</div>
<div>
<br /></div>
</div>
Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com1tag:blogger.com,1999:blog-7384142322553850856.post-24090385480491957432013-12-16T15:29:00.000-08:002013-12-16T15:29:00.752-08:00The next chapter: Prepared Mind InnovationsA little over two months ago, in early October, I finished my last day working at Facebook. Although that last Friday working lasted less than 10 hours, it had been almost a year in planning to ensure a smooth transition for the teams I worked with, and I'm grateful to the new leaders who didn't miss a beat as we shifted responsibilities around. I wanted to stop working full time in part to be really involved as we design and build a new house, but I can't stop working altogether. At Facebook, we have posters in every building asking (among other things):<br />
<br />
"What would you do if you weren't afraid?"<br />
<br />
<a href="http://prepared-mind.com/">Prepared Mind Innovations</a> is my answer to that: an advisory and innovation company for me to share experience, empathy, wisdom, and technical chops from my over 25 years of professional experience. I'm happy to report that I'm working with some great companies already (see the <a href="http://prepared-mind.com/">Prepared Mind Innovations page</a> for details).<br />
<br />
Also note that, separate from my company, as an individual, I'm doing a good bit more angel investing. <a href="http://www.badros.com/greg/index.html#investments">I've done over ten investments now</a>, and am still adding to that set of exciting startups. I'm looking for missions that will have a positive impact on the world, not just make a buck. (But I do believe that it's helpful to make a buck to fund a mission on the world, so I care about that, too!)<br />
<br />
Happy holidays!<br />
<br />Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com8tag:blogger.com,1999:blog-7384142322553850856.post-40063695127382324542013-01-21T09:25:00.002-08:002015-10-25T20:32:06.127-07:00My first three months with the Tesla Model S<h2>
Delivery of my Tesla Model S</h2>
<div>
(Want <a href="http://ts.la/greg681">$1,000 off your new Tesla</a>? Use <a href="http://ts.la/greg681">this code</a> which expires October 31, 2015.)<br />
<br />
In early 2010, I started looking for a new car to replace my beloved 2006 Infiniti M35x. I love technology and I love gadgets, and the M in 2006 was well ahead of its time with adaptive cruise control, steerable headlights, lane-departure warning, voice recognition, Bluetooth integration, and more. I looked for a couple years for a car that I thought would be a worthy successor to the M, but no luck. The closest candidate I debated was when the M itself got redesigned in 2011 as the M37, but even that seemed like too little of a tech upgrade for the 6 years that had passed since I first drove the M in 2005.</div>
<div>
<br /></div>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnxsc2E3_eNeiP9eMyVf4HCbCJemterxc-yIFcN41h9MksAfM9fyb52_qu52YZgepp5TKDKuDg0lhSK38rtJZv9sRddboBr2fEsLVgwQDnqp6-kGdsVnDwWmYpxcXRbc36CEbBMRPZDQMe/s1600/tesla-version-1-15-14.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnxsc2E3_eNeiP9eMyVf4HCbCJemterxc-yIFcN41h9MksAfM9fyb52_qu52YZgepp5TKDKuDg0lhSK38rtJZv9sRddboBr2fEsLVgwQDnqp6-kGdsVnDwWmYpxcXRbc36CEbBMRPZDQMe/s400/tesla-version-1-15-14.jpg" width="250" /></a>Finally in September 2011, I decided I'd place a bet on the newly announced Model S sedan from Tesla, the Silicon Valley auto maker that previously developed the Roadster. Because I was anxious to get into a new ride, I opted for the signature series, so I could be among the first 1,000 customers to take delivery, and I put down a biggish deposit to get in line.</div>
<div>
<br /></div>
<div>
Thirteen months later, my Tesla Model S Signature 85 was delivered direct to my house by an energetic customer representative and a rep-in-training shadowing her. Perhaps one of the greatest features of the car is the initial ownership experience: no haggling with car salespeople, but just configuring it online and intelligent people responding in a timely fashion to my every inquiry, all via email. Then it shows up at your door, with an in-person instruction manual.... nice!</div>
<div>
<br /></div>
<div>
That was October 17th, and now, three months later, these are my thoughts....</div>
<div>
<br /></div>
<div>
<br />
(The firmware version on which most of this is based is 1.15.x.)</div>
<h2>
Driving the Model S</h2>
<div>
Okay, so I admit, while my M was fun to drive, the tech features were really what drew me to the Infiniti almost 8 years ago. The promise of great technology also drew me to the Model S. However, what's most remarkable and actually the biggest surprise to me is...</div>
<div>
<br /></div>
<div>
<b><i>The Model S is just unbelievably fun to drive!</i></b></div>
<div>
<br /></div>
<div>
I can't stress that enough... it's whisper-quiet and the acceleration is smooth and immediate, no matter whether you're starting from a stand-still or already going 40 and wanting to hit 65 in a hurry, it's simply awesome! From a stoplight, "green" now means seeing the car that was behind you disappear into a tiny spot faster than I could've imagined.</div>
<div>
<br /></div>
<div>
Handling is really good too; the massive and heavy battery pack (total vehicle weight is over 4,500 lbs) sits very low on the car, so I can't detect any body rolling when cornering. I do miss the AWD of my Infiniti, but I probably will be driving my wife's Infiniti JX up to the mountains, anyway. Perhaps most importantly, I miss the adaptive/laser/smart cruise control feature that my M had been in 2006. That's a real step backwards and hopefully something that future Teslas will get in a hurry, though I'm told the car lacks the forward radar or laser sensors so this won't be a feature added via a software upgrade. </div>
<div>
<br /></div>
<div>
Another feature of my M that I miss on the Model S is the steerable headlights that point "around" corners. The Model S does have cornering lights on both sides that turn on when you've turned the steering wheel more than a certain amount in that direction, but it's not nearly as useful on on ramps and off ramps... it seems to be more intended for making slow-moving turns in urban areas, and it works great for that.</div>
<h2>
The Physical</h2>
<div>
Simply put, the car's exterior is beautiful and shapely. I won't spend a lot of time discussing this because it's really table stakes for a luxury sedan, but Tesla definitely succeeded in making the car attractive! The interior is very nice -- it's rather minimalist, but I very much like the styling of the glossy Obeche wood; the one negative is that the glossy finish does reflect a bunch of sunlight, but I still prefer it over the matte finish which, to my eye, looks worn and dull.</div>
<h4>
The Frunk</h4>
The frunk (front trunk) is enormous and cool, but it's tedious to close: I was warned by my delivery specialist to use two hands to close it, and instructed on the exact location of where to put each hand to avoid bending the super-light aluminum hood. The frunk is definitely not going to be your everyday go-to location for storage. At least not until it has a motor to pull itself shut like the hatchback does. It does have the advantage of being completed closed and concealed (compared to the back door's storage area which is accessible to anyone who gets inside the car).<br />
<h4>
Interior Controls</h4>
There are no interior physical controls for the sunroof nor for the locks. The lack of controls for the door locks is less big of a deal than I thought it would be since the car has some good smarts about auto-locking as you start driving, auto-unlocking when you shift back to park, and auto-locking again as the key fob gets a dozen or so feet away from the vehicle, so modulo safety concerns (see more on that below) the locking at least isn't a convenience issue.<br />
<br />
The lack of dedicated interior control of the sunroof, on the other hand, is a headache. It often is three clicks on the touchscreen to open the sunroof, and takes many moments looking at the screen (instead of the road) to make that happen. It's tedious, and I find myself using the awesomely-large sunroof less than I did in my M because it's so hard. (It sounds like the upcoming 4.x firmware updates allow you to control the sunroof using one of the dials on the steering column; that's a pretty unusual location for a physical control for the sunroof, but in California it might be okay. I'd hate to accidentally open the sunroof in Detroit during a winter snowstorm!)<br />
<h4>
Safety Concerns</h4>
The proximity key fob generally works great on the side doors as well as the hatchback. It has one quirky failing, though. If the key fob is near the driver's side, the passenger side door can be unlocked and opened. That seems like a misfeature... it's already hard enough to lock your doors immediately upon getting in the car, and even more dangerous to have someone hop in the car on the passenger side as you approach the car to get in because it makes no distinction between the two sides of the vehicle.<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
The Model S does, at least, have a physical hazard-lights button and an unusually-placed-on-the-gear-column emergency brake. That was thoughtful and good! It also has a standard cluster of controls on the driver's side armrest for the windows, the side-view (electrochromic dimming) mirrors, but no lock controls:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1yJVBiTRc7_xKocW7ED5RmEbTgf_0oS1widc0Vf3RXafG-YbMW77uvPBeyEP44KMBhti5S5_KljBDDfAoetoZpX8-qsR7h2d9bXD1kAatspKCDJhY2fDckLycjgjTYrSbegffVdfazTxx/s1600/tesla-drivers-side-controls.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="230" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1yJVBiTRc7_xKocW7ED5RmEbTgf_0oS1widc0Vf3RXafG-YbMW77uvPBeyEP44KMBhti5S5_KljBDDfAoetoZpX8-qsR7h2d9bXD1kAatspKCDJhY2fDckLycjgjTYrSbegffVdfazTxx/s320/tesla-drivers-side-controls.jpg" width="320" /></a></div>
<br />
Contrast the above driver's side with how bare the physical controls are for the other seating locations with just a window control (again, no lock controls):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqK2RG3S-svh9OBiYjplwXCyM-rYokbMDYSE0YoOvBGHsHtSsAXpb3bn2D4oxpzXNF4GkNDIfINyqH7MWvPj-6Od4Qgk0HHZ3c4g1Z1mOsnlfuUMGtRFFfqvzNUXVDSatmArI2i4qvjNUH/s1600/tesla-passenger-door-controls.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqK2RG3S-svh9OBiYjplwXCyM-rYokbMDYSE0YoOvBGHsHtSsAXpb3bn2D4oxpzXNF4GkNDIfINyqH7MWvPj-6Od4Qgk0HHZ3c4g1Z1mOsnlfuUMGtRFFfqvzNUXVDSatmArI2i4qvjNUH/s320/tesla-passenger-door-controls.jpg" width="320" /></a></div>
<br />
<h4>
Miscellany on the Interior</h4>
<div>
The Model S chose to use a non-standard connector for charging, and that's fine when charging at home. However, I'm one of the fortunate Silicon Valley folks whose employer has free electric vehicle charging at work, and that means I'm using the adapter all the time. A useful addition would be a permanent and easily accessible spot for the adapter to attach, perhaps a storage location on the inside of the left door. Right now, I put it in the smallish glovebox, and use velcro to attach my EV charging cards to the door of the glovebox. I've also written my email address in silver-colored permanent ink on the adapter (the part that is hidden in the vehicle while charging) in case I ever forget to detach the adapter when returning a public charger to its base; the EV community is fantastically supportive, so I have no doubt it'll get returned to me should I ever be careless!</div>
<div>
<br /></div>
<div>
By the way, that smallish glovebox is really the only concealed storage accessible from the driver's seat. Tesla talks about different configurations of convenience consoles, but I've not heard much more about them since delivery. Right now, I use a velcro-attached zippable squared-off bag to put my phone and other essentials into; otherwise they'd just slide all around in the very open center area. There's only very short 1.5" walls preventing these objects from sliding under the accelerator (not gas!) pedal and brake; that would be bad while driving!</div>
<div>
<br /></div>
<div>
A bunch of other little nits about the interior include:</div>
<div>
<ul>
<li>The rear window really is just a slit -- this it the price you pay for the amazing aerodynamics of this car.</li>
<li>The sunshades are really small and don't have lit mirrors. They really couldn't be much bigger, though, because of the sleek front styling.</li>
<li>The heated seats are great -- they warm up fast and it sounds like that's a recommended lower-energy way of warming up (rather than turning on the climate control system that uses fans and has to blow air around). The heaters should probably turn off automatically if there's no weight detected in the seat, though. I do miss my ventilated seats from my M, though... cooled seats sounds like an extraneous luxury, but not on hot California summer days!</li>
<li>There are rails on which the front seats move forward and backward via nice power controls. The fronts of the rails -- the part closest to your leg -- are <i>really</i> sharp... my wife scraped some skin off of her leg getting in the first time. Most cars seem to have some kind of plastic cover on the end cap of these rails, and that'd be nice to add.</li>
<li>The hatchback is supposed to have a privacy cover -- at delivery time they told me they were having supply issues on those... three months later, I'm still waiting for that. Last I've heard, I should be getting mine in February 2013 or so.</li>
<li>There are no rear-seat ceiling grips... they'd be useful to backseat passengers during cornering demos :-), but also I used to use them on my M for hanging dry-cleaning.</li>
</ul>
<div>
None of these things are terrible, they're all a little disappointing for the most expensive car I've ever owned, but probably perfectly reasonable at the entry-level price-point for the basic model.<br />
<h4>
Miscellany on the Exterior</h4>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
The retractable handles are the most talked-about unique feature of the exterior. They're super cool, and will be even cooler when they come out as you and your keyfob approach the car. In the 1.15 firmware, you have to push the handle a bit before it "wakes up" and comes out so you can then open the door. This is a little tedious at first, but soon becomes second nature. A surprise issue with the retractable handles is after a car wash or rain: they stay wet for quite a while since they're not exposed to the sun or wind to dry off.<br />
<br />
One other really nice and unexpected touch: the Tesla-supplied power cable has a button on it to trigger the opening of the charge port to plug-in. That wows friends at least as much as the handles.</div>
</div>
<h2>
The Technology</h2>
<div>
Drool... the 17" touchscreen. <br />
<br />
This is always the first thing my techie friends ask to see in the car. And it's nice. It does a good job of not reflecting sunlight while also not being obviously polarized in a way that can conflict with my sunglasses's polarization. It is multi-touch, and within the web browser and the navigation apps, the usual gestures for zooming in and out and panning around work as you'd anticipate, though they're more sluggish than even a first-generation Apple iPad (but those aren't 17", either :-) ). So far I have not noticed any web apps customizing their behaviour for the Tesla (the Qt-based browser reports "<span style="background-color: #f1f2f6; color: #333333; font-family: 'lucida grande', tahoma, verdana, arial, sans-serif; font-size: 10.909090995788574px; line-height: 12.727272033691406px;">Mozilla/5.0 (X11; U; Linux; C) AppleWebKit/533.3 (KHTML, like Gecko) QtCarBrowser Safari/533.3</span>" as its User Agent). I'll say more about the apps and the user interface of this large touchscreen in a bit.<br />
<br /></div>
<div>
There's also a smaller non-touch screen directly in front of the steering wheel. The center of that shows things like the spedometer, the odometer, the cruise-control settings, etc. On each of the right third and the left third of this smaller display, you can pick one of set of apps to show. The left-third can also show a Garmin perspective-view navigation display that's loosely sync'd to the map view on the main display. More about the navigation in a bit.</div>
<div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoJVNRCz-BtBp5eQu4AvOBsCSAr4K4U4Y4hphzrq464btcsWdN19CnnMGOk45Qa02ESOh1GmoUrEIkGJLHmRYrs61eHuiFEg8J-G6Pt7hn0u6mCmj-OuWL8CIbkI_IE2tbypgYYEztYgyH/s1600/tesla-left-non-touch-screen.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgoJVNRCz-BtBp5eQu4AvOBsCSAr4K4U4Y4hphzrq464btcsWdN19CnnMGOk45Qa02ESOh1GmoUrEIkGJLHmRYrs61eHuiFEg8J-G6Pt7hn0u6mCmj-OuWL8CIbkI_IE2tbypgYYEztYgyH/s400/tesla-left-non-touch-screen.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Left (non-touch) Screen directly in front of driver</td></tr>
</tbody></table>
<br /></div>
<div>
The system overall depends on a data connection. The current version uses a 3G connection, though folks at Tesla I've spoken to have been cagey about whether there's actually a 4G radio in the car or not. From the number of bars shown onscreen, it seems that they're using AT&T to provide the data package, and it's free, I'm told, for the first year at least. Tesla does report that there is a WiFi radio and they expect to turn on WiFi functionality in a later firmware update. This will probably be especially nice for folks who don't have good AT&T coverage in their garage. Ideally, the apps will get smarter about downloading and caching more when on a better connection. Right now the Slacker Internet radio and the navigation tiles fail when data connectivity is bad, and that is pretty terrible compared to other cars' navigation systems that hold data for the whole country (or at least a region) on a spinning hard drive or DVD.</div>
<div>
<br /></div>
<div>
Speaking of DVDs, it's worth noting that there's no optical disc capability in the Model S whatsoever. No CDs, no DVDs, no BluRay. There are two USB ports that are easily accessible in the all-too-open center console, and one 12V power port (into which I plugged a charge-only USB adapter). No video inputs, either, and it doesn't look like the car support MirrorLink yet or any other technology to use the 17" screen to show content from your phone or other devices. Bluetooth or the USB ports let you play music from your phone, including streaming from Pandora or Spotify or whatever your favorite music service is.<br />
<br />
The Bluetooth works reasonably well, although the first firmware I had took many minutes to sync all the many hundreds of contacts from my Samsung Galaxy SIII. It worked lots better with my iPhone 4, and today works fine with my iPhone 5 and passably well with my SGS3.</div>
<div>
<h2>
UI Framework</h2>
<div>
The basic user interaction framework for the main screen is straightforward. There's four stacked areas top to bottom:</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQqCGml0BfdkjJa3jor0BQ96VLhT1mF5jVSVWnY7YrgdNNflIYOXX6r0pyiMCuQ03oLjTBvUo-HzoFvtwfgiSS22MeNEnOAyMz5PBePc7KPi4XkFzY2Pkc6o7eS1YHoLuPw5HgDsdT-BeO/s1600/tesla-split-screen-view.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQqCGml0BfdkjJa3jor0BQ96VLhT1mF5jVSVWnY7YrgdNNflIYOXX6r0pyiMCuQ03oLjTBvUo-HzoFvtwfgiSS22MeNEnOAyMz5PBePc7KPi4XkFzY2Pkc6o7eS1YHoLuPw5HgDsdT-BeO/s640/tesla-split-screen-view.jpg" width="425" /></a></div>
<ol>
<li>A thin status area showing (from left to right) the current temperature, battery state (clicking opens charge controls window), the homelink control (for operating your garage door), the seating presets control, the stylized "T" Tesla logo, the Bluetooth control, data connectivity status, and the time.</li>
<li>A dock-like horizontal row of the applications showing (from left to right) Media, Nav, Energy, Web, Camera, and Phone. The app or apps that are currently visible onscreen are shown in reverse text (white on a black background).</li>
<li>The largest middle section which shows the active app or two -- each of the apps can be full height (and consume all of this area) or half-height. In the picture to the right, you see the Nav app in the top, and the Media app in the bottom, each half-height.</li>
<li>A bottom set of fixed controls showing (from left to right) "Controls", driver-side heated seat, driver-side temperature, climate, passenger side temperature, passenger-side heated seat, and volume. </li>
</ol>
<div>
The primary mechanism for getting around in the UI is using the dock-like horizontal bar second from the top. Touching any of the apps that isn't currently being displayed makes that app show in the top half of the large middle section of the display (the area described by the third bullet above). There's a couple of reconfiguration controls on those app windows, too:</div>
<div>
<ul>
<li>at the bottom left of each app window, there's a small icon that toggles that app between being half-height and full-height (apps that don't support full-height mode don't have this icon -- Energy, e.g., only runs half-height).</li>
<li>at the middle of the split screen (when and only when the main display is split) at the far right, there's a small curved arrow icon that swaps the top and the bottom app windows. </li>
</ul>
<div>
Although these controls are reasonably intuitive, there's a big usability issue lurking in this v1 design: when an app is full-height, there's no visible indication of what the "hidden" half-height app is, even though that hidden state is persistent. So, for example consider these two scenarios:</div>
<div>
<br /></div>
<div>
A) From the state in the above picture, if you maximize the Nav app to full-height, the Media app won't be visible and only the Nav app will show in reverse text in the dock. However, when you then touch, say, the Energy app, that app will take over the top half of the screen, and Media will start showing again in the bottom half. Versus...</div>
<div>
<br /></div>
<div>
B) From that above picture, if you first swapped the Nav to the bottom half (so Media is on the top half) and then clicked maximize on Nav, Nav will be full screen, also with no visible cue about the Media app. Now, though, when you touch the Energy app, that app still takes over the top half of the screen, but Nav will be showing in the bottom half.</div>
</div>
<div>
<br />
A simple but significant improvement would require two changes:<br />
<br />
<ol>
<li>simply indicate which app, if any, is hidden from view when there is a full screen app -- probably a different rendering of the text of the app itself in the dock. This would let the driver know what app will come into view if they un-maximize a full-height app; and</li>
<li>when an app is full screen and another app button is touched in the dock, always put that current full-screen app in the bottom half of the screen and put the newly selected app in the top half-height. A possible variant on this: If the new app can be full-height then it could come up full-height with the prior full-height app being switched to hidden bottom-half-height mode.</li>
</ol>
</div>
<div>
Furthermore, there's missed opportunities here. The dock buttons do absolutely nothing if the app is already one of the two visible apps. I'd prefer if the followed the following heuristic:<br />
<br />
<ul>
<li>If the user touches an app that's already in the top spot half-height, make it full-height if possible</li>
<li>If the user touches an app that's already full-height, toggle it back to half-height in the top spot</li>
<li>If the user touches an app that's already in the bottom spot half-height, switch it to the top half-height (this rule plus the first rule mean that double-tapping an app that's half-height bottom will make it full-height)</li>
</ul>
<h4>
Touchability Heatmap</h4>
<div>
Although the above aren't strictly necessary, they would be valuable additions because of one significant observation about the giant 17" display: it's hard to aim your touches when you're driving. For one, you don't want to take your eyes off the road, but second, notwithstanding the super smooth ride, as the car jostles around, small targets are just harder to hit than, say, and iPad sitting on your lap with both hands available to interact. To Tesla's credit, they've placed the smaller touch-targets at the easiest-to-hit parts of the screen. After a couple months with my Model S, here's a heatmap I threw together of what I think the easiest-to-touch (reddest) parts of the screen are:</div>
<div>
<br /></div>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidLdaOE7RQ7dVzA4SYyU7at3EgKSCgcOB8WQLveXNmOThdI-fUZ0pRPlkoO9Jc0CDoFEaIvpa2wYlk1Svr8TNpVqcDfkGtvj1UeHDursESFxZgPa2xAQXVXwyhRli8zh9QY5RWegGMXjyx/s1600/tesla-split-screen-view-touch-heatmap.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidLdaOE7RQ7dVzA4SYyU7at3EgKSCgcOB8WQLveXNmOThdI-fUZ0pRPlkoO9Jc0CDoFEaIvpa2wYlk1Svr8TNpVqcDfkGtvj1UeHDursESFxZgPa2xAQXVXwyhRli8zh9QY5RWegGMXjyx/s640/tesla-split-screen-view-touch-heatmap.jpg" width="426" /></a></div>
<br />
<br />
The corners are by far the easiest, especially the bottom left (where the "Controls" button is conveniently located -- you'll use this lots). All four sides are next easiest, with the left and bottom a good bit easier than the top and right. The bottom is easier because there's a ledge you can rest your hand on while reaching up to calibrate your touch. Similarly, the swap-windows icon is a little easier to hit in the middle of the right edge because of some of the trim adjacent to the glovebox (misnamed for California :-) ). <br />
<h3>
Popup Overlay Windows</h3>
The final significant framework component is the modal overlay window with optional tabs (sometimes both horizontal and vertical). For example, a big overlay window takes over nearly the entire screen when you click the bottom left "Control" button:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrFa5JYA8NJgCsp1xfDKkkaaGV_KCNCyfdo4JIzRHoeZy-zfTHbKcIDjZHn4KLYk6R20wRJZoe-XNHBtyfcNnMtctyg15pCmxJwVMUflP51CBSHOZOBbP0psW1ZND2WHc32gBTM2sMhttu/s1600/tesla-control-popup-overlay.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrFa5JYA8NJgCsp1xfDKkkaaGV_KCNCyfdo4JIzRHoeZy-zfTHbKcIDjZHn4KLYk6R20wRJZoe-XNHBtyfcNnMtctyg15pCmxJwVMUflP51CBSHOZOBbP0psW1ZND2WHc32gBTM2sMhttu/s640/tesla-control-popup-overlay.jpg" width="412" /></a></div>
<br />
This large window consumes nearly the entire screen -- only the top thin status bar and the bottom comfort controls bar remain on screen. The popup overlays have an "X" icon in the top left to dismiss them, however, it's generally easier to just click either on the background or, in the case of the Controls popup, clicking the easy-to-find-blind Controls button in the bottom left toggles the Controls popup overlay window away.<br />
<br />
There's lots going on in the Controls window. In particular, a whole bunch of nifty settings are hidden behind the Settings top tab. That tab selection only changes the top half of the Controls window -- the bottom half thankfully always shows the Doors & Locks and Lights section. I'd prefer if the sunroof were always visible, too: the Controls window's configuration is persistent between uses of the popup, so sometimes it can take several clicks to get to the sunroof setting which is one that I tend to use very often. Overall, the Controls window feels strange as a popup overlay -- I think it might be improved as an animated slide-up card -- that might also suggest the easy affordance of swiping it down to dismiss the screen.<br />
<h4>
Swipe Gestures</h4>
As a related quick aside on another area that feels like a missed opportunity: swipe gestures are conspicuously missing from the experience (excepting the Nav app that lets you drag the map around, pinch to zoom, etc.). For example, the thermostat controls (which can be synchronized between the driver and passenger) are nicely positioned, but require repeated touches for each degree up or down you want to change the temperature. I think it might be better to figure out how to allow drags to change the temperature in a single motion. E.g., you could click and drag up to change the temperature up based on how long you move your finger up (ignoring the speed of the movement), and a drag down would reduce the temperature. Since you can't drag down initially (there's no room under the button) you'd have to swipe up quickly and then drag down slowly to reduce the temperature. There'd need to be a visual indication of the virtual temperature slider to help as user's learn the admittedly weird gesture, but in time it'd be easier to do without having to look at the screen for each of multiple touches. (And the existing behaviour could stay, too, as it's easier to learn, but harder to do while driving.)<br />
<h2>
The Apps</h2>
Most of the apps are pretty basic, and I have some thoughts on each of them.<br />
<h4>
Energy App</h4>
This is very bare bones. I'd love to see vertical lines that are time markers so I can correlate when things happened with how energy usage changed. It would also be nice to have annotations explaining things like "Climate control turned off" or "0-60 in 4.7 seconds" :-).<br />
<h4>
Media App</h4>
The Media app is still very early. Before the first firmware update I got (to v1.15) it was almost impossible to use a high-capacity USB stick... it took forever to scan the folders. Luckily that wasn't too big of a deal because the Internet radio provided by Slacker works well as long as there's good 3G data connectivity (it skips plentifully when there is not). One of the little details that is surprisingly welcome about the Slacker radio integration is that the current song pauses as you stop the car, and starts back right where you left off when you get back in. I.e., it works more like your own music collection than FM or XM/Sirius radio. <br />
<br />
Another example of a popup overlay window (like the Controls window described above) is the fader and balance control for the Media app. It looks like this, and uses a nice and intuitive display and control of the acoustical center of the the vehicle:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrYPjydauh_yknrahnlKNjFByq6z9wSL9AoLucmF0Crk8KL_axmDvzFs8sOMQLjZEfO_J6nSdgj0qva71QWaeiiWKBfN2t484MJZ3Ckkf3ft6SDfkkUZPFySkSq6GskoXL9MVyzds2UOQv/s1600/tesla-fade-balance-controls.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrYPjydauh_yknrahnlKNjFByq6z9wSL9AoLucmF0Crk8KL_axmDvzFs8sOMQLjZEfO_J6nSdgj0qva71QWaeiiWKBfN2t484MJZ3Ckkf3ft6SDfkkUZPFySkSq6GskoXL9MVyzds2UOQv/s400/tesla-fade-balance-controls.jpg" width="260" /></a></div>
<br />
<br />
In the top down view, you simply drag the center disc up to make the front speakers louder, and to the left to make the left side louder. Basically, you drag the disc over the person whose talking you'd like to drown out! :-)<br />
<br />
Other big issues with the Media player include:<br />
<br />
<ul>
<li>No support for Slacker premium -- I can't search for artists and play them.</li>
<li>No de-duping of MP3s from a high-capacity USB stick</li>
<li>No ability to navigate a USB stick by folder and just find a specific file; if the ID3 tags are mangled or poor, this is essential. It's also really useful for a rips of foreign-language CDs where the ID3 Artist tag used isn't easily guessable.</li>
</ul>
<h4>
Camera</h4>
<div>
One unusual feature of the Camera app is that it can be turned on and stays on at any speed. It automatically pops up when you put the car in reverse, and if it popped up automatically, it'll also automatically go away when you exit reverse. </div>
</div>
</div>
</div>
<div>
<br /></div>
<div>
The big and notable missing feature for Camera is overlay lines showing the trajectory of the car given the current steering wheel position. That feature has been on lesser vehicles for almost ten years, and should be an easy software update. Sonar warning would also be nice, but the Model S lacks those physical sensors, so that's not coming as a software update.</div>
<div>
<br /></div>
<div>
Compared to other high-end cars these days, it's also missing side and front view cameras, but those won't be coming via software, either. :-(</div>
<h4>
Phone</h4>
<div>
Primary things missing here are better navigation of contacts and better integration with the phones favorite contacts list. Car-specific favorites would also be nice. This probably becomes less problematic with the newer firmware the supports voice-recognition-based dialing.</div>
<h4>
Web browser</h4>
<div>
The basics of the Webkit/Qt-based browser work reasonably well, though as mentioned before, it feels a little sluggish compared to even the first generation iPad. Unfortunately, the car really needs a valet mode (see more on this below) or some other mechanism of protecting the form-fill history and cookies of the web browser or else it's useless for logging in to any personalized services such as Facebook, or Google Calendar or Gmail -- any site with login credentials saved to the browser presents a big risk. Very welcome would be some mechanism to, e.g., PIN lock the cookies so that after each start of the car, the user has to enter a PIN the first time they need to send a cookie to a personalized site (or maybe to a list of personalized sites).</div>
<div>
<br /></div>
<div>
Other obvious wins that aren't yet implemented include:</div>
<div>
<ul>
<li>Phone numbers on web pages should be clickable to dial using your bluetooth-connected phone</li>
<li>Addresses on web pages should be clickable to enable navigation to that location</li>
</ul>
<div>
More interesting would be platform-like integrations to enable experiences such as navigating to a Facebook event that is coming up shortly after you enter the car, text-to-speech to read emails or Facebook posts, and more.</div>
</div>
<h4>
Nav</h4>
<div>
If, in hindsight, the Tesla engineers could've made more progress on any app before launch, I bet they'd wish those improvements on the Nav app. It's not that it's bad it's just that it's a place that the car could've really shined and still truly doesn't. I have no doubt it'll get great, but right now it's a mixed bag.</div>
<div>
<br /></div>
<div>
The first thing you'll notice about the Nav app is that it's really just a Google Maps app that integrates with what appears to be Garmin navigation technology and UI. Worse, that integration is very loose: the navigation features on the main screen are limited to drawing an overlay path on the main map and showing a small rectangle with contents that list the next couple driving instructions the Garmin-powered nav gives. Meanwhile, the left side of the left screen (the screen in front of the driver) shows a bird's eye perspetive view of a very-different-looking map. Perhaps worst, even though the main Google map shows traffic information, it lacks incident information (e.g., accident and construction markers) and the traffic information it shows is in no way reflected in the navigation directions or time estimates. Fail! (And I mean <b>fail</b> in a big way -- I spent three hours driving to Half Moon bay once when I could've been redirected around a major traffic backup and saved 1.5 hours.)</div>
<div>
<br /></div>
<div>
The Google Maps integration actually has several significant problems on its own, independent of its lack of integration with the navigation controls. These general seem to be limitations due to a very simple use of basically the stock (older) Google maps feature set. Among the problems are:</div>
<div>
<ul>
<li>The map is always in North-Up orientation; there's no option to set the car's current compass heading up and having the map rotate appropriately to keep that orientation. </li>
<li>The map is always in top-down view mode; there's no perspective view option</li>
<li>The street names are too small to read, though there's a somewhat wonky "zoom" setting that seems to just scale up the map by 1.5x or 2x to try to alleviate this problem.</li>
<li>The tiles don't seem to be cached at all (or certainly not enough). This is awful: if you're in a low data connectivity area, don't even think about changing the zoom level or panning around.</li>
<li>In track-the-current-position mode, the triangle representing the car stays dead-center. Ideally it'd show at least a bit more information in the direction you're driving (like older videogames do in 3rd-person top-down views).</li>
<li>The navigation path highlight makes it really hard to see the color of the traffic colorings underneath; I sometimes find myself turning off the current destination to get rid of that highlighting just so I can see the traffic pattern colors better.</li>
<li>Pinch to zoom in/out works really slowly and sometimes doesn't actually change the scale of the display: I hope someone is working on using the vector-graphics-based Maps instead of the tiles, as that's the long-term right fix to this.</li>
</ul>
<div>
Now it's not all bad. The satellite view at fully-zoomed-in scale is super fun to watch (as a passenger, of course) while driving along. Also, entering a destination is far easier than on any other car I've ever used: this seems to be integrated directly with a Google search so typing in a destination of "Stanford Hospital" actually works as you'd want it to (and if that's your destination, you'll be glad to save those few moments)!</div>
</div>
<div>
<br /></div>
<div>
Another feature idea: I'd love the ability to bookmark position/zoom-level of the map. I find that for regular commutes, what I most want is to see an overview of the traffic on the bay area roads and highways between my home and work, and I'd like to get to that configuration of the map with just a touch or two.</div>
<h2>
Firmware Updates</h2>
About a week after I received the Model S, I got an over-the-air firmware update. The car downloads the new software in the background over the 3G network, and then prompts you to schedule a time for the car to swap in the new packages. It takes a couple hours for the download to happen, but then the installation completed in only about 25-40 minutes for me. This is without a doubt the most important feature of the Tesla philosophy: that iterative improvements with customer feedback is the better way to innovate and advance the state of the art.<br />
<br />
By the way, the car seems to do minor firmware updates without notice. For example, I knowingly updated to 1.15.5, and sometime since then I got up to 1.15.14, but never took other action to update further. <br />
<br />
Tesla does recommend you roll down a window when doing the update installation, presumably as a precaution to make sure you can open the car door by reaching in the window and using the interior physical handle. Luckily I have a garage, so this isn't a big deal, but I imagine for some it could be more problematic (especially in the winter).<br />
<br />
Another outcome of this continuous-software-improvement philosophy: when I took delivery, at the very start of using the system, I was taught by the super-helpful delivery specialists how to reset both the left screen and the main screen -- they reboot independently. Significantly, neither of those subsystems that can be user-rebooted seem to contain anything critical to driving the car: I've rebooted each of them while driving, and while it's crazy dark in the interior (no speedometer, no odometer, nothing), the Model S drives just fine.<br />
<h2>
New Feature Ideas</h2>
<div>
Here's a partial list of features that I wish the Model S has and I hope Tesla is working on. Significantly, I'm trying to focus on things that I believe are possible with the current hardware.</div>
<h4>
Valet Mode</h4>
<div>
As I mentioned earlier in the web browser description, it's a bad omission that there are no security controls over the login credentials associated with websites. Minimally, there needs to be a simple valet mode that's triggered by a couple touches and requires either a custom code or a pre-saved PIN to get back to regular driving mode. The valet mode should (joint list with my friend Sean who I brainstormed with):</div>
<div>
<ul>
<li>provide basic usage details onscreen explaining how to drive the car</li>
<li>lock out access to the web browser, prior navigation destinations and phone numbers, and bluetooth pairings to phones</li>
<li>lock out access to any USB sticks and any data usage for media, limit the volume of the stereo</li>
<li>switch to auto off lights, put the car in regular creep mode, and close the sunroof</li>
<li>lock out access to the frunk (and glovebox if possible)</li>
<li>optionally lock out access to the homelink controls</li>
<li>limit the acceleration, and always switch to the highest suspension setting the current speed allows</li>
</ul>
<div>
If you forget the pin, you'd need to call customer service to unlock the valet mode; questions about recent prior destinations and other things that valet mode locks down could be used to verify your identity, along with other account information.</div>
</div>
<h4>
Presets for Windows and Sunroof</h4>
<div>
Given how awkward it is to control the sunroof, one approach that Tesla seems to be going down is making it easier to control the sunroof. Perhaps another more tantalizing option is making a windows-presets control in the top thin status bar. Analogous to the seat profiles dropdown from the status bar, the window presets would let you save the current window and sunroof configuration to a preset slot and then easily go back to one of the saved settings. This would let you, e.g., switch between all-closed and my preferred sunroof-open-all-the-way-and-back-windows-cracked-a-bit setting with just a couple of touches. That would leapfrog the much-easier physical controls for the sunroof that other cars have by simplifying what is otherwise a multi-step process.</div>
<h4>
Caldav Syncing and Send-to-Car</h4>
<div>
Ideally the car will sync with my Facebook events and Calendar and make it easy to navigate to any of the locations in the upcoming hours, being smart about how long it takes to get somewhere so that a trip to Tahoe will be called to my attention 6 hours before I'd need to leave, while a trip down the street wouldn't prompt me to start navigation until closer to the appointment time. I'd also like to be able to send a map address or set of phone numbers to the car to make it easier to navigate to those locations and dial those numbers.</div>
<h4>
Key fobs Associated with Driver Profiles</h4>
<div>
As far as I can tell, both of my key fobs work identically. I hope they're logically separate to the car so that a future update can associate a key fob with a driver profile that includes a seating configuration, media and phone favorites and presets, Bluetooth preferences, web browser cookies and form-fill auto-completes, etc.</div>
<h4>
Timed Charging and Power Sleep Mode</h4>
<div>
In California, we have electric-vehicle residential electrical rate cards that have much lower costs after midnight. It's an economically-valuable feature to allow you to configure the car to only start drawing power according to a pre-specified schedule (it can be 4x cheaper charging after midnight). I imagine this is coming, and from what I read, Tesla is having a difficult time with the super-hard problem of getting the car to go into a power-saving sleep mode. More precisely, the challenge is getting the operating system and all the hardware to come out of that mode correctly without unintended changes to the state. Right now the car loses about 8 miles of range per day when it's not plugged in, and Tesla's communications assure us owners that they're working on that problem at a high priority. </div>
<h4>
Creep Mode option to Stay-Put</h4>
<div>
The S has a optional creep mode to approximate an automatic vehicle's slow creep when your foot is off the break. Apparently, some people like this to help in maneuvering in tight spots. With creep mode turned off, the car actually feels more like a manual-transmission car, and in particular it will roll backwards on a hill, and I don't like that. But I also don't like creep mode since it's unnatural and is approximating a quirk of gas-powered cars.</div>
<div>
<br /></div>
<div>
I'd like a <i>stay-put mode</i> that would be almost like a "weak creep" mode. Specifically, it'd apply enough power to not roll backward <i>or forward</i> on a hill. If you want to go forward, you push the accelerator when in drive; if you want to go backward, you push the accelerator when in reverse. Otherwise, the car stays in place. (You'd still have to set the brake or put it in park for extended time on a hill, of course.) My guess is that this is actually what would be most natural and intuitive to first time drivers, and stay-put mode seems perfectly doable given the sensors that must exist on the car.</div>
<h4>
Charging Fail Notice</h4>
<div>
Many public charges require telling the base station to start charging before the current flows. It'd be useful if the car honked or chirped at you if it's plugged in but no current is flowing after, say, 20 seconds. That'd make sure you didn't forget to take the last step to start the car charging.</div>
<h4>
WiFi Access</h4>
<div>
Not only would WiFi access save on data usage and make downloading music, map tiles, firmware updates, etc., lots faster, it's also really useful for integrating with the rest of a smart home. I'd love to be able to use devices running on my house's WiFi network without opening up holes in my firewall and risking the security issues associated with that.</div>
<h4>
Voice Memo app</h4>
<div>
I'd love a simple voice memo app that emails me or other pre-configured recipients a WAV file of something I just said. Ideally it'd integrate with calendar, too, so that attendees of upcoming meetings would be easy to send a quick email-voice-recording to.</div>
<h4>
Access to Screenshots</h4>
<div>
An undocumented (as far as I can tell) feature of the car is that a ~5+ second press of the menu button on the steering wheel takes and saves a screenshot of both displays. (I discovered it accidentally when playing with the steering wheel controls in park.) Unfortunately, I don't know where they go (Tesla service?); I'd love to have them emailed to me automatically or provide a way for me to download them later so I could better communicate with the service folks without having to take an indirect photo using the camera on my phone (which is impossible to do safely while driving when something weird happens -- hypothetically, of course :-) ).</div>
<h4>
iOS and Android Apps</h4>
<div>
It's no secret Tesla is working on these, but I'm a beta user of the Android App and agreed when I became a beta user not to say anything about them, so I won't.</div>
<h2>
Oh, and it's Electric</h2>
<div>
I've not written much about it here (intentionally, because I did not have a goal of buying an electric car), but obviously the Model S gets lots of press about its one central feature of being 100% electric. I like not visiting gas stations, and in the 1,200 miles I've driven, I've averaged about 350 Wh/mile. That's about 420 KWh total, each of which costs between $0.04 and $0.21 from PG&E at home (depending on how much other electricity needs I have) -- if I'd charged all of those miles at home, it would've cost me between $15 and $90 total for the 3+ months since October 17, 2012. You don't have to want to be green for that to make a ton of sense!<br />
<br />
Postscript: I just got v4.2 of the firmware last night and I'm delighted to see a bunch of changes to way the dock app buttons work, similar to how I proposed. The voice command system is also great when it works, but it fails as often as it recognizes what I said. I'll have an update later.<br />
<br />
(Disclaimer: I own some shares of TSLA -- I bought them right after my test drive of the Model S, and expect to hold them for a long while.)</div>
Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com13tag:blogger.com,1999:blog-7384142322553850856.post-84729771061258109842012-06-09T16:47:00.000-07:002012-06-11T15:44:08.782-07:00SSD to Revive my PC<h2>
SSD to Revive my PC</h2>
<div>
My current primary home Windows PC is running Windows 7 on hardware from late 2009: a Quad Core Intel Core2 CPU, a variety of SATA and IDE hard drives, and 4GB of RAM (800 MHz bus). It's still very usable after booting, but the time from power-on to having logged in and getting Chrome or Firefox open is 3 minutes and 5 seconds. Here are a set of timings for the machine (all times are total from the point the BIOS recognizes the CPU):</div>
<div>
<br /></div>
<div>
To login screen visible: 40 sec</div>
<div>
To Windows 7 desktop visible: 1min 15sec (75 sec)</div>
<div>
To Chrome browser open: 3min 5sec (185 sec)</div>
<div>
<br /></div>
<div>
That's a bunch faster than it ran under Windows XP (I skipped Vista.... didn't everyone?), even with the addition of tons of drivers and startup processes that I surely only used once to play with a new gadget. Still, after experiencing my recently bought <a href="http://www.amazon.com/gp/product/B007Y8DJEA/ref=as_li_tf_tl?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=9325&creativeASIN=B007Y8DJEA">Samsung Series 3 Chromebox</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B007Y8DJEA" style="border: none !important; margin: 0px !important;" width="1" />
(see more on that below), I thought I had to be able to do better and breathe some more life into this machine.</div>
<div>
<br /></div>
<div>
After doing a bit of research on consumer SSD (Solid State Drives), I found the <a href="http://www.amazon.com/gp/product/B004W2JKZI/ref=as_li_tf_tl?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=9325&creativeASIN=B004W2JKZI">Crucial 128 GB m4 SSD</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B004W2JKZI" style="border: none !important; margin: 0px !important;" width="1" /> and thought I'd give that a try. My prior boot partition was only 80GB, so I didn't need a bigger size (I use secondary drives for my data to avoid overfilling the boot partition and to make it easier to re-install/update OS). I also ordered a <a href="http://www.amazon.com/gp/product/B0001Y8UI4/ref=as_li_tf_tl?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=9325&creativeASIN=B0001Y8UI4">StarTech 18in SATA Cable</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B0001Y8UI4" style="border: none !important; margin: 0px !important;" width="1" /> since it wasn't terribly clear whether the SSD would come with a cable.<br />
<br />
A little more research later, I decided I should also use some disk cloning software to facilitate the transfer of the boot drive and the OS from the existing SATA drive to my new SSD. There are instructions for how to do this with only free software (e.g., <a href="http://www.howtogeek.com/97242/how-to-migrate-windows-7-to-a-solid-state-drive/">http://www.howtogeek.com/97242/how-to-migrate-windows-7-to-a-solid-state-drive/</a> ) but <a href="http://www.paragon-software.com/technologies/components/migrate-OS-to-SSD/">Paragon's Migrate OS to SSD</a> tool seemed like a good deal for $20.<br />
<br />
And it was... the cloning of the OS to the new drive went flawlessly (as had the installation of the SSD). I did struggle for (far) too long with getting my complex mix of drives to allow me to consistently boot reliably off of the new SSD and learned more than I wanted to about <a href="http://support.microsoft.com/kb/927392">BootRec.exe</a> and <a href="http://technet.microsoft.com/en-us/library/cc709667(v=ws.10).aspx">BCDEdit.exe</a>. On the same night I started, though, I did get it working and was ready for the timing test. The new numbers with the <a href="http://www.amazon.com/gp/product/B004W2JKZI/ref=as_li_tf_tl?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=9325&creativeASIN=B004W2JKZI">Crucial 128 GB m4 SSD</a><img alt="" border="0" height="1" src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B004W2JKZI" style="border: none !important; margin: 0px !important;" width="1" /> were:<br />
<br />
<br />
<div>
To login screen visible: 34 sec (down from 40 sec, ok)</div>
<div>
To Windows 7 desktop visible: 44 sec (down from 75 sec -- wow!)</div>
<div>
To Chrome browser open: 49 sec (down from 185 sec -- wow WOW!)<br />
<br />
The PC now feels almost like a brand new high-end machine with a fresh OS install! <br />
<br />
Again, though, remember that the original point of comparison was my new <a href="http://www.amazon.com/gp/product/B007Y8DJEA/ref=as_li_tf_tl?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=9325&creativeASIN=B007Y8DJEA">Samsung Series 3 Chromebox</a>. If all you ever want to do is get into a Chrome web browser, it's still the champ. It goes from power-on to typing a URL into the address bar<b> in only 7 seconds</b>. It took me longer than that to write this short sentence!</div>
<div>
<br class="Apple-interchange-newline" /></div>
</div>Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com1tag:blogger.com,1999:blog-7384142322553850856.post-59080860331610343812011-05-24T21:42:00.000-07:002011-05-24T21:42:40.039-07:00Cassowary Constraint Solver in JavaScriptAs a longtime-recovering academic, I still occasionally feel the urge to make the work I did for my Ph.D. back in the '90s matter more in the real world. Recently, I had the chance to use two different and unrelated pieces of my research together.... as any fellow technologist knows, that makes doing the work way more than twice as fun!<br />
<br />
One of the projects I found a new use for is <a href="http://www.badros.com/greg/JavaML/">JavaML</a> – work I did in 1999. For my grad school career, JavaML was perhaps most notable because it was done right before I finished my Ph.D. but was not included in my thesis at all... quite literally, I went to my advisor one week before the submission deadline and said "I'm gonna disappear for a week while I work on this cool idea, ok?" Luckily, that paper was accepted to WWW in 2000 which was hosted in Amsterdam. Yes, the choice of conference was <i>somewhat</i> influenced by its being hosted in the Netherlands, but also because the project idea was grounded in something I knew the WWW community would be excited about: the use of web standards applied to developer, language, and compiler tooling. Specifically, my idea was appealingly simple: convert Java source code into XML so that one could use XML tools such as XSLT, SAX parsers, DOM trees, and more, to write code transformations and analysis and querying tools. The primary challenge was the exact design of the XML schema for the Java AST. That piece was critical in making downstream tools powerful and easy to build. As with all of my work, I backed the theory with an implementation that I built using IBM's then state-of-the-art Java compiler. All told, that <a href="http://www.badros.com/greg/papers/badros-javaml-www9.pdf">JavaML paper</a> turned out well, I made a cool <a href="http://www.badros.com/greg/images/javaml-poster.gif">poster for UW's industrial affiliates one year</a> and I had a working implementation that had plenty of reasonably compelling examples. And I had a fun visit to Amsterdam... But never did I myself use the tools I built for anything practical. Until a couple weeks ago....<br />
<br />
The other prior research I dusted off in my latest personal project is the <a href="http://www.cs.washington.edu/research/constraints/cassowary/">Cassowary Constraint Solving Toolkit</a>. That work and its applications made up the bulk of my PhD thesis. In a nutshell, Cassowary is a library and framework for expressing linear arithmetic equality and inequality relationships among variables and solving systems of those relationships incrementally and in real time. Most of my work involved applying the incremental solver to graphical applications such as the <a href="http://scwm.sourceforge.net/">Scheme Constraints Window Manager</a> or the <a href="http://www.badros.com/greg/papers/css-uist99.pdf">Cascading Style Sheet engine</a> in web browsers and <a href="http://badros.com/greg/CSVG/index.html">Scalable Vector Graphics renderers</a>, but other researchers used it for <a href="http://www.cs.washington.edu/homes/weld/papers/ijcai99-lpsat.pdf">satisfiability engines and resource planning</a>, and more. For many years, I used Cassowary daily in the Scheme Constraints Window Manager, but it never achieved widespread use on the web. In part this is because it needed to be built into the browser... although I made the Cassowary toolkit available in C++, Java, Smalltalk, Python, and Scheme, none of those languages were easy to demonstrate over the web. <br />
<br />
A few weeks ago, I realized something both obvious and important. We all know that JavaScript implementations have improved leaps and bounds over the last decade. IE6 was a disaster to work with in large part because the JavaScript implementation was good enough only for the tiny little scripts that defined JavaScript in the 90s. Today, Chrome and its V8 engine along with similar advances in Safari (Nitro) and Firefox (TraceMonkey) make JavaScript performance scream. That meant that I could run Cassowary natively in the browser with good performance if only I had a JavaScript implementation. (Or, more properly, ECMAScript.)<br />
<br />
Thus, my plan was hatched: given that I wrote a Java implementation of Cassowary, I would use JavaML to serialize that code into its XML representation. Back in 2000, I wrote an XSLT-based converter to go back from JavaML's XML representation to plaintext Java source code (in fact, round-tripping between those representations was my unit test as I was building JavaML). For this project, I'd change that converter to, instead of outputting Java source code, transliterate language constructs and generate a close-to-correct JavaScript plaintext representation of the program. Then, after some editing and debugging, I'd end up with a complete JavaScript implementation of the solver enabling me and others to explore the use of constraints for layout or other functionality within rich, modern AJAX web applications. In case you're impatient, here's the <a href="http://www.badros.com/greg/cassowary/js/quaddemo.html">bounded quadrilateral demo in JavaScript</a>. The <a href="http://cassowary.cvs.sourceforge.net/viewvc/cassowary/cassowary/js/">full JavaScript implementation of Cassowary is available via CVS at Sourceforge</a> with the rest of the Cassowary distribution.<br />
<br />
Let me go through that last paragraph in slow motion. First, I took my Java implementation of Cassowary which has code like this:<br />
<br />
<pre><span class="Apple-style-span" style="color: blue;">public </span><span class="Apple-style-span" style="color: purple;">String </span>toString()
{
<span class="Apple-style-span" style="color: purple;">StringBuffer </span>bstr = <span class="Apple-style-span" style="color: blue;">new </span><span class="Apple-style-span" style="color: purple;">StringBuffer</span>("[");
<span class="Apple-style-span" style="color: blue;">for </span>(int i = 0; i < _values.<span class="Apple-style-span" style="color: blue;">length</span>-1; i++) {
bstr.append(_values[i]);
bstr.append(",");
}
// ...
<span class="Apple-style-span" style="color: blue;">return </span>bstr.toString();
}
</pre><br />
after running it through my Jikes-JavaML tool, I get JavaML XML code like this:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"><method name="toString" visibility="public" id="meth-3340"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <type name="String"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <formal-arguments/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <statements></span><br />
<div style="margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px;"><span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <!-- StringBuffer bstr = new StringBuffer("[") --></span></div><span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <local-variable name="bstr" id="locvar-11045"> </span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <type name="StringBuffer"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <new><type name="StringBuffer"/><arguments></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <literal-string length="1">[</literal-string></arguments></new></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </local-variable></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <loop kind="for"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <init> <!-- int i = 0 --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <local-variable name="i" id="locvar-11050"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <type name="int" primitive="true"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <literal-number kind="integer" value="0"/></local-variable></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </init></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <test></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <binary-expr op="lt"> <!-- i < _values.length - 1 --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <var-ref name="i" idref="locvar-11050"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <binary-expr op="-"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <field-access field="length"><var-ref name="_values"/></field-access></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <literal-number kind="integer" value="1"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </binary-expr></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </binary-expr></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </test></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <update> <!-- i++ --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <unary-expr op="++" post="true"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <var-ref name="i" idref="locvar-11050"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </unary-expr></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </update></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <statements></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <send message="append"> <!-- bstr.append(_values[i]) --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <target><var-ref name="bstr"/></target></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <arguments><array-ref><base></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <var-ref name="_values"/></base><offset></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <var-ref name="i"/></offset></array-ref></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </arguments></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </send></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <send message="append"> <!-- bstr.append(",") --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <target><var-ref name="bstr"/></target></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <arguments><literal-string length="1">,</literal-string></arguments></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </send></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </statements></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </loop></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <!-- ... ... --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <return> <!-- return bstr.toString() --></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <send message="toString"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <target><var-ref name="bstr" idref="locvar-11045"/></target></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <arguments/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </send></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </return></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> </statements></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"></method></span><br />
<div><br />
</div>Note that I specifically designed features of the JavaML representation to support edges (via id/idref pairs) between uses of variables and their declarations (which include their static types). That means that I can convert the above XML into JavaScript, essentially transliterating from Java to JavaScript, in a type-aware way using XSLT and template precedence rules to specialize rewrites based on details of the AST. For example, these rules make sure that <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">StringBuffer.append</span> and<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> .toString</span> method invocations result in the appropriate JavaScript constructs of "<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">+=</span>" and a no-op, respectively:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"><<span class="Apple-style-span" style="color: blue;">xsl:template</span></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <span class="Apple-style-span" style="color: purple;">match</span>="send[@message='append'][id(./target/var-ref/@idref)/type/@name='StringBuffer']"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <<span class="Apple-style-span" style="color: blue;">xsl:apply-templates</span> <span class="Apple-style-span" style="color: purple;">select</span>="target"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <<span class="Apple-style-span" style="color: blue;">xsl:text</span>><b> +=</b> </<span class="Apple-style-span" style="color: blue;">xsl:text</span>></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <<span class="Apple-style-span" style="color: blue;">xsl:apply-templates</span> <span class="Apple-style-span" style="color: purple;">select</span>="arguments/*"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"></<span class="Apple-style-span" style="color: blue;">xsl:template</span>></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;">(The above template matches <b>send</b> elements with <b>message</b> attributes of 'append' that also have the <b>target/var-ref/@idref</b> pointing to a declaration that has a <b>type </b>element with <b>name attribute</b> of 'StringBuffer'. I.e., apply this rule to all sends of the "append" message to variables with static type "StringBuffer".)</span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"><br />
</span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"><<span class="Apple-style-span" style="color: blue;">xsl:template </span></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <span class="Apple-style-span" style="color: purple;">match</span>="send[@message='toString'][id(./target/var-ref/@idref)/type/@name='StringBuffer'][not(arguments/*)]"></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"> <<span class="Apple-style-span" style="color: blue;">xsl:apply-templates </span><span class="Apple-style-span" style="color: purple;">select</span>="target"/></span><br />
<span class="Apple-style-span" style="font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif; font-size: x-small;"></<span class="Apple-style-span" style="color: blue;">xsl:template</span>></span><br />
<br />
Those rules applied to the JavaML representation of the Java source code then result in this JavaScript:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> toString: <span class="Apple-style-span" style="color: blue;">function</span>() {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="Apple-style-span" style="color: blue;">var</span> bstr = "[";</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="Apple-style-span" style="color: blue;">for</span> (var i = 0; i < this._values.<span class="Apple-style-span" style="color: blue;">length</span>-1; i++) {</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> bstr += this._values[i];</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> bstr += ",";</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> }</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> // ...</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> <span class="Apple-style-span" style="color: blue;">return</span> bstr;</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> },</span><br />
<div><br />
</div><div>In this case, the converted code is perfect and ready to execute. The actual process was a bit more laborious, involving iterating on specializing the XSLT transformation for each subsequently more complex Java source program in the full library and hand-editing the output to fix the places where it was not worth my while to make the XSLT smarter (and finding a few bugs in Jikes-JavaML and my XSLT transformations along the way). I also had to make several choices of libraries and JavaScript langauge extensions as dependencies to the converted JavaScript. For a simple object-oriented framework, I used the server-side <a href="http://mootools.net/">MooTools</a>, and for a true hashtable and hashset (i.e., one that can have arbitrary objects as keys, not just strings), I used <a href="http://code.google.com/p/jshashtable/">jshashtable and jshashset</a> (with some minor modifications to support an <b>escapingEach</b> mechanism to allow <b>break</b> and non-local <b>return</b>s since the algorithm uses those patterns throughout).</div><div><br />
</div><div>After about three weekend days of working on this, I ironed out the last of the (known) bugs (which turned out to be an error in my Java implementation that I'd fixed in the C++ implementation years ago). Then I implemented the previously-mentioned <a href="http://www.badros.com/greg/cassowary/js/quaddemo.html">bounded quadrilateral demo in JavaScript</a>, learned about <a href="http://developer.apple.com/library/safari/#documentation/appleapplications/reference/safariwebcontent/HandlingEvents/HandlingEvents.html">Touch events on iOS devices</a> and made the demo work on my iPad and iPad2.</div><div><br />
</div><div>I hope developers out there are able to find creative uses of the constraint solver on their web pages and applications. And maybe someone will improve the JavaScript implementation with something that truly looks and feels like JavaScript (rather than Java hiding behind a JavaScript surface syntax)! Until then, I may port to JavaScript the string expression parser to make it easier to specify constraints just by writing something like "x + width < screen_width". The dynamic expressiveness of JavaScript should be a huge help in experimenting with tighter integration with the rest of the environment, similar to my Scheme implementation and the way it interacted with the Scwm Window Manager.<br />
<br />
</div>Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com6tag:blogger.com,1999:blog-7384142322553850856.post-19189650252025635182010-05-01T13:27:00.000-07:002010-05-01T14:33:36.547-07:00Skype as a whole-house Wireless Microphone for Home AutomationAfter buying a home (finally!) this past fall, I've been setting up a bunch of home automation capabilities. Recently I got far enough along that I started pondering how I might get a whole-house microphone system talking to my central (Windows XP-based) computer running <a href="http://www.homeseer.com/downloads/index.htm">HomeSeer </a>(v2.0.4.36). I first considered <a href="http://www.amazon.com/gp/product/B0017TRL1Y?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=390957&creativeASIN=B0017TRL1Y">xTag USB-Only System w/ One xTag Wearable Microphone</a><img src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B0017TRL1Y" alt="" style="border: medium none ! important; margin: 0px ! important;" border="0" height="1" width="1" />, but the range of just tens of feet seemed way too limiting.<br /><br />Then I had a thought... a wonderful, crazy thought... what if I could find a WiFi microphone -- one that used my existing 802.11n wireless network. After doing some quick searches, I realized that I already had such a thing: my <a href="http://www.amazon.com/gp/product/B002M3SOBU?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=390957&creativeASIN=B002M3SOBU">Apple iPod touch</a> (2nd generation since it needs a microphone on the hardware) and <a href="http://www.skype.com/">Skype</a>! All I needed to do was use my <a href="http://www.amazon.com/gp/product/B002M3SOBU?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=390957&creativeASIN=B002M3SOBU">iPod touch</a> (or iPhone, or anything that has a decent Skype client) to call a Skype account running on my home automation server. I could then configure the sound output from that call to be the voice recognition input into HomeSeer's "Speaker" application that then controls HomeSeer. (Notably, the Speaker application actually need not be running on the same machine as HomeSeer, but I'm trying to be green by having only a single low-wattage computer always-on in my house, so I run it on the same machine.)<br /><br />Now, if you've not been reading all my oh-so-infrequent posts, you may be thinking "hmmm, that sounds great, but how do you make the audio output from one program [Skype] become the microphone input of another program [Speaker]?"<br /><br />Avid readers realize that in an <a href="http://badros.blogspot.com/2008/10/streaming-itunes-music-to-my-squeezebox.html">earlier post I </a><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVWUj_8fGWSOIOwz0be_aSBtZfpqPZuuHgRMwuMigzwYNQ45ATmTCQeCZI9p1urLKdsSE67Fv01OSyEwpY5Q95wvlNU3fDSLWCTYYCzl_ACVu6ofRCGLw9PLd5ihohKAEs3VrkQi0zLdRi/s1600/after-speaker-started-highlighted.png"><br /></a><a href="http://badros.blogspot.com/2008/10/streaming-itunes-music-to-my-squeezebox.html">did exactly that to Stream iTunes musi</a><a href="http://badros.blogspot.com/2008/10/streaming-itunes-music-to-my-squeezebox.html">c to my SqueezeBox duet receiv</a><a href="http://badros.blogspot.com/2008/10/streaming-itunes-music-to-my-squeezebox.html">er</a>. Using <a href="http://software.muzychenko.net/eng/vac.html">Virtual Audio Cable 4</a>, I created two virtual audio cables -- I set up "Virtual Cable 1" as the default output channel for Speaker as shown in the first figure.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBi8heOLuI62oIcOBcp7Y4E-LE3pH9TpMR6iLNp0TZ2joODoxeYnayDSIqzKUb5Flpm_3Bv6KtXpJOfw3FSpdFa7t8AMk6D5v296ixoutA7iH7PfSRDAW6AcnTluSFyesJP_lsqKDsnDl7/s1600/speaker-output-config.png"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 400px; height: 328px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBi8heOLuI62oIcOBcp7Y4E-LE3pH9TpMR6iLNp0TZ2joODoxeYnayDSIqzKUb5Flpm_3Bv6KtXpJOfw3FSpdFa7t8AMk6D5v296ixoutA7iH7PfSRDAW6AcnTluSFyesJP_lsqKDsnDl7/s400/speaker-output-config.png" alt="" id="BLOGGER_PHOTO_ID_5466417741841689506" border="0" /></a><br /><br />Then I set up "Virtual Cable 2" as the default microphone input -- you do that from the Control Panel -> Sound page. (It varies by Windows operating system variant.) After making those changes, you need to restart the Speaker client and at that point the Virtual Audio Cable's control panel will look like the below -- notice that there is 1 recording stream on Virtual Cable 2 -- that's the Speaker application that opened that cable as its microphone. If it's not showing 1 there, you need to try again to make that cable the default input and restart the Speaker application.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhS02-fRFUvqn8iDzsYdDcLu0w2drDUFjgTFChyphenhyphenYf3pFg0Y1eW1KV9FOLcGJpUbHGpruY7La_zTAWRcKWZ2_TyxG5T3jh3XIiJXDfbLaEqITCQZw2MmfQg73YSbnJTtTbGB-CjfKWOMTr4n/s1600/after-speaker-started-highlighted.png"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 481px; height: 220px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhS02-fRFUvqn8iDzsYdDcLu0w2drDUFjgTFChyphenhyphenYf3pFg0Y1eW1KV9FOLcGJpUbHGpruY7La_zTAWRcKWZ2_TyxG5T3jh3XIiJXDfbLaEqITCQZw2MmfQg73YSbnJTtTbGB-CjfKWOMTr4n/s400/after-speaker-started-highlighted.png" alt="" id="BLOGGER_PHOTO_ID_5466407651616107026" border="0" /></a><br />After that, I set up Skype on the home automation PC, created a distinct user account, set it up to only accept calls from my specific other Skype account, and made it auto-answer calls. I set the audio settings of Skype to have the microphone input be Virtual Cable 1 (the one that the Speaker application is doing text-to-speech on, since that input will be sent over to the speaker of my iPod touch so I can hear what the computer says to me) and to have the Skype speaker output setting be Virtual Cable 2 (the one that the Speaker application is using for voice recognition, since it's gonna play out what it hears from what I say into my microphone on my iPod Touch). The audio settings end up looking like this:<br /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzlsNd_M0rYcsysg7CseuguXpMN2uSpGZZJWGM6vw25GX99deaPsXTftvCuenXKDpQsX0m0S-c9f3t5JQqDzDXgz_OWlj-vTGPfC1NrSGPmEiKGLvgcULtpxy3bpHRDC5-D4gi_YVgkQ72/s1600/skype_audio_settings.png"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 489px; height: 365px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzlsNd_M0rYcsysg7CseuguXpMN2uSpGZZJWGM6vw25GX99deaPsXTftvCuenXKDpQsX0m0S-c9f3t5JQqDzDXgz_OWlj-vTGPfC1NrSGPmEiKGLvgcULtpxy3bpHRDC5-D4gi_YVgkQ72/s400/skype_audio_settings.png" alt="" id="BLOGGER_PHOTO_ID_5466409654293543186" border="0" /></a>To test it, I made a call from my iPod Touch to that new Skype account. The Virtual Audio Cable control panel comes in handy here again, as it should, after the call is connected, look like:<br /><img src="file:///C:/greg/homeseer_skype/after-skype-connected-while-listening-highlighted.png" alt="" /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOUwTGLzIIvmTogbVKXopNF1l_P4JI45TFoItah8RiwyZnH3hewoj5Zzri5irk57Le5clnkUyzJpzcE0TDbtxtsz3yjzFF6Q5V79fVmwpvKvGdGMzZSPDgOXNCI-VV-jNwJ2-iG_0I-opg/s1600/after-skype-connected-while-listening-highlighted.png"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 526px; height: 241px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjOUwTGLzIIvmTogbVKXopNF1l_P4JI45TFoItah8RiwyZnH3hewoj5Zzri5irk57Le5clnkUyzJpzcE0TDbtxtsz3yjzFF6Q5V79fVmwpvKvGdGMzZSPDgOXNCI-VV-jNwJ2-iG_0I-opg/s400/after-skype-connected-while-listening-highlighted.png" alt="" id="BLOGGER_PHOTO_ID_5466410518166945202" border="0" /></a>Notice that now there is a recording stream on Virtual Cable 1 (Skype reading its microphone input, the output of the Speaker application) and a playback stream on Virtual Cable 2 (Skype writing its speaker out to the input of the Speaker application).<br /><br />From that point, I'm able to give voice commands to HomeSeer via my iPod Touch. I had to retrain the voice recognizer (Windows Control Panel->Speech) for this new setup, even just to get Speaker to recognize the attention phrase. It'd be nice to not require the attention phrase and to simply take a command immediately upon Skype answering a call, but I'm not an expert with Speaker so I'm not sure it's possible.<br /><br />I was also able to get this working using my new <a href="http://www.amazon.com/gp/product/B003HC8NUW?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=390957&creativeASIN=B003HC8NUW">HTC Droid Incredible Android Phone</a><img src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B003HC8NUW" alt="" style="border: medium none ! important; margin: 0px ! important;" border="0" height="1" width="1" />, and again had to retrain voice recognition and microphone levels with that set up. Unfortunately (and surprisingly) Verizon seems to only let its Skype Mobile run over 3G and not over WiFi... this seems really insane especially for the substantially more open Android platform -- remember, Skype exists in a more full-featured form for the Apple-controlled iPhone. Regardless, I got VoIP over WiFi working using <a href="http://www.fring.com/android/">Fring</a> and its Skype plug-in.<br /><br />All in all, I'm pretty psyched to now have my whole-house microphone setup! Now if only HSTouch on the iPod Touch worked with the SqueezeBox plug-in without crashing... then I might be able to control my audio using my voice (I still use the awesome iPeng application to do that, and waiting for an <a href="http://www.amazon.com/gp/product/B00365F6G4?ie=UTF8&tag=gregjbadrosscons&linkCode=as2&camp=1789&creative=390957&creativeASIN=B00365F6G4">Apple iPad Tablet</a><img src="http://www.assoc-amazon.com/e/ir?t=gregjbadrosscons&l=as2&o=1&a=B00365F6G4" alt="" style="border: medium none ! important; margin: 0px ! important;" border="0" height="1" width="1" /> version of that app).Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com37tag:blogger.com,1999:blog-7384142322553850856.post-66228869081305610662009-05-23T21:08:00.001-07:002009-05-23T22:32:51.231-07:00iPhone as a Universal Remote Using SqueezeBox Duet Controller IRAfter the grand machinations I described in my last post to enable me to use Apple's Remote iPhone application to control music around my house through the SqueezeBox Duet Receiver, a commenter pointed out the then-new <a href="http://penguinlovesmusic.de/2009/05/10/ipeng-111-now-available/">iPeng application</a>. That app lets my iPod Touch directly control SqueezeCenter on my PC to manage my various SqueezeBox Receivers, and is exactly what I was pining for. In the ensuing months, I transitioned exclusively to using the iPeng application, which left me with the question: what do I do with my now oh-so-dusty SqueezeBox Duet <span style="font-style: italic;">Controllers</span>.<br /><br />The answer was easy once I took a closer look at the hardware of those controllers... something I'd not noticed before jumped out at me: an Infrared (IR) transmitter! The IR transmitter in its shipped configuration doesn't do anything -- an intron component -- but thanks to Google and <a href="http://forums.slimdevices.com/showthread.php?t=40367">various threads like this one on the slimdevices community forums</a>, I discovered that there is rudimentary driver support for the IR device and some trivial samples of using that device. Given that I already knew the controller runs a BusyBox-based embedded Linux, I knew how I'd challenge myself that evening --- no turning my Sony IR-controlled TV on until I could do so from my PC!<br /><br />I spent the next while reading a bit about Infrared signalling protocols and conventions, and then I built a trivial little script that just called the Duet Controller's /bin/testir via ssh with a command line full of hex codes that I carefully crafted based on my new-found partial competence. I announced to the room "Here we go!" as I dramatically pressed [Enter]......<br /><br />Nothing.<br /><br />I rinsed and repeated many times, alternating between worry that the degrees of freedom in command specifications were too great and celebrating short-lived epiphanies that narrowed that search space dramatically. Finally, I decided to pay more attention to the "repeat" parameter that the Linux Infrared Config files mention in passing -- that some devices only recognize an IR command when sent two or more times juxtaposed. I saw that my Sony LIRC file used that option, so I doubled the hex codes corresponding to the command part in the line I was trying, and gave a surprisingly hopeful and still dramatic "And then there was..." as I once again lowered my hand onto [Enter]....<br /><br />POWER!<br /><br />The hum of my (awesome, but now 6 years dated) Sony HD CRT Monitor warming up was music to my ears! How cool! With that magical ~200 character command line, I then knew that I could use the SqueezeBox Duet Controller as the IP-controlled IR transmitter portion of an iPod-Touch (or iPhone) Infrared Universal Remote solution!<br /><br />The rest of the path was simple compared to the reverse-engineering and experimental IR learnings. I educated myself on <a href="http://www.lua.org/">Lua</a> and Jive (now known as <a href="http://wiki.slimdevices.com/index.php/SqueezeOS">SqueezeOS</a>), the embedded programming language and the Framework, respectively, used by the SqueezeBox controllers. Then I set out to code a small Lua applet to run on the controller. I called it IRServer and the design was simple: an HTTP-like server that just accepts simple commands on a TCP port, strips them of their HTTP artifacts (if any) and issues the appropriate os.exec call to run the C program that drives the IR device driver. By making the server understand HTTP just enough, I knew I could even just write a static HTML page that issued commands when links or buttons were clicked (e.g., just requesting an Image object at a URL corresponding to the command I wanted to issue to IrServer).<br /><br />I spent a good part of the following weekend pulling the above little design together, along with a simple Perl script that takes a set of <a href="http://www.lirc.org/">Linux IR Config</a> files and turns them into <a href="http://www.badros.com/greg/IrServer/ir.html">HTML with a link per defined IR button</a> (don't worry, links on that page won't control my devices when you click -- the page assumes you're on the LAN with the controller). I even played around with <a href="http://code.google.com/p/iui/">IUI, the iPhone User Interface framework</a>, and started making a much more iPhone-friendly UI for a bunch of the devices in my living room.<br /><br />Of course, the good news is I put this project -- <a href="http://code.google.com/p/irserversb/">IrServerSB (SB for SqueezeBox)</a> -- up on code.google.com. The README file at <a href="http://code.google.com/p/irserversb/wiki/IrServerSBReadme">http://code.google.com/p/irserversb/wiki/IrServerSBReadme</a> has more information about how to try it, and I've set up a Google Groups mailing mailing list (or use the code project page, or comments here) for feedback.<br /><br />The better news will be if others from the community can take this the rest of the way towards being a complete solution. In particular, <a href="http://forums.slimdevices.com/showthread.php?t=40367">RC5/RC6 remotes are reportedly not supported by the Jive IR platform</a> and I can confirm that numerous attempts to turn on my RC6-controlled XBox 360 left me without a satisfying Eureka moment. I'd love to have that supported, and perhaps even build the server as a background-running C server.<br /><br />Enjoy!Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com5tag:blogger.com,1999:blog-7384142322553850856.post-36744484720403605772008-10-13T21:54:00.000-07:002008-10-13T22:28:03.533-07:00Streaming iTunes Music to my Squeezebox Duet ReceiversApple's "Remote" iPhone (or iPod Touch 16GB in my case) application for controlling iTunes is so great that I wanted to see if I could use it to control my music in other rooms around my house. Buying an Apple Airport Express extender with its audio output would've been one means to that end. That hardware device costs $99 which isn't terribly expensive, but I already have Squeezebox Duet receivers in both my living room and master bedroom. Adding Yet Another Digital Music Receiver to both rooms seems like overkill (and would exhaust the inputs on one of my stereo receivers), so I decided to figure out how to control the music playing on those existing devices from an iPod Touch. (Yes, it really is a nicer UI than the SqueezeBox Duet controllers that are meant to be so great for this... sorry Logitech.)<br /><br />Several possibilities occurred to me:<br /><br />1) Reverse engineer the airport express protocol and improve the duet receiver firmware to make it look like an airport express to iTunes directly. This would be a great solution and <a href="http://www.macnn.com/articles/04/08/11/airport.express.key/">Johansen (of DeCSS fame) appears to have done some of the reverse engineering</a> and built <a href="http://nanocr.eu/software/justeport/">some tools to stream data to an airport express</a>. However, that's the wrong direction for my needs: I want something that emulates an airport express in software/firmware. (See also AirFoil that sends Mac audio output to an airport express; also the wrong direction.) This might be possible, but is more than a day's hacking.<br /><br />2) Find a plug-in for iTunes that streams in a format accessible to my SqueezeBox Duet receivers. If iTunes would broadcast a URL of a streaming mp3, e.g., all would be great. Alas, no such luck.<br /><br />3) Capture the iTunes audio output somehow (software or hardware) and use another piece of software to do #2.<br /><br />4) Forego iTunes altogether and figure out a way to approximate Apple's Remote software for Squeezebox Duet's software, the SqueezeCenter (configured using the iTunes plugin so it uses the same library). The downside of this is that any DRM protected songs won't play, but my collection is predominantly ripped off of CDs rather than bought from iTunes (and I've been paying extra for DRM-free music whenever convenient). This approach is tempting, and Google-searching led me to <a href="http://penguinlovesmusic.de/?page_id=27">iPeng</a> which is a SqueezeCenter plug-in that skins the HTML it generates to be iPhone/Safari friendly. It's pretty nice, but way slower than Apple Remote because it's going over WiFi chattily and the SqueezeCenter server and the iPeng app apparently weren't written with latency in mind.<br /><br />#4 might be a cool weekend (and more) project, though my underpowered Mac Mini makes me dread the thought of compile/download/test cycles on it. It's certainly more than a day's hack, too, but would be broadly useful in general (presuming Apple deems the App suitable for their store). The iPeng source might provide a useful starting place.<br /><br />In any case, I ended up getting #3 working, and that's what I describe below. Note that I did all this work on a Windows XP PC, not the Mac mini I mention above and have blogged about previously.<br /><br />The basic mechanism ends up being pretty simple and involves only a few components:<br /><br />1) iTunes plays music, controlled remotely (over WiFi in my house) by Apple Remote running on my iPod touch;<br />2) Virtual Audio Cable (VAC) 4.0.9 from http://www.ntonyx.com/vac.htm ($30 from https://www.regsoft.net/regsoft/vieworderpage.php3?productid=76825, free trial) to capture iTunes output and loop it back to a virtual input device that is available to other Windows programs on the same machine (note that the free trial has a voiceover of "trial" every five seconds or so);<br />3) <a href="http://www.oddsock.org/tools/">edCast DSP v3</a> plug-in for <a href="http://www.winamp.com/">WinAmp v5.52</a> to connect to the icecast32 broadcast server;<br />4) the <a href="http://www.icecast.org/">icecast win32 port</a> to stream the data it gets from the edCast WinAmp plug-in.<br /><br />At one point, I thought I needed the <a href="http://home.hccnet.nl/th.v.d.gronde/">LineIn plugin v1.80</a> for WinAmp and I'm actually using it, but the edCast DSP plug-in allows you to transmit to icecast via a microphone on a selected device. That lets me just configure it to record from the Virtual Cable's SPDIF output directly.<br /><br />The way I currently have it configured, though, is Winamp is playing from the virtual device "line://" (not its built-in linein:// -- that didn't work for me). My default audio output device is set to a Virtual Audio Cable before I start iTunes (it uses the default output device at startup for playback).<br /><br />icecast32 is running as a server, I copied the lame mp3 encoder DLL and its ini file into the "Program Files\WinAmp" directory, restarted edCast, and configured an encoder to encode in MP3 and connect to icecast32 running on port 8070, creating a stream called "/stream.mp3". I then set my squeezecenter to playback the "streaming radio station" http://192.168.0.xx:8070/stream.mp3.<br /><br />In the end, this results in about 4 seconds of lag between pausing on Apple Remote and the music actually pausing (or restarting, or whatever). A little over 3 seconds of that appears to be in the icecast server (for contrast, shoutcast server had over 30 seconds of lag -- unacceptable). I also tried the jetCast dsp PlugIn for WinAmp -- it's a simpler set up since it doesn't require a separate running broadcast server, but it was pretty flakey for me and I gave up on it after finding it just too inconsistent.<br /><br />I do think I can avoid Virtual Audio Cable, too, by just using a coupler cable to connect my sound-card's audio output to one of its line-inputs, but that would involve an extra gratuitous D->A and A->D conversion pair (but save $30). Another downside of this approach is that it ties up my soundcard more, making it hard to listen to music at my PC while different music is playing around the house.Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com5tag:blogger.com,1999:blog-7384142322553850856.post-91186246563372621242007-06-03T18:39:00.001-07:002007-06-29T09:32:46.692-07:00Finally Loving my Mac...I spent much of yesterday working on my Mac, largely with the plan of addressing all the issues that bother me. Instead of actually trying to make progress on my primary task, I willingly accepted every diversion when I encountered an annoyance. That approach proved fruitful as I am now <span style="font-weight: bold;">lots</span> more comfortable working in front of OS X!<br /><br />Here are some of the must-have things that, to my delight, I got working well:<br /><br /><span style="font-size:180%;">Emacs<br /></span><br />I tried <a href="http://www.apple.com/downloads/macosx/unix_open_source/carbonemacspackage.html">Carbon Emacs</a> and <a href="http://aquamacs.org/">Aquamacs</a>, and prefer Carbon Emacs. It's GNU Emacs 22.0.97.1, and I generally have used XEmacs since the late '90s. In about 2000, I simplified my .emacs considerably in order to make it work well on Windows-based XEmacs distributions. Luckily, with those changes, it was just a twenty minute exercise to clean up the most urgent quirks.<br /><br /><span style="font-size:180%;">Firefox 2.0</span><br /><br />I use Firefox on GNU/Linux and on Windows, and had it installed on my Mac from day one, but was always frustrated by the keybinds of Home/End and Ctrl+Left/Right. For me, Home/End must go to the start/end of a line, Ctrl+Home/End must go to the start/end of document, and Ctrl+Left/Right must go backward/forward words. This is non-negotiable. This <a href="http://www.macosxhints.com/article.php?story=20070320214340628">article</a> describes some steps to address this problem by replacing the jar containing platformHTMLBindings.xml, but I found it easier to just edit that file directly using Emacs to open the jar file /Applications/Firefox.app/Contents/MacOS/chrome/toolkit.jar. The required edits are pretty straightforward, and the new file is now a standard part of my CVS repository of my dot files. Note that I use <a href="http://doublecommand.sourceforge.net/">DoubleCommand</a> to fix those bindings for Carbon apps.<br /><br />For Firefox, it's also critical that I have all the right extensions and GreaseMonkey scripts installed. Things like <a href="http://customsoftwareconsult.com/extensions/febe/febe50.html">FEBE 5.0beta</a> look like they might let me backup and restore my extension set, but I ended up just using <a href="http://mozilla.doslash.org/infolister/">InfoLister</a> to create a nice HTML listing of my Firefox extensions on my various other boxes, and then focused on the right minimal set and installed them all. Here's part of the listing from my Windows and GNU/Linux boxes (which are now mostly the same as my Mac):<br /><ul><li><a href="http://www.mozilla.org/projects/inspector/">DOM Inspector</a> 1.8.1.4 </li><li><a href="http://fasterfox.mozdev.org/">Fasterfox</a> 2.0.0 </li><li><a href="http://www.getfirebug.com/">Firebug</a> 1.05 </li><li><a href="http://flashgot.net/">FlashGot</a> 0.5.98.070328 </li><li><a href="http://www.google.com/notebook">Google Notebook</a> 1.0.0.17 </li><li><a href="http://www.google.com/">Google Toolbar for Firefox</a> 3.0.20070420W </li><li><a href="http://greasemonkey.mozdev.org/">Greasemonkey</a> 0.6.9.20070507.0 </li><li><a href="http://imagezoom.yellowgorilla.net/">Image Zoom</a> 0.3 </li><li><a href="http://mozilla.doslash.org/infolister">InfoLister</a> 0.9f </li><li><a href="http://livehttpheaders.mozdev.org/">Live HTTP Headers</a> 0.13.1 </li><li><a href="http://www.pdfdownload.org/">PDF Download</a> 0.8 </li><li><a href="http://www.squarefree.com/extensions/search-keys/">Search Keys</a> 0.8.1 </li><li><a href="http://sessionmanager.mozdev.org/">Session Manager</a> 0.5.3.4 </li><li><a href="http://mozmonkey.com/">SwitchProxy Tool</a> 1.4.1 </li><li><a href="http://tmp.garyr.net/">Tab Mix Plus</a> 0.3.5.2 </li><li><a href="http://chrispederick.com/work/web-developer/">Web Developer</a> 1.1.4 </li></ul>My greasemonkey scripts are also important to my general happiness. My two favourites are <a href="http://userscripts.org/scripts/show/5711">Advanced Google Keys</a> (lets me use keys to navigate Google search results, similar to our experimental feature) and my own trivial <a href="http://www.badros.com/greg/widen-google-results.user.js">Widen Google Results</a> (optimize Google result snippets for bigger monitors). Those are a part of my CVS dots repository, too, so they were easy to add.<br /><br /><br /><span style="font-size:180%;">Terminal</span><br /><br />I'm fairly particular about the terminal program I use: key-bindings, supporting-escape sequences, control over what happens as I scroll back through the buffer, search functionally, and more, are all key to my productivity. My a priori goal was to make Mac's native Terminal application work for me. Right away, though, I noticed that it didn't have antialiased fonts. I don't know whether I turned them off in a misguided attempt to favour performance over eye candy, but I immediately went back into Window Settings -> Display and turned the setting back on. I also prefer a non-blinking blue block cursor, and lots of scrollback buffer (but don't jump to the bottom just because I start typing).<br /><br />The default keyboard bindings in Terminal are very much biased towards rudimentary use. I generally want to be able to run even an 'emacs -nw' in a Terminal and have a reasonable set of modifiers available that send keystrokes to the application rather than being interpreted by the Terminal. In particular, Home/End, Ctrl+Left/Right, and PageUp/PageDown were set to manipulate the scrollback buffer, rather than send escape sequences for those keystrokes. Although I did the bindings on my own based on my .Xresources-cpp file that I've evolved over the years for xterm, <a href="http://fdiv.net/2007/05/12/keybindings-in-macosx-terminal-app/">this article</a> has a reasonable couple of suggestions. I also recommend Edward Moy's definitive reference to <a href="http://www.kitebird.com/csh-tcsh-book/ctlseqs.pdf">xterm escape sequences</a>.<br /><br />The last of my Terminal tweaks was to change my shell to Zsh; I chose to do this just for interactive shells started by Terminal rather than changing my login shell -- that conservatism is mostly a holdover from bad experiences with very minor but subtle incompatibilities between bash and earlier versions of Zshell.<br /><br /><br /><span style="font-size:180%;">Shell</span><br /><br />My Mac has $ZSH_VERSION = 4.2.3, and my only real issue in getting it going was that the Makefile I use to generate my .zshrc relied on one of my scripts that depends on nawk or GNU AWK. (I know what you're thinking:<span style="font-style: italic;"> You use a Makefile to generate your .zshrc?!?! </span> It actually makes a lot of sense to strip that down and do dead code elimination since shells parse it every time they startup... maybe a contemporary shell has a better solution like dumping itself after your base config is loaded, but I did this almost 15 years ago and it's worked great for me.)<br /><br />Unfortunately, 10.4 seems to ship only with a pretty bare-bones AWK, and the base install of <a href="http://finkproject.org/">Fink</a> (and even the FinkCommander view of the available packages) didn't show gawk. Moreover, I didn't want my basic Terminal+Zsh configuration to depend on binaries in /sw/bin. Thus, I simply re-wrote the one script in Perl. For the curious, the script in question is now called dedup-path and simply removes multiple occurrences of a path component in a path (after normalizing the components) and optionally removes path components that match a regular expression. There were a couple other little things I needed to tweak for some shell tools (e.g., <span style="font-family:courier new;">export LESS='-i -X' </span>to make less's search case-insensitive and have it not clear the screen when exiting), but generally my configuration was pretty comfortable right away.<br /><br />BTW, after the fact, I did install Fink's gawk using <span style="font-family:courier new;">sudo /sw/bin/apt-get install gawk</span>.<br /><br /><br /><span style="font-size:180%;">Global Keybindings and Window Management</span><br /><br />One of the hardest things for me to get right on the Mac relates to the various keybindings I use to get around on the desktop, manage windows, and so forth. I try very hard to avoid using the mouse for most things as it kills my train of thought (and hence my productivity) to have to move my fingers off the keyboard. As I mentioned in an earlier post, <a href="http://homepage.mac.com/tconkling/windowdragon/">Window Dragon</a> was a real lifesaver in letting me move and resize windows without having to aim at far-too-little and often-obscured controls on window decorations. I found several other preference-panel tools that also really help.<br /><br />First, though, let me note that the "Universal Access" System Preference has lots of good stuff in it. After turning on "Enable access for assistive devices", I have Cmd+Option+= and Cmd+Option+_ do zoom in/out which helps lots when looking at smallish web page layout details or otherwise just working around ill-conceived UIs. I also turn Mouse Keys on so that I can control my pointer with the keyboard -- often that's faster than reaching up for my mouse.<br /><br />I also edited various other global keybindings using the "Keyboard & Mouse" System Preference. Most important is turning on "Full keyboard access" at the bottom of the screen. But I also tweaked lots of the keys to avoid overlap with Emacs keystrokes that my fingers have had memorized for over a decade. The <a href="http://mac.softpedia.com/get/Drivers/Microsoft-IntelliType.shtml">Microsoft IntelliType Pro keyboard</a> I use came with a nifty configuration tool that also provides some nice capabilities, and see also my earlier mention of <a href="http://doublecommand.sourceforge.net/">DoubleCommand</a>.<br /><br />I also found a couple of window/application switchers that I like better than the default Alt-Tab handling. I think I'm going to stick with <a href="http://www.petermaurer.de/filelist/">Witch</a>, but <a href="http://www.proteron.com/liteswitchx/">LightSwitch</a> and <a href="http://unsanity.com/haxies/wsx">WindowShade X</a> both have some neat features that I may want to keep around. Coupled with <a href="http://quicksilver.blacktree.com/">QuickSilver</a> for launching new applications (and a whole lot more, it seems, once I read through its capabilities [or maybe watch this <a href="http://www.youtube.com/watch?v=EBvFUhTqKK4">QuickSilver video</a>]), I'm pretty comfortable getting around my Mac.<br /><br /><br /><span style="font-size:180%;">Remaining Issues</span><br /><br />My two biggest remaining issues:<br /><br />1) I need more modifier keys -- Shift, Ctrl, Cmd, and Option are great, but on GNU/Linux I set up both Super and Hyper, too. This is important because there are often various layers of user interactions that might benefit from the same basic bindings, and modifier keys are the only recourse for disambiguating these. E.g., as I'm editing this Blog post inside Firefox, I might want Left to go cursor-left, Ctrl+Left to go left on word, Alt+Left to make Firefox go back to the prior page in the history, Cmd+Left to go to the Browser tab to the left of this one, Hyper+Left to change focus to the window to the left of the Firefox window, and Cmd+Option+Left to go to the left virtual console. I do use multiple-option keys for lots of keystrokes, but support for a bitmap of option keys is inconsistent at best, and avoiding conflict with all the various modifiers that different applications depend on is tricky (especially Emacs inside of a Terminal -- that goes through numerous different keyboard filtering/handling mechanisms before the application finally takes action on the escape sequences sent by the Terminal).<br /><br />2) I really really want my menus inside the windows so that I can have focus-follows-mouse. I tried setting <a href="http://www.macosxhints.com/article.php?story=20031029203936659">focus-follows-mouse for Terminal windows</a>, but that's not enough. Moreover, I didn't like the focus getting held by Terminal windows when I used Alt+Tab (or LiteSwitch or Witch) to switch to another window (maybe one of them can warp the pointer... I've not looked for that yet). Real focus-follows-mouse mode seems incompatible with the circumspect design decision of the application menu being tied to the top of the screen. Fitts' law is irrelevant for me because I use keybindings to manipulate the menus. Ugh... but I've mentioned that before.<br /><br />That's all for now... the outcome of these many hours of tweaking my Mac Mini is that I'm really, truly, finally loving the OS X experience!Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com4tag:blogger.com,1999:blog-7384142322553850856.post-14046101422931201132007-01-26T16:41:00.000-08:002007-01-26T16:57:03.229-08:00Happy days and book reviewsI'm thrilled to now report that I have addressed my biggest user-interface complaint about OS X: I found a package that eliminates the brain-dead resize-only-works-on-the-bottom-right-of-windows constraint! Props go to my colleague, Markus, who did some clever Google-brand searching upon hearing me bemoan my Mac's limitations. Without further ado, the answer: The <a href="http://homepage.mac.com/tconkling/windowdragon/">WindowDragon</a> module for the <a href="http://www.unsanity.com/haxies/ape">Application Enhancer (APE)</a>. Now I've got Ctrl-Shift-Cmd-Button2 assigned to drag the window (no matter where on click on it) and Ctrl-Shift-Cmd-Button3 assigned to resize the window. It's just like my <a href="http://scwm.sourceforge.net/">Scheme Constraints Window Manager (Scwm)</a> which I miss so dearly! I think it'll still be a while before I'm able to make windows dance around algorithmically by writing Scheme code... oh well!<br /><br />Another quick note: I selected a couple of books to read to soak wisdom for long-time-Mac hackers, and can now report on that experience. I bought <span style="font-style: italic;">Mac OS X: The Missing Manual, Tiger Edition</span> and find it generally useful to skim, but occasionally annoying to read. In particular, the author, David Pogue, continues to insult his readers even chapters in by always reminding them that instead of Ctrl-clicking, they can use the right mouse button. I suppose his insistence on making such remarks is only partially his fault as the Apple designers should just admit that multiple mouse buttons are substantially more expressive than one and ship the Mac with two buttons and a scroll-wheel!<br /><br />The other book that I've skimmed through is Brian Jepson and Ernest E. Rothman's <span style="font-style: italic;">Mac OS X Tiger for Unix Geeks</span>. It's got a broad but cursory treatment of lots of details about how you can find the Unix behind OS X, and how to make the best use of the various tools we know and love from that background even inside the pretty Mac exterior. The book walks a delicate line between just pointing out obvious command-line utilities such as <span style="font-family: courier new;">top</span> and providing subtle details about how OS X hides some Unix structure that would otherwise be tricky to learn. I'll recommend the book to folks who do have a Unix background, especially those who aren't really Unix Geeks.Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com2tag:blogger.com,1999:blog-7384142322553850856.post-88829748473361149312006-12-21T09:32:00.000-08:002006-12-21T09:52:51.299-08:00A couple of useful sites and things to tryI've spent a little bit of time looking around for some solutions to my complaints about Mac OS X, and, as usual, Google helped me find a variety of useful reading which I'll now help Google find better by adding some extra in-links:<br /><br /><ul><li><a href="http://www.xvsxp.com/index.php">X Vs. XP</a> compares OS X to Windows XP in a reasonably unbiased and undoubtedly thorough way. There are lots of useful tidbits here if you read carefully, though I wouldn't read too much into any final conclusions.</li><li><a href="http://www.codetek.com/ctvd/">CodeTek's VirtualDesktop Pro</a> looks like it may be an interesting and helpful way to improve the usability of the OS X desktop. I've not tried it yet, so definitely don't consider this a strong endorsement.</li><li><a href="http://www.macworld.com/">MacWorld</a> is a good online resource; it's <a href="http://www.macworld.com/weblogs/macgems/2006/09/mondomouse/index.php">MondoMouse article</a> was my greatest hope of being able to resize a window by its other corners (i.e., without having to use the bottom-right corner). Unfortunately, that dream didn't play out the way I'd hoped -- MondoMouse still only resizes that corner, but simply enables you to do the resizing without aiming for that corner so precisely.</li><li><a href="http://www.google.com/mac.html">Google's Mac Software page</a> is a good list of the software we make available for the Mac. <span style="font-style: italic;">Be sure to notice the Custom Search Engine box at the top of the page</span> that does a search over only Mac-related sites. And yes, I too look forward to more of Google's great client-side software being available for OS X.</li><li>An <a href="http://www.bioneural.net/2006/05/17/boot-an-intel-mac-into-ubuntu-linux-from-ipod/">imagination-inspiring article on booting an Intel-based Mac into Ubuntu Linux using an iPod</a>. Cool!</li><li>And, of course, there's <a href="http://www.parallels.com/">Parallels</a> (or <a href="http://www.apple.com/macosx/bootcamp/">BootCamp</a>) in case I can't get over my Mac Annoyances and decide to run Ubuntu or even Vista :-)</li></ul>I've definitely felt the smaller market share impacting the availability of workarounds for obvious problems. With respect to Windows, almost anytime I have the thought "somehow has to have fixed this," I can find some creative solutions. Even with Linux, I have a similar experience (perhaps because of the greater capabilities and curiousities of the hacker community around GNU/Linux). With Mac OS X, I seem to less often be satisfied with what I can find out there.<br /><br />Maybe if more Mac wizards wrote blogs.... :-)Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com0tag:blogger.com,1999:blog-7384142322553850856.post-88364501215359892572006-12-09T09:55:00.000-08:002006-12-09T10:45:39.393-08:00First experiences with my new Mac MiniFive days after placing my order, my Mac Mini arrived on Thursday November 30th, 2006. First some details about the system I configured and the accessories I orderd:<br /><ul><li>Mac Mini with 1.83 GHz Intel Core Duo processor, 2GB RAM, 120GB HD, SuperDrive 8x, and OS 10.4 [Tiger] (~$1200)<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieSJqHEqgabWmEH0SPTnoV7GUXUPa2E7JkqdFnhbZAAmGsM9bbE9YISdlGVk3kcjkqy9Sdi0PfqMuWOt-jxbmTi1GReRq-42rIoofYP_SmHmlslSKdtZi1V0f-9ZOwsfZVuF1fceZ7GohE/s1600-h/DSC01235.JPG"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 209px; height: 158px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieSJqHEqgabWmEH0SPTnoV7GUXUPa2E7JkqdFnhbZAAmGsM9bbE9YISdlGVk3kcjkqy9Sdi0PfqMuWOt-jxbmTi1GReRq-42rIoofYP_SmHmlslSKdtZi1V0f-9ZOwsfZVuF1fceZ7GohE/s320/DSC01235.JPG" alt="" id="BLOGGER_PHOTO_ID_5006590285333991666" border="0" /></a></li><li>Apple Cinema Display 23" LCD (~$1000)</li><li>Miglia TVMax TV Tuner/PVR USB device ($260 from Apple's Store)</li><li>500GB NewerTech MiniStack USB drive ($260 from Other World computing)</li></ul>I didn't bother with buying a new keyboard or mouse -- I have a couple extra USB keyboards and a USB 900 MHz (but not Bluetooth) wireless mouse lying around that I figured I could use until I decide if I want something more Mac-specific. I've since found the <a href="http://homepage.mac.com/ydkm/mskeybddiagram.htm">MS Wireless Keyboard for Mac</a> and might try it (except I'd really prefer a wired keyboard -- since it doesn't move, I'm happy to trade the cord for not having to change batteries).<br /><br />That Thursday night, I eagerly opened the packages (everything except the external drive, which was due to arrive the following week) and set it all up. Within a half hour, I was booting. Comically, the one time I briefly paused was in trying to figure out how to turn it on! My non-Mac keyboard lacks a power button, and I had located the stylish Mac Mini box and matching PVR device atop my old PC (magnifying the amazing contrast in design). In that location, it was hard to find the power button on the back of the Mini. My short search was rewarded with the recognizable and satisfying Mac-booting sound... it's surprising that PC manufacturers haven't duplicated that strategy and provided an equally distinctive and comforting audio snippet as the BIOS takes off.<br /><br />Once booted, I got started with the requisite configuration and software upgrades. Tiger (i.e Mac OS X 10.4) was very user-friendly in asking me questions to set itself up properly, and it took little time. What then took longer was installing all the updates since the software was installed at the factory. There was about 400MB of updates needed, which took about 45 minutes to download and install. (I should point out that the Mini first detected the Internet via my somewhat flaky Hawking 802.11g Wireless Bridge it was connected to via a router, rather than using its built-in AirPort to talk directly to my Linksys WAP downstairs -- it was a reasonable decision, but I've since switched my default route to use en1, the AirPort interface.)<br /><br />While the OS upgrades were in process, I downloaded other essential software: Firefox, various Firefox extension (especially the Google toolbar), and Google Earth. I also tried getting a Yamaha USB Midi adapter to work, only to discover that Yamaha hasn't yet updated the drivers to work with Intel-based Macs (wow, PowerMacs already seem so old! I solved that problem by just buying a different USB Midi adapter made by M-Audio from Amazon -- I downloaded the driver for it first to double-check that it was Intel-ready!)<br /><br />After a reboot or two, I had what Apple claimed was the latest install of all the Right Stuff. I only had another hour or so that night to experience the new system and had just a few significant senses and discoveries:<br /><ul><li>Having a Unix-like terminal program with all my comfortable Unix and GNU tools available out of the box was great (of course, Linux distributions have perfected this long ago, but somehow it <span style="font-style: italic;">felt</span> different because of the snazzy Aqua interface)</li><li>The modifier keys were a bit confusing to me... not only was my Windows-intended keyboard mis-labeled for a Mac, but certain shortcuts for, e.g., Firefox were switched from being on Control to being on Option or some other button. I'll have to spend some time customizing the keys, but I'm betting this is surmountable.<br /></li><li>The Cinema Display is somehow even more amazing than the Dell 24" widescreen displays I have at work... its brightness and the sub-pixel rendering of text that the OS does makes it simply extraordinary! (To be fair, Vista's rendering on the Dells is comparable -- my RedHat-based X distribution at work [from 2004ish] just looks lots worse.)<br /></li></ul>Now, between getting my Mac Mini setup and writing this post, it's been over a week (and a hectic one at that). With that extra time to reflect, I can now confidently point at my top two complaints so far:<br /><ul><li>Why, why, <span style="font-weight: bold;">WHY </span>can't I resize a window by dragging on arbitrary corners of the window?!? I shouldn't have to move and resize just to keep the bottom-right of a window in the same place and make the window smaller. [Disclaimer: I spent a fair amount of four years earning my Ph.D. working on the Scheme Constraints Window Manager (SCWM) which supports amazing capabilities for window arrangement and which I customized to my personal preferences very heavily. It's safe to conclude that I do not have typical requirements in terms of windowing support.] There are a couple of shareware tools that let you resize windows without having to grab on the control at the bottom-right, but they all still seem to only resize from that corner. Ugh!<br /></li><li>I don't like the Application menu being on the top of a giant screen. That design decision made sense on a 9" screen in the mid-1980s, but it's clearly wrong now: when I have an application window in the bottom right of the screen, I shouldn't have to drag the pointer all the way across all that screen real estate in order to tell the application what I want to do. I've held off looking for a solution to this so far, thinking maybe I'd get used to it and like it, but I'm becoming more skeptical that I'll ever come around.</li></ul>I'm looking forward to digging in more to the Darwin Unix side of my Mac this weekend and over the holiday. I'll write more when I have a sense of how that's going.Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com1tag:blogger.com,1999:blog-7384142322553850856.post-82384780436571693972006-11-27T10:50:00.000-08:002006-11-28T10:15:57.547-08:00My first Macintosh ComputerI've been programming computers since 1980, starting with a class at RadioShack learning BASIC on TRS-80 Model IIIs. Those machines pack a whopping 2 MHz of processing power, but it was enough to amaze me (hell, I was 7, so lots of things amazed me!) Over my 26 years of working with computers, I've accomplished lots of things over a wide variety of areas -- I've programmed in a dozen different languages including BASIC, 6502, Z80 and MIPS assembly languages, Pascal, C, C++, Java, Perl, Awk, Sed, Python, Ruby, Smalltalk, Haskell, Miranda, CLP(R), Prolog, Scheme, Lisp, JavaScript, XSLT, and others that I can't even remember. I've published in both industry magazines (e.g., Compute's Gazette, Run, InformIT) and in top tier international conferences and academic journals (e.g., WWW Conference, UIST, IEEE DCC, Software Practice and Experience, AAAI, TOCHI, IEEE Transacations on Software Engineering). For Google, I've built systems that are used by millions of people, and led teams that are responsible for a huge portion of the online performance advertising market. But one thing I've never done is own an Apple Macintosh computer...<br /><br />There's never been any compelling reason not to buy Macs, I just always found the raw computer performance of PCs more appealing to my sense of value than the premium that Apple forces consumers to pay for their glitzy white machines. I should clarify that I've used Macs a bit here and there through my professional career. Perhaps most notably I taught Algorithms and Data Structures to a great bunch of junior high-school students at Johns Hopkins University's Center for Talented Youth using Macs back in the summer of 1993.<br /><br />Two primary changes in technology made me finally consider the Mac as a viable alternative purchase. First, Mac's OS X, first released in 2001 (so I'm still behind the game) is based on BSD Unix -- finally, a real operating system on top of which all the great UI features are built. I've been a Linux nut since 1993, so OS X, at least under the hood, is already very familiar to me. Second, Apple finally switched to using Intel x86 processors in the Mac. This more recent change means that I can dual- or triple- boot the other OSs I care about on the same machine -- if I decide the Mac is not for me, I'll just install a new GNU/Linux distribution to play with or try Windows Vista in the spring. That means there's no risk of the new machine becoming a doorstop like my Commodore Vic-20 became 20 years ago when my mother bought a Commodore 128.<br /><br />So, this past long holiday weekend, CyberMonday came early to me, and I decided to order my first Apple Macintosh computer -- a decked out Mac Mini I accessorized with a TV tuner, a 23" HD CinemaDisplay monitor, a .5TB external drive, and a MIDI cables to USB adapter so I can plug in my music keyboard, too. Apple still hasn't sent me confirmation of the mini shipping, but everything else is on its way, and I'll report back here later about my first experiences.Greg Badroshttp://www.blogger.com/profile/00523413025773185278noreply@blogger.com0