Reaching Smith Lake requires ascending over 1,500 feet in just over 4 miles, but once at the lake the sweaty climb from the Twin Lakes trailhead is quickly forgotten.

While it’s one of the smaller lakes in Desolation Wilderness, at over 8,000 ft elevation and with an unobstructed view far, far to the west, Smith Lake is a must-visit spot.

The elevation profile clearly shows the challenge but at least has a constant slope once the incline begins:

To get to Smith Lake, drive to the Twin Lakes trailhead (off route 50 near Tahoe), and take the trail towards Twin Lakes until reaching the fork for Twin Lakes / Smith Lake, and take the route to Smith Lake. The route leads past Grouse Lake and Hemlock Lake, before taking a final steep and faded trail up to Smith Lake. The last ascent may require some route finding or scrambling off trail, but as long as you keep ascending and don’t go too far to the east you’ll reach the edge of Smith Lake.

The final ascent:

The reward for reaching the lake: spectacular views to the west.

I used the Gaia GPS app on my iPhone to record the route, and then exported the data points into a custom Google map:

You can download the kml file generated by the Gaia app.

Desolation Wilderness is extremely popular and permits are required for overnight visits. Quotas are in place for the destination zones inside the Wilderness, and you need plan your route in advance, at least for the first night’s stop, as well as entrance and exit areas. It’s a bit difficult to read some of the maps published online; this is the one I’ve found to be most useful: https://www.recreation.gov/nrso/ca/dslt/DSLT.pdf More information about the permit system is available on the recreation.gov site.

Breakfast

The local conbini, or convenience store, usually provided breakfast, unless there was a French-style bakery to visit/judge or the mind was dead-set on a morning natto-don. Jet lag also forced breakfast hand, often leaving no choice but to arrive at the food markets early in the morning, before most of the other tourists.

Lunch

Lunch choices can be dizzying. Plan on having two lunches per day if you can fit it in. It’ll all be walked off anyway.

Dinner

Many days were planned starting with the final destination, dinner, and from there backwards in time. Dinner isn’t the end of an evening, but it can be a highlight.

Part of the Los Padres National Forest, the Ventana Wilderness feels remote, shows off incredible views of the Pacific coastline and the Santa Lucia mountains, and contains a quite varied environment. It’s close enough to the San Francisco Bay Area to be accessible for one– or two–night outings.

At 5,164 feet, Cone Peak overlooks the ocean from just 3.2 miles off the coastline. Cone Peak is ringed by a few campsites and is a highlight of the Ventana Wilderness — and recent repairs to surrounding trails have improved accessibility. The ascent to the peak is steep enough to present a bit of a challenge, but not so steep to trigger regret when carrying a backpack and a day’s supply of water.

We spent two nights in the Ventana Wilderness, making it up to and then around Cone Peak. We left the San Francisco Bay Area on an October Friday morning and arrived at the trailhead by 1pm. We followed this route:

Day 1:

• Drive to the west end of Nacimiento-Fergusson Road at Highway 1 near Kirk Creek Campground or the east end of the road at Fort Hunter Liggett.
• Take Nacimiento-Fergusson Road to unpaved Cone Peak Road (watch for signs for Cone Peak and the road, also called Coast Ridge Road), and drive ~5.5 miles north. Note Cone Peak Road is closed due to weather during the rainy season. Park near the Cone Peak trailhead or alternatively closer to the Vicente Flat trialhead, on Cone Peak Road about one mile before the Cone Peak Trailhead.
• Take Cone Peak trail to Cone Peak, then down to Trail Spring Campsite and onto Gamboa trail.
• Follow Gamboa trail to Goat camp for the night. We chose this site because there’s usually water flowing from a stream about a quarter of a mile away. In the 2015 California drought, water access was especially important for a multi–day trip. If you’re planning to go during the dry season, check for water condition reports on the Ventana Wilderness Alliance forums.
• Time: 05:51:11 hours, though we backtracked at one point and hiked about a mile more than necessary
• Distance: 8.98 miles, including the backtracking
• Ascent: 1,859.32 ft
• Min/max altitude: 2,467.18 ft / 4,801.25 ft

Day 2:

• Hike from Goat camp to Vicente Flat trail
• Take Vicente Flat trail to Vicente Flat campground and camp there. Note there is usually water about a half mile from the camp site.
• Time: 07:32:24 hours
• Distance: 8.23 miles
• Ascent: 1,365.82 ft
• Min/max altitude: 1,527.11 ft / 2,553.76 ft

Day 3 (half day):

• Hike on Vicente Flat trail from Vicente Flat campground to Cone Peak Road and back to the car
• Time: 02:09:32 hours
• Distance: 3.32 miles
• Ascent: 1,845.56 ft
• Min/max altitude: 1,602.94 ft / 3,332.28 ft

I used the Gaia GPS app on my iPhone to track our progress, and then exported the GPS data points into a custom Google map:

You can download a zip of my GPS files exported by the Gaia app.

We met few people along the trail, and we were the only ones staying at Goat camp on the night of Day 1. All along, we enjoyed solitude and only the sounds the forest and ocean wind brought us.

It took some time to piece together the information needed to plan this route. There is a lot of information online, but much of it is more than five years old. The sites below were some of the most useful for planning, especially for finding water sources and picking campsites:

• http://www.backpackinglight.com/cgi-bin/backpackinglight/forums/thread_display.html?forum_thread_id=98658
• http://idahoalpinezone.com/index.php?p=4_75
• https://pantilat.wordpress.com/2013/12/03/cone-peak-marathon-loop/
• Ventana Wilderness forum, one of the best places to find out about trails and camps in the Ventana Wilderness.

An important tip we learned from the Ventana Wilderness forum was to bring mosquito head nets. Many large and pesky flies were thwarted by our nets as we continued along our way in peace.

At Vicente Flat campground we met volunteers who maintain the trails, clearing fallen trees, making sure signs and paths are easy to see, and putting in hard physical labor so that everyone can enjoy the forest. As we planned our route, we learned the trail conditions have varied quite a bit over the years, and in recent years roadblocks have been removed and the trail has been easier to follow in areas where previously one might have lost it.

I’m really grateful that the Ventana Wilderness is available to anyone, and for volunteers who keep it so accessible. I donated to the Ventana Wilderness Alliance and if you’ve enjoyed the forest, you might consider doing the same.

Music from Outer Space

Music from Outer Space, self–described as “your synth DIY headquarters,” is a fantastic place to spend hours learning about analog sound synthesis. Run by Ray Wilson, MFOS includes tutorials and guides on analog synthesizer theory & operation, has a collection of projects ranging from introductory single–purpose musical toys to advanced, cabinet–sized modular synths, and even has an online store that sells specialty parts and whole synthesizer kits. Many projects caution that at least a basic level of electronics proficiency and knowledge is assumed and projects do not have step-by-step instructions to follow. The site’s mid 90s–style design fits it perfectly. There’s even a warning on the home page for mobile users: “You may experience difficulties navigating MFOS using a mobile device…We have our top people working on this.” Awesome.

Sound Lab Mini–Synth

I built the Sound Lab Mini–Synth, described by MFOS as “fun for someone with intermediate to advanced electronics skills who wants to make cool sounds.” It has two VCOs, an LFO, a variable filter, a VCA, an attack–release envelope generator, and a noise generator—all on a single PC board. There are plenty of cool knobs and switches to play with, but not so many as to overwhelm someone new to synths.

Parts

MFOS includes artwork drawings to make PC boards from scratch, but you can also purchase naked etched boards for your project. This makes it easy to get started—and while you wait for the PC board to arrive you can order all the electronic components to complete the kit (and wait for those, too).

To get the most precise tuning performance, stick to the recommended tolerances for resistors and other components. Otherwise, you can substitute 5% for the recommended 1% tolerance components. Things will still work, though you may not get linear voltage across an octave range, for example.

Enclosure and Panel

MFOS provides a very useful panel wiring diagram and a suggested front panel design, and he also includes front panel designs contributed by other builders. Of course, if you use someone else’s front panel design, the panel wiring diagram from MFOS won’t directly apply, but it’s you can still use it as long as you’re methodical and double–check your work. MFOS doesn’t give any more guidance on enclosures other than the panel designs, so this means you have to make or find your own. The enclosure, and especially the panel, are a key part of the design and should be carefully considered.

I used a 6" x 12" x 2" bamboo drawer organizer from The Container Store to house the synth. Its open–top design allows a front panel to be easily fitted, and it costs only $7.

After studying the user–contributed panel designs on MFOS, I designed my own in Adobe Illustrator, mostly because I needed to fit the front panel to the drawer organizer’s top opening. I made the panel from 1/8" white acrylic. I’m lucky to have access to a laser engraver at my office. Here’s a preliminary test fitting with a few knobs, switches, and buttons in place:

This was the first time I tried to use the laser cutter to engrave (instead of just cut) on white acrylic, and I wasn’t sure how readable the engraved design would be the acrylic, so I first tried a few pieces of scrap acrylic. The laser engraver doesn’t burn acrylic like it does wood, so it was nearly unreadable because it doesn’t produce any contrast (like burnt wood edges) to show the lettering. I used black acrylic paint to fill in the engraved areas, quickly wiping off the excess with a cloth before it dried. I later realized it’s not necessary to quickly remove the excess before it dries, and I found it slightly easier to wait for the paint to dry, and then use a paper towel to scrub off the excess. Getting the paint into all the lettering’s nooks and crannies required two or three coats (and wiping), with final edits gently made with a very fine–tipped paint brush.

Close–up of the lettering:

Integration and Testing

The clean front panel design hides a mess of wires behind it. Assembling the panel starts off easily with the mounting of switches, knobs, and potentiometers. From then all the work is done on the underside. It’s a clean, organized start:

Midway through wiring:

It’s pretty busy once connected to the PC board:

After finishing wiring I applied power and started testing. Not everything worked exactly as expected, so I carefully rechecked every single wire from point to point. MFOS provides very useful troubleshooting information with instructions on PC board probe points and the expected voltage or waveform to verify on an oscilloscope. I found and corrected a couple of mis–routed wires and one incorrectly placed resistor (I misread the color bands).

Sounds

Here’s a short video of the synth.

Enclosure

I decided not to use a wooden hardboard enclosure because I don’t have the tools to make precise cuts into it. Instead, the enclosure is made of 1/8 inch white acrylic, laser–cut to make a box about 12"x6"x3". The speaker grill is made of a grid of tiny holes each about 2mm in diameter. I wasn’t sure if the laser cutter would be able to cut fine holes with enough precision they wouldn’t bleed into each other, and I wasn’t sure if the acrylic would be strong enough to hold together once the holes were cut. It worked out very well and the acrylic is surprisingly strong, even with the dense perforation of the holes.

Reception

The telescoping FM antenna receives local stations quite well. Sometimes it needs to be rotated to get very clear reception but this is usually easy and doesn’t require much trial and error.

The internal ferrite loop AM antenna is heavily affected by local noise. It’s frequency dependent, drowning out most of the lower half of the AM band including strong local stations like KNBR 680, sometimes called “The 50,000 Watt Flamethrower,” according to Wikipedia. The noise may be coupled in through the power supply. I’m using an inexpensive AC/DC adapter delivering 12V and up to 1.5A, and it’s probably unregulated and poorly filtered. When powering the radio through USB (via the Arduino directly), most of the noise goes away.

Improvements and What’s Next

AM reception could use a lot of improvement. I could find a quieter power supply or add filtering on the supply line. A higher quality antenna would probably help, too. Adding internal shielding could also mitigate noise being coupled in through wires and boards inside the enclosure.

While it’s fun to listen to local favorites, I also enjoy listening to distant radio stations, like WWOZ from New Orleans or the BBC. I’m thinking of overhauling the radio to add streaming audio, probably with a handful of pre–programmed stations instead of a searchable directory. I’m not sure the Arduino UNO can handle streaming audio, so this might require a different controller, and a (preferably wireless) network connection. Ideally, the radio would have AM/FM and streaming audio. A slightly bigger speaker would probably be in order—maybe even stereo?

I recently built a device that displays MUNI bus and train arrival predictions in real-time. I live in a rare pocket of MUNI route richness in San Francisco, the Lower Haight, and am lucky to be within short walking distance of nine MUNI lines. When a line is delayed, has three buses backed up in a row, or I simply need to go downtown, I can pick from a number of options. The NextBus website is easy to use, and iPhone transit apps abound, but it’s still nice to just glance up to a screen as I’m running out the door and see which bus or train I should take.

The display shows up to three upcoming times for inbound and outbound routes I selected. Each route and its predictions are shown for three seconds before the display shows the next route. At the end of the route list, the display returns to the beginning of the list and starts over. A single button lets the user cycle quickly to a route of interest instead of waiting through each route in the list. The prediction times are updated about once per minute.

Hardware

The core is an Arduino Uno Ethernet, which is very similar to the Arduino Uno with an Ethernet shield, except the Ethernet interface is integrated into the same board as the Uno. A protoboard shield holds a few peripheral components, such as a potentiometer for setting the LCD contrast and hardware button de-bouncing. The LCD is a 16×2 character amber-on-black display to keep in style with SFMTA’s bus shelter indicators. The single button is an SPST momentary on-on switch.

Software

The NextBus API is reasonably documented and gives not only MUNI predictions, but also info for transit agencies in a few other cities. The API can give route details, such as a list of all the stops in a route, or even the list of all the routes in a transit agency, from which one could systematically build a list of all the stops for all the routes. All that’s needed for this project is to call the API for a particular route and stop ID pair, and handle the response from the server. For the curious, I’ve made the Arduino code available on Github.

The API response for a single route and stop, which can hold several prediction times, is often in the range of 800 bytes. Since the Uno has only 1 kB of RAM, it’s a bit of a challenge to accept and manipulate the response to find the times of interest. I started using Arduino String objects, but wasn’t able to get it to work reliably with the long string responses returned by the NextBus API. I eventually used character buffers and C string.h substring matching and copying instead. Rather than holding the entire response in memory, the device captures single line responses from the server and extracts prediction times from each line, if available. After the third route time is extracted, the rest of the route API response is discarded.

The list of route and stop pairs is hard-coded into the Arduino sketch. The Nextbus API format is also hard-coded into the UNO, but I assume this changes very infrequently.

Enclosure

The enclosure is made of layers of laser-cut 0.5mm thickness wood from Ponoko (timber core with a timber veneer on each side). I designed each layer in Illustrator, stacked them on top of each other, and glued them together. This is how the laser-cut wood came shipped from Ponoko:

Prototype showing the layers before they were glued together:

After gluing the layers together, I added wood fill to smooth the gaps between the layers, then sanded it down to smooth it further, and stained it to give it a darker look.

Enhancements

I’m considering adding a few things now that the display works reliably and is installed in the hallway:

• A great enhancement would be to add a software-based route and stop selector / menu that updates the UNO’s sketch so users can add or delete routes and stops.
• Over-the-wire firmware updates would allow for tweaks, such as updating routes and stops, without needing to connect a computer directly to the device.
• Moving from wired to wireless: this certainly isn’t necessary, but would allow for installation in many more locations and without requiring connecting the device with an Ethernet cable.

Not long ago, I would have built a traditional receiver design using discrete components, probably a superheterodyne (used in the vast majority of inexpensive alarm clocks and portable AM/FM radios lying around in homes) or perhaps a regenerative receiver for novelty. Today, the popularity of small, digital, low-cost devices with integrated AM/FM radios has driven more parts of the radio into integrated circuits (though often keeping the superheterodyne design).

Radio hardware

I was excited to find the Silicon Labs Si473x family of radio ICs. The Si4735 includes AM/FM/SW/LW reception in a single IC that requires just a few external components and two external antennas. Essentially, the chip replaces what traditionally requires a few dozen discrete components and it eliminates manual adjustment of tuning sub-circuits.

The reduced bill of materials doesn’t simplify everything. Instead of sourcing, assembling, and tuning discrete RF/IF components, the radio IC must be interfaced over a serial, 2-wire, or 3-wire connection. Silicon Labs provides guidance through its datasheet and a few application notes, but I found some steps, such as the power-on sequence, could have been more clearly documented.

Sparkfun sells the Si4735 AM/FM IC and an Arduino shield that includes the IC, a crystal, and a few components that support the AM and FM antennas. Using the shield absolves one of any SMD soldering or source the diodes, crystals, and inductors that support the chip.

Sparkfun wrote some code to help users get started with the shield, and a few others have extended it and added some very cool features, such as querying the IC to get its current frequency and adding RDBS processing to get FM station and programming information off the air. Wagner Sartori Junior (Trunet) and Jon Carrier added quite a bit of this functionality, so it’s thanks to them and Sparkfun that getting the Si4735 up and running with these cool features is so easy.

Other hardware

I needed some other hardware besides the Si4735 shield and Arduino board. For the display I used a serial enabled white-on-black 16×2 character LCD display. A rotary encoder is used as the tuning knob and band-switching push button, and a simple 10 kΩ potentiometer controls the volume. The Si4735 support volume adjustments via software, but I chose an analog potentiometer for simplicity.

The Si4735 shield outputs a very small audio signal (I haven’t tried to measure it yet), so an off-shield audio amplifier is mandatory. I opted for a mono amplifier based on a single NE5534 op amp. It has good performance, is reasonably priced, and doesn’t require too many supporting components. I considered adding stereo support but the radio didn’t require very high quality FM output, and there is a stereo headphone jack in case it’s really desired. The circuit I used is from http://www.electronics-diy.com/

Arduino, shield, and LCD

Software

Jon Carrier’s code is hosted on Github and includes example sketches. I played with them quite a bit before using bits and pieces in my own sketch. Jon put a lot of effort into making RDBS work, and while it’s quite cool to see the station and song information scroll across the screen, I don’t use it. I do like to see the station’s callsign, so I kept this part and display the callsign on the second line of the LCD display. Many stations don’t transmit RDBS data, so in case the station callsign is unknown, my LCD doesn’t display any information on the second line. I also removed the volume and seek states, since I’m handling the volume level through an analog potentiometer and I don’t care for a separate seek mode at this time (perhaps I will later).

I experienced a few of the issues that have been brought up in the comments on Jon’s page. In particular, I frequently saw the system start or very quickly jump to its initial frequency + 0.1 MHz, and I also found the LCD screen locking up when I turned the rotary encoder very quickly. I used a serial connection of 9600 baud (the default value for the serial enabled LCD) and I wasn’t able to really troubleshoot this behavior. I’ve read in a few places that these LCDs can lock up when they receive data too quickly, so my wild guess is that this was somehow happening, perhaps from other things coming unintentionally on the serial line. A few times I also reproduced the 655.3 MHz and 0.0 MHz frequency change that Jon mentioned, even though the code is supposed to restrict the frequency to the broadcast band range when using the rotary encoder for tuning. It seems some of these issues may come from the rotary encoder, debouncing (or lack thereof), and/or the rotary callback function.

There are many sample rotary encoder implementations, some using interrupts, some using the digitalRead function in the main loop, and so on. Since the tuning knob and responsiveness is so central to the radio’s operation, I wanted to make sure the rotary encoder’s data was reliably read and displayed on the LCD, even for very fast rotation rates. I made a separate sketch to prototype this part of the system and ended up using Arduino’s built-in interrupts based on the code found here. Once this was working on its own, I integrated it as the interrupt and callback in the radio sketch.

An important note about using the shield

****The Si4735 and the shield communicate through 3.3V, not the 5V that boards like the Arduino Uno, Duemilanove, and Diecimilia expect. This is documented in the comments of the Sparkfun product page and on a few forums/blogs. The two generally accepted workarounds are to:

1. Insert a silicon diode between pin D12 and the GPO1 line on the shield. Trunet also documents how to do this on his wiki.
2. Use a level-shifter to convert between 3.3V and 5V signals.

Both are hacks for using the shield with 5V Arduino boards and require a bit of surgery in severing a trace on the shield. Using the level-shifter is the more “robust” approach and I chose this because it gives the flexibility to send and receive SPI commands to the Si4735. Inserting the silicon diode allows reading SPI data, but not writing it. However, it takes up space on the shield and limits the possibility of adding other components on top of it. Another issue with the shield is that it connects the GND and AREF pins – meaning AREF is permanently tied to ground, unless you break that connection. This is important if you need to use the Arduino’s analog pins.

Next steps

The radio works as designed, the audio amplifier is built out on a protoboard (and is certainly loud enough), and the display and tuning are smooth, so all that’s left is to connect the power supply, mount the components inside the enclosure, which is made from Ampersand Hardbord. The tuning knobs are simple diamond knurled cylindrical knobs from Digikey.

Sitting in the audience I realized some speakers kept me quite engaged and others let my attention lapse a bit during their talk. Some conclusions from notes I took during talks I attended:

• Don’t labour on the obvious
  If you represent a large, well-known company, say Intel* for example, but you are talking about a narrower topic, don’t spend five minutes repeating what your company is already known for (in Intel’s case, that would be semiconductors and processors). It’s ok to discuss the larger area to give context and perspective, but please don’t spend more than absolutely necessary (yes, Intel is in the semiconductor and processor businesses, I GET IT).

• Unreadable slides are pointless
  If I can’t read the material on the slide it’s almost pointless to present it, especially if it has all text and no visual cues. And then to go on and read the unreadable-to-me text is simply punishment.

• Easily digested slides build rapport
  As a counter to the previous point: easily read and digested slides are the opposite of pointless. Rather than alienate the audience, they help build rapport with the speaker as the audience internalizes the content and forms an opinion about it. And an easily digested slide ensures the audience won’t drown in details or attempt to listen to the speaker and read the slide at the same time, which may reduce comprehension.

• Simple is beautiful
  Be clear, confident, and concise. Make a point and back it up. Don’t try to make ten points in the space of five, and don’t make references to related points unless they are in proper context. Stay on-topic and don’t ramble.

• Make it worthwhile
  Make it valuable to the audience. If the objective is to introduce your product or service, make sure the audience leaves with a clear understanding of it. If the objective is to demonstrate complex and difficult achievements, let the audience leave with a sense of wonder. Don’t regurgitate a press release or blog. In most cases, people are in the audience attending your talk and skipped others that may be of interest because they felt yours would be more worthwhile.

By no means is this list even remotely comprehensive or unbiased, rather, it has just a few things I’ll keep in mind the next time I’m behind the podium.

<table><tr><th></th></tr><tr><td></td></tr></table>

of the web.

I kid. Mostly.

http://www.viiphoto.com/showstory.php?nID=1180

If you’re not familiar with his photography, I encourage you to see some of his other projects, such as:
Bastard Eden, Our Chernobyl or Zek: In the Prison of the East.

Free and open source full-stack enterprise framework for agile development of fast, scalable, secure and portable database-driven web-based applications.

It is written in Python and was created by Massimo Di Pierro. He is amazingly omnipresent in support and development of web2py.

When considering how to extend the tweet-dropping capability, I looked at a few options. The program to record the tweets into a SimpleGeo layer was already written in Python so I looked at web applications and frameworks built around Python or at least with Python support. Looking back a bit, it seems a lot hinged on one program which ended up being but one very small element in a larger structure. But I was enjoying fiddling with Python and there are plenty of frameworks to choose from. I didn’t have much time to evaluate and test different frameworks so I wanted to make a decision quickly.

A framework that naturally comes to mind is Django. It’s popular, widely used in production, well documented, and actively supported through the community. Others like CherryPy and webpy are also available. After the head-slap that was realizing web2py is not related to webpy, I took a closer look and decided to go with web2py. My need was mostly to quickly make a prototype, and while the others would have been fine for this also, it seemed web2py would allow me to get up and running as with minimal learning of new template languages and other overhead. It also seemed to be easy to deploy to Google App Engine. Having limited information I knew there was some risk I would run into a roadblock while carrying a lot of sunk time into the development, but I estimated it was reasonable.

Now with an application framework comes MVC, hand-in-glove. I’d never used an MVC framework so I was completely green. MVC is a pretty broad model but it’s often seen in web frameworks such as Django and yes, web2py. web2py follows the MVC model but as I would soon learn, it is quite flexible. My use case really didn’t require many views at all, and probably only a couple of controllers at most. The flexibility of web2py allowed me to rapidly prototype what I wanted while still retaining the robust structure that MVC brings, and yet at the same time mostly abstract this from the user and even myself to some extent. It really seems to be the best combination in that sense.

As I investigated and experimented with the most suitable implementation of web2py for my need, I found an explanation on the difference between push and pull frameworks by Massimo:

There are two types of frameworks push and pull. In a push framework (like web2py, Django, Rails) the URL is mapped into a function, which returns data (in the form of a dictionary) and the data is rendered by one view. In a pull framework (like Struts and JBoss) the URL is mapped into a view which calls one or more controller functions. From your question I assume you have a pull framework in mind. You can mimic a pull framework in web2py in multiple ways. One way is via ajax requests…

This was exactly the flexibility I was looking for and just didn’t know it. I needed one or two views initially, and several groups of controller functions exposed within the same view. Reading this message gave me confidence I wasn’t going down a deep framework crevasse with no rope to haul myself back out if needed.

The learning curve for web2py is quite reasonable. It doesn’t have a separate template language per se, but it does have a structure and reusable components that, along with built-in helper functions, make prototyping speedy. And it allows Python code to be typed directly into .html files, by enclosing the Python code within ``.

Extensive documentation for web2py is easily found and you can start by going through the tutorial on the web2py website. It’ll take you from rendering a simple web page to creating a wiki in one chapter! Other examples demonstrate some of the easier-to-implement functions and they really show-off how easy it is to get something functional up and running.

(http://en.wikipedia.org/wiki/User:Tlogmer/Single_serving_sites)

The mandated decay imposed on my previous experiment due to restricting Twitter searches to no more than a week (or so) in the past had me thinking of a more permanent method to archive tweets and their geo-location information. On the same day I wrapped up SF Tweets the World Cup 2010, Andrew at SimpleGeo wrote about how to map Foursquare checkins to a SimpleGeo layer. This way the checkins are all visible on one layer and the user can perform spatial queries on them.

Andrew’s example Python script looked simple enough that I took it as a base to do the same as I’d essentially done with the World Cup tweets, but instead of plotting tweets on a Google map they would be inserted into a SimpleGeo layer. This time, rather than searching on a keyword and collecting any returned results, the purpose was to collect tweets from a Twitter user, in this case, my own.

I could have accessed the Twitter API directly as I did for the World Cup tweets, but the availability of some simple and excellent Twitter clients/libraries for Python made it too easy to avoid. I quickly found a few and settled on Tweepy, by joshthecoder. It’s described as a Python library with “complete coverage” of the Twitter API. And just as cool, it has reasonably detailed documentation. Sold!

The first step was to get Tweepy returning tweets. For a single user’s timeline Tweepy provides the method tweepy.api.user_timeline, one of several timeline methods. By default this returns the last 20 tweets as objects available for manipulation, including the fields of interest here such as coordinates, if available, the tweet ID, date and time, and so on. It’s quite easy to get Tweepy talking with Twitter and in minutes I had my latest 20 tweets on the screen. Since I’m no Python guru, it took me a bit longer to get familiar with the objects returned to me and their available attributes and methods. Someone even slightly Python-adept could have breezed through this in just a few clock cycles, but it’s part of the learning process for me.

Assigning latitude and longitude is pretty easy as long as the tweet has the associated tweet.geo data. As I noted in the World Cup blog post, this is often not the case. This was an important reason to plot my own tweets: I have control over the geotagging of each tweet, and don’t have to depend on others to do it. If I were to grab tweets from the search API or the public timeline, it’s likely a small percentage would have latitude and longitude information.

But wait, what about Twitter places? Yes, users can attach place-based geographic information to their tweets. This allows the user to reveal the general location from which the tweet is sent without revealing the exact coordinates. Naturally, a bounding box described this area on a map, and Twitter has associated a unique URL for each place. In my case, many of my tweets are sent from the Richmond in San Francisco. The XML attached to my tweets sent from the Richmond and identified as such is this:

<twitter:place xmlns:georss="http://www.georss.org/georss"> <br /><twitter:id>64be9bb264eb76c1</twitter:id> <twitter:name>Central Richmond</twitter:name> <twitter:full_name> Central Richmond, San Francisco </twitter:full_name> <twitter:place_type> neighborhood </twitter:place_type> <twitter:url> http://api.twitter.com/1/geo/id/64be9bb264eb76c1.json </twitter:url> <twitter:attributes> <twitter:bounding_box> <georss:polygon>37.77212997 -122.49240012 37.77212997 -122.47178184 37.78442901 -122.47178184 37.78442901 -122.49240012</georss:polygon> </twitter:bounding_box> <twitter:country code="US">The United States of America</twitter:country> <br /></twitter:attributes></twitter:place>

A bounding box like this is not difficult to deal with, but there should be some consideration on how the information is presented when it is in the same context as markers on a map with lat and lon coordinates. But that’s a digression…

Once the proper modules are imported, the two key components to call are the SimpleGeo Client(MY_OAUTH_KEY, MY_OAUTH_SECRET) and the Tweepy tweepy.api.user_timeline(MY_TWITTER_USERNAME). All that basically remains is to throw objects from one universe into the other. Tweepy returns status objects and the SimpleGeo client adds records to a database.

My own tweets don’t always have associated coordinates, so even though I can choose to tag or not tag a tweet with coordinates, it’s smart to check and ensure a tweet has the necessary properties before trying to add it to a layer. If a tweet has coordinates, its status object will have an array in tweet.geo['coordinates'] of the form [lat,lon]. That’s simple to verify and require before calling the function to insert the object into the layer. Some other properties are useful: the tweet’s URL, the place (this could be useful later even if the lat,lon coordinates are included), the text of the tweet itself, and the time-stamp.

I ran into one hiccup with the time-stamp with regard to time zones and offsets. Each Tweepy status returns a time-stamp in GMT form. When I first added these to the SimpleGeo layer they had an offset of +7 hours (the magnitude of the difference between PDT and GMT zones). To account for this I manually subtracted the time zone offset from the tweet’s time-stamp, and that fixed it. I peeked at the Tweepy code to see if I could do it more elegantly, but I’m not sure I understand how it handles locale as there is some comment about backward-compatibility with Python 2.4.

It’s also quite simple to delete records and I did this a few times as an experiment and to check I was manipulating the layer’s objects correctly. A simple call to SimpleGeo’s client.delete_record will delete the object by passing the layer and object ID.

I didn’t change anything from Andrew’s Foursquare example to add the objects into the record and it worked! Next steps might be to add duplicate record checking, pagination, and spatial queries into the SimpleGeo layer. Thanks to Andrew for his help on getting my example to work – he gave me some pointers and was readily available to help me out.

The .py file is available here: http://gist.github.com/474741

If you’re interested, the site is here: http://www.pabloestrada.us/worldcup/

I wondered how difficult it might be to plot out local tweets on a Google map and see their geographic distribution, so I checked out the Google Maps API on a recommendation from a friend that the API is well documented, easy to understand and straightforward to implement. Indeed, throwing a map onto a web page is almost trivial and all that’s needed are some very basic parameters including latitude and longitude coordinates that define the map’s center. Easy enough, I thought. The next step was slurping some tweets from the Twitter API.

Pulling data from the Twitter search API is pretty straightforward and has been implemented elsewhere. Choosing to use jQuery not only simplified data retrieval, but also put the broad set of jQuery functions and plugins within reach. jQuery’s widespread use and well-docmuented API shortens the time to implementation for me. Naturally, I depended on Google search. However, many implementations use the JSON format instead of JSONP. I understand this is done for security reasons, and this necessitates the use of a server-side proxy. I chose JSONP simply because it doesn’t need the proxy.

Twitter’s advanced search page allowed me to construct the query and cross-checked with what I’d written based on their API documentation for the Search API. The documentation covers all the optional parameters and provides simple examples of their use, however, I would only later realize at least two parameters are missing range (allowed values): the since and until parameters; they allow the search to be restricted to a date range. I wished to restrict the search over given match dates to only return results that were tweeted during (or near) the game. The finest resolution for this restriction is entire days, so it’s not possible to restrict the search to within certain hours of a date.

At first I tried to use $.ajax to make the query as it provides a bit more flexibility than the $.getJSON function. At this point I was using mostly Google Chrome for testing and kept getting null as the returned value. In tracing this down, I found there was an unexpected character in line 1 of the HTML file, causing an Uncaught SyntaxError (Unexpected token) as a result. That seemed odd, because I certainly wasn’t changing the first (or first few) lines of the HTML file. After some searching I found this may be related to an old defect in Chrome. It seems a BOM character is inserted at the front of the JSON response, according to this: http://code.google.com/p/chromium/issues/detail?id=176. To avoid it I instead used $.getJSON. It has a bit less flexibility than $.ajax, but I worked around that by simply defining the callback function in $.getJSON to call another function, which I constructed separately. This also kept the code a bit more modular and easier to read. Once I had a proper query constructed and had a proper grip on the returned data, I saved the JSON response to a test.js file so I could keep developing without having to poll the API each and every time I, say, changed the color of some text on the page. It also froze the search results so I could test cases of tweets with different location results.

As I looked over the search results, I realized a very low percentage of tweets are tagged with geo-location data. Those that are make it very easy to jump from tweet to Google map, since the geo-tags include latitude and longitude – perfect for placing on a map! Unfortunately, most tweets don’t have this data. The Twitter search API makes it easy to restrict a search to a geographic area; my search was restricted to a 12 mile radius around San Francisco. I quickly realized the variety of location information among the tweets. Some users populate their profile with a location description: this could be San Francisco, or SF, or San Francisco, CA. Others populate a location that includes latitude and longitude, but is prefixed by a string such as “iPhone: ” or “ÜT: “. This really reduces the value of placing the tweets on a map of local (city) scale. When comparing tweets between different cities, this is perfectly valid and useful, and is probably sufficient resolution to make a geographic comparison. For example, one could plot out tweets from San Francisco and Los Angeles on one map. In that case, it may not be necessary to know exactly where in San Francisco a tweet originated. But in my case, I wanted to distinguish tweets sent from Mad Dog vs Danny Coyles, and this level of resolution just isn’t enough.

I decided not to plot tweets with unspecific location information on the map. This would not be intuitive because a user naturally associates a pinpoint on a map with that specific geographic location. Where should tweets tagged with the location, “SF,” be placed on the map? I considered placing them all in the middle, or very close to the middle, of the city (this is somewhere in or near The Castro). But then how could they be differentiated from a tweet that really did originate in The Castro and was tagged with latitude and longitude coordinates? I decided to list the tweets below the map but only plot tweets with coordinates onto the map. I used some simple regular expressions to match a few variations in location, for example, the iPhone or ÜT prefixes to coordinates.

The next step was to automatically trigger a search based on a World Cup match. The user shouldn’t have to type in “#GER” to search for Germany on the date of the Germany vs Spain semifinal. I placed all 64 matches on the right side of the page with the score for each team. Using small flag icons next to each team name, I had javascript extract the team name from the img src attribute of the flag to automatically trigger a search for the corresponding hashtag for that team. I also encoded each a href attribute with the match number so that the search could key off an array of 64 date values and restrict the search to the match date. This was pretty easy using regular expressions, but I later realized that using regex was completely overkill since the length and format of each img src string does not change for all the teams and all the matches. Only later when I tested in Internet Explorer did I realize this. Thanks, IE, for reminding me of the stings developers endured in years past when trying to write cross-browser compatible code for Netscape and IE 4, and who knows what other nightmares. Internet Explorer treats calls to String.split() with regular expressions differently than other browsers. But of course! I found a brief explanation on a page titled Inelegant JavaScript. Woohoo! In the end, it was pretty easy to slice() the string I needed and that took care of it. But hey, I learned some simple regular expressions and browser inconsistencies, right?

The next step was to take the process location information from the relevant tweets and place each one on the map. I used each team’s flag in place of the default Google map marker. If more than one team’s tweets were to be plotted on the map, this would produce a visual display of the distribution and relative magnitude between groups of tweeters mentioning each team. At some point this could even go further and plot some type of mood indicator for fans of each team, but that would require more processing and estimation of the tweet text and some guessing based on that. My map only plots one team at a time, but that’s not hard to change. In fact, I had some trouble not to do that, as I wanted to clear the map of all flags when a new search was triggered. The previous version (V2) of the Google API had a method to clear markers, but the current version requires each marker to be tracked so it can be individually erased when the clear marker function is called. Luckily, Stack Overflow came to the rescue with a solution for V3.

I made a function to take the JSON data and clicked-on team name. The function dynamically appended the contents of a `

` with the user profile image (avatar) of the Twitter user, the location information sent with the tweet, and the contents of the tweet. For tweets containing coordinate information, a function is triggered to plot the tweet on the Google map with the corresponding team flag. That function also creates an InfoWindow (tooltip) containing the user profile image and tweet contents, making each flag clickable, just like default Google markers. That adds a bit of personalization to each tweet on the map, and it’s cool to see the user’s profile next to their tweet and team flag.

After everything was working as expected, I used another function to build the search string URL based on the team name, match date, and some Twitter search parameters. Once everything was working I disconnected the test.js file and connected the code to the Twitter search API using this function to provide the input to $.getJSON.

There are still possible improvements, for example, I’ve done nothing to consider the speed of searches or anything to make the content update look a bit more smooth. There are also no graceful failures, for example, if a search yields no results or fails for another reason, or if a user clicks on a match in the future (because this triggers a search on a future date). Currently, nothing happens and no error is presented. Finally, this whole experiment will expire in its current form a few days or weeks after the end of the World Cup when the match dates become too old for the Twitter search API. Just like the tournament, it’s been fun while it lasted.

]]> 2010-07-02T00:00:00-07:00 http://pabloestrada.us/2010/gearing-up-for-arg-vs-ger And to warm up, a New York Times article on Maradona.

An Apology to Maradona, a Rollicking Genius, By ROB HUGHES

Should this match live up to expectations, it will be historic.

]]> 2010-06-25T00:00:00-07:00 http://pabloestrada.us/2010/class-abuse The introductory video on the Compass site really intrigued me – http://compass-style.org/docs/.

Compass is described on its site as:

“a stylesheet authoring tool that uses the Sass stylesheet language to make your stylesheets smaller and your web site easier to maintain. Compass provides ports of the best of breed css frameworks that you can use without forcing you to use their presentational class names. It’s a new way of thinking about stylesheets that must be seen in action!”

As I watched, I thought: Yes! It’s true! CSS can really suck sometimes! I loved the comment, “ferociously abusing your classes.” As CSS has evolved, it seems abuse has become more and more common, and maybe necessary, and people hammer away and pound it into submission. Grids! And with CSS3 grinding its cleats into the grass, I’ve seen sites emerge with complex and delicate layouts that show off the power and magic, and not least of all the promise of –the future–.

But just because you can doesn’t mean you should. I’d like to use CSS(3) as a way to move away from the extremely carefully positioned and crafted layouts that, even though technically impressive, are visually confusing and crowded. Somewhere, there must be a former HTML 4 hack who labored so hard, for so long, to implement something similar using nothing but <tr> and <td> cells who surely feels like the ability to do this with (relative) ease is better than nirvana. Liquid, fluid, blah blah blah. I don’t mean to say I don’t admire and desire a carefully, precisely positioned and crafted layout. I do. I just don’t think everything should be placed on n-th pixel resolution just because we can.

Two examples:
1) UX Magazine

Is it cleanly organized and orderly? Certainly it is. Is the text highly readable? Mostly yes, except for some of the categories that are the top heading in each box. Is it cluttered? I think it is. My eyes go nowhere and everywhere at the same time. It’s easy to understand the navigation and news sidebar, however, the center content seems all to have equal relative weight. Maybe that’s the intent, as the footer notes the content is based on articles published one-at-a-time. The quote from Bertrand Russell looks as if he tweeted it – I like that.

2) Jeff Croft

This looks just as organized and orderly, and in my view, much easier to understand visually. It lets the reader feel in control of the visual content and not barraged with everything at once. I don’t like everything there, but it’s the style of site I’d like to make.

Searching around for CSS frameworks I found this excellent article from Eric Meyer about the lessons of CSS frameworks. Great tips to keep in mind, especially before diving into frameworks.

This site is based on a WordPress theme, which itself is based on a 900 grid system. As I hacked at it, I mostly stripped away from it, partially because it had more functions than I wanted or needed, and partially to just simplify it (instead of just ignoring unused parts). My frustrations took root in the naming conventions and myriad div and span elements. <div class="span-13">, anyone? How about A collection of 224 Grid and CSS Layouts? No, thanks.

Anyway, this isn’t about being anti-grid. Compass looked cool, at least from its introduction. Naturally this led me to Saas. The ability to assign and use variables, perform simple math, and even simply nesting are appealing enough to give it a try. Functions and mixins? Woah. Now, I’m not sure if this will move away from such lovely-named classes, as was claimed (but not actually done) in the introductory video. But leaving that aside, I’m still tickled enough to give it a try. Their tutorial gives a quick overview.

]]> 2010-06-11T00:00:00-07:00 http://pabloestrada.us/2010/sounds-of-las-vegas-or-rather-inside-a-casino This week I spent a few days in Las Vegas attending a work function. The event was held at the MGM Grand Hotel/Casino/… I don’t gamble much at all but I was amused by the sights and sounds while I walked the ~mile from my hotel room to the conference center. Naturally, one must navigate myriad gaming devices along the way, each with some flashy system that attracts and entertains the gambler. Some are cheesy, some are even a bit clever.

I used my phone to record some sounds as I walked around among the slot and video machines.

Vegas – soundscape #1 (00:54)

Vegas – soundscape #2 (00:42)

]]> 2010-06-03T00:00:00-07:00 http://pabloestrada.us/2010/sinkholes-and-other-calamities A volcano burps and then the year’s first tropical storm of the season washes out. It’s been a rough week for Guatemala. Last week the volcano Pacaya, located about 25 km (15 mi) from the capital city (Guatemala City) mildly erupted, spitting ash and sand all over the metropolitan area and forcing the evacuation of residents closest to its slopes. Pacaya is roughly in a state of continuous eruption, though usually it causes little or no incident. That may be hard to imagine but indeed, at night it’s often possible to see a faint red glow at its top or during the day see a plume or puff of seeping out. It’s a tourist attraction for brave souls who are able to scamper up to its top and peer over the edge, guard rails not included.
Read more from an AP article.
When I lived there in 1998, the same volcano coughed ash all over the city, causing some disruption and again threatening those who lived closest. Twelve years later, the effect is much less benign than it was back then. A thick layer of ash and mostly sand covered the city and families were forced to evacuate or lost their homes completely. Schools closed for a week as they were temporarily reappropriated to serve as emergency shelters. The airport was shut down and left hundreds, probably thousands, stranded. It’s not of the same scale as the effect of Eyjafjallajokull on northern European airspace, but for a small country it’s not trivial.

Less than a week later, tropical storm Agatha, the first named storm of the season, rumbled in from the Atlantic, bringing strong winds and dumping over three feet of rain fell in less than 24 hours. Three feet! Widespread damange was not limited to Guatemala: Honduras and El Salvador were also under Agatha’s wrath. Sadly, hundreds of homes were swept away, mostly in the interior of the country. It’s estimated more than 150 people were killed. The term interior refers, generally, to the less densly-populated areas outside the capital region. It’s also where poverty is highest and infrastructure is weakest. (CSM article)
from Spreadit.org

And if you have seen news from Guatemala in the past week, it’s likely been related to “the sinkhole.” It sounds odd, almost in a sci-fi way, and many first reactions to the photos are simply disbelief. But yes, unbelievably, it’s true. This hole sucked out a street intersection and is estimated to be about 100m deep (or ~313 feet). Even those dimensions are hard to comprehend. What could possibly be down there? What type of soil and rock? What about the surrounding area, or even the rest of the city? It’s pretty creepy and oddly, not even the first in Guatemala City. In 2007, a similar sinkhole (father of sinkhole?) also swallowed up a chunk of the city, leaving a similar creepy vacuum behind. I’m not sure what was done to repair it. A quick

Google search shows more jaw-dropping photos of holes in the city. Fascinating, sad, and scary all at once.

]]> 2010-05-29T00:00:00-07:00 http://pabloestrada.us/2010/sadness

From ~5000 ft under the sea

BP stream