Stuff and things

Complex regexp worked exactly as expected

Sat, 02 Mar 2013 10:13:21 +0000

Submitted by Leprosy

Lessig Blog, v2: Prosecutor as bully

Sat, 12 Jan 2013 22:04:34 +0000

(Some will say this is not the time. I disagree. This is the time when every mixed emotion needs to find voice.)

Since his arresting the early morning of January 11, 2011 — two years to the day before Aaron Swartz ended his life — I have known more about the events that began this…

programming is terrible: Learning from Nostalgia

Wed, 09 Jan 2013 07:31:35 +0000

programming is terrible: Learning from Nostalgia:

This topic fascinates me, and I think tef learns and wants to learn the same way that I do. See also Bret Victor’s Learnable Programming which also cites MindStorms, and is generally a brilliant piece.

programmingisterrible:

If you ask any programmer how you should learn to program, you will get the following answer:

People should learn programming from my mistakes by repeating them exactly, because my mistakes have been refined and polished over the years

Nostalgia, and the way in which the teacher learns best—both…

Probabilistic Graphical Models on Coursera

Sun, 06 Jan 2013 20:51:00 +0000

I recently completed the Probabilistic Graphical Models course on Coursera, and thought I would note down my thoughts about it.

What?

Probabilistic Graphical Models are a tool used in statistical fields, in particular commonly in machine learning. They are:

Models because the representation of the problem is separate from its solution (meaning you can solve it in different ways).
Graphical because the models are represented as graphs; this is the most characteristic feature of PGMs, and gives one a much more direct, tangible understanding of the problem. It also makes it a little easier to have non-technical domain experts create your model.
Probabilistic because, in essence, we’re guessing. Some or even most of the values in the model can be unknown and the problem is guessing their assignment.

Here’s a quick example:

This is a Bayes Net. The way to read this is as “flow of influence”; you can read find which conditions affect which others directly from the graph. Some examples:

Meeting someone new alters the chances that you’ve fallen in love.
Receiving flowers might cause you to fall in love with someone… or it might mean that you’re being consoled for a bereavement.
Love can take your breath away, but so can being asthmatic.

The relations of probabilities between these (which we must learn or infer in some way) allow us to make judgements. If you are short of breath, it might be because you are in love; if you’re also asthmatic, the chances of your gasping being due to love are lowered. The more information one has, the better one can make judgements about the missing values.

The course also covered Markov Networks, which are similar but with undirected relationship edges.

The nice thing is that there’s a massive number of problems that can be modelled in this way; there are a few different types of questions you can answer like this:

Probability of assignments — “What is the probability of a given value?” If a patient has a set of symptoms and a set of risk factors, what is the probability they have a given condition?
Maximum a posteriori assignment — “What is the most likely joint assignment of values?” For example if you are doing optical character recognition, what word is the set of images you have most likely to represent?
Value of information — “How much is it worth to find out more information?” Is it worth my while to do market research before deciding to launch my company? How much is it likely to alter my decision, versus how much it costs to perform?
…as well as these, information can be obtained simply by learning a model. Even though one may never use the model that is created, the relationships the learning process finds may not be ones which are known.

Instant gratification

At first, a lot of these models seem quite divorced from actual problem solving. Watching the course lectures and trying to take the information in, it can seem quite theoretical and “fluffy,” not quite directly applicable to your day-to-day.

This impression is unfounded, as I found out in the first practical for the course. I was surprised to find that after wiring up a model in a relatively naive way it all just worked (given a pre-written workbench for experimenting with the values). A lot of the coursework was like this, mystical theory machines spitting out reasonable answers following an understandable process. Although the problems were largely toys, you can feel that this could be made to work on real problems, and you can see how.

My surprise with the practicality of the course didn’t end there, though. I found a few of the more self-contained topics (often presented as sidenotes) were immediately useful to me even in my day job. It helped solve problems I was only half-aware that I had.

Sacrifice

The course is gruelling in its time demands. The Coursera description puts it at “8-10 hours per week”, but I think this is lowballing it somewhat. Watching the lectures alone, at regular speed, can take around 2-4 hours, depending on the week. The actual programming assignments took, in some cases, some 18 hours or more.

This might not be the same for everyone. I didn’t have much statistics or probability knowledge that I could actively remember, definitely less than the “very little” the course recommends. Learning this on the go can be difficult given the amount of time already spent watching lectures.

I also wasn’t familiar with Matlab/Octave, and it’s a definite “experience” for anyone used to modern programming languages or paradigms. The lectures make the point that they’ve found this toolkit works better for those learning, but in my own experience a considerable amount of the exercises was spent trying to glue various types of 2D matrices together in such a way that a library function can sweep in and calculate everything you want at once, since direct processing in the language itself is often too slow to be useful. This often feels like it’s more “busy-work” than actually helping you learn the topic. It’s a little like a much less satisfying version of lining up dominoes to fall.

Disappointment, and redemption

The exercises, while very varied and conceptually interesting, do vary a lot in quality and difficulty. There’s a few places where the assignments were confusing, misleading, or incomplete. This could make it very difficult to even know where to start at times. A lot of time is spent squinting, and occasionally trying to reverse-engineer what is wanted from the test data, in the cases where the lectures and assignment notes really don’t seem to give enough information.

Also, although the assignments often included test data with which to test one’s implementation, this was sometimes a little sparse. It didn’t test everything — deliberately, so that your understanding could be graded down the line — and sometimes there was so much to do in one section that trivial bugs could be near-impossible to find or diagnose. I suspect my lack of expertise with Octave also caused trouble in these situations.

However, these problems did lead me to the course’s online forum where others taking the course along with teaching assistants (usually “TAs”) were all trying to help one another out. This forum was a ray of hope when I’d gotten myself stuck on an exercise. When there were ambiguities, you’d be sure that someone else would have encountered them too. Normally questions had been asked and often answered before I’d even gotten to the relevant part of the assignment. Each of the assignments got a handy test script to run your code against the test data automatically (I’d been doing this by hand before). The friendly TAs took the roles of providing authoritative answers when it was clear that there was widespread misunderstanding, and of ensuring that the answers never got so explicit that one might think of it as cheating, or that it gives away what one would be trying to learn.

This all came to a head with the assignment 5 (the notorious “PA5”), an assignment that was both very challenging, and written in a fairly difficult-to-follow way. In this case, the TAs had provided a rewritten and expanded version of the assignment text, and were further revising this with comments from students. It’s hard to imagine how I’d have managed to do this assignment without the rewrite (a friend of mine also on the course tried this — it sounded horrible). With the revised text one could skip past the difficulties introduced by the sometimes-lacking descriptions and skip right to the gruelling but beneficial act of working out how to solve the puzzles.

Overall

I feel like I learned a lot from this course. Furthermore I feel like I learned a lot that was useful, and that I wouldn’t have learned in the course of my job or my usual tinkering. This is, for me, one of the most important aspects of this course — sometimes it’s easy to stick to what you already know and can practice easily, and often the academic aspects are what get dropped.

The assignments’ role in showing how directly the content of the seemingly-abstract lectures could be used showed that these techniques were useful. The approachable nature of the lectures were good at conveying understanding.

Although I can gripe about the assignments, the workload, and Octave (and trust me — I can gripe about those things), this felt really worthwhile. I might wait a while (and think harder) before investing so much time in another online course, but although it was occasionally painful I’m glad that I did it, and I think others will feel the same way. Plus the people helping one another in the forums in a mostly civilised and productive way gave a little extra boost to my faith in humanity. Isn’t that reason enough?

programming is terrible: What makes good and what makes bad programmers?

Wed, 02 Jan 2013 23:04:10 +0000

programming is terrible: What makes good and what makes bad programmers?:

programmingisterrible:

Many blogs claim to elcuidate a dichotomy of programmers—normally good and bad. Upon careful inspection, most of them turn out to actually dictate the following–

Programmers who are like me are good

Programmers who are not like me are bad

The assertion is that if you cargo cult their…

"And so this is Node.js’s problem. It’s just another gun in the gunfight, but its..."

Thu, 25 Oct 2012 09:10:53 +0100

“And so this is Node.js’s problem. It’s just another gun in the gunfight, but its community thinks its a cannon.”

- A old response to Ryan Dahl on Hacker News which sums up the overall feeling I got after trying out Node.js a while ago.

I’m a man on the edge. A man with a deep, dangerous...

Tue, 31 Jan 2012 19:34:09 +0000

I’m a man on the edge. A man with a deep, dangerous hatred.

I hate my alarm clock.

It’s not a bad alarm clock. I got it as a gift, and it has served me well for a few years now. But it works hard at bugging me. For example:

If the alarm is going off, and you want to see what time it is, you turn on the screen backlight by pressing “BRIGHTNESS CONTROL”. Except that the “BRIGHTNESS CONTROL” button is also the snooze (“REPEAT ALARM”) button. So the radio stops. And you must start it again. Of course dedicated buttons for these two functions would be crazy, since you need space for a dedicated “SCAN” button
To start the radio again, you press “SOURCE” until the function (FM, DAB, or “3.5mm jack you can put into your phone”) you want is highlighted on the screen. This helpfully remembers whether you had the alarm on earlier, to excite you by changing the number of presses required to get to DAB (I think it is 1 if the radio was on, 3 otherwise). This is a fun game!
If you try to turn one of the two alarms (“AL1” and “AL2”) off using their alarm buttons, you must use the alarm button for the alarm going off. Remember that the screen is unreadable and dark at this point! If you press the other button, it will toggle the other alarm on or off for the next day. This is a great feature for people who like to be surprised by which time they’re getting up.[1]
Whenever the radio is making a sound of any kind, there’s an audible background hum. This renders the fact that it “fades in” the radio when the alarm sounds effectively useless; the clicking on of the hum is loud enough to wake me.

[1] To be completely fair there is an “AL RESET” button snuck up in the corner which apparently resets either alarm.

[Flames image is CC-BY from pgordon on Flickr.]

I always liked the Banshee music player, but their 2.0 version...

Tue, 28 Jun 2011 23:17:31 +0100

I always liked the Banshee music player, but their 2.0 version seems to have fixed all of the niggling issues I had with previous releases, and fixed most of my typo’d music tags from when I still typed those on my own. Good stuff.

"I walked through my front door with purpose and gathered my family members in the living room to..."

Sun, 26 Dec 2010 16:43:15 +0000

“I walked through my front door with purpose and gathered my family members in the living room to tell them about my vision. I was going to rewrite the birth of Jesus Christ and I was going to make it POP.”

- Hyperbole and a Half is always great, and this Christmas-related entry had me giggling like an errant child.

CSS3 - A practical introduction

Sun, 31 Oct 2010 21:23:11 +0000

CSS3 - A practical introduction:

Quite enjoyed this presentation by Lea Verou; it’s both informative and, unusually, pretty. The new CSS3 stuff looks really neat, although I wouldn’t like to speculate when we can actually use any significant amount of it.

A representative of Twilio doing a presentation about the system...

Tue, 21 Sep 2010 13:04:41 +0100

A representative of Twilio doing a presentation about the system by live coding in front of the New York Tech Meetup. Really impressive.

via. this article on avc.com

Stupid Manufactoria has stolen my last couple of lunches from...

Wed, 01 Sep 2010 17:55:10 +0100

Stupid Manufactoria has stolen my last couple of lunches from me. I don’t usually fall for these nerd-focused casual games.

Some not entirely independent variables

Mon, 30 Aug 2010 21:32:26 +0100

Nearby sell-by dates.
Stupendous special offers.
Calum gorging himself on Frubes.

Headius: My Thoughts on Oracle v Google

Sat, 21 Aug 2010 12:42:22 +0100

Headius: My Thoughts on Oracle v Google:

Not only a good primer on the recent legal challenge, but actually a good primer on the background of Android and Java over the last decade or so. Excellent article.

10 Common Mistakes Made by API Providers

Sat, 21 Aug 2010 12:40:56 +0100

10 Common Mistakes Made by API Providers:

This list of common mistakes made in web APIs is excellent; many of the points apply equally well to APIs in general, especially remote ones.

Having installed apps from QR codes a couple of times, this new...

Mon, 21 Jun 2010 19:22:50 +0100

Having installed apps from QR codes a couple of times, this new GitHub feature seems disproportionately useful to me.

An excellent whiteboard animation based on a talk by Dan...

Fri, 18 Jun 2010 13:54:07 +0100

An excellent whiteboard animation based on a talk by Dan Pink’s talk on the same subject as his book Drive: The Surprising Truth About What Motivates Us*. Found this via an entry on Coding Horror, which isn’t normally a blog I read, but I’m glad I did this time.

* Associates link, apologies for capitalism etc.

"We prove that the implementation always strictly follows our high-level abstract specification of..."

Mon, 14 Jun 2010 21:25:49 +0100

“We prove that the implementation always strictly follows our high-level abstract specification of kernel behaviour. This encompasses traditional design and implementation safety properties such as that the kernel will never crash, and it will never perform an unsafe operation. It also implies much more: we can predict precisely how the kernel will behave in every possible situation.”

- Bold claims from a proof of correctness of the seL4 kernel

BEHOLD PRINGLESCAT

Sun, 02 May 2010 21:32:17 +0100

BEHOLD PRINGLESCAT

"Dynamic programming” never meant anything; it was nothing but a political euphemism. But..."

Sat, 24 Apr 2010 14:24:32 +0100

““Dynamic programming” never meant anything; it was nothing but a political euphemism. But somehow it became attached to one of the useful concepts Bellman discovered, and to this day, students of computer science struggle to learn that simple idea, because it’s burdened with a nonsense name.”

- Interesting! I’ve always thought that term was weird.