Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

January 20 2013


Elixir: First Impressions

For the longest time now, I’ve admired Erlang from afar. It always seemed to be a bit daunting to take on. For one, there was the slightly weird and inconsistent Prolog-inspired syntax (I was always scratching my head over why this place needs a period and that place doesn’t), and then there was just plain weird stuff like one-based indexes.

While you don’t end up needing indexes very often, a nice syntax on top of Erlang is something I always kind of wanted, but nothing really could deliver. Then I saw Jose Valim demoing Elixir at Strange Loop 2012. It has a ruby-inspired (but more regular) syntax, it can do macros(!), it has protocols(!!!), and it has a very enthusiastic developer community behind it (see expm for an example of the packages that people have written/ported over to Elixir). That its data structures use zero-based index access certainly helps, too (-:

On top of all these nice things, it also lets you use any Erlang library (with only minimally less nice syntax by default). I think I’m sold.

What is all that hair on the floor?

As an initial just-for-fun project, I tried porting over the progress I’d made on a node.js-based gmail->localhost IMAP backup tool that I’d optimistically named gmail-syncer.1 So far, this has required a ton of yak shaving, but I’m enjoying the hell out of every single step down the fractal yak ranch.

  • First, there is no suitable IMAP client library. The thing that comes closest is erlmail. It is somewhat abandoned, and its IMAP client isn’t very usable for my purposes (doesn’t implement capabilities the way I need them, doesn’t really follow the one relatively sane guide to writing an IMAP client). So I’ll have to write my own IMAP interaction code.

  • To write my own IMAP code, I need to parse server responses; this requires parsing the highly weird IMAP protocol, with its somewhat lisp-inspired (but definitely not lispy) ideas of how to represent things. For example, The way a UID FETCH response looks makes it pretty impractical to tokenize & parse the response using a parser generator - unless you enjoy concatenating potentially dozens of megabytes of text that would do better to remain as an opaque binary buffer.

  • Hence, to parse server responses in a smarter way, I have to have a smarter parser. While that can use a pretty nice heuristic (despite its lispy nature, the IMAP server responses are specified to terminate in newlines at certain points), I still need it to cooperate well with something that manages buffers received from the network somewhat smartly. Aaaand that’s where I am right now.

Introducing gmail_synchronize, the tool that doesn’t do very much right now other than fill a buffer and let you read lines or N-byte-long binaries from them. But I’m sure there will be more stuff eventually (-:

To come this far, I’ve written some kilobytes of code (on various levels of the aforementioned yak stack) and thrown them away. The results in the git repo are the best I have come up with, so far. This isn’t much, and so you should take the following opinions with a mine of salt.

My impression of Elixir so far

Here’s a brain dump of what about the language stood out to me:

So far, I really like Elixir (and, by extension, Erlang). There’s a lot to be said about its pattern matching (which is as powerful as Erlang’s), but I don’t think I fully understand it yet. There’s a bit of terminology I still have to learn, but even at this level of (non-)proficiency, it’s making my job way easier.

There’s a very helpful channel on freenode, #elixir-lang. It has the creator of the language in it, and a bunch of very enthusiastic, knowledgeable and helpful people (hi, yrashk and cmn!). This has been invaluable in my learning to use the language.

I still don’t quite get why some of the decisions in it were made the way they were made. For example, it would seem natural to me to have a way to pattern-match binary buffers to test whether some bytes appear next to each other in the buffer, but there isn’t. I guess this may have to do with being able to unambiguously resolve the pattern, but it’s still a bit unsatisfactory. I’m sure this will pass as I learn more of its vocabulary and integrate it into mine.

Testing in Elixir is very cool. Instead of mocking or stubbing things like I would in, say, Ruby, I factor things such that tests can implement a protocol that the part being tested uses, and I’m set. I love protocols, and I think Elixir lets you use them in a very nice way. See here for how the tests interact with a library that follows a protocol. Note the re_buffered variable - in Ruby, I’d be using a method call expectation instead - this is way more satisfying.

Non-modifiable data structures are way less of a pain than I’d imagined (they are in fact pretty pleasing). The pattern matching makes things much easier to follow, and the way updates (which return a new object) work is also pretty cool: You can write stuff like:

…and this returns a record that is like some_record, except its buffer and number components are replaced by the values passed in the function argument list. Pretty pleasing.

I would not have been able to write code so relatively painlessly if it weren’t for the emacs mode that I’ve painfully adjusted to automatically indent Elixir code correctly. Emacs’s smie is really pretty cool, and I wish more emacs modes used it (-:

That’s all so far. I urge you to check out Elixir, and hope you have as much fun with it as I do!

  1. Why write a new tool over using offlineimap? Offlineimap is a huge pain - when used with gmail, it’ll sometimes run into UIDVALIDITY mismatches (which require a re-download of potentially huge mailboxes, which run for days), it’s slow, and its thread-based design is so horrible that it manages to mess up its own UI even when using a single worker thread, and then it can’t even exit cleanly on anything other than a SIGKILL. Arrrrgh.

Tags: Hacks

December 27 2012


Write gmail filters in a nice Ruby DSL: gmail-britta

I’ve just finished (mostly) documenting and writing tests for my latest little library, gmail-britta, so thought I should release it to the world as a sort of holiday gift.

Gmail-britta is a library that lets you write Gmail filters (hah, Britta, get it?) in a way that doesn’t drive you insane - you write them as a ruby program, run that program, and out comes XML that you can import into Gmail’s filter settings.

It does a bunch of other nice things, but I guess it’s better to let the README explain

So far, I (and a few colleagues of mine) have been successfully using this for the past few months to generate filters for work email. Just yesterday I took the step and ported my 156 filters over to a gmail-britta program (yep, that’s my filters, with sensitive email addresses stubbed out), resulting in 34, easier to maintain, more accurate filters.

If you’re interested, please give it a try. Also, please let me know in the issues if you find anything that it doesn’t do, or if you’re feeling super generous, please open a pull request and send me improvements!

Tags: Hacks

November 20 2010


Userscript for nicer l1sp.org search

I love the l1sp.org documentation redirection service. It is quick and easy to look stuff up there if you know the name, and it has a pretty good search if you don’t. However, the search results are not presented very nicely: They’re very close together, and the ones I’m looking for most often (mostly CLHS pages) are buried somewhere in the middle.

So I wrote a userscript (for Firefox through greasemonkey or Google Chrome/Chromium) to improve things a bit: It enables keyboard navigation (j/k or cursor-down/up select the next/previous result, enter opens the page, and / focuses the input field), and searches for the closest match from the results (ranks them by section and then selects the shortest matching entry).

Get the l1sp.org userscript here.

I do hope that Xach will consider adding these features to l1sp proper (maybe de-uglified a bit… that yellow highlight is really not too great). Until then, you can already enjoy the benefits of just a little javascript (-:

Tags: Hacks Lisp

September 24 2010


Visualizing the CLHS interlinking structure

Lately, I’ve been wondering what we could use the ~110k Hyperlinks in the CLHS for, other than click through the spec in a web browser. For example, given a glossary entry, how do you find out which functions refer to it?

So I wrote a little program that crawls (a local copy of) the HyperSpec, and creates an RDF graph of its link structure. This graph can be used to answer these questions, or it could be the basis of a useful research overlay on top of the HyperSpec, who knows.

Lots of tools exist to visualize and manipulate the facts in an RDF graph already; Here’s a screen shot of gruff with a small trace through the HyperSpec:

Gruff with Hyperspec (RDF)

You can get the code to create the RDF graph at github.

April 28 2010


I think this is going to be pretty sweet.


I’ve been toying around with making a Common Lisp adapter to Cucumber, a behavior-driven development tool. I think this will really be very sweet.

Here’s how the step definition file would look like (these are really just stubs; in reality, you’d put in the lisp code you want to happen for the given textual description):

(Given* #?"^I start clucumber in (.*)$" (path)
  (assert path))

(When* #?"^I define some-other-package as the test package$" ()

(Then* #?"^the current package should be \"([^\"]+)\"$" (package-name)
  (pending (format nil "package is ~A" package-name)))
Tags: Hacks Lisp
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!