Monday, May 20, 2013

Instantbird 1.4 Released!

After a bunch of l10n build problems, we've finally released Instantbird 1.4, which includes updates to libpurple 2.10.7 and Mozilla 20.  In particular this includes:
  • Updated Twitter code that uses v1.1 of their API (v1.0 will be disabled on June 11th, 2013).
  • Better character counter for Twitter (it now takes into account if URLs are embedded).
  • Updated log viewer which organizes logs by date (and nests them by week, month, etc.)
  • Better support for IRC bouncers.
  • Support for overriding self-signed/invalid/out-of-date certificates for IRC.
If you've been using some other instant messaging client (e.g. Pidgin or Adium); I'd highly suggest giving Instantbird a try, especially if you also go on IRC. Instantbird has great IRC support! (And...if you do have issues, feel free to ping me in #instantbird on irc.mozilla.org and let me know what your issue is.)

You can download it here, or view the full release notes.

Wednesday, November 28, 2012

JavaScript typed arrays pain

If you've ever tried to deal with binary data in JavaScript you know it isn't much fun and you usually resort to using strings lots of charCodeAt and related functions.  Typed arrays are supposed to solve this though!  The typed array API consists of creating a buffer of bytes (called an ArrayBuffer) and then manipulating those bytes via different views (ArrayBufferViews).  You can have multiple views of the same buffer, starting at different offsets, of different lengths and types...which is all neat from a technical point of view, but is it really useful?  It is kind of nice working with the views as if they were normal arrays though.

I've been playing with these ArrayBuffers quite a bit as I'm working on an implementation of the OSCAR protocol (used for AOL Instant Messenger and ICQ) in the chat backend (for Instantbird / Thunderbird).  (As an aside, the OSCAR protocol Wikipedia page has surprisingly good documentation of some of the underlying data structures of the protocol...)  I started by writing some test code using ArrayBuffers and views, which have been around a while: since Gecko 2.0 in fact!  I quickly ran into some tedious issues with repetitive code such as:

/*
 * A TLV (Type, Length and Value) data structure:
 *  Unsigned Short  type    Describes what the value represents.
 *  Unsigned Short  length  The length of the data block.
 *  Bytes           value   The raw payload.
 *
 * The overall length of a TlvBlock is length + 4.
 *
 * The inputs to this are:
 *  aType    The type of the TLV Block.
 *  aValue   An ArrayBuffer containing the data.
 */
function TlvBlock(aType, aValue) {
  let data = new ArrayBuffer(aValue.byteLength + 4);
  // The first two bytes are unsigned shorts.
  let view = new Uint16Array(data, 0, 2);
  view[0] = aType;
  view[1] = aValue.byteLength;
  
  // The rest just gets the data copied into it.
  view = new Uint8Array(data, 4);
  view.set(new Uint8Array(aValue));
  
  return data;
}

This actually illustrates two annoying issues I have:
  1. I end up with extra lines of code defining a new view every time I switch data types.
  2. There's no simple way to copy an ArrayBuffer into a part of an ArrayBuffer. In the above example I create a Uint8Array view of the target location, a Uint8Array view of the source location and then set the source to the target. Seems simple once you figure it out, but it took a while to figure out.
(As an aside, some of you might find the following function helpful, it is essentially a memcpy for ArrayBuffers...this isn't really tested heavily at all, however.)
/*
 * aTarget / aSource are ArrayBuffers
 */
function copyBytes(aTarget, aSource, aTargetOffset = 0, aSourceOffset = 0, aLength = aSource.byteLength) {
  // The rest just gets the data copied into it.
  let view = new Uint8Array(aTarget, aTargetOffset);
  view.set(new Uint8Array(aSource, aSourceOffset, aLength));
}

OK, so typed arrays seem good, but kind of annoying, right? Wrong...the OSCAR protocol is a "network order" protocol (aka it is big endian). At this point you're probably thinking "OK, so the ArrayBuffer constructor must take an endianess flag!"  Wrong, it does no such thing.  "Hmmm...Well do the ArrayBufferViews take an endianess flag?"  Nope, wrong again.  The only way to specify the endianess of the data is to use a DataView, a slightly different interface to the underlying bytes.  It offers an API to individually set different data elements via their offset and endianess.  (If you're too lazy to read the documentation all the way through, DataView assumes big endian: makes my life easier!)

For the curious, JavaScript typed arrays use the system endianess, which in my opinion is pretty much useless (at least if you plan on sharing data) since you can never guarantee the endianess to be either big or little endian.  (The fun part is that this isn't even documented, I found it on Stack Overflow and verified.)

So, in summary...if you plan on networking at all with ArrayBuffers, don't use ArrayBufferViews, use DataViews.  (Although Uint8Arrays and Int8Arrays should work fine!)

And to not rant the entire time, working with typed arrays certainly does beat strings + charCodeAt!

Friday, November 16, 2012

Instantbird 1.3 Released!!!

Well we finally got our release process down a bit better and were able to do a quicker release (from 1.2 to 1.3, compared to our previous few releases). This is great news, as it gives incremental changes to our users faster!  There's some new features available, which are mostly covered on the Instantbird blog, but quickly:
  • IRC now supports SASL authentication, which is required by Freenode when connecting from Tor or certain IP ranges.
  • Long messages over IRC are now smartly chopped and sent as multiple messages (instead of being truncated).
  • The "Show Nick" add-on was integrated: this allows styling of a mentioned nick in a conversation (and is extremely useful for following multiple conversations).
  • Various other minor improvements...
Instantbird 1.3 is based off of Mozilla 16.0.2, but I believe we're hoping to update to mozilla-central soon in order to benefit from cool new technologies, like WebRTC!  You should check out Instantbird now!

Tuesday, October 16, 2012

On Status

Something that comes up often about Instantbird is why we only support three statuses: Available, Unavailable and Offline.  (We do actually support a fourth one too, Idle, but that is set automatically, not chosen by the user.)  Frequently this discussion is in the context of wanting an "Invisible" status, but I'll get to that later...

Many users have talked to us on IRC, email or via bugs and complained about wanting an "Away" status or a "Do Not Disturb" status.  There's a few issues with this:
  1. What's really the difference between "Unavailable", "Away" and "Do Not Disturb"?  Do you really need to choose them individually?  (Other things that fit into here: "Not at my desk", "on the phone", "busy", "stepped out".  It is amazing how some protocols have so many ways to describe being unavailable!)
  2. A technical issue that we often run into is trying to shoehorn different protocol implementations into our abstract protocol interface.  (We already have some fairly complicated interfaces around joining chat rooms, creating different account, etc. because of this.)
  3. Setting yourself as "Away" or "Invisible" is a lie.  Perhaps this is me being overly idealistic, but why would you set yourself as "Away"?  It seems that this is something that should be done automatically (when you lock your display, perhaps?).  You can't be "Away" and using your computer at the same time!
    Again, perhaps being idealistic, but what is the point of the "Invisible" status?  If you wish to be hidden from someone (everyone?) why not just block those users.  Or ignore them when they send you a message.  If you are busy, set yourself to "Unavailable" and people should understand that they should not talk to you...if they don't, well...do you really want them talking to you ever?  (Are they really your friend?  I guess you don't get to choose your co-workers, but still.)  Now, perhaps this is just my opinion as being someone who never really hard an "Invisible" status (I actually remember it being added to the AIM client at some point).
At this point you probably don't believe me that there's really that many different protocol statuses out there, so I figured I'd illustrate a few protocols in a matrix.  Note that this isn't meant to be exhaustive, just to show how complicated of a situation this really is.  All protocols can obviously be "offline" as well, but that's not shown in the table.

Available Unavailable Phone Invisible
Oscar (AIM/ICQ) Online Away Invisible
IRC Online Away
Microsoft Lync Online Busy
Away
In a meeting
In a call
Yahoo! Available Busy
Stepped out
Be right back
Not at my desk
On the phone Invisible
Windows Live Messenger Online Busy
Away
Be Right Back
Out to lunch
On the phone Appear offline
XMPP (Google Talk) Available Busy Invisible

Wednesday, August 8, 2012

Instantbird 1.2 Released (with awesome new IRC features)!

If you haven't seen the announcement...Instantbird 1.2 has been released!  It's got a ton of great new features that I'm excited for: better tab complete, a marker showing the last viewed messages, support for Bonjour and more.  But the most exciting bits to me are our JavaScript implementations of XMPP (used for Facebook Chat and GTalk, so far) and IRC!

Why am I so excited for them? Mostly because they're extendable!  (Well...and I guess because I wrote most of the IRC code.) I've written a bit about this before for IRC...but it will let add-ons do whatever they want to the IRC protocol.  You should check out the implementations (links above), they're very hackable.  Hopefully we can remove libpurple XMPP and fully switch to Instantbird's XMPP for the next release, once a few Mozilla bugs are fixed.

Did I also mention that these implementations (including the raw XMPP and Twitter, which Instantbird has supported since 1.0) are going to be included in Thunderbird 15, as part of it's new chat feature?  Florian has done a great job of integrating our chat code there and it gives quite a different user experience than Instantbird, so don't be worried about Instantbird going away!

Now of course, we always think of the future here (after all, releasing itself isn't really exciting when most of the features have been in nightly builds...forever), so we started making a list of some of the stuff we'd like to implement in future Instantbirds, you can check it out here. Some of them are very exciting, feel free to grab one and work on it.

Monday, June 11, 2012

IRC Auto-Performs

There have been a few requests to support "auto-performs" (sending commands to the IRC server after connection that the user types into a box or whatever). Personally I find this to be:
  1. A fairly awful user experience.
  2. Confusing to new users.
  3. Unnecessary.
I additionally don't like this idea since it requires us to have commands for all the common tasks you'd want to do in an auto-perform (or support sending absolutely raw messages to the server, which we actually do already in the /quote command). Essentially what I just described is writing our own scripting language...that seems pointless (and frankly, I have better things to do). I'm hoping to convince you with this post (and maybe a series of posts) that auto-performs aren't necessary and a trivial restartless extension can replace them.

Design

Part of the desire to replace the libpurple IRC protocol plugin with a new JavaScript one built specifically for Instantbird (which is also now used in Thunderbird!) was to make the protocol fully extensible. There are many revisions and unofficial extensions to IRC and we might not necessarily want to support them all (especially if they only apply to a single network). Allowing all parts of the protocol implementation to be touched and extended seemed like a great way to handle this.

Initially I tried to do this by making the IRC account into an XPCOM component (well it is one already, it's an prplIAccount, but I meant an IRC specific one: implementing ircIAccount, if you will). Unfortunately, this seemed to have a lot of overhead and got complicated extremely quickly. Anything I'd want to touch from a message handler (wait, wait...what's a handler?! I'll get back to that) would need to have methods written and exposed to access internal data of the account...does that sound very extensible to you? Well, it doesn't to me...

Onto design two! (Well actually my first design...) Lots of JavaScript objects! The entire protocol is implemented as a set of JavaScript objects and the handlers directly touch and modify the account's data (of course there's methods for abstraction, etc.). This means that an extension has absolutely FULL access to every about an account...this also means an extension could seriously mess with and cause the protocol to stop working or do really crazy things, etc. Unfortunately there isn't really a way to avoid that. Hopefully people write good code.

Messages

I'm going to go into an aside about messages right now, even though it doesn't quite seem relevent yet. It will. IRC has a bunch of sub-protocols embedded within the IRC protocol (see the link above about unofficial extensions). We attempt to parse all the string messages and make pretty JavaScript objects out of them. I've actually identified five (yes, count that: five) different sub-"protocols" within IRC that we deal:
  1. IRC itself (i.e. RFC 1459 / RFC 2812 / various numeric extensions)
  2. CTCP (the Client-to-Client Protocol),embedded in PRIVMSG commands of IRC
  3. DCC (Direct Client-to-Client), a subprotocol of CTCP
  4. ISUPPORT (also known as Numeric 005), a method of negotiating capabilities between a client and server
  5. And finally, handling of IRC Services (there's a lot of them and no specification, but we treat them specially)
Briefly what happens when we receive a raw message over the wire, we create an ircMessage object out of it using a variety of regular expressions. This object has a variety of fields (see the link for details), including the command, who sent the message and the parameters.

If the message is identified as a CTCP message, we then morph the ircMessage into a CTCPMessage, which can be morphed into a DCCMessage. Additionally, a 005 reply can be parsed into a isupportMessage. And last, but not least, a received PRIVMSG can also be parsed into a ServiceMessage. Each of these extends the IRC message without destroying information. (Yes, I'm realizing now that my choice of whether to use capitals is all messed up...)

Well, why do we care...? By preparsing the strings into objects (as defined by any "specifications" that exist), we keep extensions from having to parse messages over and over again from strings.

Handlers

A handler is simply what I call the object that contains the methods to deal with an incoming message. Pretty much, you get to say "Only send me ISUPPORT messages!" or "Only send me CTCP messages!" and voila, you only get that type of message. Each message type has a field that is used to choose the method to run (for the IRC messages, the "command", for CTCP the "CTCP command", ISUPPORT the "parameter", etc.) This sounds a lot more complicated than it is, I think a brief example is in order:
var ircSimpleExample = {
  // The name here is really only used in error messages.
  name: "IRC Simple Example",
  // Slightly above the default priority so we run before the main IRC handler.
  priority: ircHandlers.DEFAULT_PRIORITY + 10,
  // Run this for all accounts (note that the 'this' object in this method is
  // the JavaScript account object.
  isEnabled: function() true,

  // The commands we want to handle. For each of these, the account object is
  // bound to 'this' and the single parameter is of the type that you've
  // registered your handle.
  commands: {
    "001": function(aMessage) {
      // At the 001 response we've successfully connected to the server.
      // Send an IDENTIFY command to NickServ.
      this.sendMessage("PRIVMSG", ["NickServ", "IDENTIFY <your password>"]);

      // Return false so the default handler still runs.
      return false;
    }
  }
}

Just like that we've designed a handler! Whenever the 001 method is received from the server, this function will run and attempt to identify with the NickServ (of course this could use a bit more security on it, but it's to demonstrate the possibilities). (The sendMessage function takes the command to send and an array of parameters to send.)

As this is already a long post, I think I'll cut this off now and continue this at another time, but I hope I'm beginning to convince you that allowing directy access to the account and protocol implementation is a more powerful (and even simpler in many ways, in my opinion) alternative to "auto-performs". The one major downside I see to this, is that it requires a bit more understanding of the actual protocol level implementation, I don't feel that knowing you need to use "PRIVMSG" as a command instead of /msg is a huge issue, however.

Sunday, January 15, 2012

Instantbird Contact List Hack #2

There was a request on the Instantbird Bugzilla to always show contacts in the contact list as the "big" contact (as shown when a contact is selected).  Similarly to my last post, this can easily be done with userChrome.css.  See the post if you don't know what userChrome.css is.

Again, we're simply going to always apply a specific CSS style to the contacts, namely we'll be modifying the behavior of blist.css.  I'm sure you don't really care about that and just want the code, well I'll oblige:

/* Expand all contacts to the big contact. */
contact {
  -moz-binding: url("chrome://instantbird/content/contact.xml#contact-big") !important;
  -moz-box-orient: vertical !important;
  -moz-box-align: stretch !important;
}
And that's it!  Restart Instantbird and you should always have big contacts.  I haven't seen any issues of using this (missing or wrong behavior), but of course your mileage might vary.  Have fun!