Archive for the 'Bugdar' Category
It’s been a while since I last wrote about Bugdar2 development, so I thought I’d post an update.
Over this summer, I managed to get a lot of work done for Bugdar2 and started working through the roadmap to the Alpha 1 milestone. The purpose of the Alpha 1 milestone was to do as much of the core backend as possible, without writing much templated UI. While I did end up writing some skeleton templates to test basic functionality, most of the backend work was tested through unit tests.
When I got to the point where I couldn’t do much more backend work without having interfaces to add and modify data, Alpha 1 was complete. The goal of the next milestone Alpha 2, which I’m currently working, is to get the basic UI shell up. I’ve started building the CRUD templates for entities like bugs, settings, usergroups, and attributes. In this milestone, the initial look and feel will come together and a round of string extraction (the first step towards localization) will occur.
Here’s two screen shots of Bugdar2 as it currently exists. These are very rudimentary and will evolve as more time is put into them.
In a previous post, I stated that Bugdar2 is powered by a brand-new PHP framework called phalanx that I had written from scratch. In this post, I’m going to talk a little about what’s been going on with Bugdar2. The past month or so have been very busy, but I’ve managed to start writing unit tests and do some work on MacGDBp 1.4.
Phalanx is not a typical MVC framework. While it aims to keep that style of separation, it does so in a very different way than most web frameworks. Zend Framework, for instance, uses a Controller class that has various action methods. In phalanx, the controller unit is replaced by an Event object.
What is an event? An event is essentially a three-stage function with explicit input and output values. When defining an event, clients must explicitly list the input values the event requires and the output values it will produce. This is important because it makes unit testing extremely easy: if you know what should happen, it is not difficult to test it. The three stages an Event goes through are WillFire(), Fire(), and Cleanup(). The only required method is Fire(), which is where the event does the work of transforming the inputs to the outputs. WillFire() provides an opportunity to cancel an event if preconditions are not met, and Cleanup() is called regardless of if an event was successfully fired or not.
Here is an example. This is the event that creates new bugs. It takes in a title and the body of the first comment (description). On return, it provides the bug and comment IDs. From here, it’s easy to verify the results: create an input dictionary, create the event, and post it to the pump to run it.
In the code, references are made to the EventPump. This is the class that is responsible for firing events. The following is the description of control flow in a web app: In the index.php page, a HTTPDispatcher is created; this class parses HTTP requests and synthesizes the Event object for that request. The resulting event is then passed to the EventPump via PostEvent(). Since that is the first event in the pump, it begins processing immediately. If another event is sent to the pump via PostEvent() or RaiseEvent() [preempting the current event], it will be processed, too. Once all events finish processing, the OutputHandler is notified to take the list of events and figure out which one(s) it needs to display output from. The ViewOutputHandler simply takes the last successfully completed event, gathers the output from the event, and passes that information to the template. When unit testing, we can do away with the Dispatcher and OutputHandler. Just synthesize events, send them to the pump, and test the outputs.
Currently, work on Bugdar2 consists mostly of writing these unit test cases. Currently I’m blocked by engineering the API for bug attributes. Because attributes are at the heart of Bugdar2, I want to make sure I get this part right. Attributes and usergroups are the last two features before Alpha 1 is complete.
Bugdar 2 is now sufficiently far along that I’m comfortable posting the git repository. The Phalanx repository has been public from the beginning (albeit unnanounced) at Github: http://github.com/rsesek/phalanx. Bugdar 2’s repository is now also available there: http://github.com/rsesek/Bugdar2. At the same time, they will be availiable at their canonical home in http://bluestatic.org/git/. The Github mirrors are just easier for most people.
Development of Bugdar 2 is still moving fast. In the past few days I’ve laid down the core of the search system and added support for resubmitting a POST request after login (don’t lose your work if you’re not logged in). There’s two more major milestones before I consider us past the ‘first alpha’ stage: usergroup permissions and bug attributes. What are bug attributes?
In Bugdar 1, there were a set of fields that Bugdar compelled you to use: product, version, status, resolution, priority, severity, and assignment. If you didn’t want to use one of these fields, you couldn’t remove it — you just had to ignore it. There were also custom fields, but the architecture under them was always shaky. By having all these required fields, we were defining a bug as so:
CREATE TABLE bug (
bugid int unsigned NOT NULL AUTO_INCREMENT
userid int unsigned NOT NULL
dateline bigint unsigned NOT NULL
product int unsigned NOT NULL
component int unsigned NOT NULL
version int unsigned NOT NULL
summary varchar(255) NOT NULL
priority int unsigned NOT NULL
severity int unsigned NOT NULL
status int unsigned NOT NULL
resolution int unsigned NOT NULL
assignedto int unsigned NOT NULL
duplicateof int unsigned NOT NULL
dependency text NOT NULL
hidden smallint unsigned NOT NULL
initialreport int unsigned NOT NULL
-- [A lot of other cache information here]
That’s a lot of data! If a bug isn’t going to use one of those fields, it’s still going to carry around all that weight. In Bugdar 2, this is how we define a bug:
CREATE TABLE bugs (
bug_id int NOT NULL PRIMARY KEY AUTO_INCREMENT,
title varchar(250) NOT NULL DEFAULT '',
reporting_user_id int NOT NULL DEFAULT 0,
reporting_date int NOT NULL DEFAULT 0,
hidden boolean NOT NULL DEFAULT 0,
first_comment_id int NOT NULL DEFAULT 0
That’s a huge difference. You’ll notice all the attributes — the metadata — are not stored on the bug itself anymore. Instead, that’s done through a tuple:
CREATE TABLE bug_attributes (
bug_id int NOT NULL DEFAULT 0,
attribute_title varchar(50) NOT NULL DEFAULT '',
PRIMARY KEY (bug_id, attribute_title)
The implications of this are that an single (unique), arbitrary attribute can be attached to a bug at the user’s will. The requirement of data is now imposed by the administrator rather than the developer. It also helps to normalize the database schema. So how do attributes work?
Attributes can be defined formally in an administrative panel, or they can be defined ad-hoc while editing bugs. Attributes can be titled to create a field, or they can just be values, which will be represented as tags. The goal with this system is to create a highly flexible data model in which all the stock Bugdar 1 fields could be represented as attributes.
Formally defined attributes can be one of five types: text, boolean, list, date, or user. Text fields are self explanatory. Booleans are interesting because they don’t necessarily have binary value; they have ternary value: true, false, and unset — but that’s more of a semantic detail. Lists are a set of selectable, pre-defined options. The last two types are new since Bugdar 1. Date fields will have a calendar selection widget. Finally, user fields allow the bug to be attached to some user in Bugdar’s system; this is how assignments work.
This is a rendering of how attributes will be arranged on the bug display:
That same information takes up more than half the page in Bugdar 1. That’s all for now.
So I know I promised at Thanksgiving an update on the state of things. Well I lied/got busy. Sorry; these things happen. The past few months have been a whirlwind. So here’s a brief personal update before I talk about software.
In May 2009 the Mac port of Chromium (the open source project that Google uses for their Chrome browser) was starting to gear up. I’d been interested in working on a Mac browser project for a while; but Firefox’s code seemed beastly and I never have been a fan of XUL. Safari is great, but it isn’t open source. Enter Chromium: an open-source, truly native, WebKit-based browser. Consider it love at first sight. Fast forward 12 patches and four months later to August when I am nominated and made a committer. My work has all involved bridging the cross-platform C++ model (as in MVC) to Objective-C/Cocoa land. I wrote the Mac the page info window, history menu, cookie manager, font and language settings, download settings, and added favicons throughout various parts of the UI. Now, present day. Last week Google offered me an internship position in Manhattan. I’m thrilled at the opportunity and am excited to start work there in May with such smart, interesting, and talented people. I expect to learn a lot!
So now let’s talk software.
Last time I wrote about the future of Bugdar 2, I posted a link to a list of phases development that would occur. The original plan was to create 1.3 as a stepping stone to 2.0. That line of development has since been abandoned. Instead, I decided to greenfield the project. There were many reasons for doing this; but here are the two big ones:
1. I have a better, more focused vision of what I want Bugdar 2 to be. When writing Bugdar 1, I didn’t have a clear goal of what I wanted — there was no philosophy to the design, I was merely thinking “copy Bugzilla, but do it better”. Overall, I think I was successful with this. But this is not Bugdar 2, at all. Bugdar 2 will be a refined, incredibly flexible system that I am excited to share with you. But because the two projects will be so different, trying to create a large, temporary scaffolding system (version 1.3) was extremely time consuming and kludgey.
2. ISSO. Originally dubbed the “Iris Studios Shared Objects” framework. This thing is ancient. The first version dates back to when I was a rather n00b PHPer. The first commit was from 12 January 2005! Since that time I’ve had a lot more development experience and completed a Minor in Computer Science. A better, more-efficient architecture could be devised.
I wrote an entirely new PHP framework (that will require PHP 5.3+) that is a bare-minimum, no-frills framework. I call this Phalanx and the code is currently available on Github; it is licensed under the GPL version 3. Phalanx is event-driven. Rather than writing another MVC framework, I decided to look at the problem of web frameworks through the lens of functional programming. The problem with MVC on the web is the lack of state; HTTP is not persistent. Functional programming focuses more on inputs and outputs (and the transformations between them), which is how Phalanx works. Each Event class defines a set of inputs that it accepts and the outputs it will return. This also makes unit testing much more clear-cut. I won’t get into further details here, though. That’s for another post.
So before I could start moving again on Bugdar, I had to write this new framework. That took a matter of months (July – January). But I’m now back on Bugdar. Phalanx makes development really fast, and in the past two days I’ve implemented the basic user system, a preliminary Auth2 API, and initial submission and listing of bug reports. I’ve posted a new roadmap for Bugdar 2.
I’ve been refactoring and cleaning up MacGDBp’s code over the past few months. It’s also been accumulating various fixes for non-critical bugs. I’d also like to re-tackle the socket/network layer for 1.4, as I tried and failed to improve this in 1.3. You can probably expect this update before Spring.
I’m also working on redesign the Blue Static site. It really hasn’t changed since 2006 (insert the ‘learned a lot’ bit here) and it’s time. I’m not very far on this, yet.
Wow that’s a lot. More in the next few days. UI mocks to come.
Happy Thanksgiving to all you US citizens!
Today I’m releasing a security update to Bugdar, version 1.2.4. This version addresses a possible SQL injection vector in search.php. The details of the report can be found here. Thanks to Adam DiCarlo for disclosing the issue.
Please update to 1.2.4 as soon as you can. The release links can found on the normal Bugdar webpage. Alternatively, here are the direct ZIP and Tar links.
I apologize for the lack of posts, but I’d rather be coding rather than talking about coding ;). I’ll post an update on the state of Bugdar 2.0 in a few days.