Posts Tagged ‘MacGDBp’
Today I spent some time debugging MacGDBp 1.4. The issue I was having was that socket data was being processed out of order, and I couldn’t figure out why. Until I stuck it in the debugger. It turns out that when you call
-[NSAttributedString initWithHTML:documentAttributes:], it spins the run loop internally (making a nested run loop). If, for example, you have a socket scheduled on the main run loop in common modes, the socket will signaled if you create the NSAttributedString on the main thread. Here’s a stack trace:
#0 0x0000495e in -[DebuggerConnection handlePacket:] at DebuggerConnection.m:597
#1 0x000044e8 in -[DebuggerConnection readStreamHasData] at DebuggerConnection.m:508
#2 0x00003243 in ReadStreamCallback at DebuggerConnection.m:76
#3 0x9021cdd3 in _signalEventSync
#4 0x9021d7be in _cfstream_solo_signalEventSync
#5 0x9021ca88 in _CFStreamSignalEvent
#6 0x9021d707 in CFReadStreamSignalEvent
#7 0x906e7cd9 in SocketStream::dispatchSignalFromSocketCallbackUnlocked
#8 0x90694819 in SocketStream::socketCallback
#9 0x90694721 in SocketStream::_SocketCallBack_stream
#10 0x901d91ae in __CFSocketDoCallback
#11 0x901d8c97 in __CFSocketPerformV0
#12 0x90192ff1 in __CFRunLoopDoSources0
#13 0x90190c1f in __CFRunLoopRun
#14 0x901900f4 in CFRunLoopRunSpecific
#15 0x9018ff21 in CFRunLoopRunInMode
#16 0x989ee6e8 in -[NSHTMLReader _loadUsingWebKit]
#17 0x989e2ddb in -[NSHTMLReader attributedString]
#18 0x98842585 in _NSReadAttributedStringFromURLOrData
#19 0x9883f910 in -[NSAttributedString(NSAttributedStringKitAdditions) initWithData:options:documentAttributes:error:]
#20 0x98887a35 in -[NSAttributedString(NSAttributedStringKitAdditions) initWithHTML:options:documentAttributes:]
#21 0x988879a9 in -[NSAttributedString(NSAttributedStringKitAdditions) initWithHTML:documentAttributes:]
#22 0x0000acf7 in -[BSSourceView setFile:] at BSSourceView.m:93
#23 0x0000af91 in -[BSSourceView setString:asFile:] at BSSourceView.m:120
#24 0x00007bae in -[DebuggerController updateSourceViewer] at DebuggerController.m:264
#25 0x000081e4 in -[DebuggerController sourceUpdated:] at DebuggerController.m:353
#26 0x00005a6b in -[DebuggerConnection setSource:] at DebuggerConnection.m:839
#27 0x00004ecf in -[DebuggerConnection handleResponse:] at DebuggerConnection.m:694
#28 0x000049cc in -[DebuggerConnection handlePacket:] at DebuggerConnection.m:600
#29 0x000044e8 in -[DebuggerConnection readStreamHasData] at DebuggerConnection.m:508
#30 0x00003243 in ReadStreamCallback at DebuggerConnection.m:76
#31 0x9021cdd3 in _signalEventSync
#32 0x9021cd58 in _cfstream_shared_signalEventSync
#33 0x9019315b in __CFRunLoopDoSources0
#34 0x90190c1f in __CFRunLoopRun
#35 0x901900f4 in CFRunLoopRunSpecific
#36 0x9018ff21 in CFRunLoopRunInMode
#37 0x944ed0fc in RunCurrentEventLoopInMode
#38 0x944eceb1 in ReceiveNextEventCommon
#39 0x944ecd36 in BlockUntilNextEventMatchingListInMode
#40 0x98605135 in _DPSNextEvent
#41 0x98604976 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
#42 0x985c6bef in -[NSApplication run]
#43 0x985bec85 in NSApplicationMain
#44 0x00002c44 in main at main.m:21
At frame 30, the socket callback gets signaled because data is ready. This happens on the outermost loop invocation, which is supposed to happen (yay!). At frame 22, the data for this most recent network packet is still being processed. At frame 21, there’s the call to initWithHTML for NSAttributedString. And at frame 15, trouble starts when the run loop gets spun again, while still processing sources from the first/outermost run loop invocation. At frame 2, while still processing the packet from frame 30, the socket source gets signaled again and new data is read and processed, while still processing the first piece of data. Sigh.
I haven’t yet decided how I’m going to get around this problem. The issue is that more data gets read from the socket before the current packet finishes handling, making a hot mess (it screws up internal state). The easiest conceptual solution is to push the socket stuff into its own thread on its own run loop, but that will require a fairly significant refactoring. Another option would be to schedule the socket in its own run loop mode, but then I’d be responsible for spinning the loop myself and would have to manage that carefully.
Update: I bit the bullet and refactored. In the long run, this is probably a good thing because it forced a separation of components that deal with CFNetwork itself and the response to the data received from it. It also split a 1032 line file into two files, one about 600 lines and one about 500 lines.
In between working on Bugdar 2, Phalanx, and Chromium, I’ve also been working on MacGDBp 1.4.
With the 1.3.0 release, I tried to address a couple of issues with the underlying network communication layer (called SocketWrapper, which was an Obj-C bridge to a BSD socket). The primary issue I tried to address was with Unicode characters; MacGDBp really only knows how to deal with ASCII text, which has proven problematic for international users. I tried to address this with the 1.3 branch by switching how data was stored and processed. But the 1.3.0 release was an utter failure from this perspective. It crashed extremely often, which lead to the 1.3.1 release that reverted the new network-layer changes. (That said, the new features introduced by 1.3 have been well-received.)
But the issues with the network layer ran far deeper than just dealing with Unicode text. MacGDBp uses synchronous communication with the debugger engine. When you issue a command (step in/out/over, get source, get properties, etc.), the command would be sent and then the thread would block until it received a response. This is bad, especially because MacGDBp does not use background threads; all the communication happens on the main UI thread. This can lead to beach balling and a bad user experience. So, why did I do this? Because synchronous communication makes for a dead-simple API. Asynchronous communication requires a lot more bookkeeping and works through callbacks.
So, for 1.4.0, I decided it was time to revisit the debugger backend. Rather than just trying to adjust how data is transmitted (ASCII vs. Unicode), I decided to rewrite from scratch the entire Xdebug communication layer. Rather than using raw BSD sockets, the 1.4 branch uses the CFNetwork API. This yields two huge benefits. The first is that all network activity is asynchronous because socket stream events are scheduled on the run loop. When data is available, a callback is executed and the UI is updated (as opposed to blocking the thread while waiting for a response). This also makes the application more robust, because if the response never comes, it will not lock up the entire UI. Secondly, by using a Foundation-level framework, UTF8 support comes pre-baked via the toll-free bridge to NSString.
I’ve been rewriting the back end in my spare time and it’s finally starting to stabilize. This release is still fairly far out, though. This core network change is also leading to a large refactoring of the entire application to make it more easily unit testable in the future. One of the results of this is creating a new LoggingController to record all network activity. The goal with this “hidden feature” is to make it easier to track down causes of bug reports. The log looks like this; it’s not pretty, but it should get the job done:
I haven’t yet selected the set of feature enhancements that are scheduled for this milestone. Before any new feature work can be started, the 1.4 trunk needs to become as stable as the 1.3 branch. If you have any feature requests, please search for or file new issues and vote them up in the bug tracker.
Current SHA1: a7725f8
Version: 18.104.22.168 ÃŸ
Last Release Build: 2010-03-29 17:51:39
So I know I promised at Thanksgiving an update on the state of things. Well I lied/got busy. Sorry; these things happen. The past few months have been a whirlwind. So here’s a brief personal update before I talk about software.
In May 2009 the Mac port of Chromium (the open source project that Google uses for their Chrome browser) was starting to gear up. I’d been interested in working on a Mac browser project for a while; but Firefox’s code seemed beastly and I never have been a fan of XUL. Safari is great, but it isn’t open source. Enter Chromium: an open-source, truly native, WebKit-based browser. Consider it love at first sight. Fast forward 12 patches and four months later to August when I am nominated and made a committer. My work has all involved bridging the cross-platform C++ model (as in MVC) to Objective-C/Cocoa land. I wrote the Mac the page info window, history menu, cookie manager, font and language settings, download settings, and added favicons throughout various parts of the UI. Now, present day. Last week Google offered me an internship position in Manhattan. I’m thrilled at the opportunity and am excited to start work there in May with such smart, interesting, and talented people. I expect to learn a lot!
So now let’s talk software.
Last time I wrote about the future of Bugdar 2, I posted a link to a list of phases development that would occur. The original plan was to create 1.3 as a stepping stone to 2.0. That line of development has since been abandoned. Instead, I decided to greenfield the project. There were many reasons for doing this; but here are the two big ones:
1. I have a better, more focused vision of what I want Bugdar 2 to be. When writing Bugdar 1, I didn’t have a clear goal of what I wanted — there was no philosophy to the design, I was merely thinking “copy Bugzilla, but do it better”. Overall, I think I was successful with this. But this is not Bugdar 2, at all. Bugdar 2 will be a refined, incredibly flexible system that I am excited to share with you. But because the two projects will be so different, trying to create a large, temporary scaffolding system (version 1.3) was extremely time consuming and kludgey.
2. ISSO. Originally dubbed the “Iris Studios Shared Objects” framework. This thing is ancient. The first version dates back to when I was a rather n00b PHPer. The first commit was from 12 January 2005! Since that time I’ve had a lot more development experience and completed a Minor in Computer Science. A better, more-efficient architecture could be devised.
I wrote an entirely new PHP framework (that will require PHP 5.3+) that is a bare-minimum, no-frills framework. I call this Phalanx and the code is currently available on Github; it is licensed under the GPL version 3. Phalanx is event-driven. Rather than writing another MVC framework, I decided to look at the problem of web frameworks through the lens of functional programming. The problem with MVC on the web is the lack of state; HTTP is not persistent. Functional programming focuses more on inputs and outputs (and the transformations between them), which is how Phalanx works. Each Event class defines a set of inputs that it accepts and the outputs it will return. This also makes unit testing much more clear-cut. I won’t get into further details here, though. That’s for another post.
So before I could start moving again on Bugdar, I had to write this new framework. That took a matter of months (July – January). But I’m now back on Bugdar. Phalanx makes development really fast, and in the past two days I’ve implemented the basic user system, a preliminary Auth2 API, and initial submission and listing of bug reports. I’ve posted a new roadmap for Bugdar 2.
I’ve been refactoring and cleaning up MacGDBp’s code over the past few months. It’s also been accumulating various fixes for non-critical bugs. I’d also like to re-tackle the socket/network layer for 1.4, as I tried and failed to improve this in 1.3. You can probably expect this update before Spring.
I’m also working on redesign the Blue Static site. It really hasn’t changed since 2006 (insert the ‘learned a lot’ bit here) and it’s time. I’m not very far on this, yet.
Wow that’s a lot. More in the next few days. UI mocks to come.
I’m pleased to announce MacGDBp 1.3. This release features a new tool that allows you inspect values in the variables list via a HUD (screenshot below). This inspector will allow you to see long strings and to select/copy text values. To access this feature, go to Window → Inspector (Shift+Cmd+I). The currently-selected variable (which now, in 1.3, remembers itself across debugger stepping) will have its full value displayed in the HUD window.
Under the hood, there are two significant changes. The first is a rewrite of the receiving side of the socket layer (it now precisely uses the message length when buffering memory), which should eliminate the “buffer is incomplete” errors that occasionally popped up. The second is that the PHP stack will no longer be managed internally. After every debugger step, the entire stack will be now re-created because Xdebug proved to be unpredictable for keeping in sync with an internal state. Debugger actions may now seem a little less snappy, but I think accuracy is more important than speed here. In 1.4 I’m going to use a caching mechanism that will speed up performance by reducing network access.
This release also features a bunch of bug fixes, a couple of which would cause the debugger display to not work under certain circumstances. As always, the change log and commit log have all the details on the release.
It’s been a while since I discussed pipelines, so I’m going to talk in this post about the state of various projects, as well as some talk on new projects!
Released this summer, MacGDBp has been a tremendous success. This week version 1.1.2 was released, which address two bugs, one of which was a crasher. I’ve got big plans for version 1.2 and 1.3, but these will have to wait for other projects to be completed.
In my last pipeline update, I mentioned the CMS I was building for my university’s TV station. This project launched only two weeks ago, so the past month and a half I’ve been working feverishly to get this done. It’s now complete and so this will no longer be affecting development.
The first milestone in Bugdar 2’s development phases was completed a few weeks ago. Since then, not much has happened because all of my effort went to deploying the aforementioned CMS. However, work will be resuming on phase 2 and development will be continuing. This week I went through the Blue Static bug tracker and triaged a bunch of bugs that had been left unresolved.
I’ve started working on this project in the past few days. As stated in the previous pipeline posts, this is a Usenet downloader for NZB files. However, I’m writing this as a cross-platform application. Most of the code is being written using C++ and Boost, which allows it to be platform-portable. I intend to then create native interfaces for various platforms (initially, only the Mac one will be created).
This project is mostly complete. A few interfaces need to be updated and created, but on the whole this is largely done. I haven’t actively worked on this in a few months, and I consider it to be one of my lower priorities.
I realized the other day that the Mac lacks a good, free ID3 tag editor. I quickly found id3lib and wrote a simple Mac interface for it. Version 1.0 will be released in a few days, but will be very minimal. I intend to add more features (and more tags to edit) in the future, but the necessary fields will all be present in v1.0.
This is the project I’m most excited about. It’ll be a unique take on launchers (e.g. QuickSilver, Butler, LaunchBar) that is for pro-users only. I won’t be going into the details until I have something to show, but this will be one awesome app!