Today I spent some time debugging MacGDBp 1.4. The issue I was having was that socket data was being processed out of order, and I couldn’t figure out why. Until I stuck it in the debugger. It turns out that when you call -[NSAttributedString initWithHTML:documentAttributes:], it spins the run loop internally (making a nested run loop). If, for example, you have a socket scheduled on the main run loop in common modes, the socket will signaled if you create the NSAttributedString on the main thread. Here’s a stack trace:
#0 0x0000495e in -[DebuggerConnection handlePacket:] at DebuggerConnection.m:597
#1 0x000044e8 in -[DebuggerConnection readStreamHasData] at DebuggerConnection.m:508
#2 0x00003243 in ReadStreamCallback at DebuggerConnection.m:76
#3 0x9021cdd3 in _signalEventSync
#4 0x9021d7be in _cfstream_solo_signalEventSync
#5 0x9021ca88 in _CFStreamSignalEvent
#6 0x9021d707 in CFReadStreamSignalEvent
#7 0x906e7cd9 in SocketStream::dispatchSignalFromSocketCallbackUnlocked
#8 0x90694819 in SocketStream::socketCallback
#9 0x90694721 in SocketStream::_SocketCallBack_stream
#10 0x901d91ae in __CFSocketDoCallback
#11 0x901d8c97 in __CFSocketPerformV0
#12 0x90192ff1 in __CFRunLoopDoSources0
#13 0x90190c1f in __CFRunLoopRun
#14 0x901900f4 in CFRunLoopRunSpecific
#15 0x9018ff21 in CFRunLoopRunInMode
#16 0x989ee6e8 in -[NSHTMLReader _loadUsingWebKit]
#17 0x989e2ddb in -[NSHTMLReader attributedString]
#18 0x98842585 in _NSReadAttributedStringFromURLOrData
#19 0x9883f910 in -[NSAttributedString(NSAttributedStringKitAdditions) initWithData:options:documentAttributes:error:]
#20 0x98887a35 in -[NSAttributedString(NSAttributedStringKitAdditions) initWithHTML:options:documentAttributes:]
#21 0x988879a9 in -[NSAttributedString(NSAttributedStringKitAdditions) initWithHTML:documentAttributes:]
#22 0x0000acf7 in -[BSSourceView setFile:] at BSSourceView.m:93
#23 0x0000af91 in -[BSSourceView setString:asFile:] at BSSourceView.m:120
#24 0x00007bae in -[DebuggerController updateSourceViewer] at DebuggerController.m:264
#25 0x000081e4 in -[DebuggerController sourceUpdated:] at DebuggerController.m:353
#26 0x00005a6b in -[DebuggerConnection setSource:] at DebuggerConnection.m:839
#27 0x00004ecf in -[DebuggerConnection handleResponse:] at DebuggerConnection.m:694
#28 0x000049cc in -[DebuggerConnection handlePacket:] at DebuggerConnection.m:600
#29 0x000044e8 in -[DebuggerConnection readStreamHasData] at DebuggerConnection.m:508
#30 0x00003243 in ReadStreamCallback at DebuggerConnection.m:76
#31 0x9021cdd3 in _signalEventSync
#32 0x9021cd58 in _cfstream_shared_signalEventSync
#33 0x9019315b in __CFRunLoopDoSources0
#34 0x90190c1f in __CFRunLoopRun
#35 0x901900f4 in CFRunLoopRunSpecific
#36 0x9018ff21 in CFRunLoopRunInMode
#37 0x944ed0fc in RunCurrentEventLoopInMode
#38 0x944eceb1 in ReceiveNextEventCommon
#39 0x944ecd36 in BlockUntilNextEventMatchingListInMode
#40 0x98605135 in _DPSNextEvent
#41 0x98604976 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
#42 0x985c6bef in -[NSApplication run]
#43 0x985bec85 in NSApplicationMain
#44 0x00002c44 in main at main.m:21
At frame 30, the socket callback gets signaled because data is ready. This happens on the outermost loop invocation, which is supposed to happen (yay!). At frame 22, the data for this most recent network packet is still being processed. At frame 21, there’s the call to initWithHTML for NSAttributedString. And at frame 15, trouble starts when the run loop gets spun again, while still processing sources from the first/outermost run loop invocation. At frame 2, while still processing the packet from frame 30, the socket source gets signaled again and new data is read and processed, while still processing the first piece of data. Sigh.
I haven’t yet decided how I’m going to get around this problem. The issue is that more data gets read from the socket before the current packet finishes handling, making a hot mess (it screws up internal state). The easiest conceptual solution is to push the socket stuff into its own thread on its own run loop, but that will require a fairly significant refactoring. Another option would be to schedule the socket in its own run loop mode, but then I’d be responsible for spinning the loop myself and would have to manage that carefully.
Update: I bit the bullet and refactored. In the long run, this is probably a good thing because it forced a separation of components that deal with CFNetwork itself and the response to the data received from it. It also split a 1032 line file into two files, one about 600 lines and one about 500 lines.
In a previous post, I stated that Bugdar2 is powered by a brand-new PHP framework called phalanx that I had written from scratch. In this post, I’m going to talk a little about what’s been going on with Bugdar2. The past month or so have been very busy, but I’ve managed to start writing unit tests and do some work on MacGDBp 1.4.
Phalanx is not a typical MVC framework. While it aims to keep that style of separation, it does so in a very different way than most web frameworks. Zend Framework, for instance, uses a Controller class that has various action methods. In phalanx, the controller unit is replaced by an Event object.
What is an event? An event is essentially a three-stage function with explicit input and output values. When defining an event, clients must explicitly list the input values the event requires and the output values it will produce. This is important because it makes unit testing extremely easy: if you know what should happen, it is not difficult to test it. The three stages an Event goes through are WillFire(), Fire(), and Cleanup(). The only required method is Fire(), which is where the event does the work of transforming the inputs to the outputs. WillFire() provides an opportunity to cancel an event if preconditions are not met, and Cleanup() is called regardless of if an event was successfully fired or not.
Here is an example. This is the event that creates new bugs. It takes in a title and the body of the first comment (description). On return, it provides the bug and comment IDs. From here, it’s easy to verify the results: create an input dictionary, create the event, and post it to the pump to run it.
In the code, references are made to the EventPump. This is the class that is responsible for firing events. The following is the description of control flow in a web app: In the index.php page, a HTTPDispatcher is created; this class parses HTTP requests and synthesizes the Event object for that request. The resulting event is then passed to the EventPump via PostEvent(). Since that is the first event in the pump, it begins processing immediately. If another event is sent to the pump via PostEvent() or RaiseEvent() [preempting the current event], it will be processed, too. Once all events finish processing, the OutputHandler is notified to take the list of events and figure out which one(s) it needs to display output from. The ViewOutputHandler simply takes the last successfully completed event, gathers the output from the event, and passes that information to the template. When unit testing, we can do away with the Dispatcher and OutputHandler. Just synthesize events, send them to the pump, and test the outputs.
Currently, work on Bugdar2 consists mostly of writing these unit test cases. Currently I’m blocked by engineering the API for bug attributes. Because attributes are at the heart of Bugdar2, I want to make sure I get this part right. Attributes and usergroups are the last two features before Alpha 1 is complete.
Due to a server misconfiguration issue, the site couldn’t connect to its database. As a result, downloads stopped working since yesterday’s rollout. This has now been fixed. Sorry for any inconvenience.
After about two months of work, I’ve finally completely redesigned the website. I think it sucks a lot less than the previous version, which is always a good thing. It’s written in HTML5 using CSS3. The best part is that I have no idea whether or not it works in Internet Explorer. My traffic is always less than 10% IE, so I’m not going to bend over backwards to support it. Anywhoo, check it out! You’ll also notice that the entire site is now being served over HTTPS.
In between working on Bugdar 2, Phalanx, and Chromium, I’ve also been working on MacGDBp 1.4.
With the 1.3.0 release, I tried to address a couple of issues with the underlying network communication layer (called SocketWrapper, which was an Obj-C bridge to a BSD socket). The primary issue I tried to address was with Unicode characters; MacGDBp really only knows how to deal with ASCII text, which has proven problematic for international users. I tried to address this with the 1.3 branch by switching how data was stored and processed. But the 1.3.0 release was an utter failure from this perspective. It crashed extremely often, which lead to the 1.3.1 release that reverted the new network-layer changes. (That said, the new features introduced by 1.3 have been well-received.)
But the issues with the network layer ran far deeper than just dealing with Unicode text. MacGDBp uses synchronous communication with the debugger engine. When you issue a command (step in/out/over, get source, get properties, etc.), the command would be sent and then the thread would block until it received a response. This is bad, especially because MacGDBp does not use background threads; all the communication happens on the main UI thread. This can lead to beach balling and a bad user experience. So, why did I do this? Because synchronous communication makes for a dead-simple API. Asynchronous communication requires a lot more bookkeeping and works through callbacks.
So, for 1.4.0, I decided it was time to revisit the debugger backend. Rather than just trying to adjust how data is transmitted (ASCII vs. Unicode), I decided to rewrite from scratch the entire Xdebug communication layer. Rather than using raw BSD sockets, the 1.4 branch uses the CFNetwork API. This yields two huge benefits. The first is that all network activity is asynchronous because socket stream events are scheduled on the run loop. When data is available, a callback is executed and the UI is updated (as opposed to blocking the thread while waiting for a response). This also makes the application more robust, because if the response never comes, it will not lock up the entire UI. Secondly, by using a Foundation-level framework, UTF8 support comes pre-baked via the toll-free bridge to NSString.
I’ve been rewriting the back end in my spare time and it’s finally starting to stabilize. This release is still fairly far out, though. This core network change is also leading to a large refactoring of the entire application to make it more easily unit testable in the future. One of the results of this is creating a new LoggingController to record all network activity. The goal with this “hidden feature” is to make it easier to track down causes of bug reports. The log looks like this; it’s not pretty, but it should get the job done:
I haven’t yet selected the set of feature enhancements that are scheduled for this milestone. Before any new feature work can be started, the 1.4 trunk needs to become as stable as the 1.3 branch. If you have any feature requests, please search for or file new issues and vote them up in the bug tracker.
Tracking Info:
Current SHA1: a7725f8
Version: 1.4.0.40 ß
Last Release Build: 2010-03-29 17:51:39
