Core Data in Motion - the Gems

At long last, Core Data in Motion, my book that helps RubyMotion developers get up to speed with Core Data in their RubyMotion projects, is complete.

So, what is RubyMotion? RubyMotion enables you to write cross-platform applications for iOS, Android, and OS X in Ruby. Instead of using the painfully verbose syntax of Objective-C, or taking a chance on the not-quite-ready-for-production Swift language, you can use the programming language that was designed to make programming fun to develop your iOS applications.

And what is Core Data? Core Data is to iOS and OS X development what ActiveRecord is to Rails. It solves all the hard problems involved in your data stack. To quote Drew Crawford

If you are saving more than 3 objects to disk without linking to CoreData.framework, you’re doing it wrong.

This final chapter of Core Data in Motion is an epic look at the available gems and CocoaPods that can simplify and streamline your Core Data code. It weighs in at more that double the size of any of the other chapters in the book.

My reviewers have given fantastic feedback on this chapter, and I know you will find it extremely helpful, when trying to figure out which gem or CocoaPod will be most useful to speed along the development of your application.

Each gem is demonstrated, as I walk you through the process of transforming the base Core Data API's used in the book's sample project, into the gem's specific API. I also discuss the problems and issues I discovered along the way, so you can see what happens, and avoid those problems in your project.

It's been a long time coming, but Core Data in Motion will definitely give you the boost into using Core Data that you need, and save you hours and days of investigation and frustration.

2014 Year in Review + 2015 Goals

Last year was just crazy. I was attempting to finish writing my book, work some contracts, speak at two conferences, teach my Rails workshop, mentor and teach at a couple of Ladies Learning Code/ChicGeekYYC events, and just for shiggles, my husband and I picked up and moved to a new (old) city - Lethbridge.

In retrospect, it isn't really surprising that I didn't actually finish primary writing on the book until Dec 31. Other than finishing the book, and planning to move out of Calgary, I didn't really have a master plan last year. And you know what they say about planning...

Failing to plan is planning to fail. - Alan Lakein

This year should be a little more sane. I'm laying out some goals, and trying to concentrate on building up my "products" business, finally making progress on my 30x500 learnings.

Before looking forward, however, I need to get an idea of where I currently stand. Taking the cue from some of my fellow 30x500 alumni, I decided I should post my product business results from last year. Got onto Stripe, downloaded my transaction history, and summed everything up:

WNDX Products Revenue

Ok, that % increase on the book is kind of lame, since I only sold one copy in Dec 2013, but the overall revenue numbers were still a nice uptick, year over year. I guess I’m still in that constant battle with impostor syndrome, and I didn’t keep track regularly, because I didn’t think I was doing that well, considering that I was still pretty much half-assing my way along last year.

Now I have the numbers on which to base my goals for the next year. Drum roll, please...

  1. Marketing the Core Data in Motion book
  2. Create new Advanced Rails Workshop
  3. Schedule/deliver Rails/Advanced Rails/Rails4iOS workshops quarterly
  4. Finish my damn iOS application - WIMBY
  5. Establish a Ladies Learning Code Lethbridge Chapter & and host at least one workshop

Marketing the now-finished (just waiting for comments from one more reviewer before pushing the final) book is pretty much a no-brainer.

Creating the new workshop is one of the "brick building" exercises we learn from 30x500, and is also a no-brainer, considering how many requests I have had for it.

Obviously, I can't make any money from workshops if I don't get them scheduled. Seems like I get busy, and then forget, and then it's a bad time of year or whatever. I mean... I scheduled a workshop on Grey Cup weekend one time (#facepalm). So, we'll get those lined up, with real dates, sooner rather than later this year.

The iOS application is a bit of a soft goal. I need to finish it, because it's been languishing at the 80% finished level for too long. It's not really planned as a revenue generator, directly, but it was the spur behind my book, so there's that. And then I can say I am a published iOS developer, so there's that, too. iOS and Android developers seem to have a LOT of pains, so that is a fruitful area for future sales safaris.

I also want to continue my volunteer work in conjunction with Ladies Learning Code, to improve the levels of diversity in software development. Lethbridge may be small, but it has both a university and a college, so I fully expect to be able to host workshops here quarterly, as I get the Lethbridge Chapter ramped up. (Contact me if you are interested in helping out!)

So there we have it. Last year I had a 300% increase in revenue. I don't think it's a stretch to go for 500% this year (not a goal, but a good measuring stick of progress), since I am buckling down with these definite goals. Instead of leaving my revenue checkup for the end of the year, this year I'll be doing that at least quarterly, so I can tell what sort of progress I am making.

If you'd like to keep in touch, you can always contact me directly, or you can sign up for my mailing list, and get advance notice about workshops and special discounts for my products not offered elsewhere.

Screw you, too, Rogers

I've been a Rogers customer since my first cell phone, which was a very long time ago. Sure, every once in a while, I get this twinge because their coverage sucks in rural areas (like where all my in-laws live), but I've put up with that, because when I am visiting family I mostly don't want to be interrupted by work anyway.

I've also experienced the super extreme rip off of international roaming. But Telus does the same, and that's an easily solved problem. I got a Roam Mobility SIM for my US travels ($20 SIM + $14.95/GB), and a nice little mifi device that I can load up with 3 GB of data for 30 euro when I am in the UK, the equivalent of 10 euro/GB. That's a little better than Rogers, where I'd be spending $400/GB with one of their "special" travel packs. Ouch. So ... same shit, different provider, whatever.

But the last straw came with my most recent Rogers bill. It seemed unusually high, so I double checked with my previous bill. And yup, it was almost $50 more. After digging into the details, I discovered that I was charged $48.49 for a data overage.

Now, I get that you have a limit, and you get charged to go over. What I don't get, is the EXTREME ripoff of those overage charges. Let's review this in detail shall we?

I get charged $35/6GB right now. This is the original iPhone data plan, that I have been using since my original iPhone. Since that first, locked, phone I got from Rogers, I've never "upgraded", and I've always gotten my own unlocked phone from Apple. So they aren't subsidising my phones, either. I'll even leave out the fact that I get ripped off on my iPad data plan, since I last upgraded and the new device was "inelligible" for data sharing on that 6GB plan, so I had to get a separate plan for it.

Since I am good at the maths, let me illustrate exactly WHY I feel ripped off. On that plan, I get $35/6GB or:

$5.83 / 1 GB

I was over my 6GB (6000MB) limit by 900 MB. That would make the $48.49/0.9GB equivalent to:

$53.87 / 1 GB

Yah. That. The fuckers are charging me almost 10 times the original rate for an overage. And that is why Rogers just lost me as a customer. Not because they charge extra for going over. I would understand that. But double or triple the rate. Not 10 fucking times the rate. That's just unconscionable price gouging.

So, screw you, too, Rogers. Next week, when I have the time, I'll be hitting up the Telus store and they'll have my $150/month account from now on.

Static Tables in Code

While there are a lot of great gems and tools to help us with creating great looking user interfaces in code in RubyMotion, sometimes I still like to explore how to do stuff at the iOS API level without using all the magic. Especially when just writing the code turns out to be pretty straightforward and elegant all on its own.

Creating a static table in code, to display some detail data in your application is one of these cases.

For my specific example, I have 6 items of information about a well to display, and this data logically groups into 3 sections of 2 items in each section. So, we'll set up the table view using sections like so:

class WellDetailsController < UITableViewController

  SECTIONS = %w(Name Status Location)

  def viewDidLoad
    super
    navigationItem.title = "Well Details"
  end

  def viewWillAppear(animated)
    navigationController.setNavigationBarHidden(false, animated:true)
  end

  def numberOfSectionsInTableView(tableView)
    SECTIONS.size
  end

  def tableView(tableView, numberOfRowsInSection:section)
    2
  end

  def tableView(tableView, titleForHeaderInSection:section)
    SECTIONS[section]
  end

With this code, I have set up my 3 sections (Name, Status, and Location) as a constant array. Then we implement the necessary methods of the table view (numberOfSectionsInTableView, tableView:numberOfRowsInSection, and tableView:titleForHeaderInSection) to deal with these sections.

Next up, we need to fill the actual table cells with data:

  CellID = self.class.name

  def tableView(tableView, cellForRowAtIndexPath:indexPath)
    cell = tableView.dequeueReusableCellWithIdentifier(CellID) || begin
      cell = UITableViewCell.alloc.initWithStyle(UITableViewCellStyleValue2,
                                                 reuseIdentifier:CellID)
    end
    cell.textLabel.text = @details[indexPath.section][indexPath.row][:label]
    cell.detailTextLabel.text = @details[indexPath.section][indexPath.row][:value]
    cell
  end

  def showDetailsForWell(well)
    @details = [
      [
        {label: 'UWI', value: well.uwi_display},
        {label: 'Well Name', value: well.well_name},
      ],
      [
        {label: 'Current', value: well.status},
        {label: 'Updated', value: well.status_date.strftime('%Y-%m-%d')},
      ],
      [
        {label: 'Latitude', value: well.latitude.stringValue},
        {label: 'Longitude', value: well.longitude.stringValue},
      ]
    ]
    tableView.reloadData
  end
end

Here, we set up the traditional tableView:cellForRowAtIndexPath. In it, we pull the data for the cell textLabel and detailTextLabel out of our @details data structure. This data structure is an array of arrays. More specifically, it is an array of sections, and each section array contains an array of rows. Each row is a hash, with a label/value pair.

The @details data structure itself gets populated with new values (the labels don't change) when a well is selected in another view (from a list of wells, or a map of wells). The showDetailsForWell gets called, and then this view is displayed.

That's all folks.

Certainly, it would be far simpler to code this up using Promotion, assuming you already knew and used Promotion. But it wasn't necessary to add another gem to my project for this. And I now understand exactly how sections and rows work in UITableViewController, so it wasn't a total loss ;-)

If you liked this post, you may find the prerelease of my Core Data in Motion book of interest. Certainly if you plan to use Core Data in your RubyMotion project it will save you many hours of head-scratching. Chapter 4 (Load Optimization) is now available. This will be the last chapter before the book goes up to full price, so get it now, and you will receive the last couple chapters (and maybe some bonus content) as they are completed, at a bargain price.

Core Data has selects

After my last post, there was an interesting side discussion on Twitter (with @macfanatic and @kastiglione) about some alternatives. It was interesting enough that I thought it would make good material for another post. So here ya go!

So, to revisit the last post Core Data in Motion - Chapter 2, we discussed the notion that Core Data doesn't really have the notion of a "select" clause.

What is the impact of this limitation?

If you have a large monolithic model, but just wanted a subset of the attributes for your current view, like, say a MAP, you'd probably want to do something similar to the ActiveRecord query:

     Well.select(:uwi, :well_name, :surface_latitude, :surface_longitude)

But you can't do that in Core Data. When you execute a Core Data fetch, you get the option of getting back object ids only or whole objects only.

Or can you?

That was the point under discussion on Twitter. There is actually another option, but it comes with its own limitations. So let's take a look at that other option, and see what it gives us, and what it takes away.

If you look at the SDK docs for NSFetchRequest, you will see a method called setPropertiesToFetch. The SDK description for this method is:

Specifies which properties should be returned by the fetch. The 
property descriptions may represent attributes, one-to-one
relationships, or expressions. The name of an attribute or
relationship description must match the name of a description 
on the fetch request’s entity.

Parameters:
    values (Array) — An array of NSPropertyDescription objects that 
    specify which properties should be returned by the fetch.

Returns:
    (Object)

Hey! So you can get your NSFetchRequest to return only this specified subset of properties! Isn't that just like a select?

Well, not quite. You see, when you run the NSFetchRequest with the propertiesToFetch specified, it will still ignore your propertiesToFetch, unless you have also specified a resultType of NSDictionaryResultType. This is detailed in the answer to this Stack Overflow question.

Then, when you run the NSFetchRequest with some propertiesToFetch and a resultType of NSDictionaryResultType you get back an array of NSDictionary items, NOT your NSManagedObjects. So your results won't have any of the methods you've declared on those NSManagedObjects, like, for instance, if you implemented the MKAnnotation interface so that you can drop those results on an MKMapView (which was my particular issue with this).

I will further note that the Stack Overflow post also indicates that the NSFetchedResultsController doesn't play nicely with NSDictionaryResultsType, which would limit it's usefulness with most standard UITableViewControllers implementations, too.

So... I can't use these results on my map, and it won't play nicely with my list view, either. I'm sure it's useful in some use cases, but its limitations made it useless for both of mine, so this is where NSFetchRequest.propertiesToFetch and I parted company.

Chapter 3 of Core Data in Motion (Preloading Data) is now available, and I've added the information from this post as an update to Chapter 2 as an extra bonus. As always, I welcome your feedback, especially when it helps me improve the book like this discussion has done.

Core Data in Motion - Chapter 2

So, Core Data and relationships. Lots of iOS apps are small, so why wouldn't you just smoosh all your data into one big model?

Performance. Memory. Speed.

With Core Data, you cannot simply retrieve only certain attributes from a object. For instance, in my WIMBY application, we have a rather huge monolithic well table, with about 20 attributes for each well. When we want to display these on a map, we only need 4 of those 20 attributes. A unique identifier, "label" for the pin, and the latitude and longitude values for map coordinates. To be memory/bandwidth efficient, we would only select the 4 attributes we actually need when retrieving the data from the database. In Rails ActiveRecord parlance, that would be something like:

Well.select(:uwi, :well_name, :surface_latitude, :surface_longitude)

In this Well.select statement, we have only 4 attributes out of our original 20. This saves us approximately 80% of the memory/bandwidth of a full record retrieval, which most programmers would think was a Good Thing™. The only problem here is, we can't do that in Core Data.

WAT?

Yup, in Core Data, fetches will return either object ID's only or whole objects only. There is no way to refine that fetch with a select clause, like we have in SQL or ORM tools like ActiveRecord.

In our particular case of the well table, Core Data will either fetch entire objects, chewing up 5x the amount of memory that we actually need, or fetch only object ID's, and then lazily fetch each object as it is accessed. In case you haven't run into THIS particular problem before, it's called the N+1 Query problem, and it will have a significant effect on the speed of your application. Your choice here is between MOAR MEMORY or MOAR SLOW.

So how do we make make our object fetches more performant in Core Data?

This is accomplished through decomposing our monolithic models into several distinct models with one-to-one relationships between each model (entity). Each model contains only the information necessary for the view in which it will be used. In the hot-off-the-presses Chapter 2 of Core Data in Motion, we examine, in detail, how to accomplish this.

Core Data with NSFetchedResultsController in RubyMotion

Today we'll be finishing off our series on Core Data in RubyMotion, discussing table view optimization of large amounts of data in your RubyMotion application. If you've missed the earlier posts, you can find them here:

Introduction to Core Data in Motion

Core Data Basics in RubyMotion

Core Data Relationships in RubyMotion

Core Data Pre-loading in RubyMotion

Core Data Load Optimization in RubyMotion

Once again, we turn to Ray Wenderlich for inspiration and instruction. His Core Data tutorial wraps up with a post on the usage of NSFetchedResultsController, so that is where we will end as well.

Why do we want to use NSFetchedResultsController, anyway? What's so special about it? When we started this series, with a relatively small sample dataset, it didn't really need much optimization. Now that we've loaded our database up with all 244,292 wells, it definitely needs some help, because I don't want my customers to wait minutes for the table view to load, which is what it does at this point.

So, we'll need to reduce memory overhead, and improve the response time of our table view, now that we have all that data. Ideally, in a table view, we would only load up the data that is actually visible to the user. And that is exactly what the utility class NSFetchedResultsController provides for us. So let's see how that is accomplished in RubyMotion.

First of all, we create an NSFetchedResultsController. Because this object requires access to the NSManagedObjectContext, which is in our store class, that's where we will put it.

  def fetched_results_controller
    fetch_request = NSFetchRequest.alloc.init
    fetch_request.entity = NSEntityDescription.entityForName('FailedBankInfo', inManagedObjectContext:@context)
    sort = NSSortDescriptor.alloc.initWithKey("details.close_date", ascending: false)
    fetch_request.sortDescriptors = [sort]
    fetch_request.fetchBatchSize = 20

    NSFetchedResultsController.alloc.initWithFetchRequest(fetch_request,
                                                managedObjectContext:@context,
                                               sectionNameKeyPath:nil,
                                               cacheName:"Root")
  end

The key to the construction of the NSFetchedResultsController is providing a base NSFetchRequest. This request needs to know which entity is being fetched, and also requires an NSSortDescriptor so it knows in what order to return the requested objects. The fetchBatchSize simply limits the number of objects returned on any single query to the database.

Now that we can create our NSFetchedResultsController, where do we put it? In this case, we will be creating it in our table view controller's viewDidLoad method.

  def viewDidLoad
    super
    error_ptr = Pointer.new(:object)
    @fetch_controller = FailedBankStore.shared.fetched_results_controller
    @fetch_controller.delegate = self
    unless @fetch_controller.performFetch(error_ptr)
      raise "Error when fetching banks: #{error_ptr[2].description}"
    end
  end

Here we create the NSFetchedResultsController, set it's delegate to be self, and trigger the initial fetch to populate the table view.

Next, we need to update the table view to get it's data from the NSFetchedResultsController.

  def tableView(tableView, numberOfRowsInSection:section)
    @fetch_controller.sections.objectAtIndex(section).numberOfObjects
  end

  def configureCell(cell, atIndexPath:index)
    bank = @fetch_controller.objectAtIndexPath(index)
    cell.textLabel.text = bank.name
    cell.detailTextLabel.text = "#{bank.city}, #{bank.state}"
    return cell
  end

  CellID = 'CellIdentifier'
  def tableView(tableView, cellForRowAtIndexPath:indexPath)
    cell = tableView.dequeueReusableCellWithIdentifier(CellID) || UITableViewCell.alloc.initWithStyle(UITableViewCellStyleSubtitle, reuseIdentifier:CellID)
    configureCell(cell, atIndexPath:indexPath)
  end

These methods translate over from Ray's tutorial pretty much intact, without much change, other than the "rubyization".

I did sort of skip a step back there, so let's not forget about that. In viewDidLoad we set the NSFetchedResultsController's delegate to be self. Now, we have to implement the NSFetchedResultsControllerDelegate's signature methods. Ray simply copied his implementation from an Apple sample. I've simply converted his code into a Ruby module.

module NSFetchedResultsControllerDelegate

  def controllerWillChangeContent(controller)
    self.tableView.beginUpdates
  end

  def controller(controller, didChangeObject:object, atIndexPath:path, forChangeType:type, newIndexPath:new_path)
    tableView = self.tableView
    case type
      when NSFetchedResultsChangeInsert
        tableView.insertRowsAtIndexPaths([new_path], withRowAnimation:UITableViewRowAnimationFade)
      when NSFetchedResultsChangeDelete
        tableView.deleteRowsAtIndexPaths([path], withRowAnimation:UITableViewRowAnimationFade)
      when NSFetchedResultsChangeUpdate
        configureCell(tableView.cellForRowAtIndexPath(path), atIndexPath:path)
      when NSFetchedResultsChangeMove
        tableView.deleteRowsAtIndexPaths([path], withRowAnimation:UITableViewRowAnimationFade)
        tableView.insertRowsAtIndexPaths([new_path], withRowAnimation:UITableViewRowAnimationFade)
    end
  end

  def controller(controller, sectionIndexTitleForSectionName:sectionName)
  end

  def controller(controller, didChangeSection:section, atIndex:index, forChangeType:type)
    case type
      when NSFetchedResultsChangeInsert
        self.tableView.insertSections( NSIndexSet.indexSetWithIndex(index), withRowAnimation:UITableViewRowAnimationFade)
      when NSFetchedResultsChangeDelete
        self.tableView.deleteSections( NSIndexSet.indexSetWithIndex(index), withRowAnimation:UITableViewRowAnimationFade)
    end
  end

  def controllerDidChangeContent(controller)
    self.tableView.endUpdates
  end
end

And then we include the module in our table view controller:

class FailedBankTableViewController < UITableViewController

  include NSFetchedResultsControllerDelegate

It looks like a lot of code, but since it's doubtful you will need to change it much, you should just be able to reuse this module when required.

And that, as they say, is that. We now have a working implementation of NSFetchedResultsController, and the data will only be loaded 20 objects at a time. This speeds things up immensely, and reduces memory usage in our app from get-killed-immediately to just fine ;-) The complete example can be downloaded, compiled, loaded with data and run. Alas, I am unable to provide the "large data load" that I used, as that data is not mine to give away. I encourage you to come up with your own large data set, and plug it in, and see how it works.

If you found this series interesting, you might want to take a look at my Core Data in Motion ebook. It is now available in pre-sales, with a sample chapter.

Core Data Pre-Loading in RubyMotion

<p>This week, we are delving into loading up that data store with a pre-populated set of data.  Once again, Ray Wenderlich's Core Data tutorial offers us a great starting point. <a href="http://www.raywenderlich.com/12170/core-data-tutorial-how-to-preloadimport-existing-data-updated">Core Data on iOS 5 Tutorial: How To Preload and Import Existing Data</a></p>

Read More…

And the survey says...

Thanks to everyone who participated in my Ruby Workshop survey. Although I didn't get a tonne of responses, the ones I did get formed some distinct patterns, so I got what I needed.

In general, you prefer weekends or don't care, so weekend it is.

You distinctly prefer November, with a secondary preference of January, so we'll schedule it that way.

No preferences emerged for location, so I'll just see what's available/comfortable/cheap. ;-) (One write in for Dubai/Muscat… sorry, but probably not)

You really don't like "Mostly lecture", and lean strongly toward a mix of lecture and interactive work-along (which is what we've always done, good to know).

And last but not least, you want the straight up Ruby on Rails for Real workshop most of all, followed by the Rails for iOS Developers, so that will be what we schedule for November and January. A reasonable number of people are wanting the Intro to Ruby as well, and I am certain that Ladies Learning Code will be running that workshop again soon as well, so keep an eye out for that. Oh, and I had one "write in" for RubyMotion. That's on my radar for next year.

Thanks again. I'll post a schedule as soon as I've firmed up the dates. If you are not already on my mailing list, please do signup, and you'll be the first to hear as well as get special, list only, discount codes for workshops.