Common SQLite Operations

In nearly any program with persistent data that can be manipulated, I find myself writing three types of routines: insert, update, and delete. In the example below, I focus on using SQLite to perform these operations on some generic data.

To create a simple table for demonstration purposes, we can use the command-line ‘sqlite3’ utility.  In one of my previous posts, I discuss how to automate the (re)generation of your database, but this example is contrived enough that it doesn’t warrant such attention.  In the shell, use the following commands:

   $> sqlite3 db.sqlite
   sqlite> CREATE TABLE myObject (primaryKey INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT, amount NUMERIC);
   sqlite> .quit
   $>

Getting this new database into your XCode project, and adding the SQLite3 framework would be next, but I’m going to skip over it, as it’s covered elsewhere and not difficult to put together.

Here’s a simple Objective-C object as the data model for our newly created table.

MyObject.h

@interface MyObject : NSObject
{
    NSInteger primaryKey;
    NSString *name;
    float amount;
}

@property (nonatomic, assign, readonly) NSInteger primaryKey;
@property (nonatomic, retain) NSString *name;
@property (nonatomic, assign) float amount;

// Returns: new MyObject with primary key
- (id)initWithPrimaryKey:(NSInteger)pk database:(sqlite3*)db;

// Updates the database with the object values
- (void)updateDatabase:(sqlite3*)db;

// Delete an object with primary key 'pk'
+ (void)removeWithPrimaryKey:(NSInteger)pk database:(sqlite3*)db;

// Returns: primary key of newly inserted object
+ (NSInteger)addMyObjectIntoDatabase:(sqlite3*)db;

// Clean up, called on app termination
+ (void)finalizeStatements;

@end

The class defines “insert” using addMyObjectIntoDatabase:, “update” using updateDatabase:, and “delete” with removeWithPrimaryKey:database:

The class also has a initialization member presumably called by the controller that has better visibility to all of the table elements.  I’ll fill in the method bodies, but leave the actual hook-up to higher level classes as an “exercise for the reader”.

In the pattern above, generally the controller class will issue a “SELECT primaryKey FROM myObject” statement, then iterate through the returned result.  With each row, the controller factory method creates a new MyObject using the row’s primary key.  But, there are a number of valid approaches depending on your implementation requirements.

Here’s the implementation body of the MyObject class.  I took the design choice of preparing the various queries and storing the result as a class static pointer.  The advantage is performance since the queries are parameterized and “pre-compiled” for SQLite.  The disadvantage is that they consume resources that must be released at some point – in our case using the finalizeStatements method.

MyObject.m

static sqlite3_stmt *init_MyObj_statement = nil;
static sqlite3_stmt *update_MyObj_statement = nil;
static sqlite3_stmt *delete_MyObj_statement = nil;
static sqlite3_stmt *insert_MyObj_statement = nil;

@implementation MyObject

@synthesize primaryKey;
@synthesize name;
@synthesize amount;

- (id)initWithPrimaryKey:(NSInteger)pk database:(sqlite3*)db
{
   if (self = [super init])
   {
      primaryKey = pk;
      if (init_MyObj_statement == nil)
      {
         const char *sql = "SELECT * FROM myObject WHERE primaryKey=?";
         int result = sqlite3_prepare_v2(db, sql, -1, &init_MyObj_statement, NULL);
         NSAssert1(result == SQLITE_OK, @"initWithPrimaryKey: failed to prepare statement with err '%s'", sqlite3_errmsg(db));
      }

      sqlite3_bind_int(init_MyObj_statement, 1, primaryKey);

      if (sqlite3_step(init_MyObj_statement) == SQLITE_ROW)
      {
         int columnID = 1; // 0 is primary key
         char *nameStr = (char*)sqlite3_column_text(init_MyObj_statement, columnID++);
         name = (nameStr) ? [[NSString stringWithUTF8String:nameStr] retain] : @"";

         amount = sqlite3_column_double(init_MyObj_statement, columnID);
      }

      sqlite3_reset(init_MyObj_statement);
   }

   return self;
}

- (void)updateDatabase:(sqlite3*)db
{
   if (update_MyObj_statement == nil)
   {
      const char *sql = "UPDATE myObject SET name=?, amount=? WHERE primaryKey=?";
      int success = sqlite3_prepare_v2(db, sql, -1, &update_MyObj_statement, NULL);
      NSAssert1(success == SQLITE_OK, @"updateDatabase: failed to prepare with message '%s'", sqlite3_errmsg(db));
   }

   int columnID = 1;

   sqlite3_bind_text(update_MyObj_statement, columnID++, [name UTF8String], -1, SQLITE_TRANSIENT);
   sqlite3_bind_double(update_MyObj_statement, columnID++, amount);
   sqlite3_bind_int(update_MyObj_statement, columnID++, primaryKey);

   int success = sqlite3_step(update_MyObj_statement);
   NSAssert1(success == SQLITE_DONE, @"updateDatabase: update failed with message '%s'", sqlite3_errmsg(db));

   sqlite3_reset(update_MyObj_statement);
}

+ (void)removeWithPrimaryKey:(NSInteger)pk database:(sqlite3*)db
{
   if (delete_MyObj_statement == nil)
   {
      static char *sql = "DELETE FROM myObject WHERE primaryKey=?";
      int result = sqlite3_prepare_v2(db, sql, -1, &delete_MyObj_statement, NULL);
      NSAssert1(result == SQLITE_OK, @"removeWithPrimaryKey: failed with message '%s'", sqlite3_errmsg(db));
   }

   sqlite3_bind_int(delete_MyObj_statement, 1, pk);
   int success = sqlite3_step(delete_MyObj_statement);
   NSAssert1(success == SQLITE_DONE, @"removeWithPrimaryKey: failed with message '%s'", sqlite3_errmsg(db));
   sqlite3_reset(delete_MyObj_statement);
}

+ (NSInteger)addMyObjectIntoDatabase:(sqlite3*)db
{
   if (insert_MyObj_statement == nil)
   {
      static char *sql = "INSERT INTO myObject (name) VALUES ('')";
      int result = sqlite3_prepare_v2(db, sql, -1, &insert_MyObj_statement, NULL);
      NSAssert1(result == SQLITE_OK, @"addMyObjectIntoDatabase: failed to prepare statement with err '%s'", sqlite3_errmsg(db));
   }

   int success = sqlite3_step(insert_MyObj_statement);
   if (success != SQLITE_ERROR)
   {
      return sqlite3_last_insert_rowid(db);
   }

   NSAssert1(0, @"addMyObjectIntoDatabase: failed with message '%s'", sqlite3_errmsg(db));
   return -1;
}

+ (void)finalizeStatements
{
   if (init_MyObj_statement)
      sqlite3_finalize(init_MyObj_statement);
   if (update_MyObj_statement)
      sqlite3_finalize(update_MyObj_statement);
   if (delete_MyObj_statement)
      sqlite3_finalize(delete_MyObj_statement);
   if (insert_MyObj_statement)
      sqlite3_finalize(insert_MyObj_statement);
}

@end

App Analytics

One of the more daunting aspects of app development is the lack of direct contact with your customers.  With Apple serving as an intermediary, it can be difficult to know what folks like (or dislike!) about your product.  While we get a fair share of constructive feedback about our products, people occasionally post to iTunes or elsewhere online which isn’t regularly monitored.  Thus, it’s possible that we miss a really good idea or could prioritize features better based on usage.

Some make a claim for analytics as a developer tool to combat piracy but that seems less clear to me.  Knowing – even expecting – that a popular app will attract the attention of pirates should be something most developers understand and appreciate.  It’s not like the problem is unique to the App Store, nor is there a unique iPhone solution to combat it.  Piracy exists for all major computer platforms including consoles.  I’d like to think that the majority of iPhone users are honest and would rather pay a nominal amount for an app rather than jump through the required hoops (possibly making their phone less secure) to obtain something illegally.

I’ve been playing around with Pinch Media and their analytics package.  I put together a quick test app to poke around with the API and see how it performed.  It was pretty cool.  The basic process of getting it into an app was easy:

  1. Get the library.
  2. Copy the lib and header into your project.
  3. Include three frameworks: libsqlite, SystemConfiguration and CoreLocation.
  4. Start the data capture on startup.
  5. Stop the data capture on terminate.

Rather than call directly into the Pinch API, I wrote a simple singleton to wrap the interface.  I didn’t want events being triggered during development in the simulator, or in the event we want to add an app option/preference to “opt in”.

The implementation file is a bit “loose” – especially the handling of the static singleton which I’d probably cleanup if attempting to productize this code.  However, for experimentation purposes, the wrapper was a snap and looks something like this:

Analytics.h


#import <Foundation/Foundation.h>
@class CLLocation;

@interface Analytics : NSObject
{
}

+ (id)initAnalytics:(NSString*)appCode useCoreLocation:(BOOL)coreLocation useOnlyWiFi:(BOOL)wiFi;
+ (void)endAnalytics;
+ (id)shared;
- (void)startSubWithName:(NSString*)name timeSession:(BOOL)timed;
- (void)endSubWithName:(NSString*)name;
- (void)setLocation:(CLLocation*)location;

@end

Analytics.m


#import "Analytics.h"
#import "Beacon.h"

#if (!TARGET_IPHONE_SIMULATOR)
static Analytics *obj_Analytics = nil;
#endif

@implementation Analytics

#if (!TARGET_IPHONE_SIMULATOR)

- (id)init
{
   self = [super init];
   obj_Analytics = self;
   return obj_Analytics;
}

+ (id)initAnalytics:(NSString*)appCode useCoreLocation:(BOOL)coreLocation useOnlyWiFi:(BOOL)wiFi
{
   return [Beacon initAndStartBeaconWithApplicationCode:appCode useCoreLocation:coreLocation useOnlyWiFi:wiFi];
}

+ (void)endAnalytics
{
   [obj_Analytics release];
   obj_Analytics = nil;

   [[Beacon shared] endBeacon];
}

+ (id)shared
{
   if (obj_Analytics == nil)
   {
      obj_Analytics = [[self alloc] init];
   }

   return obj_Analytics;
}

- (void)startSubWithName:(NSString*)name timeSession:(BOOL)timed
{
   [[Beacon shared] startSubBeaconWithName:name timeSession:timed];
}

- (void)endSubWithName:(NSString*)name
{
   [[Beacon shared] endSubBeaconWithName:name];
}

- (void)setLocation:(CLLocation*)location
{
   [[Beacon shared] setLocation:location];
}

#else

+ (id)initAnalytics:(NSString*)appCode useCoreLocation:(BOOL)coreLocation useOnlyWiFi:(BOOL)wiFi { return nil; }
+ (void)endAnalytics {}
+ (id)shared { return nil;}
- (void)startSubWithName:(NSString*)name timeSession:(BOOL)timed {}
- (void)endSubWithName:(NSString*)name {}
- (void)setLocation:(CLLocation*)location {}

#endif

@end

After hooking in the startup/shutdown code in your app delegate, the idea is you scatter “markers” throughout your app to track or even time various events using the startSubWithName:timeSession method.

To use the API, you’ll need to register with the Pinch Media site to obtain a unique app-id that will aggregate various statistics on their server.  The library collects a number of stats by default including:

  1. OS version
  2. Device type
  3. Demographic information (if known)
  4. Regional usage (if using CoreLocation)
  5. Lifecycle data (app version, time used, number of actions, etc.)

And – all the various actions you scattered too.  The default charts on the web site are good but basic.  You can export the data to CSV or access it online via a nifty API.

While running a test over a few minutes, I noticed the library (r69) leaked a small amount of memory during the asynchronous send.  During the transfer, I didn’t detect a noticeable performance impact but then, the test app wasn’t exactly doing real time calculations either.  Instruments reported about 1 kb in the library thread.  The library appears to periodically send data perhaps after a certain time period or once the event queue fills.    The data is also sent when the app terminates – which apparently can cause Springboard to “force quit” your app and lead to a misleading crash report.

Viewing the data online took a while.  It was about 24 hours for the test data to show up on the website, and nearly 48 hours for the custom events to appear.

Linking with the library (and the other frameworks) added about 200 kb to the binary, although that may be slightly misleading as libsqlite3 was already being linked into the tester app.

I’m glad that the library also offered a feature to enable developers to expose “opt in” features as users can sometimes be sensitive to profiling, even if it’s anonymous.  Before hooking up the service to your app, it’d be worth checking out their privacy policy and making sure you’re not passing user specific data (which Pinch discourages and won’t allow) to their site.

If you’re looking for some insight into what your customers are doing with your app, linking in Pinch Media’s analytics library may be the answer.  From an ease of integration point of view, it was definitely very easy and worked as advertised.

Free App Landscape

Last week Apple reversed policy and made an announcement regarding a change to now allow free apps the option of supporting in-app purchases via the Store Kit.  It remains to be seen how the change plays out, but I suspect it’s a win for consumers and Apple and something of a mixed bag for developers.

Previously, the revenue model in the app store was narrow: paid apps could continue to charge; and free apps needed to stay free.  This produced a bounty of “lite” apps where developers would strip out functionality from a paid app and offer it for free in order to entice consumers.  While it’s hard to gauge the affect of this duplicity, the previous policy undoubtedly has lead to a fair amount of app bloat in the store as developers released multiple versions of the same app.

For developers, the older model required maintaining a completely different app or conditionally building source.  Both choices require additional testing and support.  Now, the code can be unified but the burden shifts to adding and integrating in-app purchase features into the app itself.  While the APIs are pretty straight-forward, the UI experience isn’t and each developer (based on their needs) will need to create a meaningful interface within their title and do a bit of bookkeeping to manage purchases.

While this announcement benefits newly arriving apps, it leaves those functioning under the older model somewhat out in the cold.  Given multiple apps with similar functionality, wouldn’t a consumer rather try the free app over the paid one at least initially?  One could argue that nothing precludes developers using the current approach of multiple releases and in some cases, it may still make sense.

However, it puts further pressure on developers to deliver more substantial content to consumers in a free format as the download rate of free apps is typically significantly higher than a paid app alone – especially given the market saturation.  With nearly 100,000 apps available at the moment, my suspicion is that this will make competition amongst 2nd tier developers more fierce but largely leave the major developers unaffected.  I can’t imagine we’d see a free version of an app from major commercial developer who’s price points hover well north of the $0.99 median of most apps.  Maybe that’s a good thing, as there are a plethora of shovelware apps out there already.

By encouraging in-app purchases, Apple helps to reduce the App Store clutter and duplication of effort in approving multiple titles.  Further, each in-app purchase nets Apple another 30%, which for the providing a distribution mechanism and platform, starts to seem pretty steep.  I can see why the music industry balked a few months ago at Apple’s cut and renegotiated the terms.  While the major studios had a reasonable block, I can’t see that happening with the 22,000+ different app developers.

Perhaps the biggest winners of this change are the consumers.  New apps that offer “more” for a price can now be unified as one app rather than multiple versions.  This will help remove some bloat and allow people to better find what they’re looking for.  Plus, it provides a business model that might support more innovative apps – like a “chapter based” adventure, or subscription based games.

Adding Build Versions via Xcode

One of the things I was keen on setting up early in development was the automatic injection of a build number for our products.  Having a build number is vital when handing out early “pre release” versions of the app to testers for review.  During development, there are likely to be a number of builds and release candidates – and you’ll need a way to differentiate between them.

It turns out I wasn’t alone as a quick search on Google provided a wealth of possible options.

AGVTOOL

Apple provides a generic versioning build tool.  It provides a number of useful functions, like hooking directly into CVS or Subversion to commit modified code.  In addition, it automatically creates a CFBundleVersion key in your info.plist file.

By default, CFBundleVersion is 1.0 in the info.plist file.  But, there’s another key, CFBundleShortVersionString which can be used for this same purpose.  Thus, you could set the short version to “1.0” and let the agvtool set the CFBundleVersion key to the current build number.

If you automate your builds on a neutral machine, this technique is definitely something to consider.  It seems the most natural fit for generating a build number that can be referenced in an app.  There are a number of other folks who have tutorials and walkthroughs for exactly how to set this tool up for use in development:

Inner Exception

Chris Hanson’s Blog

Jamie Montgomerie’s Blog

SCRIPTING

While the agvtool has some definite benefits, it felt a bit “manual” to me.  Sure, you can hook it up as a post-build script or on a separate box that does nightly builds (or whatever) but I was looking for something else – like the revision number instead of a build number, per say, that was always updated without a ton of extra work.

Using the revision number from Subversion seemed an easier way to link the code state with the build number without the extra hassle of tagging or branching specifically for the purpose of sharing a build with testers or external stakeholders.  So, I turned my attention to script based solutions.

Luckily, there are a number of posts that describe how to do the exact same thing with a variety of scripting tools and languages.  Most hinge on creating a custom post-build step, then running either some Perl, Ruby, or shell script to substitute in the desired number into a plist file associated with your project.

Another choice in scripting tools is “PlistBuddy” which can read/write values to plists – but you’d need to wrap it in a shell script and at that point it’s debatable whether using Perl (or the shell) to perform the insertion isn’t just as easy.  PlistBuddy doesn’t solve the potential problem of parsing the revision number from Subversion either.

Adding something like this into your Xcode is trivial.  Under “Groups & Files” expand the Targets section and right-click the target to select Add –> New Build Phase –> New Run Script Phase.  A window will pop up where you can insert your own commands into the overall build.  Here’s some sample code from other folks:

Ruby – Jeff LaMarche, author of “Beginning iPhone Development.”

Git – Marcus Zarra.

Perl – Stackoverflow.

In the end, I took the route proposed by Daniel Jalkut on Red Sweater using Subversion with a couple minor modifications:

# Xcode auto-versioning script for Subversion
# by Axel Andersson, modified by Daniel Jalkut to add
# “–revision HEAD” to the svn info line, which allows
# the latest revision to always be used.

use strict;

die “$0: Must be run from Xcode” unless $ENV{“BUILT_PRODUCTS_DIR”};

# Get the current subversion revision number and use it to set the CFBundleVersion value
my $REV = `/usr/local/bin/svnversion -n ./`;
my $INFO = “$ENV{BUILT_PRODUCTS_DIR}/$ENV{WRAPPER_NAME}/version.plist”;

my $version = $REV;

# (Match the last group of digits and optional letter M/S):
($version =~ m/\d+[MS]*$/) && ($version = $&);

die “$0: No Subversion revision found” unless $version;

open(FH, “$INFO”) or die “$0: $INFO: $!”;
my $info = join(“”, <FH>);
close(FH);

$info =~ s/r0000/r$version/;
print $version;
print $INFO;

open(FH, “>$INFO”) or die “$0: $INFO: $!”;
print FH $info;
close(FH);

The major change from the original is that I write to a separate plist file (version) instead of Info.plist with a default key of “r0000” which is used for the substitution.  Really there’s not much difference and the choice of which scripting language is largely a personal choice for maintenance sake.

Automating SQLite Database Creation

“If you want more effective programmers, you will discover that they should not waste their time debugging, they should not introduce the bugs to start with.”
Edsger Dijkstra

During the software development process, the more non-deterministic processes that enter into your workflow the greater chance of introducing unintended bugs or functional regression.  There’s nothing more frustrating than to fix a bug (or make an improvement) only to have something else in your app break.  Thus, it’s generally considered good practice to automate as much of your pipeline (from build to asset creation) as possible.  If you can automate something, do so!  It’ll save you time and effort later.

If you decide to use SQLite in your iPhone app, there are a few steps to follow in order to automate re-creation of your database.

For Daily Value, I used a couple of tools to construct and later manipulate the database.  On the PC, the process started with Microsoft Access to transform the relational database from the USDA into a number of Excel spreadsheets – one for each table.  From Access, this was accomplished by right clicking the tables and selecting “Export | Excel”.

Microsoft Excel 2007 was used heavily to manipulate and process the input data before it was ready to export into SQLite.  There are some subtle and sometimes frustrating differences between Excel 2007 and 2003.  It felt to me like exporting was more flexible in the earlier version.  Basically, the “standard” CSV is “comma separated values” which created an input parsing problem for both SQLite Manager for Firefox and the command line tool, sqlite3, because the commas used for field delimiters.

I dimly thought that in older versions of Excel, when exporting to CSV, a series of dialogs took you through the process and allowed you to specify a custom delimiter.  In a file full of strings with commas already, the ability to set the token was great.  In my case, I wanted to export the data using the pipe (|) symbol.  It took some digging but exporting to CSV in 2007 to a flavor that SQLite liked required changing the output separator token via the Control Panel!

Here’re the basic steps:

  1. With Excel closed, open the Control Panel
  2. Double click the Regional and Language Options
  3. On the Regional Options tab, click on the Customize button
  4. On the Numbers tab, change the List separator to “|” (without the quotes)

Now you can “Save As” in Excel using CSV with the custom separator.

I used SQLite Manager early in the development process because it was very convenient to not only import data, but it was nice working in a visual environment as the tables continued to undergo design changes.  Even after I automated the database creation, the tool was useful to inspect run-time changes from the Simulator.  It was also nice to see the SQL the tool generated when creating tables, indices, or other operations on the data.

Once the table structure solidified, I wrote a simple “text input file” to re-create the database using the sqlite3 schema command.

  1. Gather up the structure of your tables by using either SQLite Manager or sqlite3.  In the Firefox plug-in, SQLite Manager has a tab that describes the table structure.  Or, launch the Terminal, and open your database with sqlite3, then type “.schema <table_name>” to get the structure.
  2. Copy this information to TextEdit (or an editor of your preference).  The top portion of this file should issue all the table creation commands required to setup your database structure.
  3. Then add the sqlite3 import commands to import the CSV files.

.mode csv

.separator |

.import table_1.csv table_1

And so on.  If you need to insert data into the tables, or create indices on any tables, you can insert these commands at the end of the file.  My input file, named schema.txt, looked something like this:

CREATE TABLE “food” (“pk” INTEGER PRIMARY KEY NOT NULL, “group” INTEGER, “name” TEXT, “desc” TEXT);

CREATE TABLE “version” (“ver” INTEGER);

.mode csv

.separator |

.import food.csv food

CREATE INDEX idx_food_pk ON food (“pk” ASC);

INSERT INTO version (ver) VALUES (“2”);

A couple notes on the file format.  You can span SQL commands on multiple lines in order to make the file more human readable.  The semi-colon is important for any SQL statements used in the file.  You should remove any column header information from the CSV file prior to using the “.import” command as the column types most likely won’t match and cause the import to fail.  Also, the CSV files reside in the same directory as where you’ll create the database and issue the sqlite3 command as shown below:

%sqlite3 database.sqlite < schema.txt

If you really wanted to get more sophisticated, you could hook running (and copying) this freshly created database into Xcode as a build step.  In another entry, I’ll talk about how I use something similar to auto version builds based on the repository revision.

Data persistence choices

In roughing out the early data framework for Daily Value, one of the early considerations was data persistence – particularly user created data.  The app’s requirements included supporting user entered food data that could be treated “on par” with shipped food data.

Looking around at what the iPhone SDK supported, a couple choices quickly presented themselves: archiving, property lists, and SQLite.  While the temptation was to leap right into a relational database solution, some consideration was made for the other choices and in truth, Daily Value employs all of these techniques for various pieces of data.

There’s a nice write up in “Beginning iPhone Development” by Dave Mark and Jeff LaMarche so I won’t dive too deeply into it except to say property lists and archiving were discarded.  Although writing data in a binary format with archiving had some appeal for data obfuscation purposes, it would have also meant writing a bunch of “database-like” functionality that didn’t seem a good technical investment in time or resources.  So, SQLite was what I went with.

SQLite offers a lot of upside:

  • The API is written in C and is easy to navigate.
  • Storage and manipulation of the data was offloaded to the database framework.
  • There are a number of free tools that can be used to browse and quickly inspect the data (ex. SQLite Manager for FireFox)
  • Creation can be automated
  • Good performance on complex queries

The biggest drawback is an apparent lack of authentication or way to protect the data placed into the database.  Since our data was publicly available, this wasn’t much of an issue but it might be for some other apps.

The initial design called for all the data – both shipped and user generated – to be stored in a single database file.  The thought being that it’d be easier to code queries against and manage.  User data would contain some differentiator (either a column identifier or field) to denote it as “editable” (you don’t want folks mucking about with your data).

Apps that use persistent data, or write back cannot do so to files stored within their application bundle or else it breaks the code signing and things stop working.  So, when the app initially loads, the source data is copied to the Documents directory and opened from there.  Of course, the app checks first to see if the file already exists so it doesn’t overwrite an existing file.

Test rigs built for the simulator seemed to indicate that this was viable, but putting the app on the actual device proved otherwise.  Apple requires that apps fully launch quickly after no more than 5 seconds.  The first database prototypes were nearly 4 MB.  The bulk initial copy caused startup to be unacceptable taking over 10 seconds!

Trimming the database helped.  With a little bit of effort in making tables more relational and dropping columns that weren’t strictly required, the size got squeezed down to just under 2 MB.  Even so, the copy took too long – so what to do?

After digging about through the SQLite documentation, I came across the SQL “attach” command.  One neat feature in SQLite is that it supports the notion of opening multiple databases and treating them (generally) as a single entity.  Care was taken to create a “template” user database with pre-created tables that were different than the app supplied database.  While not strictly necessary since SQLite will allow you to have duplicate table names, it seemed prudent and an easy “design time” step.

When the app launches, if it doesn’t already exist, the “empty” user database (all 12 kb of it) is copied from the main bundle to the Documents directory and opened.  Then, the app “attaches” to the large bundle database with the restriction that none of the bundled data can be modified.  Startup times were acceptable and the additional complexity was very minimal due to the SQLite abstraction support.

Here’s a bit of code to get the job done – assuming that dbHandle is a member variable of your data class.

- (void)copyDataIfNeeded
{
   NSFileManager *fm = [NSFileManager defaultManager];
   NSArray *docPath = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
   NSString *docDir = [docPath objectAtIndex:0];
   NSString *writePath = [docDir stringByAppendingPathComponent:@”user.db”];

   BOOL success = [fm fileExistsAtPath:writePath];
   if (success)
      return;

   NSError *error;
   NSString *dbPath = [[[NSBundle mainBundle] resourcePath] stringByAppendingPathComponent:”user.db”];
   success = [fm copyItemAtPath:dbPath toPath:writePath error:&error];
   if (!success)
      NSAssert1(0, @”failed to create writable database with msg ‘%@’”, [error localizedDescription]);
}

- (void)attachDatabase
{
   [self copyDataIfNeeded];

   NSArray *path = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
   NSString *docDir = [path objectAtIndex:0];
   NSString *openPath = [docDir stringByAppendingPathComponent:”user.db”];

   if (sqlite3_open([openPath UTF8String], &dbHandle) == SQLITE_OK)
   {
      NSString *sysDBPath = [[[NSBundle mainBundle] resourcePath] stringByAppendingPathComponent:”main.db”];
      NSString *sqlString = [NSString stringWithFormat:@”ATTACH DATABASE ‘%@’ AS main”, sysDBPath];

      const char *sql = [sqlString UTF8String];
      sqlite3_stmt *statement;
      int success = 0;

      success = sqlite3_prepare_v2(dbHandle, sql, -1, &statement, NULL);
      NSAssert1(success == SQLITE_OK, @”Failed to prepare ATTACH with msg ‘%s’”, sqlite3_errmsg(dbHandle));

      success = sqlite3_step(statement);
      NSAssert1(success == SQLITE_DONE, @”Failed to execute ATTACH with msg ‘%s’”, sqlite3_errmsg(dbHandle));

      sqlite3_finalize(statement);
   }
   else
       NSAssert1(0, @”Failed to open database with msg ‘%@’”, sqlite3_errmsg(dbHandle));
}

Unexpected additional time for review…

Not really the first (well, second) entry I was thinking of making – but sometimes events aren’t as one would expect.

Daily Value has been in development since April ’09 and was approved for sale in Aug ’09 (see the blog or main page for details) and I’ve been working on an update for the past few weeks.  Not only are there some neat new features planned, but a number of “quality issues” were discovered after our first binary came out.  The first submission went very smoothly – as I recall it took about 6 days for the app to go “live” in the stores.

Obviously, it’s very disappointing that problems weren’t discovered during the testing and stabilization phase and even worse reached our customers.  Not to rationalize too much, but it’s challenging to fully regress a product and anticipate a nearly infinite range of error states and conditions.  That said, I’ve started adopting unit tests to help cover corner cases and prevent functional regression or re-introduction of bugs.  Unit testing is a broad topic worthy of a longer and dedicated post or two – but suffice to say, I’ve found and fixed a number of problems and would like to deliver some new features to our users too.  Exciting!

Daily Value was resubmitted to Apple for approval with some bug fixes and new features.  I didn’t expect a quick turn-around this time, however.  Recent guidance from Apple mentioned that “90% of apps are approved within 14 days.”  It had only been about a week, but I received a rather vanilla email from the app review team that Daily Value “is requiring unexpected additional time for review.”  Not the kind of email to read before quitting for the day as my mind considered largely negative possibilities!

Unfortunately, the email is all of two lines and vaguely promises to “update you with further status as soon as we are able.”  So, I did a bunch of Google searches to see if others have encountered the same email and what the general results are – it seemed a largely mixed bag, but many of the posts were pretty old.  So, I thought I’d weigh in with my conjecture and update this post as I get more information.

In the earlier days (a year or so ago) of app development, this email seemed to indicate some sort of problem with your app – usually procedural.  Folks would use language, subject matter, or content that in Apple’s opinion was inappropriate.  In some cases, it appeared Apple may have held up an app if it may have encroached on another company’s trademarks or patents to not only protect themselves but the developer.

In fact earlier this month, Looking Glass submitted an app, iGive Blood, that not only received this same “unexpected time” email but was also rejected.  The problem was the app, and app store text mentioned that proceeds from the app’s sales would be donated to charity.  It wasn’t appropriate and the text had to be removed from the app and store text before being resubmitted.  No problem – a new binary was made and was resubmitted nine days ago.

Daily Value, however, was approved earlier in the month.  The changes in version 1.1 are feature enhancements, new functionality, and quality improvements – so, I’m curious about why it’s possibly being held up.

While I don’t have any concrete evidence other than anecdotal experience, I’d like to believe that the app review team is being flooded with apps.  When Daily Value first was being designed, there were fewer than 24,000 apps available – but four months later the number has reached nearly 65,000.  That’s not even counting updates and fixes to existing applications.  I thought I read on Apple’s web site that they get roughly 4,000 submissions (updates and new apps) per day.  Approval time has gone from 7-10 days to nearly 14.

Anyway, optimistically, I’m hoping that the form email is generated by an automated system after an app submission has been in the queue for a week – and with the deluge of apps to review and pour over – more and more developers will be receiving this email that is just serving notice “you app is in the queue – be patient” – and it’s not always as ill an omen as it used to be back when app approval was much quicker.  Fingers crossed, anyway!

Hello world!

Welcome to the Looking Glass development blog.

It’s my hope to share insight, challenges, and solutions to some of the problems I’ve faced while developing applications for Apple’s iPhone.

In many ways, the post title is quite appropriate.  There’s a lot of work involved in putting together a blog, web site, and most importantly – apps!  The first thing you do/try when learning a new language or environment is put together a simple test program.  While I’ve got a reasonable amount of experience as a software developer, web publishing and development is another thing entirely.

Time grants wisdom with practice and dedication – so, I’m optimistic with further posts and entries – the content of this blog will equally evolve and improve too!  Maybe when I get a break from app coding, I’ll poke about the blog framework and make some customizations… expect some changes!