bsdtalk186 - Soft Updates w/ Journaling & ULE Sched

FreeBSD Soft Updates with Journaling, ULE Scheduler

Very good short interview with Jeff Roberson, FreeBSD Committer.Discussion on the addition of journaling to soft updates and the development of the ULE scheduler.

BSD Talk Blog - Jeff Roberson (February 05, 2010)

You can also listen on your Android device. Just download Google Listen from the marketplace. After you setup Google Listen on your Android Device, search for bsdtalk or FreeBSD from within Google Listen.

Single Thread Object Communication

When objects in a single thread are loosely coupled and need to communicate with each other, Notifications allow you to broadcast messages on a synchronous or asynchronous basis and still retain object independence.

When you implement Store Kit in your application, it is most likely a reasonable assumption that you want to re-use your in-app store in a future iPhone application without rewriting the object(s) that implement the SKPaymentTransactionObserver and SKProductsRequestDelegate protocols. We know that these objects receive asynchronous notifications via the delegate protocol(s) that they implement. So in a nutshell, the object that implements the SKProductsRequestDelegate protocol will receive a response asynchronously when the product identifier request that it previously sent is successfully processed. The payment transaction observer object, which is added as an observer to the payment queue upon application launch, again receives asynchronous notifications when there's stuff to do in the payment queue.

When your transaction observer receives notifications from the payment queue that there are updated transactions, completed transactions, or failures, it is most likely the responsibility of the observer object to process these notifications. The processing of these notifications most likely entails application specific logic. So rather than loading up your transaction observer with application specific logic, the NSNotification center provides a clean way of synchronously or asynchronously(via Notification Queues) notifying other objects within the same thread. There is also an NSDistributedNotificationCenter for broadcasting messages to objects in other threads.

By using the NSNotification center, you can separate the application-specific logic from the observer object that receives notifications from the payment queue. This allows for a cleaner and more modular design, as each object can focus on its specific responsibilities without being tightly coupled to other objects. In addition, the NSNotification center provides a flexible way to communicate between objects within the same thread or across different threads using the NSDistributedNotificationCenter. This makes it easier to reuse and refactor code, which can save time and effort in the long run.

Creating a Mobile App for the iPhone

In the coming weeks, I plan on covering the development process from start to finish. I will be talking about code organization, memory management, useful xcode features, core data, performance optimization, debugging, iterative development, and any other useful topics that I come across.

iLogMiles was approved by Apple for sale on the App store, on April 6, 2010. iLogMiles is free to download. The app quickly hit the #2 spot on the top free business apps list. The user can upgrade to the full version and purchase themes from within the free application. We utilized the Apple Store Kit framework to provide the in-app store. The Store Kit framework provides the necessary means to securely process payments from within the application. Products are identified via product identifiers. Apple clearly defines the types of products that may be sold via the Store Kit framework. More on this later but keep in mind that it is very important to read through all of the documentation.

Over the next few weeks, we will be talking about the mobile application development process. To give you some background on me, I started with C++ in 1997. At the time, I mostly used the GNU compilers on FreeBSD and Linux and Sun workshop on a SPARCstation. And for the C++ courses I was taking, Borland in a Win environment. To get up to speed on iPhone OS, I purchased the WWDC 2009 sessions and watched them all (at the gym on my iPhone). I attended the iPhone tech talk tour in San Jose and I listened to any and all of the available iPhone development podcasts (at the gym) that I could find. I poured through all of the books that I could get my hands on. I read the iPhone OS programming guides on the iPhone OS developer connection. XCode also has integrated documentation for the Apple API's. I've always used vi/vim with ctags, so I had to force myself to watch the xcode screencasts (available via third-party on the Internet). There are lots of great xcode tips out there and you can print out a summary page with all of the cute shortcuts - if you are so inclined to do so. I found it quite helpful to print the shortcuts page.  Design patterns are at the core of how the Cocoa frameworks interact. Apple provides solid documentation on Cocoa design patterns in the iPhone OS Reference Library and these patterns can be understood at a high-level without delving into any code. This is my suggested approach. If you are not familiar with design patterns, the gang of four book (the GOF book as most often termed in the software world) - Design Patterns: Elements of Reusable Object-Oriented Software, by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, is the official source for design patterns. The theory behind Cocoa's design patterns can be found in this book and for that matter, the use of design patterns in any modern object-oriented software system can be found in this book. The GOF book describes design patterns in a language independent manner. Familiarizing yourself with Cocoa's design patterns from a language independent perspective is a key starting point for building an iPhone application.

Continuing on the topic of cocoa design patterns, Apple's frameworks are designed for re-use. There are folks out there who are very experienced with design patterns and are able to layer design patterns on top of the cocoa frameworks. These individuals understand all of the memory implications. They understand MVC backward and forward and they understand the Cocoa design patterns backward and forward. If you do not fall into this category, then keep it simple. If there is one problem that I have seen across forums and code samples on the Internet, it is that of "fighting the apis". It is very important to not fight the apis. The apple documentation describes the proper use of the iPhone OS frameworks and associated API's. The Apple developer connection provides plenty of sample code so it is important to read through this code to understand the proper use of the iPhone OS APIs.

From inception to app store approval, we spent 5 months developing iLogMiles. While this may seem longer than the standard "2 month" development lifecycle, I can assure you that iLogMiles uses a wide variety of the iPhone OS frameworks. In the coming weeks, we plan on covering the development process from start to finish. I will be talking about code organization, memory management, xcode features, core data, performance optimization, debugging, iterative development, and any other topics that we see useful.

Apple, the Apple logo, iPhone, iPod touch, and iTunes are trademarks of Apple Inc., registered in the U.S. and other countries. iPad is a trademark of Apple Inc. App Store is a service mark of Apple Inc.

Objective-C Categories

Categories are a powerful tool for extending the functionality of existing classes in Objective-C. However, it's important to follow Apple's guidelines when using them to avoid any undefined behavior.

Categories allow you to extend the functionality of an existing class without subclassing. Before I get into an example, here are a few guidelines to follow when using categories. Apple recommends to not use categories to override methods in other classes, inherited classes, or other categories. Categories are not meant to add instance variables to classes either. If used in such contexts, the behavior of a program is undefined. Apple recommends using categories for adding functionality to large, existing classes and then grouping that functionality accordingly. Overriding existing Cocoa methods that are inherited from within the Cocoa hierarchy is a no-no as is extending methods that are already part of a category within the Cocoa hierarchy. Categories allow the addition of methods to existing classes while retaining the scope of class instance variables. So here's an example. Categories are quite simple to create and very powerful.

myCategory.h

@interface UIViewController(myCategory)
- (void)messageName:(double)arga keywordA:(double)argb keywordB:(NSString *)argc;

myCategory.m

#import "myController.h"

@implementation UIViewController(myCategory)
- (void)messageName:(double)arga keywordA:(double)argb keywordB:(NSString *)argc {
// process arguments
// do something with ivar
// ...
}

myController.h

#import "myCategory.h"
@interface myController : UIViewController {

   someType ivar;
}

- (void) someMessage;
...
@end

myController.m

#import "myController.h"

@implementation

- (void)someMessage {

   [self messageName];  // send message to self - i.e. call method defined by category
}

@end

Core Data

I was fortunate to attend the iPhone Tech Talk conference, hosted by Apple Inc. in San Jose last October, 2009. One of the sessions I attended was on Core Data. At a very high level, Core Data is an Apple™ software framework for storing data on the computer - namely, the iPhone and the Mac. Please note that I don't plan on covering database design theory in this post so I'm assuming a familiarity with it.

Core data is an Object graph management and persistence framework. Core data maintains a graph of objects (otherwise known as a managed object graph or managed object context). References to managed objects are stored in the object graph. Core Data serializes these objects and persists them to a data store. Core data can use several different underlying data stores - xml, binary, sqlite. The managed object context is a key concept in Core Data, as it provides a proxy between the application objects and the underlying data store(s). The persistent store coordinator is responsible for managing the relationship between the managed object context and the persistent store(s). A managed object context wraps a collection of managed objects and sits on top of a persistent store coordinator which houses one or more persistent stores. A managed object context is created via an instance of the NSManagedObjectContext class.

It's important to note that the Core Data sqlite file should not be accessed directly from within code, as the schema may change and it's not intended for direct manipulation.

In Summary, a managed object context is a collection of managed objects. Managed objects are instances of the entities that are defined in the data model. A managed object represents a row of data in a table. The managed objects context sits on top of a persistent store coordinator. The managed object context is a proxy between the objects in the application and the underlying data store(s). A persistent object store is represented by a xml, binary, or sqlite file. The persistent store coordinator sits between the managed object context and the persistent store(s). The persistent store coordinator in turn abstracts one or more persistent object stores as a single aggregate. Please note, a persistent store coordinator can only be associated with one managed object context. But a persistent store coordinator can have many persistent object stores. The managed object context talks directly to the persistent store coordinator. The managed object context is a scratchpad of managed objects.
After working with Core Data, it is a worthwhile exercise to open the core data sqlite file and dig through the tables. A quick look through the tables will explain how Apple is handling referential integrity. The Core Data sqlite file is "not meant" to be accessed directly from within code. Do not modify it or attempt to write queries against it. There is no guarantee that the schema will not change. Core Data is meant to abstract implementation dependent details (i.e. the implementation may change upon any future releases).

The following notes were derived from three sources: Apple WWDC 2009, the iPhone Core Data session in San Jose last October, and my experience working with object/relational databases.

Creating a Core Data Stack

1. Load the Data Model
2. Create the persistent store coordinator and set the model
3. Add the data store
4. Create a managed object context and set the persistent store coordinator

Core Data Model

1. A Core Data model does not need to be normalized. Define data contracts early on. Don't over tighten the data model - design with usage patterns in mind. Define entities or tables with Xcode's visual editor. (core data objects represent rows in these tables). In other words, a Core Data object (row of data) of type PERSON is an instance of the PERSON entity in the Core Data model. Of course, a person object will have relationships with other objects in the database. A Core Data Managed object context manages these objects, their relationships, and persists them to the underlying data store (via the persistent store coordinator). Core Data is simple to setup. Don't complicate it.
2. NSManagedObjectContext defines the verbs (CRUD or fetch/insert/delete/save) that act on managed objects (nouns)
3. NSManagedObjectContext tracks changes to properties in the data model
4. Core Data fits very well in the MVC architecture.
5. An NSPersistentStoreCoordinator can have multiple persistent stores(NSPersistentStore) and multiple Managed Object Contexts can use a single NSPersistentStoreCoordinator
6. It is possible to delete the sqlite store file if the stored data needs to be deleted. This file is stored in $HOME/Library/Application Support/iPhone Simulator/User/Applications/SOMEAPPLICATIONGUID/Documents
7. Use Core Data migration or built-in versioning for model changes. easy to use. zero code.

Multithreading

1. Core data assigns one managed object context per thread; use notifcations for changes made to managed objects within a managed object context.
2. never pass managed objects between threads, pass object IDs (actually the primary key)
3. within a thread, create a new Managed object context and then tell the new Managed object context what objects IDs to fetch

Fetching

Use NSFetchedResultsController (break up fetches into sections) - it is fast, efficient, and easy.

1. Create an NSFetchedResultsController
2. Set the fetch request, predicate, and sort descriptors. fetch request provides predicate and sort descriptors). fetch request and predicate are immutable.
3. the keypath returns the section name (section information is cached)
4. set yourself as its delegate and implement your tableview methods
5. Create separate controllers for different data sets

NSFetchRequest

1. set the entity that you want to fetch against
2. create NSEntityDescription - provide managed object context and entity you want to work with
3. set the predicate (if applicable)
4. Fetch - pass in fetch and error

Batching

1. setup fetch request (only need some of the objects at a time so setBatchSize:)
2. set batch size
3. you get back an NSArray of results

Prefetching

1. use setRelationshipKeyPathsForPrefetching:

Managed Object Interaction

NSManagedObject

1. Use accessors
2. Objective-C properties
3. To Many relationships are sets
4. NSManagedObjects provides Key value coding/observing out of the box

1. Managed Object Context observes object changes. Leverage change tracking notifcations
2. Register for Object Level Changes (KVO)
3. Register for Graph Level Changes (NSNotifications)

Iterate over Custom UITableViews

UITableViews on the iPhone often contain a variable number of cells. The following code snippet should be helpful for those who need to iterate over a variable length table and subsequently read or set data model and cell state.

   //loop through cells in each section and make sure the data model is in sync 
   NSInteger numSections = [self numberOfSectionsInTableView:someTable];
   for (NSInteger s = 0; s < numSections; s++) { 
      NSLog(@"Section %d of %d", s, numSections); 
      NSInteger numRowsInSection = [self tableView:someTable numberOfRowsInSection:s]; 
      for (NSInteger r = 0; r < numRowsInSection; r++) {
         NSLog(@"Row %d of %d", r, numRowsInSection); 
         MyCell *cell = (MyCell *)[someTable cellForRowAtIndexPath: 
                                  [NSIndexPath indexPathForRow:r inSection:s]]; 
         DataModelObject *theObject = (DataModelObject *)[fetchedResultsController objectAtIndexPath:
                                                         [NSIndexPath indexPathForRow:r inSection:s]]; 
         [theObject setValue:[NSNumber numberWithBool:MyCell.someProperty] forKey:@"someEntityKey"]; 
      } 
   }

---

Mobile Technology - Where Are We Going?

OS X and iOS contain open source components.

1. Open Source Components that ship with Mac OS X 10.6.2 and their corresponding Open Source Projects (src avail for dl/browse) - http://www.apple.com/opensource/
2. Open Source Components that ship with Mac OS X 10.6.2, iPhone OS 3.1.2 and Developer Tools 3.2.1 (source avail for dl) Also browseable by prior versions - http://opensource.apple.com

The open source components are typically released under various open source licenses, such as the Apache License, the GNU General Public License, and the BSD License, among others. By including open source components, Apple is able to leverage the work of the open source community and provide a more robust and secure platform for its users. The open source components can also be used by developers to build applications for macOS and iOS, as well as for other platforms.

On to Android...so what is it? Android is an open-source mobile operating system. It uses a modified Linux 2.6 Kernel - tuned for embedded devices. To quote the Android Web site - "Android relies on Linux version 2.6 for core system services such as security, memory management, process management, network stack, and driver model. The kernel also acts as an abstraction layer between the hardware and the rest of the software stack." (http://developer.android.com/guide/basics/what-is-android.html).

Linux is known for its stability, security, and flexibility. It has become a popular choice for servers, workstations, and even desktops. Linux is widely used in the cloud computing industry and powers many of the world's largest websites and services. It has a strong developer community and a vast selection of open-source software available for free. Additionally, Linux is highly customizable, allowing users to modify and tailor their operating system to their specific needs.

Linux (GNU/Linux) is a UNIX-like operating system that comes in many flavors or distributions. The distributions typically differ in how package (applications) distribution is handled. Oftentimes, a custom or stripped down kernel is shipped with the distribution. There are over 600 GNU/Linux distributions. They run on a myriad of devices from embedded controllers to large scale grids and supercomputers.  GNU/Linux has been around a while - Linuz Torvalds coined the term in '91. The GNU Project, created by Richard Stallman in 1983, had the goal of creating a complete Unix-compatible software system composed entirely of free software". Hence came GNU/Linux. Read more here - http://www.gnu.org/.

To clarify, the GNU/Linux operating system consists of the Linux kernel (created by Linus Torvalds in 1991) and GNU software, which was developed by the GNU Project (launched by Richard Stallman in 1983) and includes tools such as the GNU C Compiler (GCC), GNU Debugger (GDB), and Bash shell.

The Linux kernel was based on the Minix operating system and Unix principles, but it was not directly derived from SVR4 or BSD. It was created as a free and open-source alternative to commercial operating systems and has since become widely used on servers, desktops, and mobile devices.

The process, thread, and memory models in GNU/Linux are based on the Unix principles and have been tested and refined over several decades. They are known for their stability and scalability, making GNU/Linux a popular choice for high-performance computing and enterprise applications.

Many GNU userland tools, such as Bash, GCC, GNU coreutils, and GNU Emacs, are included in most GNU/Linux distributions. These tools are often used by developers and system administrators to perform various tasks, such as writing and compiling code, managing files and directories, and configuring system settings. The availability of these tools on GNU/Linux systems is a result of the close relationship between the GNU Project and the Linux kernel.

Android uses a modified version of the Linux kernel, which is a Unix-like operating system kernel, but its userland libraries are not based on GNU/Linux. Instead, Android uses a custom, BSD-like userland developed by Google. This allows Android to have more control over the code and reduce dependencies on GNU/Linux. However, Android still maintains compatibility with many GNU/Linux tools and libraries.

As opposed to Objective-C, which developers use to write applications for the iPhone, Java is used for writing applications on Android devices. Android provides libraries on top of the kernel for 2D and 3D rendering, type support, sqlite access, media, libc (actually called Bionic), etc. While the Linux kernel provides memory management, process and threading support, etc., Google uses a byte-code interpreter (well, not exactly, but similar) called DalvikVM which transforms java class files into a second type of byte code format - .dex. An Android application gets loaded into a single process and is allocated a DalvikVM instance. DavlikVM was chosen because of its memory/processor efficiency on embedded devices.

Android also provides an application framework that allows developers to use various pre-built components such as activities, services, content providers, and broadcast receivers to build their applications. These components help developers to manage the lifecycle of their applications and handle various system events, such as incoming phone calls or text messages.

The Android SDK (Software Development Kit) provides all the necessary tools and libraries for developers to build, test, and debug their applications. The SDK includes an emulator, which allows developers to test their applications on a virtual device before deploying them on an actual Android device.

In addition, Android is built with security in mind. It includes various security features such as app sandboxing, which isolates each application and its data from other applications, and permissions, which allow users to control what data and resources an application can access on their device.

In 2007, Google created the open handset alliance with the following objective: "The Open Handset Alliance is a group of 47 technology and mobile companies who have come together to accelerate innovation in mobile and offer consumers a richer, less expensive, and better mobile experience. Together we have developed Android™, the first complete, open, and free mobile platform. We are committed to commercially deploy handsets and services using the Android Platform (http://www.openhandsetalliance.com)". By working together, the alliance aims to create a more open and innovative mobile ecosystem that benefits both developers and consumers.

The list of member companies is huge including but not limited to, TI, NVidia, Quallcomm, Acer, Sprint, Toshiba, LG, etc. Shortly thereafter, in 2008, Google released the Android source code under an Apache license. This move attracted and will continue to attract top developers from all over the world who are comfortable with the Java programming language in a Linux type environment.

By releasing the Android source code under an Apache license, Google allowed developers to modify and distribute the code, which led to the creation of numerous custom ROMs (modified versions of the Android operating system) and a thriving community of developers. It also allowed manufacturers to use the Android operating system on their devices without any licensing fees, which helped to accelerate the adoption of Android.

Google's initiative with the release of the Android source code and the formation of the Open Handset Alliance were huge moves. The Open Handset Alliance will aid its member companies with deploying the Android operating system on their respective devices. In other words, we will see the Android operating system on many different brands of mobile phones.

The open-source nature of Android and the formation of the Open Handset Alliance have allowed for the widespread adoption of the Android operating system across many different mobile devices from various manufacturers. This has helped to create a more competitive and diverse mobile market, which ultimately benefits consumers.

By the end of 2009, there will be over 15 mobile devices with the Android operating system. There are over 15,000 apps on the Android market (as of November 2009). There were 10,000 apps on the Android Market 2 months ago. There are over 100,000 apps on the Apple app store (as of November 2009). The Apple app store is 16 months old. There have been over 2 billion total downloads since inception.

The Android operating system is solid and it isn't going anywhere. Here are some good resources on the Internet.

Getting Started with Android Development

The GNU Operating System

Free Software Foundation

Open Handset Alliance

What is Android?

Dalvik Virtual Machine insights

Android Notes

Inside the Android Application Framework