How JSPatch works

JSPtach is a hot fix framework on iOS platform. You can use JavaScript to call any native Objective-C method just by importing a tiny engine. Now, you will enjoy the advantages of scripting language, such as integrating modules into project dynamically. You can also replace original codes so that you can fix any bug.

I wrote two blogs to explain how JSPatch works. However, with the evolution of JSPatch, they are out-of-date now.In this article, I will show the implementation and details of JSPatch to help you understand and use JSPatch easier.

Outline

Basic Theory
Method invocation
    1. require
    2. JS Interface
        i. Encapsulation of JS Object
        ii. `__c()` Metafunction
    3. message forwarding
    4. object retaining and converting
    5. type converting

Method replacement
    1. Basic theory
    2. Use of va_list(32-bit)
    3. Use of ForwardInvocation(64-bit)
    4. Add new method
        i. Plan
        ii. Protocol
    5. Implementation of property
    6. Keyword: self
    7. Keyword: super

Extension
    1. Support Struct
    2. Support C function
    

Basic Theory

The fundamental cause of calling and changing Objective-C method with JavaScript code lies in the fact that Obejctive-C is a dynamic language. All invocations of method and generation of class are handled at runtime. So, we can use reflection to get corresponding class and method from their names:

Class class = NSClassFromString("UIViewController");
id viewController = [[class alloc] init];
SEL selector = NSSelectorFromString("viewDidLoad");
[viewController performSelector:selector];

We can also replace the implementation of some method with a new one:

static void newViewDidLoad(id slf, SEL sel) {}
class_replaceMethod(class, selector, newViewDidLoad, @"");

We can even create a new class and add some methods for it:

Class cls = objc_allocateClassPair(superCls, "JPObject", 0);
objc_registerClassPair(cls);
class_addMethod(cls, selector, implement, typedesc);

There are a lot of execellent blogs talking about object model and dynamic message sending in Objective-C, so I won't talk about them. Theoretically, you can call any method at runtime with the class name and method name. You can also replace the implementation of any class and add new classes. In a word, the basic of JSPatch is the transmission of string from JavaScript to Objective-C. Then the Objective-C side can use runtime to call and replace methods.

This is only the basic theory, however, to put it into practice, we still have to solve a lot of problem. Now, let's take a look at each of them.

Method invocation

require('UIView')
var view = UIView.alloc().init()
view.setBackgroundColor(require('UIColor').grayColor())
view.setAlpha(0.5)

With JSPatch, you can create an instance of UIView with JavaScript code, you can also set the background color and the value of alpha.

The code above covers the following five topics:

1. using 'require' keyword to import a class
2. using JavaScript to call Objective-C method
3. message passing
4. object retaining and converting
5. type converting

Now, let's talk about them one by one.

1. require

With require('UIview'), you can call class method of UIView now. The require keyword is very simple, it just creates a global variable with the same name. The variable is an object, whose __isCls is set to 1, which means this is a class, and __claName is the name of this class. These two properties will be used during method invocation.

var _require = function(clsName) {
  if (!global[clsName]) {
    global[clsName] = {
      __isCls: 1,
      __clsName: clsName
    }
  }
  return global[clsName]
}

Therefore, when you call require('UIview'), what you are actually doing is creating a global object looks like this:

{
  __isCls: 1,
  __clsName: "UIView"
}

2. JS Bridge

Now, let's take a look at how UIView.alloc() is called.

i Encapsulation of JS Object

At the very beginning, in order to comply with JavaScript syntax, I tried to add a method called alloc to the object UIView, otherwise, you will get an exception when calling the method. This is different from Objective-C runtime since you won't have any chance to pass the method invocation. Based on the analysis above, on calling require method, I passed the class name to Objective-C, gave all methods of this class back to JavaScript and then create corresponding JavaScript method. In these JavaScripte method, I used the method name to call corresponding Obejective-C method.

So the UIView object now looks like this:

{
    __isCls: 1,
    __clsName: "UIView",
    alloc: function() {…},
    beginAnimations_context: function() {…},
    setAnimationsEnabled: function(){…},
    ...
}

In fact, I have to get methods from not only current class itself, but also its superclass. All methods in the inheritance chain will be added into JavaScript. However, I got a serious problem about memory usage because a class may have several hundred methods. To reduce memory usage, I made some optimization such as using inheritance chain in JavaScript to avoid adding method of superclass repeatedly. However, there is still too much memory consumption.

ii. __c() Metafunction

This is the solution which complies with JavaScript syntax. But it doesn't mean that I have to comply with JavsScript syntax strictly. But just think about CoffieScript and JSX, they have a parser to translate them into stand JavaScript. This technology is absolutely feasible in my case, and I only need to call a particular method (MetaFunction) when an unknown method is called. Therefore, as the final solution, before evaluating JavaScript in Objective-C, I translate all the method invocation to __c() function with the help of RegEx and then evaluate the translated script. This looks like the message forwarding in OC/Lua/Ruby.

UIView.alloc().init()
->
UIView.__c('alloc')().__c('init')()

I add a __c property to the prototype of base Object so that any object can access it:

Object.prototype.__c = function(methodName) {
  if (!this.__obj && !this.__clsName) return this[methodName].bind(this);
  var self = this
  return function(){
    var args = Array.prototype.slice.call(arguments)
    return _methodFunc(self.__obj, self.__clsName, methodName, args, self.__isSuper)
  }
}

In method _methodFunc, relative information will be passed to Objective-C, call corresponding method at runtime and give back the result of this function.

In this way, I don't need to iterate over every method of a class and save them in JavaScript object. With this improvement, I reduced 99% memory usage.

Message forwarding

Now, we are going to talk about the communication between JavaScript and Objective-C.In fact, I used the interface declared in JavaScriptCore. On starting JSPatch engine, we will create an instance of JSContext which is used to execute JavaScript code. We can register an Obejctive-C method to JSContext and call it in JavaScript later:

JSContext *context = [[JSContext alloc] init];
context[@"hello"] = ^(NSString *msg) {
    NSLog(@"hello %@", msg);
};
[_context evaluateScript:@"hello('word')"];     //output hello word

JavaScript talks to Objective-C with methods registered in JSContext and get information of Objective-C from the result of method call. In this manner, JavaScriptCore will convert the type of parameters and result automatically. This means NSArray, NSDictionary, NSString, NSNumber, NSBlock will be converted to array/object/string/number/function. This is how _methodFunc method passes class name and method name to Objective-C.

4. Object retaining and converting

Now, you may have known how UIView.alloc() is executed:

• When you call require('UIView'), you will create a global object called UIView
• When you call alloc() method of the object UIView, what you are actually calling is __c() method, in which class name and method will be passed to Objective-C to complete the method invocation.

This is the detail process of invocation of instance method and what about class method? We will receive an instance of UIView after calling UIView.alloc() but how can we represent this instance in JavaScript? How can we call its instance method UIView.alloc().init()?

For an object of type id, JavaScriptCore will pass its pointer to JS. Although it can't be used in JS, it can be given back to Obejctivce-C later. As for the lifecycle of this object, in my opinion, its reference count will increase by 1 when a variable is retained in JavaScript and decrease by 1 when released in JavaScript. If there is no Objective-C object refers to it, its lifecycle deponds on JavaScript context and will be released when garbage collection takes place.

As we talked before, object passed to JS can be given back to OC when calling __call() method. If you want to call a method of the Objective-C object, you can use the object pointer and method name as parameters of __call() method. Now, there is only one question left: How can we know whether the caller is an Objective-C pointer or not.

I have no idea about this and my solution is wrapping the object into a dictionary before passing it to JS:

static NSDictionary *_wrapObj(id obj) {
    return @{@"__obj": obj};
}

The object now is a value inside the dictionary, it is represented as below in JS:

{__obj: [OC Object Pointer]}

In this way, you can know whether an object is Objective-C object easily by checking its __obj property. In __c() method, if this property is not undefined, we can access it and pass to OC, this is how we call instance method.

5. Type converting

After sending class name, method name and method caller to Objective-C, we will use NSInvocation to call corresponding OC method. In this process, there are two thinks to to:

• Get type of parameters of the OC method you want to call and convert the JS value.
• Get the result of method invocation, wrap it into an object and send back to JS

For example, think about the code above view.setAlpha. The parameter on JS side is of type NSNumber, however, with calling OC method NSMethodSignature, we can know the parameter should be a float. So we should call OC method after converting NSNumber to float. In JSPatch, I mainly handle the convertion of number type such as int/float/bool. Besides, I paid some extra attention to some special type like CGRect and CGRange.

Method replacement

In JSPatch, you can use defineClass to replace any method of any class. To support this, I made a lot of effort. At the beginning, I use va_list to get parameters, which turns out to be infeasible in arm64. It also took some time to add new method to a class, implement property and add support to self/super keyword. Here I will introduce them one by one.

1. Basic theory

In OC, every class is actually a struct below:

struct objc_class {
  struct objc_class * isa;
  const char *name;
  ….
  struct objc_method_list **methodLists;
};

The type of elements in methodLists is Method:

typedef struct objc_method *Method;
typedef struct objc_ method {
  SEL method_name;
  char *method_types;
  IMP method_imp;
};

A Method object contains all information of a method, including the name of SEL, types of parameters and return value, the IMP pointer to its real implementation.

When calling a method via its Selector, you are actually find a Method in the methodList. It is a linked list whose elements can be replaced dynamically. You can replace the function pointer(IMP) of a Selector with a new IMP, you can link one IMP with another Selector as well. OC runtime provides some API to do this, as an example, let's replace viewDidLoad of UIViewCOntroller:

static void viewDidLoadIMP (id slf, SEL sel) {
   JSValue *jsFunction = …;
   [jsFunction callWithArguments:nil];
}

Class cls = NSClassFromString(@"UIViewController");
SEL selector = @selector(viewDidLoad);
Method method = class_getInstanceMethod(cls, selector);

//get function pointer to viewDidLoad
IMP imp = method_getImplementation(method)

// get type of parameters of viewDidLoad
char *typeDescription = (char *)method_getTypeEncoding(method);

// add a new method called ORIGViewDidLoad and points to the original viewDidLoad
class_addMethod(cls, @selector(ORIGViewDidLoad), imp, typeDescription);

//viewDidLoad IMP now points to the new IMP
class_replaceMethod(cls, selector, viewDidLoadIMP, typeDescription);

With the code above, we can replace viewDidLoad with a new customized method. Now, if you call viewDidLoad in your app, you will call viewDidLoadIMP, in which you will call a method from JS. This is how we can call method which is written in JS code. Meanwhile, we add a new method called ORIGViewDidLoad which points to the original viewDidLoad, it can be called in JS.

If a method doesn't need any parameter, this is all we need to do to replace a method. However, what if a method needs several parameters, how can we pass the value of parameter to the new IMP? For example, to call viewDidAppear of UIViewController, the caller will specify a BOOL value and we have to get this value in our customized IMP. If we only need to write a new IMP for a single method, it's quite easy:

static void viewDidAppear (id slf, SEL sel, BOOL animated) {
   [function callWithArguments:@(animated)];
}

However, we want a general IMP which can be used as an interchange station for any method with any parameters. In this IMP, we need to get all parameters and call into JS.

2. Use of va_list(32-bit)

At the beginning, I use mutable parameter va_list:

static void commonIMP(id slf, ...)
  va_list args;
  va_start(args, slf);
  NSMutableArray *list = [[NSMutableArray alloc] init];
  NSMethodSignature *methodSignature = [cls instanceMethodSignatureForSelector:selector];
  NSUInteger numberOfArguments = methodSignature.numberOfArguments;
  id obj;
  for (NSUInteger i = 2; i < numberOfArguments; i++) {
      const char *argumentType = [methodSignature getArgumentTypeAtIndex:i];
      switch(argumentType[0]) {
          case 'i':
              obj = @(va_arg(args, int));
              break;
          case 'B':
              obj = @(va_arg(args, BOOL));
              break;
          case 'f':
          case 'd':
              obj = @(va_arg(args, double));
              break;
          …… //Other types
          default: {
              obj = va_arg(args, id);
              break;
          }
      }
      [list addObject:obj];
  }
  va_end(args);
  [function callWithArguments:list];
}

In this case, whatever the number and type of parameters, I can use methods of va_list to get them and put into a NSArray object, which will be passed to JS. It works very well until I run these code and get crashes in my arm64 device. After looking up for some information, it turns out that the architecture of va_list will change in arm64 so that I can't get parameters like this. For more details, please look at this article

3. Use of ForwardInvocatoin(64-bit)

Finally I played a trick and solve this problem. I used the message forward in OC.

When calling a non-exist method, you won't get an exception immediately, but get several chances instead. These method will be called in order: resolveInstanceMethod, forwardingTargetForSelector, methodSignatureForSelector, forwardInvocation. In the last method forwardInvocation, you will create a NSInnovation object which contains all information about the method call, including the name of Selector, parameters and return value. The most important thing is that you can get the value of parameters in NSInvocation. The problem can be solved if we can call forwardInvocation when a method is replaced in JS.

As an example, let's try to replace viewWillAppear of UIViewController to show the details:

1. Use class_replaceMethod to point viewWillAppear to _objc_msgForward. This is a global IMP which will be called when a non-exist method is called. With this replacement, you will actually call forwardInvocation when you call viewWillAppear.
2. Add two method ORIGviewWillAppear and _JPviewWillAppear for UIViewController. The first one is the original implementation and the last one is the new implementation in which we will execute JS code.
3. Replace forwardInvocation of UIViewController with our customized implementation. When the viewWillAppear method is called, forwardInvocation will be called and we can get a NSInvocatoin object which contains the value of parameters. Then you can call the new method JPviewWillAppear with these parameters and call the implementation in JS.

The whole process is illustrated as the flow-chart below:

There is one problem left, as we replaced the -forwardInvocation: of UIViewController, what if a method really needs it? First of all, before replacing -forwardInvocation:, we will create a new method called -ORIGforwardInvocation: to save the original IMP. There will be a judgement in the new -forwardInvocation:: begin method forward if the method is asked to be replaced, otherwise call -ORIGforwardInvocation: and work normally.

4. Add new method

i. Plan

When JSPatch becomes open-source, you can't add methods to a class because I think the ability of replacing existing method is all what we need. You can add new method to JS object and run in JS context. Also, the type of parameters and return value should be figured out if we want to add new method to an OC class because these information is necessary in JS. This is a troublesome but wildly-concerned problem since we can't use target-action pattern without adding new method and I started to find a good way to add methods to add methods. Finally, my solution is that all value is of type id because the methods added are used only in JS(except Protocol) and we won't need to worry about the type if all values are of type id.

Now, defineClass is wrapped in JS, it will create an array with number of parameters and methods themselves, then pass this array to OC. If the method exists in OC, it will be replaced, otherwise class_addMethod() is called to add this method. We can create a new Method object with knowing the number of parameters and set the type to id. If the new method is called in JS, you will finally call forwardInvocation.

ii. Protocol

If a class conforms to some protocol which has an optional method and not all the types of parameters are id, for example, the method below in UITableViewDataSource:

- (NSInteger)tableView:(UITableView *)tableView sectionForSectionIndexTitle:(NSString *)title atIndex:(NSInteger)index;

If this method is not declared in OC but added in JS, all the parameters are of type id, which doesn't match the declaration in protocol and leads to an error.

In this case, you have to specify the protocol you are implementing in JS, so that types of parameters can be known according to the protocol. The syntax looks like OC:

defineClass("JPViewController: UIViewController <UIAlertViewDelegate>", {
  alertView_clickedButtonAtIndex: function(alertView, buttonIndex) {
    console.log('clicked index ' + buttonIndex)
  }
})

It's easy to parse this.First of all, we can get the protocol name then call objc_getProtocol and protocol_copyMethodDescriptionList to get the method from protocol if the method doesn't exist in the class. Otherwise, we can replace the original method.

5. Implementation of property

To use a property defined in OC, just like calling normal method in OC, you can call its set/get method:

//OC
@property (nonatomic) NSString *data;
@property (nonatomic) BOOL *succ;

//JS
self.setSucc(1);
var str = self.data();

You have to blaze a new trail if you want to add a new property to an OC object:

defineClass('JPTableViewController : UITableViewController', {
  dataSource: function() {
    var data = self.getProp('data')
    if (data) return data;
    data = [1,2,3]
    self.setProp_forKey(data, 'data')
    return data;
  }
}

In JSPatch, you can use these two methods -getProp: and -setProp:forKey: to add properties dynamically. Basically, you are calling objc_getAssociatedObject and objc_setAssociatedObject to simulate a property since you can associate an object with current object self and get this object later from self. It works as a property except that its type is id.

Although we can use class-addIvar() to add new properties, but it must be called before the class is registered. It means that this method can be used in JS to add properties but can't used in existing OC class.

6. Keyword: self

defineClass("JPViewController: UIViewController", {
  viewDidLoad: function() {
    var view = self.view()
    ...
  },
}

You can use keyword self in defineClass, just like in OC, it means current object. Wonder how this is possible? Actually, self is a global variable and will be set to current object before calling instance method then back to nil after calling the method. With this little trick, you can use self in instance method.

7. Keyword: super

defineClass("JPViewController: UIViewController", {
  viewDidLoad: function() {
    self.super().viewDidLoad()
  },
}

Super is a keyword in OC which can't be accessed dynamically, so how can we support this keyword in JSPatch? As we all know, in OC, when calling method of super, you are actually calling method of super class and take current object as self. All we need to do is to simulate this process.

FIrst of all, we have to tell OC that we want to call the method of super class, so when calling self.super(), we will create a new object in __c which holds a reference to OC object and has a property __isSuper set to 1:

...
if (methodName == 'super') {
  return function() {
    return {__obj: self.__obj, __clsName: self.__clsName, __isSuper: 1}
  }
}
...

When you call method of this returned object, __c will pass this __isSuper to OC and tell OC to call method of super class. In OC, we will find IMP in super class and create a new method in current class which points to the IMP in superclass. Now, calling the new method means calling method of super class. Finally, we should replace the method called with the new method.

static id callSelector(NSString *className, NSString *selectorName, NSArray *arguments, id instance, BOOL isSuper) {
    ...
    if (isSuper) {
        NSString *superSelectorName = [NSString stringWithFormat:@"SUPER_%@", selectorName];
        SEL superSelector = NSSelectorFromString(superSelectorName);

        Class superCls = [cls superclass];
        Method superMethod = class_getInstanceMethod(superCls, selector);
        IMP superIMP = method_getImplementation(superMethod);

        class_addMethod(cls, superSelector, superIMP, method_getTypeEncoding(superMethod));
        selector = superSelector;
    }
    ...
}

Extension

Support Struct

Struct should be converted when passed between OC and JS. At the beginning, JSPatach can only handle four native structs: NSRange/CGRect/CGSize/CGPoint and other structs can't be passed. Users have to use extension to convert customized structs. It works but can't be added dynamically because these codes must be written in OC in advanced, also it's quite complicated to write these codes. Now, I choose another implementation:

/*
struct JPDemoStruct {
  CGFloat a;
  long b;
  double c;
  BOOL d;
}
*/
require('JPEngine').defineStruct({
  "name": "JPDemoStruct",
  "types": "FldB",
  "keys": ["a", "b", "c", "d"]
})

You can declare a new struct in JS and give a name to it. You also need to specify the name and type of every member. Now this struct can be passed between JS and OC:

//OC
@implementation JPObject
+ (void)passStruct:(JPDemoStruct)s;
+ (JPDemoStruct)returnStruct;
@end

//JS
require('JPObject').passStruct({a:1, b:2, c:4.2, d:1})
var s = require('JPObject').returnStruct();

To support this syntax, I take the value inside the struct in order and wrap it into a NSDictionary with its key. To read each member in order, we can get the length of each member according to its type and copy each value:

for (int i = 0; i < types.count; i ++) {
  size_t size = sizeof(types[i]);  //types[i] is of type float double int etc.
  void *val = malloc(size);
  memcpy(val, structData + position, size);
  position += size;
}

To pass struct from JS to OC works almost the same as above. We only need to allocate the memory(accumulate length of each member) and copy value from JS to this memory section.

This solution works well since we can add a new struct dynamically without declaring it in OC in advance. But this relies strictly on the arrangement of each member in the memory space, and won't work as expected if there is byte alignment in some specific device. Fortunately, I have not encountered such a problem.

Support C function

Functions in C can't be called with reflection so we have to call them manually in JS. In detail, you can create a new method in the JavaScriptCore context whose name is the same as C function. In this method, you can call C function. Take memcpy() as an example:

context[@"memcpy"] = ^(JSValue *des, JSValue *src, size_t n) {
    memcpy(des, src, n);
};

Now you can call memcpy() in JS. In fact, here we get a problem about the conversion of parameters between JS and OC and we just it ignore it temporarily.

We have another two problems:

1. There will be too many source codes if all C functions are written in JSPatch in advance
2. Two many C functions will affect the performance.

Therefore, I chose to use extension to solve these problems. JSPatch will only provide a context and methods to convert parameters, the interface looks like this:

@interface JPExtension : NSObject
+ (void)main:(JSContext *)context;

+ (void *)formatPointerJSToOC:(JSValue *)val;
+ (id)formatPointerOCToJS:(void *)pointer;
+ (id)formatJSToOC:(JSValue *)val;
+ (id)formatOCToJS:(id)obj;

@end

The +main method exposes a context to the external so that you are free to add functions to this context. The other four methods formatXXX are used to convert the parameters. Now, the extension of memcpy() looks like this:

@implementation JPMemory
+ (void)main:(JSContext *)context
{
    context[@"memcpy"] = ^id(JSValue *des, JSValue *src, size_t n) {
        void *ret = memcpy([self formatPointerJSToOC:des], [self formatPointerJSToOC:src], n);
        return [self formatPointerOCToJS:ret];
    };
}
@end

Also, with +addExtensions: method, you can add some extension dynamically when necessary:

require('JPEngine').addExtensions(['JPMemory'])

In fact, you can choose another way to add support to C function: you can wrap it in a OC method:

@implementation JPCFunctions
+ (void)memcpy:(void *)des src:(void *)src n:(size_t)n {
  memcpy(des, src, n);
}
@end

And then in JS:

require('JPFunctions').memcpy_src_n(des, src, n);

In this case, you don't need extension or parameter conversion, but you will use runtime mechanism which is only half as fast as using extension. So, for better performance, I decided to provide an extension.