Apple CoreML: Introduction to Machine Learning in Mobile App Development
Nowadays, the progress doesn’t stand still and nearly every day new technologies are being developed, including the most sophisticated, such as machine learning. Most of the people take it for granted and don’t try to look behind the scenes to understand how these technologies work. However, this doesn’t relate to us.
Having read our article, you will find out what machine learning is and understand the way it changed our life. Specifically, we’ll tell you about an amazing technology , which uses machine learning: CoreML.
Almost every current application is based on machine learning. Our intelligent assistants are capable of identifying our voices, finding answers to our questions, and anticipating our actions. All of these would have been possible without CoreML.
CoreML is a framework that was presented by Apple in 2017. CoreML also allows simplifying the integration of machine learning models in applications ecosystem. Based on CoreML were implemented various products of the company including Siri, QuickType, Camera and so on. CoreML contributed to developing “clever” features. All these features are based on machine learning and don’t use any 3d parties libraries.
This framework is based on the native technologies, such as Accelerate and Metal. Since it allows giving full play to the hardware of a device, performance becomes better and battery’s life longer.
What are the reasons behind CoreML development?
- data protection;
- traffic economy;
- no need to use servers;
This technology allows iOS 11 apps to run machine learning models locally on your device. Basically, this allows using such features as images recognition, analysis of the text/sounds without using any Internet connection. As a result, this allows conserving traffic, battery life, and more importantly time.
Thanks to CoreML, we’ve implemented the following functionality:
real time images recognition;
predictive text input;
pattern recognition;
sentiment analysis;
handwriting recognition;
search ranking;
texts translation;
emotions recognition;
image styling;
facial recognition;
voice identification;
music identification;
text annotation;
We only scratch the surface and there’s a great number of other features that can be easily implemented using machine learning.
CoreML model is an open data model that allows converting models from such popular machine learning systems as TensorFlow, Onxx, and so on. As a result, iOS developers get simple and usable Swift code.
Let’s take a look at the example of real-time objects detection. For this purpose, we’ll be using ARKit and Vision libraries.
First of all, let’s take a trained model with an extension *.mlmodel and add it into the project.
Now let’s add all the necessary libraries into the ViewController, where we’ll be working.
We use SpriteKit for rendering the caption over the images output from the camera.
A working controller should realize delegate methods for working with AR. We need to mention it in the declaration as well as at the loading stage.
Let’s run a session and pass the camera frames, which were received from ARKit, to Vision.
Now, make image serialization in order to improve the performance in real-time.
The next step is to display all the data, which we got from AR, on the screen.
Basically, our application is ready to go now. The source code can be found here.
In 2018, was presented the logical continuation to the machine learning framework, that was called CoreML 2.
The module sizes were scaled down the module sizes using quantum scales. Additionally, was improved the performance and added new customization possibilities.
Another interesting feature is Natural Language Framework (a supplement to the CoreML). This feature allows carrying out a lexical analysis of the text. It also looks for the names into the text. Besides, you can use your own models of the text analysis. For example, it could identify a context or mood of a text as well as part of speech and so on.
Here’s an example of using NLP for managing the tabs.
Here’s an example of identifying the text mood.
Furthermore, there’s a framework CreateML that allows training your own models managed by MacOS Mojave. This allows developers to train models without studying or using any third-party features.
To start training, you need to add data into the project and just write a few lines. As a result, you’ll get a final model that can be used on any devices of the ecosystem.
Let us consider the example of the model training process using CreateML (basing on the animals classifier).
Here you can see a scheme that shows the way our app should work.
We need to select an image with an animal we want to identify. Hereafter, we need to send it to our model, which will specify the animal species.
If we want our model to identify the animals as accurate as possible, we need to prepare some samples (at least 10 images). All the images should be grouped according to the animals species. Don’t train certain types with a large amount of data.
Now let’s create our own application that will be trained. Open xCode and create an empty MacOS Playground and add there the following code:
Open the panel with an assistant editor, run playground and you’ll see a panel for choosing the images for the training. Drag all the catalogues with images and your model will start training.
You can see the status after the training.
Our training came to the end and now we can use it in our application.
let model = try VNCoreMLModel(for: AnimalClassifier().model)
Before we end our post, we need to mention an alternative of CoreML. This solution was presented by Google, which is known as MLKit.
MLKit is a framework that presents a high-level solution for cross-platform work with a machine learning. MLKit is a part of Firebase that can be used both on Android and iOS platforms. All the trained models can be used on a mobile devices and use cloud computing for improving accuracy.
Additionally, MLKit allows using the same models on both platforms. By the way, all the models will be updated without the necessity to recompile the application.
If we take a look at Xamarlabs test as well as any other non-synthetic tests, we’ll see its pros and cons.
Pros:
- it can be used both on iOS and Android platforms;
- there’s only one platform both for iOS and Android platforms, including the cloud one.
Cons:
- the necessity to add third-party libraries into the project;
- Lower speed comparing to CoreML;
- there’s no possibility to use in TVOs, MacOS, WatchOS.
The Bottom Line
Machine learning continues to gain momentum and finds its application nearly in every field of our life. Nobody knows what we can expect in the near future.
One thing’s for certain, machine learning is a promising technology that is capable of changing our experience in the technological world (actually, machine learning already changed our life). Nevertheless, developers need to perceive the information in a way the computers do in order to make the most of it. Such an approach will eventually lead to an even more sophisticated level of machine learning that will contribute to the creation of new technologies.
Remember, we are always here for you, ready to tell you about the most revolutionary, sophisticated, and enthusiastic technology of our present day.