Android Treble: Blessing or Trouble?

1
Android Treble:
Blessing or Trouble?
YouTubeTM
Livestream
March 27th
2018
Karim Yaghmour
@karimyaghmour
karim.yaghmour@opersys.com

2
These slides are made available to you under a Creative Commons Share-
Alike 3.0 license. The full terms of this license are here:
https://creativecommons.org/licenses/by-sa/3.0/
Attribution requirements and misc., PLEASE READ:
● This slide must remain as-is in this specific location (slide #2), everything
else you are free to change; including the logo :-)
● Use of figures in other documents must feature the below “Originals at”
URL immediately under that figure and the below copyright notice where
appropriate.
● You are free to fill in the “Delivered and/or customized by” space on the
right as you see fit.
● You are FORBIDEN from using the default “About” slide as-is or any of its
contents.
● You are FORBIDEN from using any content provided by 3rd parties without
the EXPLICIT consent from those parties.
(C) Copyright 2018, Opersys inc.
These slides created by: Karim Yaghmour
Originals at: www.opersys.com/community/docs
Delivered and/or customized by

3
About
● Author of:
● Introduced Linux Trace Toolkit in 1999
● Originated Adeos and relayfs (kernel/relay.c)
● Training, Custom Dev, Consulting, ...

4
Agenda
1. Introduction
2. Motivations
3. Enforcement
4. Traditional Android Release Flow
5. Treble's Intended Release Flow
6. Underneath the Framework
7. Linux Kernel
8. HIDL
9. VNDK
10.VINTF
11.VTS
12.Implications
13.Recommendations

5
1. Introduction
● Goals
● Release History
● Architecture
● Distribution scheme before 8.x / Treble
● What's added by Treble?

6
1.1. Goals
● Discuss Project Treble's motivations
● Describe differences with classic release flow
● Detail Project Treble's parts
● Outline implications and recommendations

7
1.2. Release History
● 2008 - Sept.: Android 1.0 is released
● 2009 - Feb.: Android 1.1
● 2009 - Apr.: Android 1.5 / Cupcake
● 2009 - Sept.: Android 1.6 / Donut
● 2009 - Oct.: Android 2.0/2.1 / Eclair
● 2010 - May: Android 2.2 / Froyo
● 2010 - Dec.: Android 2.3 / Gingerbread
● 2011 - Jan : Android 3.0 / Honeycomb – Tablet-optimized
● 2011 – May: Android 3.1 – USB host support
● 2011 – Nov: Android 4.0 / Ice-Cream Sandwich – merge Gingerbread and Honeycomb
● 2012 – Jun: Android 4.1 / Jelly Bean – Platform Optimization
● 2012 – Nov: Android 4.2 / Jelly Bean - Multi-user support
● 2013 -- July: Android 4.3 / Jelly Bean -- BLE / GLES 3.0
● 2013 -- Oct: Android 4.4 / KitKat -- Low RAM optimizations
● 2014 -- Nov: Android 5.0 / Lollipop - ART and other pixie dust
● 2015 -- Mar: Android 5.1 / Lollipop - Multi-SIM card
● 2015 -- Oct: Android 6.0 / Marshmallow -- New permission model
● 2016 – Aug: Android 7.0 / Nougat – Multi-window
● 2016 – Oct: Android 7.1 / Nougat – A/B updates
● 2017 – Aug: Android 8.0 / Oreo – Treble / PIP
● 2017 – Dec: Android 8.1 / Oreo – Low-end device optimizations

9
1.4. Distribution scheme before 8.x
● Google releases new version of Android
● Ecosystem customizes it to new devices –
sometimes quite extensively
● Google enforces Compatibility Test Suite (CTS) –
guaranteeing developer API conformance
● Devices get certified and ship with GMS
● ....
● Devices rarely got upgrades

10
2. Motivations
● Updates: Android vs. iOS
● Android version propagation history
● “Official” Google Treble pitch
● Nature of Android

11
2.1. Updates: Android vs. iOS
Data from Apple as of January 18, 2018
https://developer.apple.com/support/app-store/
iOS Android
Data from Google as of February 5, 2018
https://developer.android.com/about/dashboards/
2.3.x Gingerbread
4.0.x Ice Cream Sandwich
4.1.x Jelly Bean
4.2.x Jelly Bean
4.3.x Jelly Bean
4.4.x KitKat
5.0.x Lollipop
5.1.x Lollipop
6.x Marshmallow
7.0.x Nougat
7.1.x Nougat
8.0.x. Oreo
8.1.x. Oreo
Earlier
IOS 10
IOS 11

12
2.2. Version propagation history
●
Implications:
●
Android comes out with cool new features
●
Perception is iOS gets them first
● Disclaimer: This is my reading
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
D
ec
2009
Feb
2010
Apr 2010
Jun
2010
Aug
2010
O
ct 2010
D
ec
2010
Feb
2011
Apr 2011
Jun
2011
Aug
2011
O
ct 2011
D
ec
2011
Feb
2012
Apr 2012
Jun
2012
Aug
2012
O
ct 2012
D
ec
2012
Feb
2013
Apr 2013
Jun
2013
Aug
2013
D
ec
2013
Feb
2014
Apr 2014
O
ct 2013
Jun
2014
Aug
2014
D
ec
2014
Feb
2015
O
ct 2014*
*nodata
Apr 2015
Jun
2015
Aug
2015
*nodata
Jul 2015*
O
ct 2015
D
ec
2015
Feb
2016
Apr 2016
Jun
2016
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Aug
2016
O
ct 2016*
D
ec
2016
*nodata
Feb
2017
Apr 2017
Jun
2017
Android version
1.1
1.5
1.6
2.0
2.0.1
2.1
2.2
2.3-2.3.2
2.3.3-2.3.7
3.0
3.1
3.2
4.0-4.0.2
4.0.3-4.0.4
4.1
4.2
4.3
4.4
Cupcake
Donut
Eclair
Froyo
Gingerbread
Honeycomb
Ice Cream
Sandwich
Jelly Bean
KitKat
Gingerbread
Eclair
Eclair
Honeycomb
Honeycomb
Jelly Bean
Jelly Bean
5.0 Lollipop
Ice Cream
Sandwich
5.1 Lollipop
6.0 Marshmallow
7.0-7.1 Nougat

13
2.3. “Official” Google Treble pitch
● From source.android.com:
“The Android 8.0 release includes Project Treble, a major re-
architect of the Android OS framework designed to make it
easier, faster, and less costly for manufacturers to update
devices to a new version of Android.”
● In short:
● Recognition that updates are a problem
● Identification of update issues:
– Difficult
– Long
– Costly

14
2.4. Nature of Android
● Anyone is free to download Android
● Google enforces certification for GMS
● Google has no ability to force upgrades
● Google can only incentivize

15
3. Enforcement
● “To ensure forward compatibility of vendor implementations, the
new vendor interface is validated by the Vendor Test Suite (VTS),
which is analogous to the Compatibility Test Suite (CTS).”
● In sum:
● Path to upgrade has been charted
● But ... Comply or we won't certify
● Not much choice – Android is the “only game in town” aside from
Apple's iOS.
● Worthy end-goal:
● 4 to 5 years of upgrades on a single device

16
4. Traditional Android Release Flow
● Release flow
● Simplified internals view
● Example phone circa early 2014
● Upgrade after 5.x/Lollipop release
● Upgrade after 6.x/Marshmallow release

18
4.2. Simplified Internals View

19
4.3. Example Phone Circa Early 2014
● Just for the sake of example ...
● Say you bought a phone in early 2014:
● Samsung device
● Qualcomm chipset
● Verizon network
● Runs Android 4.4/KitKat at time of purchase
Note: brands/companies used for illustration purposes only, no specific device is implied.

20
4.4. Upgrade after 5.x/Lollipop release
5.x Release
QualcommQualcomm
Are they still
selling this chip? 5.x BSP
Samsung
Are they still
selling this phone or
is it important?
5.x Upgrade
Verizon
Are customers
still buying this?5.x Gets to Users
November 2014

21
4.5. Upgrade to 6.x/Marshmallow?
October 2015
6.x/Marshmallow
Qualcomm
Really? ???
Samsung
Really, Really??
???
Verizon
Really, Really, Really??!?!?!

22
5. Treble's Intended Release Flow
● Initial release flow
● New internals paradigm
● Overall reworked internals
● Targeted upgrade flow
● Theory of future release cycles
● Requirement

23
5.1. Initial Release Flow
8.x Release
QualcommQualcomm
8.x BSP
Samsung
8.x-based SKU
Verizon
8.x-based Device

24
5.2. New Internals Paradigm

25
5.3. Overall Reworked Internals

27
5.5. Theory of Future Release Cycles
9.x Release
10.x Release
...

28
5.6. Requirement
● Google must continue to support old vendor
interfaces.
● For an initial 8.x device, future framework release
must support 8.x hardware enablement:
● 9.x / P
● 10.x / Q
●
11.x / R
●
...
● Aim seems to be 4-5 releases

29
6. Underneath the Framework
● Architecture
● Integration points
● Certification before Treble
● Certification after Treble
● What's added by Treble?

32
6.3. Certification Before Treble
Compatibility Test Suite

33
6.4. Certification After Treble
Compatibility Test Suite
SE
Vendor Test Suite
Vendor Test Suite

34
6.5. What's added by Treble?
● Linux kernel requirements
● HIDL – Hardware Interface Definition Language
● VNDK – Vendor NDK
● VINTF – Vendor Interface
● VTS – Vendor Test Suite

36
7. Linux Kernel
● Basics
● Release flow
● Real-life
● Treble and the kernel
● Links

37
7.1. Basics
● Necessary for Android
● Existed prior to Android
● Decision to use based on Linux succcess
● Google downstream from kernel developers

38
7.2. Release Flow
0
Linux Kernel
Release

39
7.3. Real-Life
● End-of-Life kernels very common on Android
● Security issues
● While invisible, major implications for user
● Inconsistent kernel versions between vendors
● Inconsistent updates between vendors

40
7.4. Treble and the Kernel
● Separate deliverables based on each player's role in the update
ecosystem.
● VTS-enforced aspects
● Basic rules for:
● Kernel versions – use of LTS
● Basic config
● System call ABI/API
● FS + FS features
● LTS 4.4 supported for 6 years instead of 2
● Kernel customization through:
● Kernel config overrides
● Loadable modules
● DT overlays

41
7.5. Links
● Main doc from Google:
● https://source.android.com/devices/architecture/kernel/
● LTS:
● https://www.kernel.org/category/releases.html

42
8. HIDL
● HAL's role
● Traditional HAL before 8.x
● HIDL
● Links

43
8.1. HAL's Role
● Per-device-type hardware abstraction
● Example:
● SurfaceFlinger uses hwcomposer HAL
● Location uses gps HAL
● Lights uses lights HAL
● Etc.
● Google specifies HAL signature
● Manufacturer/SoC vendor provides HAL implementation
● Reference implementations:
● Leads devices found in AOSP
● SoC vendor reference designs/boards in BSP

44
8.2. Traditional HAL Before 8.x
● Google specifies HALs as “C” header files
● HAL module author uses header in implementation
● Resulting binary shipped as part of release
● Modules loaded at boot time by system services
● Headers could (and did) change between versions
● Required reworking, rebuilding, reshipping new
version
● To update to a new version of Android:
● All modules had to be updated, be they trivial or difficult

45
8.3. HIDL
● “Hardware Interface Definition Language”
● New layer under system services
● Formalized and versioned HAL interface definitions
●
Similar to AIDL, yet different
●
Example HIDLs in 8.x/Oreo:
● graphics/composer 2.1
● gnss 1.0
● Etc.
●
Once published, a given HIDL definition is immutable:
● Even in 9.x/P, graphics/composer 2.1 and gnss 1.0 will be the same as in 8.x.
● Fresh port assumes using latest HAL sig available
● Most importantly:
● Modules created against a given signature should continue to work so long as that
signature is supported.
● Depends on Google, but incentives are aligned
● More on this in tomorrow's presentation

46
8.4. Links
● General doc:
● https://source.android.com/devices/architecture/hidl/
● C++
● https://source.android.com/devices/architecture/hidl-
cpp/
● Java
● https://source.android.com/devices/architecture/hidl-
java/

47
9. VNDK
● Before
● What VNDK does
● Links

48
9.1. Before
● “Anything goes” under the hood
● Sometimes taken to extremes
● Ex. major handset manufacturer w/ hot new device:
●
Expecting:
– Trade press scrutiny
– Benchmarks
● So ... tweak everything for performance and battery:
– Libraries
– Core components
– Anything and everything under the sun
● Result: Significant delta between canonical AOSP and real
code/binaries shipping in devices.
● Therefore: Difficult to upgrade

49
9.2. What VNDK Does
● Provides a pre-defined way to mitigate/manage
differences between Google-minted libs and vendor libs.
● Quite a few concepts introduced by VNDK to define
library categories and explain how they are to be used.
● Boring/Tedious explanation (very dry)
● Refer to official doc
● Will need to assign this to someone on your team
● Google is trying to play funambulist on this

50
9.3. Links
● Main doc:
● https://source.android.com/devices/architecture/vndk/

51
10. VINTF
● Basics
● Architecture
● Details
● Links

52
10.1. Basics
● Need to formalize version “handshake” between
device and OTA service on upgrade.
● Specifically, need to match:
● HIDL versions
● Kernel specifics
● SE policies
● Android Verified Boot (AVB) version
● Solution:
● Vendor Interface Object (VINTF)

54
10.3. Details
● Device manifest:
● Is device's HW enablement supported by framework?
● Framework manifest:
● Does new framework provide what the device needs?
● Device compatibility matrix:
● What does device expect of new framework?
● Framework compatibility matrix:
● What does new framework expect of device?
● Google doc specifies:
● Manifest format
● Compatibility matrices
● Matching rules
● Need a good understanding as well
● Likely needs a person to be responsible for this as well

55
10.4. Links
● Main doc:
● https://source.android.com/devices/architecture/vintf/

56
11. VTS
● Similar to CTS
● Ensures that all the above is followed
● Test categories:
● HIDL HAL Tests
● Kernel Tests
● LTP
● Linux Kselftest
● VNDK (Vendor Native Development Kit) Tests
● Performance Tests
● Fuzz Tests
● Security Tests
● Even w/o certification, passing CTS will simplify OTA

57
● Links:
● Main doc:
– https://source.android.com/compatibility/vts/
● More details (user manual, etc.):
– https://android.googlesource.com/platform/test/vts/
+/master/README.md

58
12. Implications
●
Significant reduction in customization for GMS
●
Non-exhaustive no-go list:
● System service internals
● HAL definition reworking/extension
● Core library replacement by tweaking dependent components' APIs
● Tweaking existing SE policies
●
Any change that impacts OTA
● Any change that breaks VTS
●
Even permitted changes are complex, ex:
● Tweaking libraries vs. VNDK rules
● Using vendor-specific services instead of tweaking existing ones

59
● Big benefit in adhering to “vanilla” even if no GMS interest:
● Simplified OTA
● If no GMS, still as open as before to tweak everything:
● Price = reduced OTA capability, likely no path to ugprade, etc.
● Greater scrutiny required over SoC/partner work to ensure Treble
“conformance”.
● Incentive is there for SoC vendors since some of their customers
need GMS certification.
● Varying degrees of quality from board vendors and outsourcing
partners:
● Do your homework

60
13. Recommendations
● Several years before Treble benefits materialize
● Solid ground established by Google
● Likely some organic learning left to iron kinks out
● In the mean time:
● Make sure you have a firm understanding of how Treble changes your Android device
design, development, integration and support plans.
● Have your technical teams review the Treble documentation made available by
Google and possibly seek training so they can start familiarizing themselves with its
intricacies.
● If certification is on your agenda, start paying special attention to the VTS in your
planning. If you have CTS as part of your checklists, a good start is adding
equivalent entries for VTS.
● When selecting partners or suppliers, make sure their work and/or the software they
deliver falls in line with Treble's requirements, especially if certification is a target.

61
Thank you ...
karim.yaghmour@opersys.com

62
Acknowledgements:
● Several diagrams distributed by Google under
CC-BY-SA license or ASL 2.0.

Android Treble: Blessing or Trouble?

More Related Content

What's hot (20)

Similar to Android Treble: Blessing or Trouble? (20)

More from Opersys inc. (20)

Recently uploaded (20)

Android Treble: Blessing or Trouble?