89

I want to rebuild the Android SDK (or rather only the android.jar) to include hidden and internal APIs.

I could not find any documentation or discussion doing on how to go about this. I have an Ubuntu CyanogenMod build environment already setup that is able to build cm7.

Now, I read that make SDK will build the SDK but I want to build an SDK that includes methods and fields that are marked as hidden using @hide. Is this possible?

What I want to do is make changes to an application that uses hidden API and in order to rebuild it I would like to use the modified SDK.

Bjarke Freund-Hansen
  • 28,728
  • 25
  • 92
  • 135
Thomas Hofmann
  • 1,046
  • 1
  • 8
  • 7
  • 2
    @Hidden just hides the javadoc, all these methods are still available – Blundell Oct 25 '11 at 11:36
  • 12
    @hide removes them from the class files. – Thomas Hofmann Oct 25 '11 at 12:17
  • 1
    Linked: http://stackoverflow.com/questions/4951146/is-it-possible-to-use-android-sdk-methods-with-hide-annotation – Blundell Oct 25 '11 at 12:19
  • I know that I could use reflection but I want to change an existing application that uses hidden API without refelction and I don't want to change all exsiting code to use refelction. – Thomas Hofmann Oct 25 '11 at 12:21
  • Thomas I've shown you the resource to explain it in my Answer, I'm just linking more information for other people coming across your question. Also have you tried extending the class and overriding the method but don't use the @override annotation – Blundell Oct 25 '11 at 12:52
  • Sorry, I didn't notice that you also provided the answer (shame on me). I will have a look at the video later this evening. Thanks so far. – Thomas Hofmann Oct 25 '11 at 13:24
  • +1, I also want to know this. None of the answers here actually answers the question, i.e. how to build the android sdk including the internal/hidden API. – Bjarke Freund-Hansen Mar 01 '12 at 14:04
  • 4
    I think you can remove the `@Hidden` label of the API you want to access, then execute `make update-api` and `make SDK` to build your own SDK. – dreamtale Mar 02 '12 at 06:24
  • even if you will access the hidden api without reflection you still need to sign your apk with system key to grant permissions. Otherwise your app will break saying permission denied. – Sunny Mar 07 '12 at 09:54
  • Theoretically you could write a script that will parse all files in the SDK and will delete all @hide labels. Of course it will take some time. And then rebuild SDK. – YanDaik Jan 17 '14 at 08:20
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:12

10 Answers10

70

This is what I always do to use hidden api.

  1. Build the repo or download jars from https://sites.google.com/site/hippunosource/home/android/androidnohide-apiwo-shi-yongsuru-rifurekushonha-wei-shi-yong
  2. copy out out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar (better to rename it as something like framework_all.jar)
  3. config your project build path-->libraries --> add this external jars. In Order and Export, move it up and before android.jar
Conn
  • 373
  • 3
  • 10
Long
  • 719
  • 5
  • 2
  • This worked for me too! now I can get on with the actual task, thanks for this – CurlyPaul Oct 11 '13 at 11:42
  • IMHO, This is the easiest and the fastest. – Patrick Cho Feb 05 '14 at 23:11
  • Isn't there a way to mark this as the proper answer? – Chris Browet Jun 24 '14 at 12:15
  • 2
    does this method work with all the devices, or does it work only with the targeted device? – Hải Phong Nov 15 '14 at 02:11
  • To define library order in android studio, you need to alter the module's `.iml` file and bring your desired lib's `` before **Android SDK**. Unfortunately this technique is not persistent as the file will be overwritten once the gradle-sync button is pressed. – waqaslam Nov 14 '19 at 14:38
  • I manaded to get it working in Android Studio this way: dependencies { compileOnly fileTree(dir: sytem_'libs', include: ['*.jar']) ... } – Andrey Egorov Jun 18 '20 at 07:05
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar @Long – Bunny Aug 12 '20 at 11:19
45

I have done some investigating into this, and my conclusion is simply: This cannot be done without quite a bit of work. Read the rest of this answer for details on what I have found.

android.jar is actually comprised of the "public api" of framework.jar and core.jar which is found in system/frameworks/ on the device. android.jar is a kind of what I would call Java library header, all implementation in the actual byte code are just a throw new RuntimeException("stub");, this allows you to build against android.jar (e.g. in Eclipse), but execution has to be performed on a device or emulator.

The public API of the Android SDK is defined by classes/methods/fields that are not prefixed with the @{hide} javadoc annotation. I.e. everything that is not annotated is included in the SDK.

android.jar is built from the sources located in out/target/common/obj/JAVA_LIBRARIES/android_stubs_current_intermediates which itself is generated by the tool DroidDoc located in build/tools/droiddoc.

DroidDoc is the tool (probably adapted from javadoc, or using javadoc) that generate the actual Android SDK documentation. As a side-effect, and probably because it is already parsing all the javadoc, it also spews out the android stubs which are then compiled into the android.jar which is distributed in the SDK.

So to include the stuff that is hidden you could, if you only want to include specific parts, just remove the @hide annotation and rebuild the SDK.

However if you want to include all the hidden parts things get a lot more complicated. You can modify DroidDoc (the relevant source is in build/tools/droiddoc/src/Stubs.java) such that nothing is detected as hidden. This is quite trivial and I have tried this, however the stubs that is then generated does not compile at all.

My conclusion by now is that this is simply not feasible. The stubs generated if you remove the part of DroidDoc that detect hidden annotations, is simply not compilable, and would require quite a bit of work to compile correctly.

So my answer to your questions is: No, this cannot be done, without doing a lot of work. Sorry.

A side note about the mkstubs tool. mkstubs are used when you build a SDK addon, i.e. the addons you can find in the Android SDK manager from vendors, e.g. Samsung providing you with an additional API for stuff specific to Samsung phones. mkstubs does much the same as the DroidDoc stubs generation process, however it does not use @hide annotations, it uses a .defs file describing which packages/classes/fields to include or exclude from your SDK addon.

However this is all irrelevant to the question, as the Android SDK build does not use the mkstubs tool. (Unfortunately.)

Bjarke Freund-Hansen
  • 28,728
  • 25
  • 92
  • 135
  • I had a look too. Besides Droiddoc there is a tool called mkstubs in /development/tools/mkstubs. It get called during the build and as far as I have seen it will modify the class files stubbing stuff out of them. In build/core/tasks/sdk-addon.mk there is the following code: define stub-addon-jar $(call stub-addon-jar-file,$(1)): $(1) | mkstubs $(info Stubbing addon jar using $(PRODUCT_SDK_ADDON_STUB_DEFS)) $(hide) java -jar $(call module-installed-files,mkstubs) $(if $(hide),,--v) \ "$$<" "$$@" @$(PRODUCT_SDK_ADDON_STUB_DEFS) endef – Thomas Hofmann Mar 05 '12 at 15:04
  • Unfortunately, I am not into this stuff but too me it looks like it all depends on a variable called hide. I havn't found were it gets set though. If you search for $(hide) in the other build files you will see that a lot depends on this value. It seems to also affect the way C libs are built. – Thomas Hofmann Mar 05 '12 at 15:04
  • @ThomasHofmann: I initially got confused by the mkstubs tool also. What I have learned is that mkstubs only gets used when you are building a (vendor) sdk addon, not when you just build the normal sdk. However, mkstubs does much the same as DroidDoc, except it does not use `@hide` annotations, it "just" uses a `.defs` file describing which packages/classes/fields to be included in the API of the addon. – Bjarke Freund-Hansen Mar 06 '12 at 07:39
  • 2
    @ThomasHofmann: About `$(hide)`, you are confusing yourself. `$(hide)` is simply a prefix in the makefiles to hide the actual command-line of the program being executed, nothing more, and it is used just about everywhere everywhere. **It has nothing whatsoever to do with the Android SDK or the `@hide` annotation in the source code.** – Bjarke Freund-Hansen Mar 06 '12 at 07:55
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:09
31

We could reconstruct the *.jar files from the Android platform.

First, connect ADB to your device. Then run:

adb pull /system/framework/core.jar .
adb pull /system/framework/framework.jar .

The core.jar contain the standard Java libraries (java.*) and the framework.jar contain the Android libraries (android.*). This is not usable yet, as the actual files are in DEX format, not JAR format.

We could convert these DEX-formatted *.jars into real JARs using tools such as dex2jar:

dex2jar core.jar
dex2jar framework.jar

Then pull in these jars using "Add External JARs..." (assuming you're using Eclipse ADT)

  • right click on Project → Properties → Java Build Path → Libraries → Add External JARs... → (Choose the core-dex2jar.jar and framework-dex2jar.jar from above).

This will enable you to use the internal and some Java 7 APIs. (The generated APK, as far as I can see, does not contain any actual code from the JARs.)

Shashank Agrawal
  • 25,161
  • 11
  • 89
  • 121
kennytm
  • 510,854
  • 105
  • 1,084
  • 1,005
  • Thank you so much, i've tried many many ways only your methods works for me. – SalutonMondo Jan 08 '14 at 05:04
  • 7
    It's important to note that this method still works with ICS and later systems, but requires some more juggling. The relevant files are `/system/framework/core.odex`, `/system/framework/framework.odex`, and probably more. These can be deodexed (`java -jar baksmali-2.0.3.jar -d system.framework -x system.framework/core.odex -o core`) and reodexed (`java -jar smali-2.0.3.jar -x -o core.dex core`), and only after that `dex2jar core.dex` does its job. – Alex Cohn Feb 18 '14 at 17:58
  • unable to find core.jar in marshmallow – mehmet6parmak Dec 29 '15 at 08:53
  • do you have any idea for core & framework jar in android p ? – Prabhakaran Sep 05 '19 at 07:35
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:09
  • i dont think these will work on normal device we will need root access or aosp device for framework folder to be there in filesystem – babbin tandukar Nov 10 '20 at 09:52
21

You can download the modified android.jar to be used as hidden APIs from this repository. Follow the instructions there.

Anggrayudi H
  • 14,977
  • 11
  • 54
  • 87
  • 4
    This answer should be upvoted more as it is the simplest solution. People, if you are looking for a solution - know that other people already did that and placed a solution to that github repo. Use it and enjoy! )) – Mixaz Dec 06 '16 at 22:24
  • 1
    While using android.jar for API 28 I am getting an error with robolectric. Raised an issue https://github.com/anggrayudi/android-hidden-api/issues/62 and thank you for your jar @Anggrayudi – Prabhakaran Sep 05 '19 at 07:34
  • i also did the same for android 10 it doesnt work for me rather i downloaded from other contributor https://github.com/aeab13/android-jar-with-hidden-api. i was able to access hdmi-cec class and build it but codes are still confusing .code link https://android.googlesource.com/platform/frameworks/base/+/4e90fcd/core/java/android/hardware/hdmi – babbin tandukar Nov 10 '20 at 09:54
18

For Lollipop the flow is little different:

  1. Get /system/framework/arm/boot.oat from lollipop device

  2. Use 'java -jar oat2dex.jar boot boot.oat'

  3. You will get two folders: dex and odex. Go to dex and make 'java -jar dex2jar.jar framework.dex'
  4. Rename resulting framework.jar to .zip, extract and find classes you need
  5. Go to [sdk_path]/platforms/[target_platform] and extract android.jar (first rename it to zip).
  6. Copy files from extracted framework to extracted android.jar. Then compress to zip and rename to .jar :)

ps: probably you need repeat steps 4-6 for 'framework_classes2.dex'

Rishikesh pathak
  • 423
  • 4
  • 18
deviant
  • 3,539
  • 4
  • 32
  • 47
  • For step 3, I can't find the dex2jar.jar at that link.... I've tried a lot of things and I can't figure it out. Is there a link to that somewhere? Is it located in my Android SDK? I can't find it. – Dwebtron Nov 13 '15 at 22:42
  • Yeah, it goes to a github project, and maybe it's me, but I can't find ANY file ending in ".jar" there... – Dwebtron Nov 15 '15 at 16:11
  • 1
    see 'releases' section – deviant Nov 16 '15 at 09:14
  • I've looked there, and unless I'm missing something there's plenty of files that end in '.jar' but none of them are 'dex2jar.jar' like you mention in your comment. When I try to run ANY of those files (as per your comment), I get an error that says "no main manifest attribute, in '. The instructions on that git page say to build something that ends in.apk.... And I've tried that, but it #1 results in a file that ends in .apk (NOT .jar) and #2 doesn't work properly. Am I seeing/doing something wrong? I'm kind of new at this.... – Dwebtron Nov 16 '15 at 14:58
  • By the way, I thank you for all of your help so far, it's been incredibly helpful. – Dwebtron Nov 16 '15 at 15:31
  • 2
    So it looks like the recent versions of dex2jar have changed the format of the download. Just unzip the download and instead of 'java -jar ...' just run the script 'd2j-dex2jar.sh' or 'd2j-dex2jar.bat', depending on your platform, directly on the framework.dex file – CalumMcCall Dec 02 '15 at 15:25
  • 1
    i have copied both jar-files into the android.jar, now android studio tells me Error:Execution failed for task ':app:processDebugResources'. > com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command 'D:\Programme\Android\android-sdk\build-tools\19.1.0\aapt.exe'' finished with non-zero exit value 1 – wutzebaer Jan 05 '16 at 21:18
  • @wutzebaer i am also facing the same issue. Have you resolved it ? – Gautam Mar 01 '16 at 12:33
  • I am getting this error when running java -jar... 01-10 14:46:24:330 Invalid dex magic 01-10 14:46:24:330 Invalid dex version Exception in thread "main" java.lang.IllegalArgumentException at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:597) at org.rh.smaliex.reader.Oat$DexFile.saveTo(Oat.java:229) at org.rh.smaliex.OatUtil.extractOdexFromOat(OatUtil.java:219) at org.rh.smaliex.OatUtil.prepareOdex(OatUtil.java:137) at org.rh.smaliex.OatUtil.bootOat2Dex(OatUtil.java:124) at org.rh.smaliex.Main.mainImpl(Main.java:101) at org.rh.smaliex.Main.main(Main.java:42) – Vishnu Jan 10 '18 at 09:16
14

DroidCon 2011

Here Erik Hellman from Sony Ericson explains how to access the hidden Android API's:

http://vimeo.com/30180393 (Hmm link doesn't appear to work).

Goto the DroidCon webpage Day 2 scroll down to Using Hidden APIs 10:15 and you can watch it there.

Links are dieing!

I've found this one: http://skillsmatter.com/podcast/os-mobile-server/hidden-api I don't know, how long it'll be up.

The official APIs in the Android SDK is usually sufficient for most normal applications. However, there are sometimes situations where a developer needs access to the internal system services, APIs and resources that are not published in the official APIs. Fortunately, these APIs are still available through some clever tricks and can often be useful when developing new and innovative solution on top of Android. In this session you will learn how to access and use these hidden and protected APIs, the limitations of their usage and some tips'n'trick on how to use them in a safe and control manner across multiple vendors devices and Android versions. The audience will see several advanced demos that you normally cannot do with Android. Expect a fairly advanced session with lots of insights in the internals of the Android platform.

peterh
  • 11,875
  • 18
  • 85
  • 108
Blundell
  • 75,855
  • 30
  • 208
  • 233
  • 1
    It's interesting but unfortunately it does not answer my question. I was asking for a way to actually rebuild the SDK with hidden stuff included. – Thomas Hofmann Oct 25 '11 at 21:25
  • It looks like I found another way to achieve what I want. I will describe it tomorrow. – Thomas Hofmann Oct 25 '11 at 22:22
  • I found out that if you want to compile the source of a project that uses hidden API in ADT you can do the following: 1) Create an Android project for the source. 2) Remove the Android classpath container from the build path 3) Define a user library (also check the system library checkbox) that includes JAR files from an ASOP ROM build, e.g. cm7). Which JAR files you use depends on what you need to reference. framework-immediates will probably be part of it. – Thomas Hofmann Oct 27 '11 at 14:23
  • 4) When the project is now built the user library classes will not be included in the APK being created. Hidden API is visible and everything will compile just fine. – Thomas Hofmann Oct 27 '11 at 14:27
  • @Blundell : Can you please update the links.. they are dead ! – zombie Jun 18 '12 at 15:25
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:11
12

Try to look at this:

The ultimate target of these articles is to give developers the power of Internal and Hidden APIs without using reflection. If you complete all the steps described in next several parts you will be able to use Internal and Hidden APIs as if they were public open APIs. There will be no need for reflection.

But if you’re using these non-public APIs then you should be aware that your application is at great risk. Basically there are no guarantees that APIs will not be broken with next update to Android OS. There are even no guarantees about consistent behavior across devices from different vendors. You are completely on your own.

There are three scenarios you may want to follow:

  1. Enable both internal and hidden APIs (scenario A)
  2. Enable only hidden API (scenario B)
  3. Enable only internal API (scenario C)

Scenario A is a sum of B and C. Scenario B is the easiest one (requires no eclipse ADT plugin modifications).

Scenario A: read parts 1, 2, 3, 4, 5

Scenario B: read parts 1, 2, 3, 5

Scenario C: read parts 1, 2, 3, 4, 5

gnat
  • 6,213
  • 108
  • 53
  • 73
Zorb
  • 726
  • 11
  • 24
  • 3
    I already read that blog entry. It mentions this: "1) Android is an open source project. We can download the source code and customize build system so it does not exclude internal & hidden classes from android.jar. This is a hard way." Unfortunately, it does not go into details. – Thomas Hofmann Nov 10 '11 at 21:55
  • 3
    Go to the last part of the blog post (https://devmaze.wordpress.com/2011/01/19/using-com-android-internal-part-5-summary-and-example/), there's a link (https://github.com/inazaruk/android-sdk/tree/master/platforms) to pre-built Android APIs with all hidden and internal APIs. – Bob Apr 02 '12 at 15:21
  • 1
    The given links not accessible , require permission from author – SHAHS May 23 '18 at 10:47
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:11
  • The links are not available anymore for public access. – Lucas Dec 27 '21 at 18:23
1

I once wrote some Groovy scripts for extracting the java files from a repo checkout from http://source.android.com/ and then compiling them without the need for a full toolchain for compiling all the android sources, including the needed other steps (packaging, generating resources etc).

They can be found here:

https://github.com/thoutbeckers/CollectAndroid

But for sure this will need updating for anything after Gingerbread, mostly by setting the correct directories in "rootdirs" in the config file (CollectConfig.groovy).

At the time I regularly used this for development with all of the hidden API and sources (also problematic at the time) available.

As mentioned elsewhere com/android/internal/** will still be hidden in recent versions of ADT due to the access rule aded.

thoutbeckers
  • 2,599
  • 22
  • 24
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:12
  • I archived that repo since it was last updated for _Gingerbread_. You'll have to find another approach unfortunatly. – thoutbeckers Aug 18 '20 at 14:03
1

Long's answer worked for me, but I was still missing some classes I needed, in particular android.provider.Telephony. I was able to add it like this:

  1. Extract the framework.jar file

    mkdir /tmp/framework
cp framework.jar /tmp
cd /tmp/framework
jar xvf ../framework.jar
mv android classes

  2. Build the Android repo, which will create the out/target/common/obj/JAVA_LIBRARIES directory

  3. Find where the missing classes are

    $ cd /path/to/out/target/common/obj/JAVA_LIBRARIES
$ find . | grep "/Telephony.class"
./telephony-common_intermediates/classes/android/provider/Telephony.class
./android_stubs_current_intermediates/classes/android/provider/Telephony.class

  4. Add the new classes and rebuild the framework JAR file

    cd /tmp/framework
cp -r /path/to/out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes .
cp -r /path/to/out/target/common/obj/JAVA_LIBRARIES/telephony-common_intermediates/classes .
cd classes
jar cvf ../framework.jar .

Or you can just be lazy and include all of the classes into one giant jar file:

cd /tmp/framework
cp -r /path/to/out/target/common/obj/JAVA_LIBRARIES/*/classes .
cd classes
jar cvf ../framework.jar .
bmaupin
  • 14,427
  • 5
  • 89
  • 94
  • out/target/common/obj/JAVA_LIBRARIES how to find this path ? – Bunny Aug 13 '20 at 09:27
  • @Bunny I've updated my answer with more details. But it seems like the answer here might be easier: https://stackoverflow.com/a/32626155/399105 – bmaupin Aug 13 '20 at 11:18
  • thanks for your response @bmaupin ..can you please help me how i can find this path out/target/common/obj/JAVA_LIBRARIES to get framework jar ... i really got stuck..I am not able to find this path or missing something...please guide – Bunny Aug 13 '20 at 11:29
0

I can't comment but this is basically a comment to @KennyTM's (https://stackoverflow.com/a/13550030/2923406) excellent answer:

If you find yourself with the following error in Eclipse:

The type com.android.internal.util.Predicate cannot be resolved. It is indirectly referenced from required .class   files

(that is, android.internal.* is not available)

Then one possible solution is to apply the same method for /system/framework/framework2.jar. Using the Android Emulator for SDK19 I have this extra jar. On my HTC One there is even a framework3.jar.

Community
  • 1
  • 1
Rolf
  • 1,129
  • 11
  • 27
  • how i can checkou this above path out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/classes.jar – Bunny Aug 17 '20 at 13:12