39

Apple occasionally uses a proprietary XIP file format, particularly when distributing versions of Xcode. It is an analog to zip, but is signed, allowing it to verified on the receiving system. When a XIP file is opened (by double-clicking), Archive Utility will expand it, but only if the digital signature is intact.

Does anyone know how to extract a XIP file from the Terminal command line to a specific folder? Is there any way to unarchive this type of file if the signature is invalid?

Geoff Nixon
  • 4,697
  • 2
  • 28
  • 34
Antony Raphel
  • 2,036
  • 2
  • 25
  • 45
  • If it is "just a zip" then presumably ordinary unzip tools will do the trick. Or are you wanting to verify the signature as well? – halfer Feb 13 '17 at 21:16
  • 2
    It's not just a signed zip file; it's a signed xar archive, typically with an LZMA-based compression applied. – Geoff Nixon Jun 13 '18 at 11:08
  • The answer given by @Geoff Nixon below works for me – rexford Jun 28 '18 at 12:44
  • @antony-raphel Does my answer below adequately answer your question? – Geoff Nixon Oct 21 '18 at 08:10
  • Does this answer your question? [How to extract xip archive using command line?](https://stackoverflow.com/questions/40414645/how-to-extract-xip-archive-using-command-line) – Geoff Nixon Jul 14 '21 at 04:42

6 Answers6

53

Maybe try:

xip -x [path to .xip file]

That will unpack the archive into your current working directory.

As for extracting into a specific directory, there is not explicitly an option for this, but xip -x will extract into the current working directory. Therefore, cding to where you would like to extract the file should work; if you specifically need to automate this, a script to the effect of:

#!/bin/sh

xipfile="$(cd $(dirname "$1"); pwd -P)/$(basename "$1")" # a portable "realpath"

cd "$2"
xip -x "$xipfile"

Should do the trick I think?

Geoff Nixon
  • 4,697
  • 2
  • 28
  • 34
  • 8
    +1 how did you find out about the -x flag? It's neither documented on the man page nor in the command's help ... – rexford Jun 28 '18 at 12:51
  • 8
    @rexford I'd dare to say *most* of Apple's utilities have hidden flags. Unfortunately, finding them is not a straightforward process. In fact, I just [asked a question myself about this myself](https://stackoverflow.com/questions/40414645/how-to-extract-xip-archive-using-command-line/50835904#50835904) (that is, if there's a better way to find these). But the short answer is to use `/usr/bin/strings` on the binary and look for likely candidates. – Geoff Nixon Jul 01 '18 at 08:31
  • 2
    Thank you! This worked for extracting Xcode_9.4.1.xip – Patrick D Jul 13 '18 at 07:12
  • @GeoffNixon I just used `strings`, and there is no mention of `-x` in the printable strings. – HappyFace Sep 07 '18 at 14:28
  • 1
    @HappyFace You're right, the `x` flag is no longer readily apparent in the output of using `strings` on itself. I believe this is as of Xcode 10. Perhaps Apple has begun to strip their binaries more thoroughly, or perhaps its intentional obfuscation. However, for the portions of Darwin for which code is available, these options can also be found there: https://opensource.apple.com/source/cctools/cctools-886/misc/strings.c.auto.html (see line 205). And there are still plenty of other utilities with hidden options to be found this way. – Geoff Nixon Oct 21 '18 at 07:39
  • 1
    Despite this now being documented behaviour in "man xip", there is a wrinkle. I've just used "xip -x" on Xcode 13.3.1's archive, when cd'd to an external drive, and $TMPDIR, at least transiently, acquired a 17Gb working copy of Xcode. This is why the install is nuts and typically requires 50Gb+: there are three different variants of Xcode in different places at the same time. – Stuart Watt Apr 30 '22 at 13:40
  • Recommended if normal extraction does not work because of the signature verification – אורי orihpt Sep 06 '22 at 18:04
  • In Monterey (12.6.3) it seems now impossible to unpack an Xcode.xib to a particular folder - it always gets extracted into a tmp folder on the boot volume first - and if that's not having enough space, the entire process will fail. There seems to be no way around that. :( – Thomas Tempelmann Feb 08 '23 at 01:52
25

I would recommend to simply extract the archive into the folder you want trying the following:

xar -xf file.xip -C /path/to/target

(and/or)

tar -zxvf file.xip -C /path/to/target

The xar and tar commands extract the .xip "Content" and "Metadata" in a raw format.

Using a pbzx stream parser you'll need to extract the "Content" which is an lzma compressed Payload; the format is similar to that found within a package installer (eg. .pkg). You can compile the pbzx source from here, or download the compiled binary and install to /usr/local/bin then invoke the pbzx command:

pbzx -n Content | cpio -i

After the command finishes parsing the Content you should get the original form of whatever it was within the .xip archive.

Useful / Additional Info:

$ pkgutil --check-signature file.xip

Xcode_9_beta_2.xip returns:

Package "Xcode_9_beta_2.xip":
   Status: signed Apple Software
   Certificate Chain:
    1. Software Update
       SHA1 fingerprint: 1E 34 E3 91 C6 44 37 DD 24 BE 57 B1 66 7B 2F DA 09 76 E1 FD
       -----------------------------------------------------------------------------
    2. Apple Software Update Certification Authority
       SHA1 fingerprint: FA 02 79 0F CE 9D 93 00 89 C8 C2 51 0B BC 50 B4 85 8E 6F BF
       -----------------------------------------------------------------------------
    3. Apple Root CA
       SHA1 fingerprint: 61 1E 5B 66 2C 59 3A 08 FF 58 D1 4A E2 24 52 D1 98 DF 6C 60

Notes:

Important: Starting with macOS Sierra, only XIP archives signed by Apple will be expanded. Developers who have been using XIP archives will need to move to using signed installer packages or disk images.

↳ OS X manual page : xip

Berry
  • 2,143
  • 4
  • 23
  • 46
l'L'l
  • 44,951
  • 10
  • 95
  • 146
  • I got an error: `Archive: Xcode_8.1.xip End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive. note: Xcode_8.1.xip may be a plain executable, not an archive` – Antony Raphel Feb 14 '17 at 17:03
  • What happens when you double-click the file? – l'L'l Feb 14 '17 at 19:01
  • 1
    it opens the default **archive utility** to validate the signature and extract. But, before finishing the validation error comes up. – Antony Raphel Feb 15 '17 at 04:08
  • Then I think the file is corrupted. – l'L'l Feb 15 '17 at 04:10
  • `unzip` does not work. It gives me (on 10.12.6): ''End-of-central-directory signature not found''. Tried with the latest Xcode 9 beta – Thomas Tempelmann Aug 29 '17 at 09:38
  • @AntonyRaphel: Please try the commands shown above and let me know the outcome. – l'L'l Feb 11 '18 at 13:01
  • I opened an [**issue**](https://github.com/NiklasRosenstein/pbzx/issues/2) for fixing a bug in `parse_args()` in **pbzx.c**. Using the patched version it can be invoked in one-liner style: `tar zxOf Xcode.xip Content | pbzx -n - | cpio -i`. You don't have to extract the **Content** first. – Meow Feb 23 '18 at 03:13
  • @Meow: Thanks for the info — it sounds like a nice fix, as piping it would've been a better way to go; maybe do a pull request for it also if you haven't already and hopefully they'll accept it, cheers! – l'L'l Feb 23 '18 at 03:17
  • 1
    Note that, undocumented in the man page you cited, there is a simple `-x` option to the `xip` utility (see my alternative answer). But this doesn't verify the signature, your "Useful / Additional Info" bit is I think the only practical way to authenticate it. So perhaps the best answer would be something like `pkgutil --verify-signature [xip file] && xip -x [xip file]`. – Geoff Nixon Jun 13 '18 at 11:17
  • Instead of compiling pbzx you can use a [python version](https://gist.github.com/pudquick/ff412bcb29c9c1fa4b8d) – XMB5 Aug 30 '18 at 19:25
22

You can open archive utility itself, go into Preferences and set a specific destination folder and then double click the file. This way you achieved expanding it to a specific destination. ;-)

Hug
  • 337
  • 2
  • 5
  • I feel safer about this solution. Just a note that one should go into "Preferences" actually :) – Gobe Apr 02 '18 at 17:42
  • In Monterey (12.6.3) it seems now impossible to unpack an Xcode.xib to a particular folder - it always gets extracted into a tmp folder on the boot volume first - and if that's not having enough space, the entire process will fail. There seems to be no way around that. :( – Thomas Tempelmann Feb 08 '23 at 01:52
7

You can change the output by opening Archive Utility (either via spotlight search or launchpad). Then Archive Utility > Preferences (Command+,). Then Save expanded files > into... and you can change the output directory. :)

More simply: Opening "Archive Utility" and opening its preferences and changing where it can extract the file to. Also, Archive Utility can open .xip files.

Avoxel284
  • 95
  • 1
  • 2
  • 9
1

(For an archive with a valid signature.)

From the destination folder, create a symbolic link to the .xip archive, then expand using the link (and then delete the link when done). Example:

cd /Applications
ln -s /Volumes/SomewhereElse/Xcode.xip
xip --expand Xcode.xip
rm Xcode.xip
chrstphrchvz
  • 657
  • 5
  • 18
  • The following doesn't involve extraneous symlinks and worked fine for me, could be a simplification of your option: `cd /Applications && xip --expand ~/Downloads/Xcode.xip` — substitute `~/Downloads/Xcode.xip` and `/Applications` for the source and destination paths as appropriate – akasaka Jan 13 '23 at 01:21
0

There seems to be no single tool to accomplish that, so far. One needs a tool that can decode pbzx streams, and there seems to be no tool for that pre-installed on macOS.

Various solutions are found in these answers:

Thomas Tempelmann
  • 11,045
  • 8
  • 74
  • 149