How are instruction sets standardized?

Question

My understanding is the AMD64 was invented by AMD as a 64 bit version of x86.

New instructions are added by both AMD and Intel (because those are the only two companies that implement AMD64). In effect, there is no central standard like there is in C++.

When new instructions are added, they are usually a part of a "set" like sse or avx.

In my research, the designation for some instructions is inconsistent, ie, it's not always clear which set an instruction belongs to.

What defines the instruction sets? Is there a universal agreement on what instructions are in which instruction sets or is it decided by convention?

Answer 1

80x86 is/was essentially the 8086 instruction set (from late 1970s) plus "optional" extensions added by different vendors; where a lot of extensions became ubiquitous (effectively "pseudo standards" supported by all vendors after a while); with a cross-licensing agreement used to allow one vendor to provide a compatible implementation of another vendor's extension.

Over time this (all the "optional but ubiquitous" extensions) grew to become awkward. To help fix that; AMD created "AMD64", which is the name given to a large set of different extensions (from different vendors) to form a new base-line. It consists of long mode (a 64-bit extension from AMD, that extends a few different extension from Intel - mainly a 32-bit protected mode extension and a "PAE paging" extension), plus SSE (from Intel), plus SYCALL (from AMD), plus lots of smaller/older things (TSC, CPUID, INVLPG, MSRs, ... - mostly from Intel). Of course the cross-licensing agreement meant that other vendors could implement all of these different extensions (and therefore implement the new "AMD64" base-line); and so both Intel and VIA added support for AMD64 in their CPUs.

Every other ISA is similar in that there's a base standard (e.g. Aarch64) and vendor specific extensions (e.g. the matrix and machine learning accelerators Apple added to their M1 chips). The fundamental difference is that there is no cross-licensing agreement in place for any other ISA (e.g. if Qualcomm wanted to implement Apple's extensions they'll probably end up fighting a legal battle for several years).

The other difference is that 80x86 has a clear way for software to identify optional extensions (via. the cpuid instruction), which is split into vendor specific ranges (e.g. Intel's extensions/features starting at 0x0000000, hypervisor extensions/features starting at 0x4000000, AMD's extensions/features starting at 0x8000000, Transmeta's starting at 0x8086000, Centaur/VIA's starting at 0xC000000). This allows vendors to add their own extensions whenever they want without causing conflicts; and allows software to work fine on all vendors CPUs (and old CPUs and new CPUs from the same vendor) by asking the CPU which features it supports and then enabling support for the extensions the CPU provides (e.g. crudely like "if( CPU_supports(thing) ) { use_thing(); } else { use_alternative(); }).

What defines the instruction sets? Is there a universal agreement what what instructions are in which sets or is it decided by convention?

Each vendor defines their own extensions (without wasting 3 years arguing with competitors to end up with a "designed by committee" compromise).

Note that sometimes this means you get competing alternative extensions (e.g. 3DNow vs. SSE, or SYSCALL vs. SYSENTER, or AMD's virtualization extensions vs. Intel's, or...). Rarely one of them becomes officially dead (e.g. 3DNow).

The other thing that's worth mentioning is that, for the ISA itself, there's a strong emphasis on backward compatibility. You can almost guarantee that next year's CPU will still be capable of running 16-bit software from 40 years ago. For better or worse, it's everything else (the OS, the devices, the firmware, ..) and not the ISA or extensions that breaks backward compatibility.

Answer 2

There is no such thing, and I cannot imagine how there would be.

One or more people at intel define their instruction sets for their products, period. If AMD happens to have been able to make legal clones (which they have) and as part of that agreement or perhaps even not but with some penalty, they add additional instructions/features. First off it is on them to do it and keep some sense of compatibility, if they even want to be compatible. Second if they want to add extensions and can get away with it it is purely within AMD one or more engineers. Then if intel goes and makes some new instructions, it is one or more intel engineers. As history played out you then have other completely disconnected parties like gnu tools folks, microsoft tools folks and a list of others, as well as operating system folks that use tools and make their products, choosing directly or indirectly what instructions get used. And as history plays out some of these intel only or amd only instructions may be favored by one party or another. And if that party happens to have influence (microsoft windows, linux, etc), to the point that it puts pressure on intel or amd to lean one way or another, they it is their management and engineering that does that, within their company. They can choose to not go with what the users want and try to push users in their direction. Simple sales of one product line or another may dictate the success or failure of each parties decisions.

I cannot think of a or many standards that folks actually agree on even though they might have representatives that wear shirts with the same logo on them that participate in the standards bodies. From pcie to java to C++, etc (C and C++ being really bad since they were written then attempts to standardize later, which are just patches and too much left to individual compiler authors choices of interpretation). You want to win at business you differentiate yourself from the others. I have an x86 clone that is much cheaper but performs 95% as well as intel. Plus I added my own stuff that intel does not have that I pay employees to add to open source stuff, making those open source things optional to gain that feature/performance boost. That differentiates me from the competition, and for some users locks me in as their only choice.

Instruction sets for an architecture (x86 has a long line of architectures over time, arm does too and they are more organized about it imo, etc) are defined by that individual or teams within that company. End of story. At best they may have to avoid patents (yep there have been patents you have to avoid, making it hard to make a new instruction set). If two competing and compatible architectures like intel and amd (or intel team a vs intel team b, amd team a vs ...) happen to adopt each others features/instructions it is more market driven not some standards body.

Basically go look at itanium vs amd64 and how that played out.

The x86 history is a bit of a nightmare and I still cannot fathom why it still even exists (has nothing to do with the quality of the instruction set but instead how the business works), and as such attempting to put labels on things and organize them into individual boxes, really does not add any value and creates some chaos. Generation r of intel has this, generation m of amd has that, my tool supports gen r of this and gen m of that. Next year I will personally choose if I want to support the next gen of each or not. Repeat forever until the products die. You also have to choose if you want to support an older generation as those may have the same instructions but with different features/side effects despite in theory being compatible.