You can think of ARP as "where the rubber meets the road" in IP networks. Many people are aware that the Internet Protocol (IP) uses a system of IP addresses, routers and DNS to get packets moved from source to destination. Fewer are familiar with the fact that the "MAC address" is the address that matters for final delivery! IP addresses are ephemeral; they can be, and often are, changed frequently. But the MAC address serves the role of a "permanent address", and is used in networks to determine which host computer/interface takes delivery of the data packet. This role as permanent address was established when it was decided that each and every Ethernet adapter would have a unique address embedded in the hardware during the manufacturing process.
Well, that was the theory at the time anyway :) Since that time, it's been decided that our systems will be allowed to "override" that permanent hardware address. It's quite easy to change the MAC address of an adapter, and there are even good reasons for doing so.
ARP is a rather arcane protocol that has been described, delineated and defined in numerous sources. The definitive specification is RFC 826, and it serves the basis for much of the ARP-specific networking software that has been coded since about 1982.
Oh... and don't get ARP confused with ARPA! ARPA was the Advanced Research Projects Agency, an agency of the US government that funded the development of the Internet! Yes, it's true... the Internet was even called ARPAnet back in the day. ARPA funded development of what would eventually become the TCP/IP networking protocol. It's the protocol that you're reading (and I'm writing) this page with. It was the Government's funding which led Al Gore to state that he "created" (not "invented") the Internet. At any rate, ARPA eventually morphed into DARPA (Defense Advanced Research Projects Agency), and adopted a more, uh... nefarious (?) agenda. However, it is interesting perhaps that initial developement of ARP was funded by ARPA :)
But I digress... It seems curious (to me at least) that although the MAC address was developed with the notion that it would be a permanent address, designed with an 'irrevocably-encoded-in-hardware' mentality, that it is used today in an even more ephemeral context than the IP address! I say this because MAC Addresses are stored in something called the "ARP cache" in your computer, and that cache typically "expires", and is deleted under software control, every 60 seconds in many systems! This seems very odd to me, but clearly there may be good reasons for such a short lifetime - reasons of which I'm unaware.
What I do know is that the ARP cache's short lifetime can make certain chores more difficult; e.g. finding my new Raspberry Pi's IP address on my local network. In this case, what I'm searching for on my network are any MAC addresses whose first three octets are b8:27:eb. These three octets are known as the OUI, and have been assigned to the Raspberry Pi Foundation. Consequently, all network interfaces on all Raspberry Pi devices manufactured under the auspices of The Foundation begin with these three octets. However, beginning with the RPi 4B, manufacturing was assigned to the Raspberry Pi Trading Co, which has added a new OUI: dc:a6:32. Since The Organization has not seen fit to do anything else to help their user base with the issue of what IP address has been assigned, these OUIs are the best clue we have for tracking the IP addresses of these devices down. I didn't understand why this was so difficult until I understood the workings of the ARP cache. With that in mind, my simple hack in pseudocode is this:
ping every address on the subnet to 'refresh' my host's ARP cache
use the `arp` utility to get MAC addresses from the cache
`grep` the `arp` output for the OUI octets
i.e. `arp -a | grep b8:27:eb | dc:a6:32`
So it's all good... good in the sense of a learning opportunity. Along the way, I've discovered some details that may help with other yet-to-be-encountered problems. I've also found what seems to be a neat tool named arping (a clever Portmanteau :) that I've added to my toolbox. Turns out there are two implementations, the plain-vanilla arping in iputils, and an improved arping from Thomas Habets. This latter version of arping is available on Raspbian via sudo apt-get install arping, and on Mac via Homebrew. What to do with all of this new-found knowledge and technology? Hmmm... I'd like to do an improved version of my shell script, or perhaps even create a customised Raspbian Lite distribution that placed its address into its neighbors ARP caches immediately after it booted up! Yeah... this could be done with a "gratuitous ARP" that was invoked during startup... that doesn't sound too hard, does it? Kinda' makes you wonder why The Organization hasn't done it already since they're rolling in cash :)
Random notes that show where my head is:
- Is The Raspberry Pi Foundation a 'proper' charity if it spends 73% of its revenue on "Other", and 27% on "Charitable activities?". Sounds more like a way to enrich the faculty members who now occupy board seats on the Raspberry Pi Foundation!