Threat Research

Analysis: Inspecting Mach Messages in macOS Kernel-Mode Part I: Sniffing the sent Mach messages

By Kai Lu | October 26, 2018

In this blog we will review how to inspect the sent Mach messages by setting up a kernel inline hook for the function mach_msg_overwrite_trap() and ipc_kmsg_send().

Mach IPC and Mach message are the foundation for many communications that occur in macOS.  The question that many threat researchers ask is, “how can we inspect these Mach messages in user-mode or kernel-mode perspective?”  In this blog, FortiGuard Labs looks at how to inspect Mach message in kernel-mode perspective by setting up an inline hook on specific kernel APIs for handling Mach messages.

This idea was inspired by a tool from Blackhat USA 2018 Arsenal: “Kemon” that is an open-source pre and post callback-based framework for macOS kernel monitoring. Kemon provides some good features, the most important of which is how to monitor macOS kernel by setting up kernel inline hooks on the kernel APIs we are concerned about. We can do a number of interesting things through kernel inline hooks. For example, we can develop an in-memory kernel fuzzer to find bugs in macOS kernel. We can also develop a tool to monitor IPC data in macOS. So I extended this framework to implement an inline hook to inspect the Mach messages in macOS.


A quick look at Mach Messages


Mach ports are a kernel-provided inter-process communication (IPC) mechanism used heavily throughout the operating system. A Mach port is a unidirectional channel that can have multiple send endpoints and only one receive endpoint. Because they are unidirectional, a common way to use a Mach port is to send a message to the receiver, and to include another Mach port on which the receiver can reply to the sender.

Mach messages are used for inter-process communication (IPC). Programs on macOS can either send and receive Mach messages directly, or they can use remote procedure calls (RPCs) generated by MIG (Mach Interface Generator).


Messaging Implementation

Let’s first look at how the user-space program and kernel handles sending and receiving Mach messages.

Figure 1. The function mach_msg()
Figure 2. The function mach_msg_overwrite()

Both functions can send or receive a message, and they are defined in xnu-4570.71.2/libsyscall/mach/mach_msg. These two functions are the APIs in user-space.  They can invoke the function mach_msg_overwrite_trap() inside them. The function mach_msg_overwrite_trap() is implemented in xnu-4570.71.2/osfmk/ipc/mach_msg.c, which resides in the kernel mode.

The following is the execution flow of the function mach_msg() and mach_msg_overwrite():

Figure 3. Sending Mach Messages

We can see that when they enter the kernel they invoke the function ipc_kmsg_send() to send a Mach message. The declaration of the function ipc_kmsg_send() is shown below:

Figure 4. The function ipc_kmsg_send()

Its first parameter, kmsg, is the type of ipc_kmsg_t that is a pointer to the ipc_kmsg structure.

This structure is only the header for a kmsg buffer; the actual buffer is normally larger. The rest of the buffer holds the body of the message, which is defined as follows:

Figure 5. The definition of the ipc_kmsg structure

As shown in Figure 5, we can see the ipc_kmsg structure’s member variable ikm_header is a pointer to mach_msg_header_t structure. which represents the header of a Mach message.

Next, let’s take a look at the structure of a Mach message.

Figure 6. The structure of a Mach message

The top bit of the member variable msgh_bits in mach_msg_header_t structure is the complex flag. If it’s equal to 1, it representthats  this Mach message is a complex message and not a simple message. In a complex mach message, the mach_msg_header_t is followed by a descriptor count(mach_msg_body_t), then an array of that number of descriptors(mach_msg_*_descriptor_t). The type field of mach_msg_type_descriptor_t indicates the flavor of the descriptor. The mach_msg_body_t structure is defined as follows:

Figure 7. The definition of the mach_msg_body_t structure

Figure 8 shows the presently defined types of descriptors.

Figure 8. The defined types of descriptors

We can see the descriptor type is positioned on the 11th byte(index from 0).

Next, let’s look at the exact definition of each descriptor. 

Figure 9. The definition of port descriptor
Figure 10. The OOL descriptor
Figure 11. The OOL port descriptor

To this point we have detailed the structure of a Mach message. Now we will look at kernel inline hooking.

Kernel Inline Hooking

Inline hooking is a method of intercepting calls to target functions. The general idea is to redirect a function to a function of our own so that we can perform processing before and/or after the function performs it. This could include checking parameters, logging, spoofing the returned data, and filtering calls.

The hooks are placed by directly modifying code within the target function, usually by overwriting the first few bytes with a jump instruction to allow execution to be redirected before the function does something. 

Let’s look at how general inline hooking works.

Figure 12. The general inline hooking

As shown in Figure 12, we can see that hooking is made up of three parts.

1.     The inline hook – we overwrite 12 bytes into the prologue of the target function. These 12 bytes includes two instructions. If they move the trampoline’s address to register RAX, the JMP instruction will jump from the hooked function to our code.

2.     The trampoline consists of exactly three parts: the original instructions, a call instruction to call the handler, and a JMP instruction to jump back to the target function to continue executing the remaining instructions.

3.     The handler takes the exact same parameters with the target function. It allows you to do things like log data, filtering parameters, etc.

Not that we have briefly outlined how inline hooking works, let’s move on to how to hook the function of sending Mach messages.

As shown in Figure 1, in kernel mode the function mach_msg_overwrite_trap() is used to send Mach messages. It can invoke the function ipc_kmsg_send() to perform sending Mach messages. This means we need to implement a kernel inline hook on these two functions.

Before detailing the hooking implementation, there are two things to be noted. This first is how to resolve the kernel API’s symbol, and the other is how to disassemble the instructions from the address of the symbol. As for how to resolve the kernel symbol, you can refer to this blog. For the second, I used Capstone as the disassembler, which is a lightweight multi-platform and multi-architecture disassembly framework. The most significant feature is support for embedding into firmware or an OS kernel.

Next we’ll look at the implementation of hooking the functions mach_msg_overwrite_trap() and ipc_kmsg_send().

The following is a snippet of the function mach_msg_overwrite_trap() from

Figure 13. The definition of the function mach_msg_overwrite_trap()

This function can check to see if the parameter option has the MACH_SEND_MSG property. If yes, it can then invoke the function ipc_kmsg_send() to send the Mach messages. The function ipc_kmsg_send() takes three parameters: the first is the type of a pointer to the ipc_kmsg structure. What we need to do is log and parse the data of the ipc_kmsg structure when the function ipc_kmsg_send() is called.  

Next, let’s go into the assembly world of the function mach_msg_overwrite_trap().

Figure 14. The assembly of the function mach_msg_overwrite_trap()

We can see that the function starts with a prologue. Here I used the following inline hook with 12 bytes length. 

Figure 15. The inline hooking instructions of the function mach_msg_overwrite_trap()

Now let’s look at the assembly instructions of the hooked mach_msg_overwrite_trap(). It can jump to the memory address 0xffffff7f87a76160, which is actually the address of its trampoline.

Figure 16. The hooked mach_msg_overwrite_trap()

At offset 0x376 relative to the start address of mach_msg_overwrite_trap(), it can invoke the ipc_kmsg_send().  We next pick seven bytes to do the inline hooking for ipc_kmsg_send(), such as the following:

Figure 17. The inline hooking instuctions of the function ipc_kmsg_send()

So far, we have selected the opcode instructions for inline hooking. Next, we need to construct the trampolines. The following is the trampoline of hooking mach_msg_overwrite_trap().

(lldb) di -b -n mach_msg_overwrite_trap_trampoline kernelmonitorkit`mach_msg_overwrite_trap_trampoline: 0xffffff7f87a76160 <+0>: 55 pushq %rbp 0xffffff7f87a76161 <+1>: 48 89 e5 movq %rsp, %rbp 0xffffff7f87a76164 <+4>: 41 57 pushq %r15 0xffffff7f87a76166 <+6>: 41 56 pushq %r14 0xffffff7f87a76168 <+8>: 41 55 pushq %r13 0xffffff7f87a7616a <+10>: 41 54 pushq %r12 0xffffff7f87a7616c <+12>: 53 pushq %rbx 0xffffff7f87a7616d <+13>: 48 83 ec 58 subq $0x58, %rsp 0xffffff7f87a76171 <+17>: 4c 8b 3f movq (%rdi), %r15 0xffffff7f87a76174 <+20>: 8b 47 08 movl 0x8(%rdi), %eax 0xffffff7f87a76177 <+23>: 90 nop 0xffffff7f87a76178 <+24>: 90 nop 0xffffff7f87a76179 <+25>: 90 nop 0xffffff7f87a7617a <+26>: 90 nop 0xffffff7f87a7617b <+27>: 90 nop 0xffffff7f87a7617c <+28>: 90 nop 0xffffff7f87a7617d <+29>: 90 nop 0xffffff7f87a7617e <+30>: 9c pushfq 0xffffff7f87a7617f <+31>: 50 pushq %rax 0xffffff7f87a76180 <+32>: 53 pushq %rbx 0xffffff7f87a76181 <+33>: 51 pushq %rcx 0xffffff7f87a76182 <+34>: 52 pushq %rdx 0xffffff7f87a76183 <+35>: 55 pushq %rbp 0xffffff7f87a76184 <+36>: 56 pushq %rsi 0xffffff7f87a76185 <+37>: 57 pushq %rdi 0xffffff7f87a76186 <+38>: 41 50 pushq %r8 0xffffff7f87a76188 <+40>: 41 51 pushq %r9 0xffffff7f87a7618a <+42>: 41 52 pushq %r10 0xffffff7f87a7618c <+44>: 41 53 pushq %r11 0xffffff7f87a7618e <+46>: 41 54 pushq %r12 0xffffff7f87a76190 <+48>: 41 55 pushq %r13 0xffffff7f87a76192 <+50>: 41 56 pushq %r14 0xffffff7f87a76194 <+52>: 41 57 pushq %r15 0xffffff7f87a76196 <+54>: ff 15 34 d7 0d 00 callq *0xdd734(%rip) ; jmp_to_mach_msg_overwrite_trap_prologue_handler 0xffffff7f87a7619c <+60>: 41 5f popq %r15 0xffffff7f87a7619e <+62>: 41 5e popq %r14 0xffffff7f87a761a0 <+64>: 41 5d popq %r13 0xffffff7f87a761a2 <+66>: 41 5c popq %r12 0xffffff7f87a761a4 <+68>: 41 5b popq %r11 0xffffff7f87a761a6 <+70>: 41 5a popq %r10 0xffffff7f87a761a8 <+72>: 41 59 popq %r9 0xffffff7f87a761aa <+74>: 41 58 popq %r8 0xffffff7f87a761ac <+76>: 5f popq %rdi 0xffffff7f87a761ad <+77>: 5e popq %rsi 0xffffff7f87a761ae <+78>: 5d popq %rbp 0xffffff7f87a761af <+79>: 5a popq %rdx 0xffffff7f87a761b0 <+80>: 59 popq %rcx 0xffffff7f87a761b1 <+81>: 5b popq %rbx 0xffffff7f87a761b2 <+82>: 58 popq %rax 0xffffff7f87a761b3 <+83>: 9d popfq 0xffffff7f87a761b4 <+84>: ff 25 1e d7 0d 00 jmpq *0xdd71e(%rip) ; jmp_back_to_mach_msg_overwrite_trap 0xffffff7f87a761ba <+90>: cc int3 0xffffff7f87a761bb <+91>: 0f 0b ud2 (lldb)


The first 23 bytes of instructions are copied from the prologue of the function mach_msg_overwrite_trap(). This trampoline first executes the prologue of the target function, then invokes its handler that sets up the hook of the function ipc_kmsg_send(). Finally, it jumps back to the target function to continue to execute the remaining instructions. We can calculate the indirect jumping address as follows:

Figure 18. Calculating the indirect jumping address

We can see that the trampoline finally jumps to 0xffffff8003d6f797(offset:+23) to execute instructions.  

Next, let’s look at the handler(mach_msg_overwrite_trap_prologue_handler).  In this handler, we first copy the instructions starting at 0xffffff8003d6faf6 (seven bytes length) to the corresponding position in the trampoline of hooking  ipc_kmsg_send(). We then overwrite the inline hooking instructions on the address starting at 0xffffff8003d6faf6.

Figure 19. The inline hooking on ipc_kmsg_send()

It contains two instructions, JMP and nop. The JMP instruction is an indirect jump. We can see it is able to jump to ipc_kmsg_send_trampoline.

Figure 20. An indirect jump to ipc_kmsg_send_trampoline


The following is the trampoline of hooking ipc_kmsg_send():


(lldb) di -b -n ipc_kmsg_send_trampoline




    0xffffff7f87a761c0 <+0>:   9c                 pushfq 


    0xffffff7f87a761c1 <+1>:   50                 pushq  %rax


    0xffffff7f87a761c2 <+2>:   53                 pushq  %rbx


    0xffffff7f87a761c3 <+3>:   51                 pushq  %rcx


    0xffffff7f87a761c4 <+4>:   52                 pushq  %rdx


    0xffffff7f87a761c5 <+5>:   55                 pushq  %rbp


    0xffffff7f87a761c6 <+6>:   56                 pushq  %rsi


    0xffffff7f87a761c7 <+7>:   57                 pushq  %rdi


    0xffffff7f87a761c8 <+8>:   41 50              pushq  %r8


    0xffffff7f87a761ca <+10>:  41 51              pushq  %r9


    0xffffff7f87a761cc <+12>:  41 52              pushq  %r10


    0xffffff7f87a761ce <+14>:  41 53              pushq  %r11


    0xffffff7f87a761d0 <+16>:  41 54              pushq  %r12


    0xffffff7f87a761d2 <+18>:  41 55              pushq  %r13


    0xffffff7f87a761d4 <+20>:  41 56              pushq  %r14


    0xffffff7f87a761d6 <+22>:  41 57              pushq  %r15


    0xffffff7f87a761d8 <+24>:  ff 15 02 d7 0d 00  callq  *0xdd702(%rip)            ; jmp_to_ipc_kmsg_send_pre_handler


    0xffffff7f87a761de <+30>:  41 5f              popq   %r15


    0xffffff7f87a761e0 <+32>:  41 5e              popq   %r14


    0xffffff7f87a761e2 <+34>:  41 5d              popq   %r13


    0xffffff7f87a761e4 <+36>:  41 5c              popq   %r12


    0xffffff7f87a761e6 <+38>:  41 5b              popq   %r11


    0xffffff7f87a761e8 <+40>:  41 5a              popq   %r10


    0xffffff7f87a761ea <+42>:  41 59              popq   %r9


    0xffffff7f87a761ec <+44>:  41 58              popq   %r8


    0xffffff7f87a761ee <+46>:  5f                 popq   %rdi


    0xffffff7f87a761ef <+47>:  5e                 popq   %rsi


    0xffffff7f87a761f0 <+48>:  5d                 popq   %rbp


    0xffffff7f87a761f1 <+49>:  5a                 popq   %rdx


    0xffffff7f87a761f2 <+50>:  59                 popq   %rcx


    0xffffff7f87a761f3 <+51>:  5b                 popq   %rbx


    0xffffff7f87a761f4 <+52>:  58                 popq   %rax


    0xffffff7f87a761f5 <+53>:  9d                 popfq  


    0xffffff7f87a761f6 <+54>:  90                 nop    


    0xffffff7f87a761f7 <+55>:  90                 nop    


    0xffffff7f87a761f8 <+56>:  90                 nop    


    0xffffff7f87a761f9 <+57>:  90                 nop    


    0xffffff7f87a761fa <+58>:  90                 nop    


    0xffffff7f87a761fb <+59>:  90                 nop    


    0xffffff7f87a761fc <+60>:  e8 ff e9 2d 7c     callq  0xffffff8003d54c00        ; ipc_kmsg_send at ipc_kmsg.c:1793


    0xffffff7f87a76201 <+65>:  85 c0              testl  %eax, %eax


    0xffffff7f87a76203 <+67>:  90                 nop    


    0xffffff7f87a76204 <+68>:  90                 nop    


    0xffffff7f87a76205 <+69>:  90                 nop    


    0xffffff7f87a76206 <+70>:  90                 nop    


    0xffffff7f87a76207 <+71>:  90                 nop    


    0xffffff7f87a76208 <+72>:  90                 nop    


    0xffffff7f87a76209 <+73>:  90                 nop    


    0xffffff7f87a7620a <+74>:  90                 nop    


    0xffffff7f87a7620b <+75>:  90                 nop    


    0xffffff7f87a7620c <+76>:  90                 nop    


    0xffffff7f87a7620d <+77>:  90                 nop    


    0xffffff7f87a7620e <+78>:  90                 nop    


    0xffffff7f87a7620f <+79>:  90                 nop    


    0xffffff7f87a76210 <+80>:  9c                 pushfq 


    0xffffff7f87a76211 <+81>:  53                 pushq  %rbx


    0xffffff7f87a76212 <+82>:  51                 pushq  %rcx


    0xffffff7f87a76213 <+83>:  52                 pushq  %rdx


    0xffffff7f87a76214 <+84>:  55                 pushq  %rbp


    0xffffff7f87a76215 <+85>:  56                 pushq  %rsi


    0xffffff7f87a76216 <+86>:  57                 pushq  %rdi


    0xffffff7f87a76217 <+87>:  41 50              pushq  %r8


    0xffffff7f87a76219 <+89>:  41 51              pushq  %r9


    0xffffff7f87a7621b <+91>:  41 52              pushq  %r10


    0xffffff7f87a7621d <+93>:  41 53              pushq  %r11


    0xffffff7f87a7621f <+95>:  41 54              pushq  %r12


    0xffffff7f87a76221 <+97>:  41 55              pushq  %r13


    0xffffff7f87a76223 <+99>:  41 56              pushq  %r14


    0xffffff7f87a76225 <+101>: 41 57              pushq  %r15


    0xffffff7f87a76227 <+103>: 48 31 db           xorq   %rbx, %rbx


    0xffffff7f87a7622a <+106>: 89 c3              movl   %eax, %ebx


    0xffffff7f87a7622c <+108>: 48 89 de           movq   %rbx, %rsi


    0xffffff7f87a7622f <+111>: ff 15 b3 d6 0d 00  callq  *0xdd6b3(%rip)            ; jmp_to_ipc_kmsg_send_post_handler


    0xffffff7f87a76235 <+117>: 41 5f              popq   %r15


    0xffffff7f87a76237 <+119>: 41 5e              popq   %r14


    0xffffff7f87a76239 <+121>: 41 5d              popq   %r13


    0xffffff7f87a7623b <+123>: 41 5c              popq   %r12


    0xffffff7f87a7623d <+125>: 41 5b              popq   %r11


    0xffffff7f87a7623f <+127>: 41 5a              popq   %r10


    0xffffff7f87a76241 <+129>: 41 59              popq   %r9


    0xffffff7f87a76243 <+131>: 41 58              popq   %r8


    0xffffff7f87a76245 <+133>: 5f                 popq   %rdi


    0xffffff7f87a76246 <+134>: 5e                 popq   %rsi


    0xffffff7f87a76247 <+135>: 5d                 popq   %rbp


    0xffffff7f87a76248 <+136>: 5a                 popq   %rdx


    0xffffff7f87a76249 <+137>: 59                 popq   %rcx


    0xffffff7f87a7624a <+138>: 5b                 popq   %rbx


    0xffffff7f87a7624b <+139>: 9d                 popfq  


    0xffffff7f87a7624c <+140>: ff 25 9e d6 0d 00  jmpq   *0xdd69e(%rip)            ; jmp_back_to_ipc_kmsg_send_call


    0xffffff7f87a76252 <+146>: cc                 int3   


    0xffffff7f87a76253 <+147>: 0f 0b              ud2    






In this trampoline, it first invokes the function ipc_kmsg_send_pre_handler(), which is intended to parse and log the Mach messages to be sent. It then invokes the target function ipc_kmsg_send(), followed by invoking the ipc_kmsg_send_post_handler(), which is intended to handle or subvert the return value of the target function ipc_kmsg_send(). Finally, it can then jump back to the function mach_msg_overwrite_trap().

Figure 21. Jumping back to the function mach_msg_overwrite_trap()

In this section we have examined how to sniff the sending Mach messages by implementing an inline hook of the function mach_msg_overwrite_trap() and ipc_kmsg_send().

Next, I will draw a picture to depict the workflow of the kernel inline hooking.

Figure 22. The workflow of the kernel inline hooking

We can briefly sum up the steps of inline hooking mach_msg_overwrite_trap() and ipc_kmsg_send() as follows:

a.    Copy the first 23 bytes of the function mach_msg_overwrite_trap’s prologue to mach_msg_overwrite_trap_trampoline.

b.    Overwrite the first 12 bytes with our inline hooking instructions, which contain a MOV instruction and a JMP instruction followed by the address (8 bytes)of ipc_kmsg_send_trampoline.

c.    When the function mach_msg_overwrite_trap() is called, the execution can be redirected to mach_msg_overwrite_trap_trampoline. In mach_msg_overwrite_trap_trampoline, it can invoke the function mach_msg_overwrite_trap_prologue_handler.

d.    In the function mach_msg_overwrite_trap_prologue_handler it sets up the inline hook for the function ipc_kmsg_send(). It then copies instructions starting at 0xffffff8003d6faf6 (seven bytes length) to the corresponding position in ipc_kmsg_send_trampoline. Note that you need to tweak the relative address of the call instruction, otherwise it can cause a kernel panic.

e.    Overwrite the instructions starting at 0xffffff8003d6faf6 with our inline hooking instructions that contain an indirect JMP instruction and a NOP instruction. The indirect address is stored as eight bytes behind the inline hooking instructions of the function mach_msg_overwrite_trap().

f.     At the end of mach_msg_overwrite_trap_trampoline, it can jump back to 0xffffff8003d6f797 to continue to execute the remaining instructions.

g.    When it executes at 0xffffff8003d6faf6, it can then jump to ipc_kmsg_send_trampoline. In ipc_kmsg_send_trampoline it first invokes the function ipc_kmsg_send_pre_handler(), which is used to parse and log the Mach messages to be sent. It then invokes ipc_kmsg_send(), followed by invoking ipc_kmsg_send_post_handler() ,which is used to handle or subvert the return value of the function ipc_kmsg_send(). Finally, it jumps back to 0xffffff8003d6fafd in the function mach_msg_overwrite_trap().

Sniffing the Mach Messages

In the previous sections, I detailed the implementation of inline hooking for the function mach_msg_overwrite_trap() and ipc_kmsg_send(). In this section I present the sniffed sending Mach messages. All core work of inline hooks are implemented in a KEXT in kernel mode. We use a client program to receive the messages from logging Mach messages from the KEXT. This tool can record a detailed structure for a Mach message sent by a specific process. Some screenshots are shown below.

Figure 23. The process WindowServer sends a complex Mach message
Figure 24. The process sends a complex Mach message
Figure 25. The process launchd sends a complex Mach message

We can see the tool recorded all fields of the ipc_kmsg structure, as well as the parsed Mach messages in detail.


In this blog, I demonstrated how to sniff the sending of a Mach message by setting up a kernel inline hook for the function mach_msg_overwrite_trap() and ipc_kmsg_send(). This is able to record all fields of an ipc_kmsg structure and each field of a Mach message. In Part II of this blog, I will present how to sniff the received Mach messages via a kernel inline hooking.

You’re invited to stay tuned! Read part II of this blog analysis

Special thanks to the researcher Wang Yu’s open source project “Kemon: An Open-Source Pre and Post Callback-Based Framework for macOS Kernel Monitoring”.



Download our latest Fortinet Global Threat Landscape Report to find out more detail about recent threat landscape trends.

Sign up for our weekly FortiGuard Threat Brief.

Know your vulnerabilities – get the facts about your network security. A Fortinet Cyber Threat Assessment can help you better understand: Security and Threat Prevention, User Productivity, and Network Utilization and Performance.