Arm架构下的Synchronization概述和案例分析白皮书 ｜ 附下载

白皮书下载链接：Arm架构下的Synchronization概述和案例分析

1. 简介

随着近年来Arm服务器的应用越来越广泛，越来越多的云厂商开始提供基于Arm架构的云实例，越来越多的开发人员正在为Arm平台编写软件。

Synchronization是软件迁移和优化过程中的热门话题。基于Arm架构的服务器通常具有比其他架构更多的CPU内核，对Synchronization的深入理解显得更为重要。

Arm和X86 CPU之间最显著的区别之一是它们的内存模型：Arm架构具有与x86架构的TSO（Total Store Order）模型不同的弱内存模型。不同的内存模型可能会导致程序在一种架构上运行良好，但在另一种架构上会遇到性能问题或错误。Arm服务器更宽松的内存模型允许更多的编译器和硬件优化以提高系统性能，但代价是它更难理解并且可能更容易编写错误代码。

我们创作此文档是为了分享有关Arm架构的Synchronization专业知识，可以帮助其他架构的开发人员在Arm系统上进行开发。

2. Armv8-A架构上的Synchronization方法

本文档首先介绍了Armv8-A架构上的Synchronization相关知识，包括原子操作、Arm内存顺序和数据访问屏障指令。

2.1 原子操作

锁的实现要求原子访问，Arm架构定义了两种类型的原子访问：

• Load exclusive and store exclusive
• Atomic operation, which is introduced in armv8.1-a large system extension (LSE)

2.1.1 Exclusive load and store

LDREX/LDXR - The load exclusive instruction performs a load from an addressed memory location, the PE (e.g. the CPU) also marks the physical address being accessed as an exclusive access. The exclusive access mark is checked by store exclusive instructions.

STREX/STXR - The store exclusive instruction tries to a value from a register to memory if the PE (e.g. the CPU) has exclusive access to the memory address, and returns a status value of 0 if the store was successful, or of 1 if no store was performed.

2.1.2 LSE Atomic operation

LDXR/STXR使用了try and test机制，LSE不一样，它直接强制原子访问，主要有如下指令:

• Compare and Swap instructions, CAS, and CASP. These instructions perform a read from memory and compare it against the value held in the first register. If the comparison is equal, the value in the second register is written to memory. If the write is performed, the read and write occur atomically such that no other modification of the memory location can take place between the read and write.
• Atomic memory operation instructions, LD<OP>, and ST<OP>, where <OP> is one of ADD, CLR, EOR, SET, SMAX, SMIN, UMAX, and UMIN. Each instruction atomically loads a value from memory, performs an operation on the values, and stores the result back to memory. The LD<OP> instructions save the originally read value in the destination register of the instruction.
• Swap instruction, SWP. This instruction atomically reads a location from memory into a register and writes back a different supplied value back to the same memory location.

2.2 Arm内存顺序

Arm架构定义了一种弱内存模型，内存访问可能不会按照代码顺序：

2.3 Arm数据访问屏障指令

Arm架构定义了屏障指令来保证内存访问的顺序。

DMB – Data Memory Barrier
Explicit memory accesses before the DMB are observed before any explicit access after the DMB

• Does not guarantee when the operations happen, just guarantee the order

    LDR  X0, [X1]        ;Must be seen by memory system before STR
    DMB  SY
    ADD  X2, #1        ; May be executed before or after memory system sees LDR
    STR  X3, [X4]        ;Must be seen by memory system after LDR

DSB – Data Synchronization Barrier
A DSB is more restrictive than a DMB

• Use a DSB when necessary, but do not overuse them

No instruction after a DSB will execute until:

• All explicit memory accesses before the DSB in program order have completed
• Any outstanding cache/TLB/branch predictor operations complete

    DC   ISW        ; Operation must have completed before DSB can complete
    STR  X0, [X1]    ; Access must have completed before DSB can complete
    DSB  SY
    ADD  X2, X2, #3    ;Cannot be executed until DSB completes

DMB和DSB是双向栅栏，对两个方向都限制，Armv8-a也设计了一种单向栅栏：load-acquire和store-release机制，只在一个方向上做限制。

Load-Acquire (LDAR)

• All accesses after the LDAR are observed by memory system after the LDAR.
• Accesses before the LDAR are not affected.
  

Store-Release (STLR)

• All accesses before the STLR are observed by memory system before the STLR
• Accesses after the STLR are not affected
  

3. C++内存模型

有了语言层面的内存模型，对于大多数情况，开发者不需要去写依赖于具体架构的汇编代码，而只需要借助于良好设计的语言层面的内存模型来编写高质量代码，不必担心架构差异。
C++ memory model:
https://en.cppreference.com/w/cpp/header/atomic

我们做了一个C++内存模型与Armv8-A实现之间的映射：

4. 总结

在白皮书中，为帮助读者更好地理解，我们选取了三个典型案例进行深入分析。由于与Synchronization相关的编程非常复杂，因此我们必须仔细权衡其正确性和性能。我们建议首先使用较重的屏障指令保证逻辑的正确性，然后通过移除一些冗余屏障或在必要时切换到较轻的屏障来继续提高性能。对Arm内存模型和相关指令的深入理解，是对实现准确和高性能的Synchronization编程非常有必要的。

在附录部分，我们还介绍了内存模型工具（The litmus test suite），它可以帮助理解内存模型并在各种架构上验证程序。

关于以上内容更完整的讲解，请参考“Arm架构下的Synchronization概述和案例分析白皮书”。

参考文献

1. Arm®, “Arm® Architecture Reference Manual Armv8, for Armv8-A architecture profile Documentation” https://developer.arm.com/docs/ddi0487/latest
2. “The software suite diy7”
   http://diy.inria.fr/
3. “A working example of how to use the herd7 Memory Model Tool”
   https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/how-to-use-the-memory-model-tool
4. “How to generate litmus tests automatically with the diy7 tool”
   https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/generate-litmus-tests-automatically-diy7-tool
5. “Running litmus tests on hardware using litmus7”
   https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/running-litmus-tests-on-hardware-litmus7