Skip to content

GH-115869: Make jit_stencils.h reproducible - #127166

Merged
brandtbucher merged 6 commits into
python:mainfrom
brandtbucher:justin-reproducible
Nov 24, 2024
Merged

GH-115869: Make jit_stencils.h reproducible#127166
brandtbucher merged 6 commits into
python:mainfrom
brandtbucher:justin-reproducible

Conversation

@brandtbucher

@brandtbucher brandtbucher commented Nov 22, 2024

Copy link
Copy Markdown
Member

At the core dev sprint, we discussed the possibility of hosting pre-built jit_stencils.h files for common platforms and build configurations.

While there's a lot of work involved in making that happen (I'll post a rough plan shortly), this is the first logical step in that direction. This PR replaces absolute file paths in the generated code with relative ones, and processes stencils in sorted order.

@brandtbucher brandtbucher added build The build process and cross-build 3.14 bugs and security fixes topic-JIT labels Nov 22, 2024
@brandtbucher brandtbucher self-assigned this Nov 22, 2024
Comment thread Tools/jit/_stencils.py
offset = len(self.body)
padding = -offset % alignment
self.disassembly.append(f"{offset:x}: {' '.join(['00'] * padding)}")
if padding:

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unrelated cleanup.

@savannahostrowski savannahostrowski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@brandtbucher
brandtbucher merged commit 17c16ae into python:main Nov 24, 2024
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…ythonGH-127166)

llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory().  That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:

     // /tmp/tmp164olvmi/_BINARY_OP.o:  file format elf64-x86-64
    +// /tmp/tmpmufvewt2/_BINARY_OP.o:  file format elf64-x86-64

Replace the full path with the bare file name.

Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.

(cherry picked from commit 17c16ae)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
Keep the build tmpdir out of jit_stencils.h (pythonGH-127166)

llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory().  That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:

     // /tmp/tmp164olvmi/_BINARY_OP.o:  file format elf64-x86-64
    +// /tmp/tmpmufvewt2/_BINARY_OP.o:  file format elf64-x86-64

Replace the full path with the bare file name.

Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.

(cherry picked from commit 17c16ae)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…7166)

Keep the build tmpdir out of jit_stencils.h

llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory().  That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:

     // /tmp/tmp164olvmi/_BINARY_OP.o:  file format elf64-x86-64
    +// /tmp/tmpmufvewt2/_BINARY_OP.o:  file format elf64-x86-64

Replace the full path with the bare file name.

Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.

(cherry picked from commit 17c16ae)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
@bedevere-app

bedevere-app Bot commented Jul 31, 2026

Copy link
Copy Markdown

GH-154988 is a backport of this pull request to the 3.13 branch.

bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…7166)

Keep the build tmpdir out of jit_stencils.h

llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory().  That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:

     // /tmp/tmp164olvmi/_BINARY_OP.o:  file format elf64-x86-64
    +// /tmp/tmpmufvewt2/_BINARY_OP.o:  file format elf64-x86-64

Replace the full path with the bare file name.

Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.

(cherry picked from commit 17c16ae)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…7166)

Keep the build tmpdir out of jit_stencils.h

llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory().  That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:

     // /tmp/tmp164olvmi/_BINARY_OP.o:  file format elf64-x86-64
    +// /tmp/tmpmufvewt2/_BINARY_OP.o:  file format elf64-x86-64

Replace the full path with the bare file name.

Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.

(cherry picked from commit 17c16ae)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.14 bugs and security fixes build The build process and cross-build topic-JIT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants