pythongh-78214: marshal: Stabilize FLAG_REF usage (pythonGH-8226)

Use FLAG_REF always for interned strings. Refcounts of interned string is very unstable. When compiling same source, refcounts of interned string in the output may be 1 or >1. It makes FLAG_REF usage unstable. To help reproducible build, use FLAG_REF for interned string even if refcnt(obj)==1.
evildmp · May 4, 2022 · 6dcfd6c · 6dcfd6c
1 parent dfb1b9d
commit 6dcfd6c
Show file tree

Hide file tree

Showing 3 changed files with 17 additions and 10 deletions.
diff --git a/Misc/NEWS.d/next/Core and Builtins/2018-07-14-16-58-00.bpo-34093.WaVD-f.rst b/Misc/NEWS.d/next/Core and Builtins/2018-07-14-16-58-00.bpo-34093.WaVD-f.rst
@@ -0,0 +1,2 @@
+``marshal.dumps()`` uses ``FLAG_REF`` for all interned strings.  This makes
+output more deterministic and helps reproducible build.
diff --git a/Programs/test_frozenmain.h b/Programs/test_frozenmain.h
diff --git a/Python/marshal.c b/Python/marshal.c
@@ -298,9 +298,14 @@ w_ref(PyObject *v, char *flag, WFILE *p)
     if (p->version < 3 || p->hashtable == NULL)
         return 0; /* not writing object references */
 
-    /* if it has only one reference, it definitely isn't shared */
-    if (Py_REFCNT(v) == 1)
+    /* If it has only one reference, it definitely isn't shared.
+     * But we use TYPE_REF always for interned string, to PYC file stable
+     * as possible.
+     */
+    if (Py_REFCNT(v) == 1 &&
+            !(PyUnicode_CheckExact(v) && PyUnicode_CHECK_INTERNED(v))) {
         return 0;
+    }
 
     entry = _Py_hashtable_get_entry(p->hashtable, v);
     if (entry != NULL) {