`test_error_on_parser_stack_overflow` from `test_syntax` triggers triggers-out-of-bounds memory protection for a debug build under WASI

[brettcannon]

Detected with WASI-SDK 20 and wasmtime 14.

Linked PRs

• GH-111807: Lower the parser stack depth under WASI debug builds #112225

[brettcannon]

Looks like an out-of-bounds memory access:

         1325: 0x36592 - <unknown>!factor_rule
         1326: 0x36027 - <unknown>!term_rule
         1327: 0x351fb - <unknown>!sum_rule
         1328: 0x34a30 - <unknown>!shift_expr_rule
         1329: 0x34265 - <unknown>!bitwise_and_rule
         1330: 0x32dc2 - <unknown>!bitwise_xor_rule
         1331: 0x2dc0e - <unknown>!bitwise_or_rule
         1332: 0x47aed - <unknown>!comparison_rule
         1333: 0x47867 - <unknown>!inversion_rule
         1334: 0x43af8 - <unknown>!conjunction_rule
         1335: 0x2f2c9 - <unknown>!disjunction_rule
         1336: 0x14fde - <unknown>!expression_rule
         1337: 0x4ac44 - <unknown>!star_expression_rule
         1338: 0x24e67 - <unknown>!star_expressions_rule
         1339: 0x1fcd4 - <unknown>!simple_stmt_rule
         1340: 0x1321b - <unknown>!simple_stmts_rule
         1341: 0x114c0 - <unknown>!statements_rule
         1342: 0xd694 - <unknown>!_PyPegen_parse
         1343: 0x76a8 - <unknown>!_PyPegen_run_parser
         1344: 0x7ba3 - <unknown>!_PyPegen_run_parser_from_string
         1345: 0x8f1b1 - <unknown>!_PyParser_ASTFromString
         1346: 0x318dec - <unknown>!Py_CompileStringObject
         1347: 0x242f85 - <unknown>!builtin_compile
         1348: 0x13e0bd - <unknown>!cfunction_vectorcall_FASTCALL_KEYWORDS
         1349: 0xc8db9 - <unknown>!_PyObject_VectorcallTstate
         1350: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1351: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1352: 0x26f5dd - <unknown>!_PyEval_Vector
         1353: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1354: 0xcc3f4 - <unknown>!_PyObject_VectorcallTstate
         1355: 0xcc231 - <unknown>!method_vectorcall
         1356: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1357: 0xc9ff7 - <unknown>!_PyObject_Call
         1358: 0xca159 - <unknown>!PyObject_Call
         1359: 0x2675a1 - <unknown>!_PyEval_EvalFrameDefault
         1360: 0x26f5dd - <unknown>!_PyEval_Vector
         1361: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1362: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1363: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1364: 0x18c442 - <unknown>!slot_tp_call
         1365: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1366: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1367: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1368: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1369: 0x26f5dd - <unknown>!_PyEval_Vector
         1370: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1371: 0xcc3f4 - <unknown>!_PyObject_VectorcallTstate
         1372: 0xcc231 - <unknown>!method_vectorcall
         1373: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1374: 0xc9ff7 - <unknown>!_PyObject_Call
         1375: 0xca159 - <unknown>!PyObject_Call
         1376: 0x2675a1 - <unknown>!_PyEval_EvalFrameDefault
         1377: 0x26f5dd - <unknown>!_PyEval_Vector
         1378: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1379: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1380: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1381: 0x18c442 - <unknown>!slot_tp_call
         1382: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1383: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1384: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1385: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1386: 0x26f5dd - <unknown>!_PyEval_Vector
         1387: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1388: 0xcc3f4 - <unknown>!_PyObject_VectorcallTstate
         1389: 0xcc231 - <unknown>!method_vectorcall
         1390: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1391: 0xc9ff7 - <unknown>!_PyObject_Call
         1392: 0xca159 - <unknown>!PyObject_Call
         1393: 0x2675a1 - <unknown>!_PyEval_EvalFrameDefault
         1394: 0x26f5dd - <unknown>!_PyEval_Vector
         1395: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1396: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1397: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1398: 0x18c442 - <unknown>!slot_tp_call
         1399: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1400: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1401: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1402: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1403: 0x26f5dd - <unknown>!_PyEval_Vector
         1404: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1405: 0xc9282 - <unknown>!_PyObject_VectorcallDictTstate
         1406: 0xca3fe - <unknown>!_PyObject_Call_Prepend
         1407: 0x18d4e5 - <unknown>!slot_tp_init
         1408: 0x18197a - <unknown>!type_call
         1409: 0xc90af - <unknown>!_PyObject_MakeTpCall
         1410: 0xc8da3 - <unknown>!_PyObject_VectorcallTstate
         1411: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1412: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1413: 0x248cc4 - <unknown>!PyEval_EvalCode
         1414: 0x243a85 - <unknown>!builtin_exec
         1415: 0x13e0bd - <unknown>!cfunction_vectorcall_FASTCALL_KEYWORDS
         1416: 0xc8db9 - <unknown>!_PyObject_VectorcallTstate
         1417: 0xc9f44 - <unknown>!PyObject_Vectorcall
         1418: 0x2695da - <unknown>!_PyEval_EvalFrameDefault
         1419: 0x26f5dd - <unknown>!_PyEval_Vector
         1420: 0xca244 - <unknown>!_PyFunction_Vectorcall
         1421: 0xc9e3e - <unknown>!_PyVectorcall_Call
         1422: 0xc9ff7 - <unknown>!_PyObject_Call
         1423: 0xca159 - <unknown>!PyObject_Call
         1424: 0x343f99 - <unknown>!pymain_run_module
         1425: 0x343461 - <unknown>!Py_RunMain
         1426: 0x3445e9 - <unknown>!pymain_main
         1427: 0x344692 - <unknown>!Py_BytesMain
         1428: 0x5987 - <unknown>!main
         1429: 0x52c681 - <unknown>!__main_void
         1430: 0x5960 - <unknown>!_start
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    2: memory fault at wasm address 0x10000002c in linear memory of size 0xb30000
    3: wasm trap: out of bounds memory access

[brettcannon]

For some reason this problem screams @pablogsal to me. 😉 I wonder if there's a missing check on whether memory allocation succeeded or failed?

[pablogsal]

This looks like the builder is stack overflowing on this platform. We do have a protection based on depth, but seems like the stack in the platform is not big enough. We can lower it flor WASI if we are able to detect it at compile time 🤔

[pablogsal]

If is not this, we need more debugging information here to understand how the out of memory access is happening, but my bet is certainly on a stack overflow.

[pablogsal]

Ah, we already have it set to 4000 here:

https://github.com/python/cpython/blob/31c90d5838e8d6e4c47d98500a34810ccb33a6d4/Parser/parser.c#L11

maybe we need to lower it even more for debug builds?

[brettcannon]

Ah, another stack depth setting!

I can play with the value to see where things trigger. Is there a "reasonable" minimum to decide whether it's worth differentiating between a debug build or not under WASI?

[pablogsal]

The debug mode shouldn't affect parser depth or at least I don't see how that may be possible. It may affect stack size of every frame but the difference shouldn't be too different.

With this I mean that it should not matter a lot so maybe is worth checking empirically what depth causes it to crash in both modes.

[brettcannon]

It looks like the value needs to be 1325 or lower. I'll do a PR to set it to 1000 under WASI when in a debug build.