Ayon menu crashing a lot

[MustafaJafar]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior:

a client is using a lot of external addons and scripts and it seems something is colliding with Ayon menu,
There is no log provided, unfortunately.

Expected Behavior:

It should not error on collect and instead provide a nice validation report that the output node is not set or incorrectly configured (which I think may already be the case if the error on the collector itself is resolved.)

Version

1.0.0

What platform you are running on?

Windows

Steps To Reproduce:

..

Are there any labels you wish to add?

• I have added the relevant labels to the bug report.

Relevant log output:

No response

Additional context:

No response

[BigRoy]

external addons and scripts

What external addons and scripts?
When does it crash? How often? All machines or just some? Does it crash (reproducable) in certain cases by doing certain actions?

Or better...

Is this Houdini 20.5+? (Since it's in ayon-houdini) Because there are crashes that I've had (and others like @krishna8008) with Houdini 20.5 when clicking on any of the AYON menu entries. Is that the error here?

[BigRoy]

Just reporting some findings here:

I don't have a fix, but it does seem like a SideFX Houdini Bug.

Details

1. I have a bug reported under ticket number 140456 and have found others that also faced the issue - one of their bug reports is ticket number 136866.
2. It seems to not be AYON specific - and can occur on just accessing the QtWidgets.QApplication.instance() call.

Potential workaround
I haven't found a workaround that solves it except that I've heard - it does not occur with the Houdini 20.5+ Py3.10 builds. So if that works for your production - a workaround may be to install the Hou 20.5 releases using the Py3.10 versions (note that you'll need the renderer plug-ins to also match the Py3.10 versions).

---

And here's a part of my bug report to SideFX:

Steps to reproduce

After launch of Houdini, run this:

from PySide2.QtWidgets import QApplication
QApplication.instance()

It's unfortunately hard to reproduce when it happens exactly. When it didn't crash - it seems that it'll continue to work for that session but whenever it crashes it's gone instantly. There's a brief moment Houdini Console shows a "SegmentationFault" message before it disappears.

I can't seem to reproduce this easily from a Python Shell in Houdini, but I can OFTEN reproduce it from a custom houdini menu (.xml) file with a script item like:

            <scriptItem id="my_crash_debug">
                <label>DEBUG</label>
                <scriptCode><![CDATA[

from PySide2 import QtWidgets
QtWidgets.QApplication.instance()
]]> </scriptCode>
            </scriptItem>

Best way to reproduce is launch Houdini 20 times with such a custom menu with that script item in there. Then on each of them directly after launch, go to the menu and click the button. Sometimes it crashes most of the times, then it seems rarely... then you retry it later and again 80 of the sessions crash. Hard to put a finger on.

[BigRoy]

Together with SideFX support we may have found another workaround.

When setting env var HOUDINI_DISABLE_BACKGROUND_HELP_INDEXING to 1 the crashing does not seem to happen. Both me and SideFX were unable to reproduce it with that env var set - even in AYON. ❤️

So, setting that env var in application environment settings should resolve the crahes.

It's in ayon+settings://applications/applications/houdini/environment.

For example see this line in the screenshot:

    "HOUDINI_DISABLE_BACKGROUND_HELP_INDEXING": "1",

[krishnaavril]

Together with SideFX support we may have found another workaround.

When setting env var HOUDINI_DISABLE_BACKGROUND_HELP_INDEXING to 1 the crashing does not seem to happen. Both me and SideFX were unable to reproduce it with that env var set - even in AYON. ❤️

So, setting that env var in application environment settings should resolve the crahes.

It's in ayon+settings://applications/applications/houdini/environment.

For example see this line in the screenshot:

    "HOUDINI_DISABLE_BACKGROUND_HELP_INDEXING": "1",

Colorbleed is awesome!

[MustafaJafar]

Looks like it solved the issue for @krishnaavril!
@BigRoy Should we add it to environment variables for Houdini 20.5 in application settings ?

[BigRoy]

Should we add it to environment variables for Houdini 20.5 in application settings ?

We could - but I'm still getting some crashes. Plus, from the tech community I've just received another "reproducable" that apparently can still make it crash with this environment variable set. So I'll be testing that soon and if it also crashes, follow up to SideFX for that.

This is the reproducable that I got:

This seems pretty reproducable on my machine right now, but haven't tested it over a longer time or on other PCs:
In your userprefs create a python3.11libs\pythonrc.py file with this content:

import sys
from PySide2.QtWidgets import *
pluginPath = r"C:\tmp"
sys.path.append(pluginPath)
import houcrashtest
QApplication.instance()

Then create an empty file here: C:\tmp\houcrashtest.py and start Houdini.
It will create a __pycache__ folder in C:\tmp and then crash Houdini. When starting Houdini again it works fine. When deleting the __pycache__ folder and starting Houdini, it crashes again.

If that does crash for me. I'll also start testing whether setting PYTHONDONTWRITEBYTECODE to 1 would make any difference on the crashing since that should disable Python to write the byte code.

And of course, will share the details with SideFX again.

Anyway, it seems the environment variable HOUDINI_DISABLE_BACKGROUND_HELP_INDEXING is not the holy grail yet.

[krishnaavril]

Did you heard any response from the Sidefx team? @BigRoy

[BigRoy]

Nothing new - so for now my best recommendation is use the Py3.10 build of Houdini because that is stable it seems.

I've just now followed up on this to them and asked whether there's any news since beginning of last week.

[BigRoy]

I've had steady updates from SideFX about this bug report - they've gone deep into trying to figure out what the solution is. The issue they are facing is that as soon as they make a debug build, the crash is gone. It only happens with the release builds.

They've tried different internal builds of Houdini to see if any updates in PySide2 fixes it:

• PySide2 5.15.2 crashes
• PySide2 5.15.11 crashes
• PySide2 5.15.15 crashes
• PySide6 6.5.3 works

The issue hence seems Python 3.11 + PySide2 related (which officially had limited support as a combination anyway and only ever got an official release with PySide2 5.15.11 with Python 3.11 support - SideFX was using a custom build of their own of 5.15.2 to make it Py3.11 compatible).

So - where does it leave us? They can go down the rabbit hole further with little to go on (due to not being able to attach a debugger to the crash) OR decide to move their efforts on the switch to Qt6 / PySide6 which is on their roadmap anyway. I've told them - moving on and focusing there is time likely better spent.

That leaves us:

• Use Houdini 20.5 Python 3.10 builds as long as that ships with PySide2. This works now!
• Once Houdini ships with PySide6 the move to Python 3.11 builds makes more sense again.