JupyterHub base URL does not load until user data cleared

JupyterHub
1

Hi all :waving_hand:,

Recently, I’ve been seeing a lot of issues with JupyterHub not loading with a couple of deployments (both running v5.3.0). We’ve had occasional issues of this type in the past, however, they have become much more frequent of late.

The most common symptom is a blank screen when visiting the JupyterHub URL (no content on page at all).

The situation is generally resolved/worked-around by:

  • Clearing cookies and site data for the JupyterHub domain.
  • Using a private tab.
  • Using another web browser.
  • Clearing cookies and site data, then restarting the browser (fix for Firefox, associated with the NS_BINDING_ABORTED message).

I suspect that there is some issue with (outdated?) authentication data in the browser, that is blocking the usual authentication/redirection process? Are there any known issues in this area (nothing obvious in GH issues)?

Details:

  • JupyterHub 5.3.0 running on Linux.
  • PAM authenticator.
  • Jupyter Server 2.16.0 (requests don’t get through to the server).
  • Issue reported with a mix of web browsers on Windows/Linux.
  • Hub and servers routinely restarted.

There aren’t any obvious log messages associate with this and the browser connection errors are unclear. I’m getting daily reports of this issue so can easily obtain diagnostics. I’m not sure how to proceed with debugging this, any ideas?

2

Try turning on debug logging in JupyterHub, and share your logs when a failure occurs, along with the logs from your browser console. Include the logs preceding the error as they’ll often contain useful context.

Do you have a proxy in front of JupyterHub, or a firewall/proxy/antivirus elsewhere on your network or computer?

You can also try narrowing down the problem, e.g. temporarily replace PamAuthenticator with the DummyAuthenticator to rule out PAM being the problem.

2 Likes
3

Thanks for the suggestions.

Try turning on debug logging in JupyterHub

Will do.


Do you have a proxy in front of JupyterHub

Nothing I’m aware of. Anything I should ask about / look for if there is one?


You can also try narrowing down the problem, e.g. temporarily replace PamAuthenticator with the DummyAuthenticator to rule out PAM being the problem.

Will try. The issue is highly intermittent, it reproduces regularly in a live deployment because there are lots of users with lots of browser tabs, likely hard to reproduce in a test deployment.

4

If you’re not sure you could try asking your organisation’s IT department? They might have access to logs, or explain what software is installed on devices or the network. If the problem is in JupyterHub you’d normally see an error message somewhere-is the window completely blank, or is it hanging?

Are the operating system users managed locally, or is the system using LDAP/AD?

5

Thanks for your help.

It’s a frustrating problem to debug as the issue never reproduces when you need it to.

Anecdotally, it’s possible that issues may be associated with Jupyter Hub restarts, but I cannot confirm or deny this at present.


Do you have a proxy in front of JupyterHub

Have asked, there is no proxy involved (beyond configurable-http-proxy).


is the window completely blank, or is it hanging?

The window is completely blank, the browser tab displays as loading. I’ll try and gather some more detailed evidence from the users.


Are the operating system users managed locally, or is the system using LDAP/AD?

LDAP/AD.


Try turning on debug logging in JupyterHub

Have done, will take effect when the system restarts at the weekend.