Perform large-scale calculations on the remote Jupiter notebook to continue calculating even if the ssh runs out.

Asked 2 years ago, Updated 2 years ago, 104 views

I started Jupiter notebook on the screen command window of the server machine as follows and connected remotely on the local machine to use Jupiter notebook.
However, after performing a calculation that takes 2-3 weeks on the Jupiter notebook, disconnecting the ssh, and then connecting to the same Jupiter notebook running on the remote screen window again, the calculation ended in the middle.Is there any way that the cell execution of the Jupiter notebook will not be interrupted even if the ssh expires and the window of the Jupiter notebook drops?
Thank you for your cooperation.

*Remote to

screen
ipython notebook --no-browser --port=8889

After opening the jupyter notebook in
* Local to

 ssh-N-f-L 8888:localhost:8889 remote_user @remote_host

You are connecting to the jupyter notebook in .
After you turn off the ssh,

 ssh-N-f-L 8888:localhost:8889 remote_user @remote_host

You are back to the notebook where you were connecting to and performing the calculation.

python ssh jupyter-notebook

2022-09-29 22:39

2 Answers

I have a similar question in the English version.
Keep Jupiter notebook running after closing browser tab

According to this, the calculation continues even if the browser is closed, but the output destination disappears, so the result is not output.

The easiest solution is to capture stdout, stderr, and IPython's display()calls using the cell magic %%capture.

% capture output
# write a time-consuming code below

This saves all output to the variable output, so you can see all output with the following command after reconnecting:

output.show()


2022-09-29 22:39

https://github.com/NII-cloud-operation/Jupyter-LC_wrapper

This allows the wrapper kernel to save the output of the running (running instructions have been sent to Kernel) cell to a file even if the browser drops, so you can check the results.

Another possible approach would be to use ...papermil (https://github.com/nteract/papermill).


2022-09-29 22:39

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.