When needing to perform some long-running command, potentially with lots of output, on a remote server, there are a few things you can do to make your life easier. Given the tooling is available (which it should), this works with any kind of server or connection.
screen for Long-Running Commands
You might have come across the
screen utility before. For any long-running command on a remote server, where you are prone to time-outs or other ways of having your connection closed (e.g., due to a network disconnect), I would strongly advise to use a
If you want to read more about that, I can recommend this nice little tutorial, as well as a much more complete blog post on using
screen. For your everyday usage, you only really need two/three commands, though.
screen available on your remote server, you can start a new session by just typing:
If you want to execute multiple commands in dedicated sessions, or just as a general best practice, you can name the session when creating it. For this, you need to pass the
-S option, followed by the unique name.
screen -S media
If you have been disconnected or closed your terminal deliberately, you can always connect back to the server. Depending on how things are set up and what has happened since when you started the
screen session, you may or may not automatically “land” in your
To see all running sessions, you can use:
In the output, you will see a unique ID, as well as the session name, if provided.
Reattach to a
If you do not connect back to the context of the session, or in case you explicitly detached from your session, you can use the following to reattach to it:
screen -r media
Please note that, in the above example,
media is the session name we provided before. If you did not provide a session name, you can reattach using the unique ID (see
Occasionally, I found that reattaching with
-r simply doesn’t work. I’m not sure why this is, maybe because the setup at hand was not a native linux server that I connected to and then executed a command, but instead a connection across multiple servers, using AWS tooling…?
Anyway, the way to definitely reattach to a
screen session is to explicitly detach from anything else—using
screen -dr media
Write the Output to a File
Depending on the command you are running, there may be lots and lots of output. Even when using
screen, you might run into issues when there is more output than what the buffer can hold.
While there are ways to configure the output buffer and/or
screen directly, another much more straightforward (one-off) way is to write the output to a file. You can do so by delegating it via
>, or if you want to append, use
[COMMAND] > [FILE]
wp media regenerate > media.txt
Also Redirect Error Output
If you also want to write any errors printed to the screen to your file, you have to reroute
stderr. You can do this like so:
[COMMAND] > [FILE] 2>&1
Essentially, this means that you want all errors to be written to the standard primary output. Actual example:
wp media regenerate > media.txt 2>&1
Write to File and Screen
Sometimes you want to write all outpt to the file, but also be able to just connect to the server and see what’s going on. Using
>>, all output is redirected, so you don’t see anything in your terminal.
As always, there are multiple ways to have a command write to both terminal and file, but a really easy one is to use the
Pipe all output to
tee and write to a file:
[COMMAND] | tee [FILE]
wp media regenerate | tee media.txt
Again, if you are interested in errors as well, you can do like so:
[COMMAND] 2>&1 | tee [FILE]
wp media regenerate 2>&1 | tee media.txt
Download the File from the Server
If you are wondering how you can get the file from the server, you should be able to use something like
rsync. For example, like so:
scp [SERVER]:[FILE] .
Profile the Command
Sometimes you are interested in how long something took to execute. A simple-enough way to do this is to use the
Granted, this is not specific to a remotely-executed command, but nonetheless useful.
Putting It All Together
Where that makes sense, you can also do all of the above. When working on migrating a large site from its dedicated stack ot another existing one, which also involved migrating all the uploads, and regenerating all the media files due to changed image sizes in the new theme, I did that quite a few times.
Kick it off:
screen -S media time wp media regenerate 2>&1 | tee media.txt
Re-attach, if necessary:
screen -dr media
Finally, pull down the file:
scp [SERVER]:media.txt .